CN113688181A

CN113688181A - Data processing method and device, electronic equipment and computer readable medium

Info

Publication number: CN113688181A
Application number: CN202111041081.6A
Authority: CN
Inventors: 张志君
Original assignee: Beijing Jingdong Century Trading Co Ltd; Beijing Wodong Tianjun Information Technology Co Ltd
Current assignee: Beijing Jingdong Century Trading Co Ltd; Beijing Wodong Tianjun Information Technology Co Ltd
Priority date: 2021-09-07
Filing date: 2021-09-07
Publication date: 2021-11-23

Abstract

The embodiment of the disclosure discloses a data processing method, a data processing device, an electronic device and a computer readable medium. One embodiment of the method comprises: in response to receiving a request for target data sent by a target client, acquiring first metadata information associated with the target data; determining at least one data interface associated with the target data; generating at least one second metadata information associated with the at least one data interface; generating a data set associated with the at least one second metadata information; and performing data arrangement processing on at least one data associated with the first metadata information in the data set to obtain the target data. The implementation method can quickly and efficiently generate the target data required by the target client.

Description

Data processing method and device, electronic equipment and computer readable medium

Technical Field

Embodiments of the present disclosure relate to the field of computer technologies, and in particular, to a data processing method and apparatus, an electronic device, and a computer-readable medium.

Background

At present, a front-end and back-end interactive call operation often exists in an application system, and the acquisition of front-end and back-end data can be realized through the interactive call operation. For the acquisition of data, the following methods are generally adopted: the server provides an Application Programming Interface (API) for the client, and the client may transmit parameters by requesting the API, so as to obtain data required by the client.

However, when the data is acquired in the above manner, there are often technical problems as follows:

the data returned by the application program interface often has field information such as redundant fields, historical version fields and the like, and the required fields and data structures cannot be acquired as required, so that the returned data is often complex, the performance is reduced, and the version maintenance is difficult. In addition, the redundant field and the field information such as the history version field also increase the Input/Output (I/O) consumption, which affects the processing load of the computer.

Disclosure of Invention

This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the detailed description. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.

Some embodiments of the present disclosure propose data processing methods, apparatuses, electronic devices and computer readable media to solve one or more of the technical problems mentioned in the background section above.

In a first aspect, some embodiments of the present disclosure provide a data processing method, including: in response to receiving a request for target data sent by a target client, acquiring first metadata information associated with the target data; determining at least one data interface associated with the target data; generating at least one second metadata information associated with the at least one data interface; generating a data set associated with the at least one second metadata information; and performing data arrangement processing on at least one piece of data associated with the first metadata information in the data set to obtain the target data.

Optionally, the obtaining, in response to receiving a request for target data sent by a target client, first metadata information associated with the target data includes: and in response to receiving a request for target data sent by the target client, instructing the interface request gateway to acquire first metadata information associated with the target data, wherein the request meets a request format requirement corresponding to the interface request gateway.

Optionally, the generating at least one second metadata information associated with the at least one data interface includes: analyzing each data interface in the at least one data interface to generate analyzed data information to obtain a data information set; performing data conversion on each data information in the data information set according to the data conversion configuration information to generate converted data information, and obtaining a converted data information set as at least one second metadata information; and carrying out structured storage on the at least one piece of second metadata information.

Optionally, the generating a data set associated with the at least one second metadata information includes: performing parameter conversion on the at least one second metadata information to obtain at least one converted second metadata information; and calling the at least one data interface according to the transformed at least one second metadata information and the interface execution sequence of the at least one data interface to generate the data set.

Optionally, the analyzing each data interface of the at least one data interface to generate analyzed data information, so as to obtain a data information set, includes: and analyzing the data interface in a mode corresponding to the interface type according to the interface type of each data interface in the at least one data interface to generate analyzed data information and obtain a data information set.

Optionally, before analyzing each data interface of the at least one data interface to generate analyzed data information and obtain a data information set, the method further includes: and performing registration processing on each data interface in the at least one data interface.

In a second aspect, some embodiments of the present disclosure provide a data processing apparatus comprising: an acquisition unit configured to acquire first metadata information associated with target data in response to receiving a request for the target data sent by a target client; a determining unit configured to determine at least one data interface associated with the target data; a first generating unit configured to generate at least one second metadata information associated with the at least one data interface; a second generating unit configured to generate a data set associated with the at least one piece of second metadata information; and the data arrangement processing unit is configured to perform data arrangement processing on at least one data associated with the first metadata information in the data set to obtain the target data.

Optionally, the obtaining unit is further configured to: and in response to receiving a request for target data sent by the target client, instructing the interface request gateway to acquire first metadata information associated with the target data, wherein the request meets a request format requirement corresponding to the interface request gateway.

Optionally, the first generating unit is further configured to: analyzing each data interface in the at least one data interface to generate analyzed data information to obtain a data information set; performing data conversion on each data information in the data information set according to the data conversion configuration information to generate converted data information, and obtaining a converted data information set as at least one second metadata information; and carrying out structured storage on the at least one piece of second metadata information.

Optionally, the second generating unit is further configured to: performing parameter conversion on the at least one second metadata information to obtain at least one converted second metadata information; and calling the at least one data interface according to the transformed at least one second metadata information and the interface execution sequence of the at least one data interface to generate the data set.

Optionally, the first generating unit is further configured to: and analyzing the data interface in a mode corresponding to the interface type according to the interface type of each data interface in the at least one data interface to generate analyzed data information and obtain a data information set.

Optionally, the apparatus further comprises: and performing registration processing on each data interface in the at least one data interface.

In a third aspect, some embodiments of the present disclosure provide an electronic device, comprising: one or more processors; a storage device having one or more programs stored thereon, which when executed by one or more processors, cause the one or more processors to implement the method as described in any of the implementations of the first aspect.

In a fourth aspect, some embodiments of the disclosure provide a computer readable medium having a computer program stored thereon, where the program when executed by a processor implements a method as described in any of the implementations of the first aspect.

The above embodiments of the present disclosure have the following beneficial effects: the data processing method of some embodiments of the present disclosure can quickly and efficiently generate target data required by a target client. Specifically, field information such as redundant fields and historical version fields often exists in data returned by an application program interface, and required fields and data structures cannot be acquired as required, so that the returned data is often complex, the performance is reduced, and the version maintenance is difficult. In addition, the redundant field and the field information such as the history version field also increase the Input/Output (I/O) consumption, which affects the processing load of the computer. Based on this, the data processing method of some embodiments of the present disclosure may first obtain, in response to receiving a request for target data sent by a target client, first metadata information associated with the target data. Here, the first metadata information is acquired for the acquisition of the subsequent target data. Target data required by the target client can be acquired from the data set according to needs through the first metadata information. Then, at least one data interface associated with the target data is determined. At least one data interface associated with the target data is determined for subsequent acquisition of the target data. Here, a relatively full amount of the data set for constituting the target data is available via the at least one data interface. And further, generating a data set associated with the at least one second metadata information for generating the target data. And finally, performing data arrangement processing on at least one data associated with the first metadata information in the data set to obtain the target data. At least one piece of data associated with the first metadata information in the data set is subjected to data arrangement processing, so that target data required by a target client can be generated quickly and efficiently. Furthermore, fields such as redundant fields and historical version fields can be eliminated, required fields and data structures can be obtained as required, and the complexity of returned data is reduced.

Drawings

The above and other features, advantages and aspects of various embodiments of the present disclosure will become more apparent by referring to the following detailed description when taken in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numbers refer to the same or similar elements. It should be understood that the drawings are schematic and that elements and features are not necessarily drawn to scale.

FIG. 1 is a schematic diagram of one application scenario of a data processing method according to some embodiments of the present disclosure;

FIG. 2 is a flow diagram of some embodiments of a data processing method according to the present disclosure;

FIG. 3 is a schematic illustration of second metadata information corresponding data in some embodiments of a data processing method according to the present disclosure;

FIG. 4 is a flow diagram of further embodiments of a data processing method according to the present disclosure;

FIG. 5 is a schematic block diagram of some embodiments of a data processing apparatus according to the present disclosure;

FIG. 6 is a schematic structural diagram of an electronic device suitable for use in implementing some embodiments of the present disclosure.

Detailed Description

Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it is to be understood that the disclosure may be embodied in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the disclosure are for illustration purposes only and are not intended to limit the scope of the disclosure.

It should be noted that, for convenience of description, only the portions related to the related invention are shown in the drawings. The embodiments and features of the embodiments in the present disclosure may be combined with each other without conflict.

It should be noted that the terms "first", "second", and the like in the present disclosure are only used for distinguishing different devices, modules or units, and are not used for limiting the order or interdependence relationship of the functions performed by the devices, modules or units.

It is noted that references to "a", "an", and "the" modifications in this disclosure are intended to be illustrative rather than limiting, and that those skilled in the art will recognize that "one or more" may be used unless the context clearly dictates otherwise.

The names of messages or information exchanged between devices in the embodiments of the present disclosure are for illustrative purposes only, and are not intended to limit the scope of the messages or information.

The present disclosure will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.

Fig. 1 is a schematic diagram of one application scenario of a data processing method according to some embodiments of the present disclosure.

In the application scenario of fig. 1, in response to receiving a request for target data 103 sent by target client 102, electronic device 101 may first obtain first metadata information 104 associated with the target data 103. The electronic device 101 may then determine at least one data interface 105 associated with the target data 103. In this application scenario, the at least one data interface 105 may include: data interface 1051, data interface 1052, and data interface 1053. Further, the electronic device 101 may obtain at least one second metadata information 106 associated with the at least one data interface 105. In this application scenario, the at least one second metadata information 106 includes: second metadata information 1061 corresponding to the data interface 1051, second metadata information 1062 corresponding to the data interface 1052, and second metadata information 1063 corresponding to the data interface 1053. Next, the electronic device 101 may generate a data set 107 associated with the at least one second metadata information 106; in the application scenario, the data set 107 may include: data 1071, data 1072, and data 1073. Finally, the electronic device 101 may perform data layout processing on at least one data associated with the first metadata information 104 in the data set 107 to obtain the target data 103. In the application scenario, the at least one data may include: data 1071 and data 1072.

The electronic device 101 may be hardware or software. When the electronic device is hardware, the electronic device may be implemented as a distributed cluster formed by a plurality of servers or terminal devices, or may be implemented as a single server or a single terminal device. When the electronic device is embodied as software, it may be installed in the above-listed hardware devices. It may be implemented, for example, as multiple software or software modules to provide distributed services, or as a single software or software module. And is not particularly limited herein.

It should be understood that the number of electronic devices in fig. 1 is merely illustrative. There may be any number of electronic devices, as desired for implementation.

With continued reference to fig. 2, a flow 200 of some embodiments of a data processing method according to the present disclosure is shown. The data processing method comprises the following steps:

step 201, in response to receiving a request for target data sent by a target client, obtaining first metadata information associated with the target data.

In some embodiments, in response to receiving a request for target data sent by a target client, first metadata information associated with the target data is acquired, and an execution subject (e.g., an electronic device shown in fig. 1) of the data processing method may acquire the first metadata information associated with the target data through a wired connection manner or a wireless connection manner. The target client may be a certain client that is to obtain data. The client may be a module that accomplishes the objective task. The target data may be data required by the target client. The above-mentioned first metadata information may be metadata information describing the target data.

As an example, the first metadata information may be: "" skuId ": 212332433333" & "skuId": 233332433222"&" skuId ": 212111133333" ". The first metadata information may characterize data fusion between an item image and item description for item identification "12332433333", an item image and item description for item identification "233332433222", and an item image and item description for item identification "212111133333".

In some optional implementations of some embodiments, in response to receiving a request for target data sent by the target client, the execution body may instruct the interface request gateway to obtain first metadata information associated with the target data. Wherein, the request meets the request format requirement corresponding to the interface request gateway. The interface request gateway is preset corresponding to the request format requirement. The interface request gateway can be a universal layer of a universal appearance mode, standardizes input and output and other contents, and appoints an interactive description language and the like. The interface request gateway unifies the request standards of the client, so that a plurality of client requests have unified standards, and the maintenance of subsequent standards is facilitated.

At step 202, at least one data interface associated with the target data is determined.

In some embodiments, the execution agent may determine at least one data interface associated with the target data. Wherein the at least one data interface may be an interface for subsequently generating the target data. Here, the data set can be acquired via at least one data interface. The target data may be generated from the data set. For example, the target data may be data obtained by performing data processing on a target data subset in the data set.

As an example, the execution subject may first determine a data set associated with the target data. The execution agent may then query the target database for at least one data interface of the data set with which the target data is associated. Wherein, the target database stores a table representing the association relationship between the data interface and the data.

Step 203, generating at least one second metadata information associated with the at least one data interface.

In some embodiments, the execution agent may generate at least one second metadata information associated with the at least one data interface. The at least one second metadata information may be information describing an attribute of each interface of the at least one data interface. The interface attribute may be attribute information of the data interface. For example. Interface attributes of the data interface may include, but are not limited to, at least one of: request predicates (POST/GET), domain names (domainx.) com, module names (md), component names (cmp), method names (method), business parameters (bizparam), and system parameters (uuid, pin).

As an example, the execution subject may directly parse the interface attribute information of each data interface of the at least one data interface through the interface attribute splitting rule to generate parsed metadata information, and obtain a parsed metadata information set as the at least one second metadata information. The interface attribute splitting rule may be written in advance.

As yet another example, the data interface may be: "http:// domainx. jt. com/md/cmp/method 1? bizparam { "appId":4521, "stock eId":234, "pageNum":1, "pageSize":20, "skuId": 212332433333, "" platform ":1," bizType ":1," source ": source1," "feature flag":0, "token": "" deviceUU Id ": df6051b8e2994b673a1c012c01c4dce f823de0," "systembaseinfofo": { "SDKVersion": 1.10.2.2, "" hostsionname ":"2, "" system ": step": ios13.6, "" ospe ": 39599 } duid { (3975) 6051b 673 b 6712 { (3975 { -3 j 3 b673 j 3 b 6712.

The second metadata information corresponding to the data interface may be: "domain": domain, jd, com ",

and step 204, generating a data set associated with the at least one second metadata information.

In some embodiments, the execution subject may generate a data set associated with the at least one second metadata information. The second metadata information represents each interface attribute of the corresponding data interface. Some of the various interface attributes of the data interface are associated with data to which the data interface is linked. The data in the data set may be data in various data formats. For example, the data set includes: image data, text data, video data.

Here, the generation of the data set may be done in the task execution engine. The task execution engine can perform multitask data arrangement and aggregation according to the at least one second metadata information and the at least one data interface.

As an example, the data interface may be: "http:// domainx. jt. com/md/cmp/method 1? bizparam { "appId":4521, "stock eId":234, "pageNum":1, "pageSize":20, "skuId": 212332433333, "" platform ":1," bizType ":1," source ": source1," "feature flag":0, "token": "" deviceUU Id ": df6051b8e2994b673a1c012c01c4dce f823de0," "systembaseinfofo": { "SDKVersion": 1.10.2.2, "" hostsionname ":"2, "" system ": step": "ios13.6," "ospe": 39599 ": and } duuiod 6051b 673a 2964 b 6712 {" 3975 j 3 b673 b 6712 { (3975 j 3 b 1213 b 3 b 1213 b 3 c 1213 d 3 b 1213 b 3 c4 465 f823d 75 f823d 3 f 823. The interface attribute "skiid" may characterize the identification information of the item. "" skuId ": 212332433333" "may correspond to data for an item identification of" 212332433333 ".

For example, data for item identification "212332433333" may be a description of the item image and the item image correspondence. Reference may be made in particular to fig. 3.

In some optional implementations of some embodiments, the generating the data set associated with the at least one second metadata information may include:

and firstly, performing parameter conversion on the at least one second metadata information to obtain at least one converted second metadata information.

As an example, the executing entity may perform format conversion between entering and leaving parameters on the at least one second metadata information according to the parameter conversion configuration information, so as to obtain at least one converted second metadata information. Wherein, the parameter conversion configuration information may be preset.

As another example, the execution body may perform a chinese format conversion on the input parameter and the output parameter of the at least one second metadata information to obtain the transformed at least one second metadata information.

And a second step of calling the at least one data interface according to the transformed at least one second metadata information and the interface execution sequence of the at least one data interface to generate the data set. Wherein, the interface execution sequence may include: the interfaces are executed side by side and the interfaces are executed serially.

As an example, the execution subject may call the at least one data interface in a serial manner, and obtain a called data set. Then, the executing entity may determine, from the invoked data set, a data set associated with at least one piece of second metadata information according to the transformed at least one piece of second metadata information.

Step 204, performing data layout processing on at least one data associated with the first metadata information in the data set to obtain the target data.

In some embodiments, the executing entity may perform data arrangement processing on at least one data associated with the first metadata information in the data set to obtain the target data. The data layout processing may be processing for executing a corresponding data operation on at least one data according to a data presentation format of the data in the at least one data. The data operations may include, but are not limited to, at least one of: splicing data and operating the data.

As an example, in response to determining that the data presentation format of each of the at least one data is an image format, the execution subject may perform image stitching combination on the at least one data.

With further reference to fig. 4, a flow 400 of further embodiments of a data processing method according to the present disclosure is shown. The data processing method comprises the following steps:

step 401, in response to receiving a request for target data sent by a target client, obtaining first metadata information associated with the target data.

At step 402, at least one data interface associated with the target data is determined.

In some embodiments, the specific implementation of

steps

401 and 402 and the technical effects thereof can refer to

steps

201 and 202 in the embodiment corresponding to fig. 2, and are not described herein again.

Step 403, analyzing each data interface in the at least one data interface to generate analyzed data information, so as to obtain a data information set.

In some embodiments, an executing agent (e.g., the electronic device shown in fig. 1) may parse through each of the at least one data interface to generate parsed data information, resulting in a data information set. The analyzing the data interface may be analyzing an interface attribute of the data interface.

As an example, the execution subject may parse the interface attribute of each data interface of the at least one data interface through the interface attribute parsing script to generate parsed data information, so as to obtain a data information set.

As yet another example, the parsing of the data interface may be to indicate that the metadata information parsing module is complete.

In some optional implementation manners of some embodiments, according to an interface type of each data interface of the at least one data interface, the execution main body may analyze the data interface in a manner corresponding to the interface type to generate analyzed data information, so as to obtain a data information set.

As an example, for a general interface, the execution subject may parse the metadata information of the necessary interface at the metadata information parsing module through an automation process, and structurally describe such information to obtain a data information set.

For a relatively complex interface, on the operation of analyzing the general interface, the execution subject can manually supplement the part which is not completely analyzed through manual intervention to obtain a data information set.

Particularly, for a hypertext Transfer Protocol (HTTP) data interface, the execution body may instruct the metadata information parsing module to check parsing in and out parameters, a header request, a predicate, and the like of the data interface, so as to parse core information of the entire interface.

For some Remote Procedure Call (RPC) interfaces, because the parsing logic and the hypertext transfer protocol are different, the execution body may extend the parsing protocol in a manner of customizing an adapter, and support to customize multiple parsers. The data information set is generated by customizing various parsers.

In some optional implementation manners of some embodiments, before the parsing each data interface of the at least one data interface to generate parsed data information and obtain the data information set, the step further includes:

the execution subject may perform registration processing on each data interface of the at least one data interface.

As an example, the execution agent may register each data interface in the form of a relationship table.

The complicated operation caused by the subsequent re-calling of each data interface is reduced by performing the registration processing on each data interface.

Step 404, performing data conversion on each data information in the data information set according to the data conversion configuration information to generate converted data information, and obtaining a converted data information set as at least one second metadata information.

In some embodiments, the executing entity may perform data conversion on each data information in the data information set according to the data conversion configuration information to generate converted data information, and obtain a converted data information set as the at least one second metadata information. The data conversion configuration information may be pre-written configuration information requiring an interface attribute format.

As an example, the data information may be: "skuId" and "212332433333.00". The data conversion configuration information may include: the decimal point of the skiId holds 1 bit. That is, the converted data information is: "skuId" and "212332433333.0".

And step 405, performing structured storage on the at least one piece of second metadata information.

In some embodiments, the execution subject may structurally store the at least one second metadata information. Wherein the second metadata information may be metadata information in a certain structural format.

Here, by structurally storing at least one piece of second metadata information, the structural information of each piece of second metadata information can be effectively retained. In addition, the generation of each second metadata information is facilitated when the second metadata information is used again later.

As an example, the executing entity may store the at least one second metadata information in a database such as an association database or a column database.

Step 406, generating a data set associated with the at least one second metadata information.

Step 407, performing data layout processing on at least one data associated with the first metadata information in the data set to obtain the target data.

In some embodiments, the specific implementation of steps 406 and 407 and the technical effects thereof can refer to step 204 and 205 in the embodiment corresponding to fig. 2, which are not described herein again.

As can be seen from fig. 4, the flow 400 of the data processing method in some embodiments corresponding to fig. 4 highlights the specific step of generating the at least one second metadata information more than the description of some embodiments corresponding to fig. 2. Therefore, the solutions described in the embodiments can effectively and completely retain the content and the data structure of the metadata information by analyzing, converting and storing at least one data interface, and the generation efficiency of the subsequent target data is greatly improved laterally.

With further reference to fig. 5, as an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of a data processing apparatus, which correspond to those shown in fig. 2, and which may be applied in particular to various electronic devices.

As shown in fig. 5, a data processing apparatus 500 includes: an acquisition unit 501, a determination unit 502, a first generation unit 503, a second generation unit 504, and a data arrangement processing unit 505. The acquiring unit 501 is configured to, in response to receiving a request for target data sent by a target client, acquire first metadata information associated with the target data; a determining unit 502 configured to determine at least one data interface associated with the target data; a first generating unit 503 configured to acquire at least one second metadata information associated with the at least one data interface; a second generating unit 504 configured to generate a data set associated with the at least one piece of second metadata information; and a data arrangement processing unit 505 configured to perform data arrangement processing on at least one data associated with the first metadata information in the data set to obtain the target data.

In some optional implementations of some embodiments, the obtaining unit 501 in the apparatus 500 may be further configured to: and in response to receiving a request for target data sent by the target client, instructing the interface request gateway to acquire first metadata information associated with the target data, wherein the request meets a request format requirement corresponding to the interface request gateway.

In some optional implementations of some embodiments, the first generating unit 503 in the apparatus 500 may be further configured to: analyzing each data interface in the at least one data interface to generate analyzed data information to obtain a data information set; performing data conversion on each data information in the data information set according to the data conversion configuration information to generate converted data information, and obtaining a converted data information set as at least one second metadata information; and carrying out structured storage on the at least one piece of second metadata information.

In some optional implementations of some embodiments, the second generating unit 504 in the apparatus 500 described above may be further configured to: performing parameter conversion on the at least one second metadata information to obtain at least one converted second metadata information; and calling the at least one data interface according to the transformed at least one second metadata information and the interface execution sequence of the at least one data interface to generate the data set.

In some optional implementations of some embodiments, the first generating unit 503 in the apparatus 500 may be further configured to: and analyzing the data interface in a mode corresponding to the interface type according to the interface type of each data interface in the at least one data interface to generate analyzed data information and obtain a data information set.

In some optional implementations of some embodiments, the apparatus 500 further includes: a registration unit (not shown). Wherein the registration unit may be configured to: and performing registration processing on each data interface in the at least one data interface.

It will be understood that the elements described in the apparatus 500 correspond to various steps in the method described with reference to fig. 2. Thus, the operations, features and resulting advantages described above with respect to the method are also applicable to the apparatus 500 and the units included therein, and are not described herein again.

Referring now to FIG. 6, a block diagram of an electronic device (e.g., the electronic device of FIG. 1) 600 suitable for use in implementing some embodiments of the present disclosure is shown. The electronic device shown in fig. 6 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present disclosure.

As shown in fig. 6, electronic device 600 may include a processing means (e.g., central processing unit, graphics processor, etc.) 601 that may perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM)602 or a program loaded from a storage means 608 into a Random Access Memory (RAM) 603. In the RAM 603, various programs and data necessary for the operation of the electronic apparatus 600 are also stored. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.

Generally, the following devices may be connected to the I/O interface 605: input devices 606 including, for example, a touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 607 including, for example, a Liquid Crystal Display (LCD), a speaker, a vibrator, and the like; storage 608 including, for example, tape, hard disk, etc.; and a communication device 609. The communication means 609 may allow the electronic device 600 to communicate with other devices wirelessly or by wire to exchange data. While fig. 6 illustrates an electronic device 600 having various means, it is to be understood that not all illustrated means are required to be implemented or provided. More or fewer devices may alternatively be implemented or provided. Each block shown in fig. 6 may represent one device or may represent multiple devices as desired.

In particular, according to some embodiments of the present disclosure, the processes described above with reference to the flow diagrams may be implemented as computer software programs. For example, some embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In some such embodiments, the computer program may be downloaded and installed from a network through the communication device 609, or installed from the storage device 608, or installed from the ROM 602. The computer program, when executed by the processing device 601, performs the above-described functions defined in the methods of some embodiments of the present disclosure.

It should be noted that the computer readable medium described above in some embodiments of the present disclosure may be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In some embodiments of the disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In some embodiments of the present disclosure, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, optical cables, RF (radio frequency), etc., or any suitable combination of the foregoing.

In some embodiments, the clients, servers may communicate using any currently known or future developed network Protocol, such as HTTP (HyperText Transfer Protocol), and may interconnect with any form or medium of digital data communication (e.g., a communications network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), the Internet (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed network.

The computer readable medium may be embodied in the electronic device; or may exist separately without being assembled into the electronic device. The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: in response to receiving a request for target data sent by a target client, acquiring first metadata information associated with the target data; determining at least one data interface associated with the target data; generating at least one second metadata information associated with the at least one data interface; generating a data set associated with the at least one second metadata information; and performing data arrangement processing on at least one piece of data associated with the first metadata information in the data set to obtain the target data.

Computer program code for carrying out operations for embodiments of the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The units described in some embodiments of the present disclosure may be implemented by software, and may also be implemented by hardware. The described units may also be provided in a processor, and may be described as: a processor includes an acquisition unit, a determination unit, a first generation unit, a second generation unit, and a data arrangement processing unit. The names of these units do not in some cases constitute a limitation on the units themselves, and for example, the acquiring unit may also be described as a "unit that acquires first metadata information associated with target data in response to receiving a request for the target data sent by a target client".

The functions described herein above may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), systems on a chip (SOCs), Complex Programmable Logic Devices (CPLDs), and the like.

The foregoing description is only exemplary of the preferred embodiments of the disclosure and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the invention in the embodiments of the present disclosure is not limited to the specific combination of the above-mentioned features, but also encompasses other embodiments in which any combination of the above-mentioned features or their equivalents is made without departing from the inventive concept as defined above. For example, the above features and (but not limited to) technical features with similar functions disclosed in the embodiments of the present disclosure are mutually replaced to form the technical solution.

Claims

1. A method of data processing, comprising:

in response to receiving a request for target data sent by a target client, acquiring first metadata information associated with the target data;

determining at least one data interface associated with the target data;

generating at least one second metadata information associated with the at least one data interface;

generating a data set associated with the at least one second metadata information;

and performing data arrangement processing on at least one piece of data associated with the first metadata information in the data set to obtain the target data.

2. The method of claim 1, wherein the obtaining first metadata information associated with the target data in response to receiving a request for the target data sent by a target client comprises:

in response to receiving a request for target data sent by the target client, instructing the interface request gateway to obtain first metadata information associated with the target data, wherein the request meets a request format requirement corresponding to the interface request gateway.

3. The method of claim 1, wherein the generating at least one second metadata information associated with the at least one data interface comprises:

analyzing each data interface in the at least one data interface to generate analyzed data information to obtain a data information set;

performing data conversion on each data information in the data information set according to data conversion configuration information to generate converted data information, and obtaining a converted data information set as the at least one second metadata information;

and carrying out structured storage on the at least one second metadata information.

4. The method of claim 1, wherein the generating the data set associated with the at least one second metadata information comprises:

performing parameter conversion on the at least one second metadata information to obtain at least one converted second metadata information;

and calling the at least one data interface according to the transformed at least one second metadata information and the interface execution sequence of the at least one data interface to generate the data set.

5. The method of claim 3, wherein the parsing each of the at least one data interface to generate parsed data information resulting in a set of data information comprises:

and analyzing the data interface in a mode corresponding to the interface type according to the interface type of each data interface in the at least one data interface to generate analyzed data information and obtain a data information set.

6. The method of claim 3, wherein prior to said parsing each of the at least one data interface to generate parsed data information resulting in a set of data information, the method further comprises:

and performing registration processing on each data interface in the at least one data interface.

7. A data processing apparatus comprising:

an acquisition unit configured to acquire first metadata information associated with target data in response to receiving a request for the target data sent by a target client;

a determining unit configured to determine at least one data interface associated with the target data;

a first generating unit configured to generate at least one second metadata information associated with the at least one data interface;

a second generating unit configured to generate a data set associated with the at least one second metadata information;

and the data arrangement processing unit is configured to perform data arrangement processing on at least one data associated with the first metadata information in the data set to obtain the target data.

8. The apparatus of claim 7, wherein the first generating unit is further configured to:

performing data conversion on each data information in the data information set according to the data conversion configuration information to generate converted data information, and obtaining a converted data information set as at least one second metadata information;

9. An electronic device, comprising:

one or more processors;

a storage device having one or more programs stored thereon,

when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-6.

10. A computer-readable medium, on which a computer program is stored, wherein the program, when executed by a processor, implements the method of any one of claims 1-6.