CN113626512A - Data processing method, device, equipment and readable storage medium - Google Patents

Data processing method, device, equipment and readable storage medium Download PDF

Info

Publication number
CN113626512A
CN113626512A CN202110942760.4A CN202110942760A CN113626512A CN 113626512 A CN113626512 A CN 113626512A CN 202110942760 A CN202110942760 A CN 202110942760A CN 113626512 A CN113626512 A CN 113626512A
Authority
CN
China
Prior art keywords
data
service
target
service data
data format
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110942760.4A
Other languages
Chinese (zh)
Inventor
吴辰侣
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Weikun Shanghai Technology Service Co Ltd
Original Assignee
Weikun Shanghai Technology Service Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Weikun Shanghai Technology Service Co Ltd filed Critical Weikun Shanghai Technology Service Co Ltd
Priority to CN202110942760.4A priority Critical patent/CN113626512A/en
Publication of CN113626512A publication Critical patent/CN113626512A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/25—Integrating or interfacing systems involving database management systems
    • G06F16/258—Data format conversion from or to a database
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/18—File system types
    • G06F16/182—Distributed file systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor

Landscapes

  • Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • Information Transfer Between Computers (AREA)

Abstract

The embodiment of the application discloses a data processing method, a device, equipment and a readable storage medium, wherein the method comprises the following steps: receiving a data acquisition request sent by a service system, wherein the data acquisition request comprises a target data identifier of service data requested to be acquired and a target data format of the service data; acquiring candidate service data according to the target data identification, and determining the data format of the candidate service data; if the data format of the candidate service data is different from the target data format, acquiring a target data conversion rule parameter of the service data, and generating a data format conversion component based on the target data conversion rule parameter; and performing data format conversion on the candidate service data based on the data format conversion component to obtain target service data with a target data format, and sending the target service data to the service system. By adopting the embodiment of the application, the service processing efficiency can be improved.

Description

Data processing method, device, equipment and readable storage medium
Technical Field
The present application relates to the field of computer technologies, and in particular, to a data processing method, apparatus, device, and readable storage medium.
Background
When a service system needs to acquire service data for service processing, the service system needs to acquire the service data from a data source end, and because the data formats of different data source ends are different, the service system needs to develop a set of data interface for each format of data, and extract data in a corresponding data source based on the data interface. After the service data is acquired, the service system needs to convert the service data based on each format, and converts the service data in different formats into a target format required by the service system, so as to process the service data based on the target format. If the format of the service data acquired by the service system is not the target format, the accuracy of subsequent service processing is affected.
In the prior art, when a service system acquires data in a data source, a set of interfaces needs to be developed for each data format, and data in each data format needs to be converted, so that the service processing efficiency is low. Moreover, for a plurality of service systems, the same processing is required for each service system (i.e., a set of interface needs to be developed and converted for each data format), the method is complicated, the cost is high, and the service processing efficiency is low.
Disclosure of Invention
The embodiment of the application provides a data processing method, a data processing device, data processing equipment and a readable storage medium, and can improve service processing efficiency.
In a first aspect, the present application provides a data processing method, including:
receiving a data acquisition request sent by a service system, wherein the data acquisition request comprises a target data identifier of service data requested to be acquired and a target data format of the service data;
acquiring candidate service data according to the target data identification, and determining the data format of the candidate service data;
if the data format of the candidate service data is different from the target data format, acquiring a target data conversion rule parameter of the service data, and generating a data format conversion component based on the target data conversion rule parameter;
and performing data format conversion on the candidate service data based on the data format conversion component to obtain target service data with the target data format, and sending the target service data to the service system.
With reference to the first aspect, in a possible implementation manner, the data processing method further includes:
and if the data format of the candidate service data is the same as the target data format, determining the candidate service data as target service data, and sending the target service data to the service system.
With reference to the first aspect, in a possible implementation manner, the obtaining target data conversion rule parameters of the service data includes:
outputting parameter input prompt information to a user operation interface, wherein the parameter input prompt information is used for prompting a user to input a data format conversion rule parameter of the service data;
and acquiring the data format conversion rule parameters input by the user based on the user operation instruction on the user operation interface, and determining the data format conversion rule parameters input by the user as the target data conversion rule parameters of the service data.
With reference to the first aspect, in a possible implementation manner, the data obtaining request further includes a parameter identifier of a target data format conversion rule parameter or the target data format conversion rule parameter;
the target data conversion rule parameter for acquiring the service data comprises:
acquiring a data format conversion rule parameter corresponding to the parameter identifier from a preset data format conversion rule parameter library, and determining the data format conversion rule parameter corresponding to the parameter identifier as a target data conversion rule parameter of the service data; alternatively, the first and second electrodes may be,
and determining the target data format conversion rule parameter carried in the data acquisition request as the target data conversion rule parameter of the service data.
With reference to the first aspect, in a possible implementation manner, the generating a data format conversion component based on the target data conversion rule parameter includes:
acquiring a component generating function corresponding to the data format of the candidate service data, wherein the component generating function comprises a data format conversion rule parameter field which is empty;
and filling the target data conversion rule parameters into the data format conversion rule field, and running the component generation function to generate the data format conversion component.
With reference to the first aspect, in a possible implementation manner, before receiving a data acquisition request sent by a service system, the data processing method further includes:
acquiring initial service data corresponding to at least one data identifier from a data source end, and generating a target file based on the initial service data corresponding to the at least one data identifier;
the obtaining of candidate service data according to the target data identifier includes:
and when a data acquisition request sent by a service system is received, accessing the target file to acquire candidate service data corresponding to the target data identifier from the target file.
With reference to the first aspect, in a possible implementation manner, the acquiring candidate service data according to the target data identifier includes:
acquiring service data corresponding to the target data identifier from a specified data set according to the target data identifier, and determining the service data corresponding to the target data identifier as candidate service data, wherein the specified data set comprises at least one piece of service data corresponding to at least one data format, and the target data identifier comprises at least one of the name of the service data and the data type of the service data;
after the data format conversion component performs data format conversion on the candidate service data to obtain target service data with the target data format, the data processing method further includes:
and storing the target service data in the target data format into the specified data set, and marking the target service data as the service data corresponding to the target data format.
In a second aspect, the present application provides a data processing apparatus comprising:
the request acquisition module is used for receiving a data acquisition request sent by a service system, wherein the data acquisition request comprises a target data identifier of service data requested to be acquired and a target data format of the service data;
the format determining module is used for acquiring candidate service data according to the target data identification and determining the data format of the candidate service data;
the component obtaining module is used for obtaining a target data conversion rule parameter of the business data if the data format of the candidate business data is different from the target data format, and generating a data format conversion component based on the target data conversion rule parameter;
and the format conversion module is used for performing data format conversion on the candidate service data based on the data format conversion component to obtain target service data with the target data format and sending the target service data to the service system.
With reference to the second aspect, in a possible implementation manner, the data processing apparatus further includes: and the data sending module is used for determining the candidate service data as target service data if the data format of the candidate service data is the same as the target data format, and sending the target service data to the service system.
With reference to the second aspect, in a possible implementation manner, the component obtaining module includes:
the information output unit is used for outputting parameter input prompt information to a user operation interface, wherein the parameter input prompt information is used for prompting a user to input a data format conversion rule parameter of the service data;
and the parameter generating unit is used for acquiring the data format conversion rule parameters input by the user based on the user operation instruction on the user operation interface and determining the data format conversion rule parameters input by the user as the target data conversion rule parameters of the service data.
With reference to the second aspect, in a possible implementation manner, the data obtaining request further includes a parameter identifier of a target data format conversion rule parameter or the target data format conversion rule parameter; the component acquisition module includes:
an identifier obtaining unit, configured to obtain a data format conversion rule parameter corresponding to the parameter identifier from a preset data format conversion rule parameter library, and determine the data format conversion rule parameter corresponding to the parameter identifier as a target data conversion rule parameter of the service data; alternatively, the first and second electrodes may be,
and the parameter acquisition unit is used for determining the target data format conversion rule parameter carried in the data acquisition request as the target data conversion rule parameter of the service data.
With reference to the second aspect, in a possible implementation manner, the component obtaining module includes:
a function obtaining unit, configured to obtain a component generating function corresponding to the data format of the candidate service data, where the component generating function includes a parameter field of a data format conversion rule, and the parameter field of the data format conversion rule is null;
and the component generating unit is used for filling the target data conversion rule parameters into the data format conversion rule field and operating the component generating function to generate the data format conversion component.
With reference to the second aspect, in a possible implementation manner, the data obtaining apparatus further includes:
the file generation module is used for acquiring initial service data corresponding to at least one data identifier from a data source end and generating a target file based on the initial service data corresponding to the at least one data identifier;
the format determining module is specifically configured to, when a data obtaining request sent by a service system is received, obtain candidate service data corresponding to the target data identifier from the target file by accessing the target file.
With reference to the second aspect, in a possible implementation manner, the format determining module is specifically configured to obtain, according to the target data identifier, service data corresponding to the target data identifier from a specified data set, and determine the service data corresponding to the target data identifier as candidate service data, where the specified data set includes at least one piece of service data corresponding to at least one data format, and the target data identifier includes at least one of a name of the service data and a data type of the service data;
the data acquisition apparatus further includes: and the data storage module is used for storing the target service data in the target data format into the specified data set and marking the target service data as the service data corresponding to the target data format.
In a third aspect, the present application provides a computer device comprising: a processor, a memory, a network interface;
the processor is connected with a memory and a network interface, wherein the network interface is used for providing a data communication function, the memory is used for storing a computer program, and the processor is used for calling the computer program so as to enable a computer device comprising the processor to execute the method.
In a fourth aspect, the present application provides a computer-readable storage medium having stored thereon a computer program adapted to be loaded and executed by a processor to cause a computer device having the processor to perform the above-mentioned method.
In a fifth aspect, the present application provides a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computer device to perform the method provided in the various alternatives in the first aspect of the application.
In the embodiment of the application, a data acquisition request sent by a service system is received, wherein the data acquisition request comprises a target data identifier of service data requested to be acquired and a target data format of the service data; acquiring candidate service data according to the target data identification, and determining the data format of the candidate service data; if the data format of the candidate service data is different from the target data format, acquiring a target data conversion rule parameter of the service data, and generating a data format conversion component based on the target data conversion rule parameter; and performing data format conversion on the candidate service data based on the data format conversion component to obtain target service data with a target data format, and sending the target service data to the service system. By integrating the processing process of the service data in the distributed management platform, when the distributed management platform receives a data acquisition request sent by a service system, the service data corresponding to the data acquisition request can be acquired from a data source or a designated data set, and if the service data is not in a target data format required by the service system, the format of the service data can be converted by the distributed management platform, so that the service system can perform corresponding service processing based on the converted service data. Because the data format of the service data is processed by the distributed management platform, the service system does not need to respectively develop an interface or a plug-in for each data source end to acquire the service data, does not need to perform format conversion on the received service data, and can be directly used for subsequent service processing. When a plurality of service systems exist, each service system can acquire service data from the distributed management platform, and each service system does not need to develop an interface again, so that the cost can be saved, and the service processing efficiency can be improved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.
FIG. 1 is a block diagram of a data processing system according to an embodiment of the present application;
fig. 2 is a schematic flowchart of a data processing method according to an embodiment of the present application;
FIG. 3 is a schematic flow chart diagram of another data processing method provided in the embodiments of the present application;
fig. 4 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present application;
fig. 5 is a schematic structural diagram of a computer device according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
Referring to fig. 1, fig. 1 is a schematic structural diagram of a data processing system provided in an embodiment of the present application, where the schematic structural diagram of the system includes a service system 101, a server 102 and a data source terminal 103 corresponding to a distributed management platform, data interaction may be performed between the service system 101 and the server 102 corresponding to the distributed management platform, and data interaction may be performed between the server 102 and the data source terminal 103 corresponding to the distributed management platform. When the service system 101 needs to process a service, it may send a data acquisition request to the server 102 corresponding to the distributed management platform, so as to acquire target service data from the server 102 corresponding to the distributed management platform for service processing. The server 102 corresponding to the distributed management platform may receive a data acquisition request sent by the service system, acquire candidate service data according to a target data identifier in the data acquisition request, and determine a data format of the candidate service data. Further, if the data format of the candidate service data is different from the target data format, the server 102 corresponding to the distributed management platform may obtain a target data conversion rule parameter of the service data, and generate a data format conversion component based on the target data conversion rule parameter. The server 102 corresponding to the distributed management platform may further perform data format conversion on the candidate service data based on the data format conversion component to obtain target service data with a target data format, and send the target service data to the service system 101, so that the service system 101 performs corresponding service processing. Optionally, the data source end 103 may store the service data, and the server 102 corresponding to the distributed management platform may further obtain the candidate service data from the data source end 103 according to the target data identifier.
The server 102 corresponding to the distributed management platform may be an independent physical server, may also be a server cluster or a distributed system formed by a plurality of physical servers, and may also be a cloud server providing basic cloud computing services such as a cloud service, a cloud database, cloud computing, a cloud function, cloud storage, a network service, cloud communication, a middleware service, a domain name service, a security service, a Content Delivery Network (CDN), a big data and artificial intelligence platform, and the like. When the server 102 corresponding to the distributed management platform is an independent physical server, the server can independently perform data processing; when the server 102 corresponding to the distributed management platform is a plurality of physical servers, the plurality of physical servers may cooperate to perform data processing. The service system 101 and the data source end 103 may be an electronic device, including but not limited to a mobile phone, a tablet computer, a desktop computer, a notebook computer, a palm computer, a vehicle-mounted device, an Augmented Reality/Virtual Reality (AR/VR) device, a helmet display, a wearable device, a smart speaker, a digital camera, a camera, and other Mobile Internet Devices (MID) with network access capability, and the like; the method can also refer to an independent server, a server cluster consisting of a plurality of servers, or a cloud computing center.
Referring to fig. 2, fig. 2 is a schematic flowchart of a data processing method provided in an embodiment of the present application, and as shown in fig. 2, the method may be applied to a server side corresponding to the distributed management platform, where the method includes, but is not limited to, the following steps:
s101, receiving a data acquisition request sent by a service system.
In the embodiment of the application, the distributed management platform can receive a data acquisition request sent by the service system. The obtaining request may include a target data identifier of the service data requested to be obtained and a target data format of the service data. Specifically, the service system may be a system that needs to acquire service data for service processing, and when the service system needs to perform service processing, the service system may send a data acquisition request to the distributed management platform, and when the distributed management platform receives the data acquisition request, the distributed management platform may acquire the service data from a data source end that stores the service data, and send the service data to the service system after performing corresponding processing on the service data, so that the service system uses the service data to perform corresponding service processing. That is to say, when the service system needs to acquire the service data, the target data identifier of the service data that needs to be acquired and the target data format of the service data may be sent to the distributed management platform, so that the distributed management platform returns the service data that meets the target data identifier and the target data format, thereby performing service processing.
Alternatively, the Distributed management platform may refer to a platform that combines a storage function of a Distributed File System (HDFS) and a processing function such as a spark cluster computing platform. The HDFS is a distributed file system for storing files, and multiple servers can be combined to realize the functions of the HDFS. The spark cluster computing platform is a fast and general cluster computing platform, and functions of the spark cluster computing platform include task scheduling, memory management, and fault tolerance mechanisms. The data source end may refer to a terminal or a database storing various service data, and the distributed management platform or other terminals (such as the service system) may obtain the data from the data source end. The target data identifier of the service data may include information for uniquely identifying the service data, such as a name of the service data, a data type of the service data, a storage path of the service data, and a data number of the service data. The target data format of the service data refers to which type of data format the service data is, and specifically, the target data format of the service data can be set according to the requirements of a service system. The data format may include Structured Query Language (MySQL), distributed full-text Search server (ES), File Transfer Protocol (FTP), and other data formats.
In the embodiment of the application, if a service system needs to directly obtain data from a data source end, a set of data interfaces needs to be developed for service data, the service data of the data source end is obtained based on the data interfaces, and after the service data is obtained, the service system needs to convert (for example, perform a cleaning operation) the service data into service data in a target format required by the service system, so that the service system can implement related service processing. If the format of the service data is not the target format required by the service system, the accuracy of the service processing performed by the service system is affected. When the service system acquires the service data in the data source, a set of interfaces needs to be developed and cleaning operation needs to be performed for each type of service data, so that the service processing efficiency is low. Moreover, for a plurality of service systems, the same processing is required for each service system (i.e. a set of interface needs to be developed and cleaning operation needs to be performed for each service data), the manner is complicated, the cost is high, and the service processing efficiency is low. Therefore, the process of data conversion is integrated into the distributed management platform for processing, the distributed management platform only needs to develop a set of uniform interfaces to acquire the service data and convert the data format of the service data, the service system does not need to access each data source end through various code program layout interfaces to acquire the service data in each data source end and does not need to convert the format of the acquired service data, development and maintenance cost is reduced, and service processing efficiency is improved. Moreover, for the distributed management platform, the converted service data can be stored, multiplexing is realized, and the data acquisition efficiency is improved.
Optionally, before receiving a data obtaining request sent by the service system, the distributed management platform may obtain initial service data corresponding to at least one data identifier from the data source end, and generate a target file based on the initial service data corresponding to the at least one data identifier. That is to say, the distributed management platform may obtain at least one type of initial service data from one or more data sources in advance, one type of data identifier may correspond to one type of initial service data, one type of initial service data may include multiple data formats, and after obtaining multiple types of initial service data, the distributed management platform may generate a target file according to the at least one type of initial service data. For example, an object file may be generated according to at least one type of initial service data, or an object file may be generated according to each type of initial service data, which is not limited in this embodiment of the present application. The target file may be an HDFS file, that is, when the distributed management platform acquires various initial service data, the various service data may be stored in the HDFS file, so that when a data acquisition request sent by the service system is received subsequently, the service data is read from the HDFS file for corresponding processing.
Optionally, the distributed management platform processes the initial service data in different ways for different ways of obtaining the initial service data of the data source. Specifically, when the manner of obtaining the initial service data at the data source end is the message queue, the distributed management platform may process the initial service data in a stream processing manner, and the stream processing may implement real-time processing or near real-time processing on the initial service data, so that the processing efficiency is higher. When the manner of obtaining the initial service data of the data source end is the relational database, the distributed management platform may process the initial service data in a batch processing manner, and the batch processing may implement processing of the offline initial service data. That is, the two ways of acquiring the initial service data in the data source are different, and when the data acquisition way is a message queue, the initial service data needs to be processed in real time; and when the data acquisition mode is a relational database, offline initial service data can be processed.
S102, obtaining candidate service data according to the target data identification, and determining the data format of the candidate service data.
In the embodiment of the application, the distributed management platform can acquire the candidate service data according to the target data identifier and determine the data format of the candidate service data. The number of the data source ends may be one or more, and each data source end may store service data in one or more data formats, that is, the data formats of candidate service data determined according to the target data identifier may include multiple data formats. For example, the candidate service data a includes 3 data formats, which are data format a1, data format a2, and data format a3, where the data format a1 may refer to service data in MySQL format, the data format a2 may refer to service data in ES format, and the data format a3 may refer to service data in FTP format. It is understood that the contents of the candidate service data corresponding to the 3 data formats may be the same. In a specific implementation, the distributed management platform may determine the data format of the candidate service data according to the file type of the candidate service data acquired from the data source.
Optionally, the distributed management platform may obtain, from the specified data set, service data corresponding to the target data identifier according to the target data identifier, and determine the service data corresponding to the target data identifier as candidate service data. The specified data set may be a data set associated with the distributed management platform, and may refer to a specified data set in a local storage of the distributed management platform, for example. The designated data set comprises at least one piece of business data corresponding to at least one data format, and the target data identification comprises at least one of the name of the business data and the data type of the business data.
Optionally, after the distributed management platform processes the service data, the processed service data may be stored in the designated data set, that is, the designated data set may store service data in multiple data formats, and when the data format of the service data acquired by other subsequent service systems or other terminals is the same as the data format of a certain service data stored in the designated data set, the service data in the designated data set may be directly sent to the service system or other terminal that needs to acquire the service data.
It can be understood that the designated data set may store service data in multiple formats, if the target data format of the service data acquired by the service system or other terminals is different from the service data in each data format stored in the designated data set, the distributed management platform may convert the data format of the service data in the designated data set to obtain the service data in the target data format, store the target service data in the target data format in the designated data set, and mark the target service data as the service data corresponding to the target data format, so that other subsequent service systems can acquire the service data corresponding to the target data format. Wherein, the specified data set may refer to an inner table associated with the HDFS file, i.e. a hive inner table. The HDFS file comprises a hive inner table and a hive outer table, the hive outer table is used for storing data acquired from a data source end, and the hive inner table is used for storing service data in a converted target data format.
Optionally, because the distributed management platform may obtain initial service data corresponding to at least one data identifier from the data source end, and generate the target file based on the initial service data corresponding to the at least one data identifier, a manner that the distributed management platform obtains candidate service data according to the target data identifier may include: when a data acquisition request sent by a service system is received, candidate service data corresponding to the target data identifier is acquired from the target file by accessing the target file. That is, since the distributed management platform obtains a plurality of service data from the data source end in advance and stores the service data in the target file, when receiving a data obtaining request sent by the service system, the distributed management platform can obtain candidate service data corresponding to the target data identifier from the target file by accessing the target file.
In a specific implementation, the distributed management platform may extract initial service data from a data source end into an HDFS file (i.e., a target file). When a data acquisition request sent by a service system is received, a hive appearance associated with an HDFS file can be created by the distributed management platform, the HDFS file is accessed through the hive appearance in an SQL mode to read the content in the HDFS file, namely, initial service data, the initial service data is converted and the like, and then the processed initial service data is stored in a hive inner table (namely, a designated data set), so that the service data in the hive inner table can be sent to the service system subsequently, the service system can perform corresponding service processing based on the service data, namely, the data in the designated data set can be directly sent to the service system.
Optionally, after acquiring the initial service data from the data source, the distributed management platform may perform corresponding processing based on a historical data format of the historical service data and a data format of the initial service data, and store the processed service data into a specified data set. Specifically, the distributed management platform may convert the data format of the initial service data according to the historical data format of the historical service data in the historical data acquisition request sent by the service system, convert the data format of the initial service data into the historical data format, and store the initial service data having the historical data format into the designated data set. That is to say, the distributed management platform may convert the data format of the initial service data into service data in multiple data formats according to the historical data format conversion rule, and store the service data in each data format to the specified data set. The method comprises the steps that when a data acquisition request sent by a service system is received subsequently, service data in a corresponding target data format can be acquired from a designated data set, the data format of initial service data is processed in advance, and when the data acquisition request sent by the service system is received subsequently, the corresponding service data can be directly sent to the service system for service processing, so that the efficiency of service processing is improved. The service data in various data formats is stored, so that multiplexing can be realized, namely, each service system can acquire the required service data from the distributed management platform, the cost is saved, and the service processing efficiency is improved.
S103, if the data format of the candidate service data is different from the target data format, acquiring a target data conversion rule parameter of the service data, and generating a data format conversion component based on the target data conversion rule parameter.
In the embodiment of the application, if the data format of the candidate service data is different from the target data format, the distributed management platform may obtain a target data conversion rule parameter of the service data, and generate the data format conversion component based on the target data conversion rule parameter. If the data format of the candidate service data is different from the target data format, it may be that the data format of the candidate service data acquired by the distributed management platform from the data source end is different from the target data format. That is to say, when the distributed management platform acquires candidate service data required by the service system, because the format of the candidate service data is not the target data format, and the distributed management platform needs to convert the data format of the candidate service data, the distributed management platform may acquire the target data conversion rule parameter of the service data, and generate a data format conversion component based on the target data conversion rule parameter, so that the data format of the candidate service data is converted into the target data format based on the data format conversion component, and is sent to the service system for corresponding service processing. The target data conversion rule parameter may be used to set a rule for data format conversion, and the data format conversion component generated based on the target data conversion rule parameter may perform data format conversion based on the set rule for data format conversion. Alternatively, the target data conversion rule parameters may include conversion rule parameters such as preprocessing of data, removal/completion of missing data, removal/modification of format and content error data, removal/modification of logical error data, removal of unnecessary data, and correlation verification.
Optionally, the distributed management platform may obtain the target data conversion rule parameter of the service data in the following three ways:
in the first mode, the distributed management platform outputs parameter entry prompt information to a user operation interface, wherein the parameter entry prompt information is used for prompting a user to enter a data format conversion rule parameter of service data; and acquiring a data format conversion rule parameter input by a user based on a user operation instruction on the user operation interface, and determining the data format conversion rule parameter input by the user as a target data conversion rule parameter of the service data.
In specific implementation, the distributed management platform can output the parameter entry prompt information through the user operation interface, and the user can check the parameter entry prompt information through the user operation interface, so that the user can input information in the designated area of the user operation interface, and when the condition that the user triggers a 'submit' button/key in the user operation interface is detected, the information input by the user in the designated area of the user operation interface is used as the target data conversion rule parameter of the service data. Because the user can input the target data conversion rule parameters through the user operation interface, the data conversion mode can be more flexible.
In the second way, if the data obtaining request further includes a parameter identifier of the target data format conversion rule parameter, the distributed management platform may obtain the data format conversion rule parameter corresponding to the parameter identifier from a preset data format conversion rule parameter library, and determine the data format conversion rule parameter corresponding to the parameter identifier as the target data conversion rule parameter of the service data.
In a specific implementation, a user may store at least one data format conversion rule parameter in a preset data format conversion rule parameter library in advance, each data format conversion rule parameter corresponds to a parameter identifier, and the parameter identifier may include information for uniquely identifying the data format conversion rule parameter, such as a name of the data format conversion rule parameter, an effect of the data format conversion rule parameter, and a conversion content that the data format conversion rule parameter can implement, and the like. By storing at least one data format conversion rule parameter in a preset data format conversion rule parameter base, when a data acquisition request sent by a service system is subsequently received, a target data format conversion rule parameter corresponding to a parameter identifier can be directly acquired, so that the parameter acquisition efficiency is improved, and the subsequent service processing efficiency is further improved.
In a third mode, the data acquisition request further includes a target data format conversion rule parameter, and the distributed management platform may determine the target data format conversion rule parameter carried in the data acquisition request as the target data conversion rule parameter of the service data. That is to say, when the service system sends the data acquisition request to the distributed management platform, the data acquisition request may carry the target data format conversion rule parameter, and then the target data format conversion rule parameter of the service data may be determined according to the target data format conversion rule parameter carried in the data acquisition request.
Optionally, the manner of generating the data format conversion component by the distributed management platform based on the target data conversion rule parameter may be: the distributed management platform acquires a component generating function corresponding to the data format of the candidate service data, wherein the component generating function comprises a data format conversion rule parameter field, and at the moment, the data format conversion rule parameter field is empty. The distributed management platform can fill the target data conversion rule parameters into the data format conversion rule fields in the component generation function, and run the component generation function to generate the data format conversion component.
The component generating function may refer to a plug-in for data format conversion or a data format conversion tool such as a component, the component generating function may refer to some basic functions, and is equivalent to a frame, the data format conversion rule field in the frame is filled by using the target data conversion rule parameter, and the data format conversion component may be generated by running the filled component generating function. Optionally, the candidate service data in each data format may correspond to one component generation function, the distributed management platform may obtain the component generation function corresponding to the data format of the candidate service data, after obtaining the target data conversion rule parameter of the service data in any one of the three manners, fill the target data conversion rule parameter into the data format conversion rule field, and run the component generation function to generate the data format conversion component.
In a specific implementation, the distributed management platform may generate the data format conversion component according to a target data conversion rule parameter of the service data by using a code generation technology. The code generation techniques may include, for example, groovy dynamic language techniques or other code generation techniques. By adopting the code generation technology, the data format conversion component can be automatically generated, the component generation efficiency can be improved, and a user does not need to manually write codes, so that the component generation mode is more flexible. Specifically, after the distributed management platform generates the data format conversion component, the data format conversion component may be stored in a local directory of the distributed management file, and when the data format conversion component is subsequently required to convert the format of the service data, the data format conversion component stored in the local directory may be called to perform format conversion on the candidate service data.
And S104, performing data format conversion on the candidate service data based on the data format conversion component to obtain target service data with a target data format, and sending the target service data to the service system.
In the embodiment of the application, since the data format conversion component is generated in the above steps, the distributed management platform may perform data format conversion on the candidate service data based on the data format conversion component to obtain the target service data with the target data format, and send the target service data to the service system. The data format conversion component may be configured to convert a data format of the service data. Optionally, a data format conversion component may correspond to a conversion rule, and by converting the candidate service data using the corresponding data format conversion component, a field in the candidate service data that meets a preset rule may be extracted and converted into structured data, so as to be sent to a service system for corresponding service processing.
The target data format may include any one of a MySQL format, an ES format, an FTP format, and the like, and may be specifically determined according to the requirements of the service system. For example, if the target data format is the MySQL format, when the data format of the candidate service data is not the MySQL format, it indicates that the data format corresponding to the candidate service data is not the target data format, and the data format corresponding to the candidate service data may be converted by the data format conversion component to obtain the target service data with the target data format.
In a specific implementation, when the distributed management platform acquires the initial service data from the data source, the data format of the initial service data may be determined according to the file type of the initial service data, for example, the target data format is MySQL format, and when the distributed management platform acquires the initial service data in the ES format or the FTP format, it is determined that the initial service data is not in the target data format, and the distributed management platform may extract the initial service data into an HDFS file for storage. When a data acquisition request sent by service data is received, the distributed management platform can create a row of hive appearances associated with the HDFS file, access the data content in the HDFS file in an SQL mode through the hive appearances, and convert the data format of the initial service data in the HDFS file based on a data format conversion component to obtain target service data with a target data format. Optionally, the distributed management platform may further store the target service data in the target data format into a specified data set (hive internal table), so that when other subsequent service systems need to acquire the target service data, the target service data can be quickly sent to the corresponding service systems.
Optionally, if the data format of the candidate service data is the same as the target data format, the distributed management platform may determine the candidate service data as the target service data, and send the target service data to the service system.
In a specific implementation, if the data format of the candidate service data is the same as the target data format, for example, the target data format is the MySQL format, and the data format of the candidate service data is the MySQL format, the candidate service data may be determined as the target service data, and the target service data is sent to the service system. Optionally, the distributed management platform may further map the candidate service data to a specified data set (i.e., a hive internal table) for storage, so that when other subsequent service systems need to acquire the target service data, the candidate service data may be quickly sent to the corresponding service systems.
In the embodiment of the application, a data acquisition request sent by a service system is received, wherein the data acquisition request comprises a target data identifier of service data requested to be acquired and a target data format of the service data; acquiring candidate service data according to the target data identification, and determining the data format of the candidate service data; if the data format of the candidate service data is different from the target data format, acquiring a target data conversion rule parameter of the service data, and generating a data format conversion component based on the target data conversion rule parameter; and performing data format conversion on the candidate service data based on the data format conversion component to obtain target service data with a target data format, and sending the target service data to the service system. In the embodiment of the application, by integrating the processing process of the service data in the unified distributed management platform, when the distributed management platform receives a data acquisition request sent by a service system, the service data corresponding to the data acquisition request can be acquired from a data source or a designated data set, and if the service data is not in a target data format required by the service system, the format of the service data can be converted by the distributed management platform, so that the service system can perform corresponding service processing based on the converted service data. Because the data format of the service data is processed by the distributed management platform, the service system does not need to respectively develop an interface or a plug-in for each data source end to acquire the service data, does not need to perform format conversion on the received service data, and can be directly used for subsequent service processing. When a plurality of service systems exist, each service system can acquire service data from the distributed management platform, and each service system does not need to develop an interface again, so that the cost can be saved, and the service processing efficiency can be improved.
Optionally, please refer to fig. 3, where fig. 3 is a schematic flow chart of another data processing method provided in the embodiment of the present application, and the method describes a data processing process as a whole; as shown in fig. 3, the method includes, but is not limited to, the following steps:
s201, the service system sends a data acquisition request to the distributed management platform.
S202, the distributed management platform acquires candidate service data based on the target data identification in the data acquisition request, and determines the data format of the candidate service data.
The distributed management platform may acquire candidate service data from the data source based on the target data identifier in the data acquisition request, or may acquire multiple service data from the data source in advance by the distributed management platform and store the multiple service data in a local database corresponding to the distributed management platform, and at this time, the distributed management platform may acquire the candidate service data from the local database based on the target data identifier in the data acquisition request, which is not limited in this application.
S203, if the data format of the candidate service data is different from the target data format, the distributed management platform obtains the target data conversion rule parameter of the service data, and generates a data format conversion component based on the target data conversion rule parameter.
S204, the distributed management platform performs data format conversion on the candidate service data based on the data format conversion component to obtain target service data with a target data format, and sends the target service data to the service system.
And S205, if the data format of the candidate service data is the same as that of the target service data, determining the candidate service data as the target service data, and sending the target service data to the service system.
In this embodiment of the application, the specific implementation manner of step S201 to step S205 may refer to the method implementation manner of step S101 to step S104 in fig. 2, which is not described herein again.
In the embodiment of the application, by integrating the processing process of the service data in the unified distributed management platform, when the distributed management platform receives a data acquisition request sent by a service system, the service data corresponding to the data acquisition request can be acquired from a data source or a designated data set, and if the service data is not in a target data format required by the service system, the format of the service data can be converted by the distributed management platform, so that the service system can perform corresponding service processing based on the converted service data. Because the data format of the service data is processed by the distributed management platform, the service system does not need to respectively develop an interface or a plug-in for each data source end to acquire the service data, does not need to perform format conversion on the received service data, and can be directly used for subsequent service processing. When a plurality of service systems exist, each service system can acquire service data from the distributed management platform, and each service system does not need to develop an interface again, so that the cost can be saved, and the service processing efficiency can be improved.
The method of the embodiments of the present application is described above, and the apparatus of the embodiments of the present application is described below.
Referring to fig. 4, fig. 4 is a schematic diagram of a component structure of a data processing apparatus provided in an embodiment of the present application, where the data processing apparatus may be a computer program (including program code) running in a computer device, for example, the data processing apparatus is an application software; the data processing device can be used for executing the corresponding steps in the method provided by the embodiment of the application. The data processing apparatus 40 includes:
a request obtaining module 41, configured to receive a data obtaining request sent by a service system, where the data obtaining request includes a target data identifier of service data requested to be obtained and a target data format of the service data;
a format determining module 42, configured to obtain candidate service data according to the target data identifier, and determine a data format of the candidate service data;
a component obtaining module 43, configured to obtain a target data conversion rule parameter of the service data if the data format of the candidate service data is different from the target data format, and generate a data format conversion component based on the target data conversion rule parameter;
and a format conversion module 44, configured to perform data format conversion on the candidate service data based on the data format conversion component to obtain target service data with the target data format, and send the target service data to the service system.
Optionally, the data processing apparatus 40 further includes: and a data sending module 45, configured to determine the candidate service data as target service data if the data format of the candidate service data is the same as the target data format, and send the target service data to the service system.
Optionally, the component acquiring module 43 includes:
the information output unit 431 is used for outputting parameter entry prompt information to a user operation interface, wherein the parameter entry prompt information is used for prompting a user to enter a data format conversion rule parameter of the service data;
a parameter generating unit 432, configured to obtain a data format conversion rule parameter input by the user based on a user operation instruction on the user operation interface, and determine the data format conversion rule parameter input by the user as a target data conversion rule parameter of the service data.
Optionally, the data obtaining request further includes a parameter identifier of the target data format conversion rule parameter or the target data format conversion rule parameter; the component acquisition module 43 includes:
an identifier obtaining unit 433, configured to obtain a data format conversion rule parameter corresponding to the parameter identifier from a preset data format conversion rule parameter library, and determine the data format conversion rule parameter corresponding to the parameter identifier as a target data conversion rule parameter of the service data; alternatively, the first and second electrodes may be,
a parameter obtaining unit 434, configured to determine the target data format conversion rule parameter carried in the data obtaining request as the target data conversion rule parameter of the service data.
Optionally, the component acquiring module 43 includes:
a function obtaining unit 435, configured to obtain a component generating function corresponding to the data format of the candidate service data, where the component generating function includes a parameter field of a data format conversion rule, and the parameter field of the data format conversion rule is null;
a component generating unit 436, configured to fill the target data conversion rule parameter into the data format conversion rule field, and execute the component generating function to generate the data format conversion component.
Optionally, the data acquiring device 40 further includes:
the file generating module 46 is configured to obtain initial service data corresponding to at least one data identifier from a data source end, and generate a target file based on the initial service data corresponding to the at least one data identifier;
the format determining module 42 is specifically configured to, when a data obtaining request sent by a service system is received, obtain candidate service data corresponding to the target data identifier from the target file by accessing the target file.
Optionally, the format determining module 42 is specifically configured to obtain, according to the target data identifier, service data corresponding to the target data identifier from a specified data set, and determine the service data corresponding to the target data identifier as candidate service data, where the specified data set includes at least one piece of service data corresponding to at least one data format, and the target data identifier includes at least one of a name of the service data and a data type of the service data;
the data acquisition device 40 further includes: a data storage module 47, configured to store the target service data in the target data format into the specified data set, and mark the target service data as service data corresponding to the target data format.
It should be noted that, for the content that is not mentioned in the embodiment corresponding to fig. 4, reference may be made to the description of the method embodiment, and details are not described here again.
In the embodiment of the application, by integrating the processing process of the service data in the unified distributed management platform, when the distributed management platform receives a data acquisition request sent by a service system, the service data corresponding to the data acquisition request can be acquired from a data source or a designated data set, and if the service data is not in a target data format required by the service system, the format of the service data can be converted by the distributed management platform, so that the service system can perform corresponding service processing based on the converted service data. Because the data format of the service data is processed by the distributed management platform, the service system does not need to respectively develop an interface or a plug-in for each data source end to acquire the service data, does not need to perform format conversion on the received service data, and can be directly used for subsequent service processing. When a plurality of service systems exist, each service system can acquire service data from the distributed management platform, and each service system does not need to develop an interface again, so that the cost can be saved, and the service processing efficiency can be improved.
Referring to fig. 5, fig. 5 is a schematic structural diagram of a computer device according to an embodiment of the present disclosure. As shown in fig. 5, the computer device 50 may include: the processor 501, the network interface 504 and the memory 505, and the computer device 50 may further include: a user interface 503, and at least one communication bus 502. Wherein a communication bus 502 is used to enable connective communication between these components. The user interface 503 may include a Display screen (Display) and a Keyboard (Keyboard), and the optional user interface 503 may also include a standard wired interface and a standard wireless interface. The network interface 504 may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface). The memory 505 may be a high-speed RAM memory or a non-volatile memory (non-volatile memory), such as at least one magnetic disk memory. The memory 505 may alternatively be at least one memory device located remotely from the processor 501. As shown in fig. 5, the memory 505, which is a kind of computer-readable storage medium, may include therein an operating system, a network communication module, a user interface module, and a device control application program.
In the computer device 50 shown in fig. 5, the network interface 504 may provide network communication functions; while the user interface 503 is primarily an interface for providing input to a user; and processor 501 may be used to invoke a device control application stored in memory 505 to implement:
receiving a data acquisition request sent by a service system, wherein the data acquisition request comprises a target data identifier of service data requested to be acquired and a target data format of the service data;
acquiring candidate service data according to the target data identification, and determining the data format of the candidate service data;
if the data format of the candidate service data is different from the target data format, acquiring a target data conversion rule parameter of the service data, and generating a data format conversion component based on the target data conversion rule parameter;
and performing data format conversion on the candidate service data based on the data format conversion component to obtain target service data with the target data format, and sending the target service data to the service system.
It should be understood that the computer device 50 described in this embodiment may perform the description of the data processing method in the embodiment corresponding to fig. 2 and fig. 3, and may also perform the description of the data processing apparatus in the embodiment corresponding to fig. 4, which is not described herein again. In addition, the beneficial effects of the same method are not described in detail.
In the embodiment of the application, by integrating the processing process of the service data in the unified distributed management platform, when the distributed management platform receives a data acquisition request sent by a service system, the service data corresponding to the data acquisition request can be acquired from a data source or a designated data set, and if the service data is not in a target data format required by the service system, the format of the service data can be converted by the distributed management platform, so that the service system can perform corresponding service processing based on the converted service data. Because the data format of the service data is processed by the distributed management platform, the service system does not need to respectively develop an interface or a plug-in for each data source end to acquire the service data, does not need to perform format conversion on the received service data, and can be directly used for subsequent service processing. When a plurality of service systems exist, each service system can acquire service data from the distributed management platform, and each service system does not need to develop an interface again, so that the cost can be saved, and the service processing efficiency can be improved.
Embodiments of the present application also provide a computer-readable storage medium storing a computer program, the computer program comprising program instructions, which, when executed by a computer, cause the computer to perform the method according to the foregoing embodiments, and the computer may be a part of the above-mentioned computer device. Such as processor 501 described above. By way of example, the program instructions may be executed on one computer device, or on multiple computer devices located at one site, or distributed across multiple sites and interconnected by a communication network, which may comprise a blockchain network.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like.
The above disclosure is only for the purpose of illustrating the preferred embodiments of the present application and is not to be construed as limiting the scope of the present application, so that the present application is not limited thereto, and all equivalent variations and modifications can be made to the present application.

Claims (10)

1. A data processing method, comprising:
receiving a data acquisition request sent by a service system, wherein the data acquisition request comprises a target data identifier of service data requested to be acquired and a target data format of the service data;
acquiring candidate service data according to the target data identification, and determining the data format of the candidate service data;
if the data format of the candidate service data is different from the target data format, acquiring a target data conversion rule parameter of the service data, and generating a data format conversion component based on the target data conversion rule parameter;
and performing data format conversion on the candidate service data based on the data format conversion component to obtain target service data with the target data format, and sending the target service data to the service system.
2. The method of claim 1, further comprising:
and if the data format of the candidate service data is the same as the target data format, determining the candidate service data as target service data, and sending the target service data to the service system.
3. The method of claim 1, wherein obtaining the target data transformation rule parameter of the service data comprises:
outputting parameter input prompt information to a user operation interface, wherein the parameter input prompt information is used for prompting a user to input a data format conversion rule parameter of service data;
and acquiring the data format conversion rule parameters input by the user based on the user operation instruction on the user operation interface, and determining the data format conversion rule parameters input by the user as the target data conversion rule parameters of the service data.
4. The method according to claim 1, wherein the data acquisition request further includes a parameter identifier of a target data format conversion rule parameter or a target data format conversion rule parameter;
the obtaining of the target data conversion rule parameter of the service data includes:
acquiring a data format conversion rule parameter corresponding to the parameter identifier from a preset data format conversion rule parameter library, and determining the data format conversion rule parameter corresponding to the parameter identifier as a target data conversion rule parameter of the service data; alternatively, the first and second electrodes may be,
and determining the target data format conversion rule parameter carried in the data acquisition request as the target data conversion rule parameter of the service data.
5. The method of claim 3 or 4, wherein generating a data format conversion component based on the target data conversion rule parameters comprises:
acquiring a component generating function corresponding to the data format of the candidate service data, wherein the component generating function comprises a data format conversion rule parameter field which is empty;
and filling the target data conversion rule parameters into the data format conversion rule field, and running the component generation function to generate the data format conversion component.
6. The method of claim 5, wherein before receiving the data acquisition request sent by the service system, the method further comprises:
acquiring initial service data corresponding to at least one data identifier from a data source end, and generating a target file based on the initial service data corresponding to the at least one data identifier;
the obtaining of candidate service data according to the target data identifier includes:
and when a data acquisition request sent by a service system is received, accessing the target file to acquire candidate service data corresponding to the target data identifier from the target file.
7. The method of claim 5, wherein the obtaining candidate service data according to the target data identification comprises:
acquiring service data corresponding to the target data identifier from a specified data set according to the target data identifier, and determining the service data corresponding to the target data identifier as candidate service data, wherein the specified data set comprises at least one piece of service data corresponding to at least one data format, and the target data identifier comprises at least one of the name of the service data and the data type of the service data;
after the data format conversion component performs data format conversion on the candidate service data to obtain target service data with the target data format, the method further includes:
and storing the target service data in the target data format into the specified data set, and marking the target service data as service data corresponding to the target data format.
8. A data processing apparatus, comprising:
the system comprises a request acquisition module, a data acquisition module and a data processing module, wherein the request acquisition module is used for receiving a data acquisition request sent by a service system, and the data acquisition request comprises a target data identifier of service data requested to be acquired and a target data format of the service data;
the format determining module is used for acquiring candidate service data according to the target data identification and determining the data format of the candidate service data;
the component obtaining module is used for obtaining a target data conversion rule parameter of the service data if the data format of the candidate service data is different from the target data format, and generating a data format conversion component based on the target data conversion rule parameter;
and the format conversion module is used for performing data format conversion on the candidate service data based on the data format conversion component to obtain target service data with the target data format and sending the target service data to the service system.
9. A computer device, comprising: a processor, a memory, and a network interface;
the processor is coupled to the memory and the network interface, wherein the network interface is configured to provide data communication functionality, the memory is configured to store program code, and the processor is configured to invoke the program code to cause the computer device to perform the method of any of claims 1-7.
10. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program adapted to be loaded and executed by a processor to cause a computer device having the processor to perform the method of any of claims 1-7.
CN202110942760.4A 2021-08-17 2021-08-17 Data processing method, device, equipment and readable storage medium Pending CN113626512A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110942760.4A CN113626512A (en) 2021-08-17 2021-08-17 Data processing method, device, equipment and readable storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110942760.4A CN113626512A (en) 2021-08-17 2021-08-17 Data processing method, device, equipment and readable storage medium

Publications (1)

Publication Number Publication Date
CN113626512A true CN113626512A (en) 2021-11-09

Family

ID=78386027

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110942760.4A Pending CN113626512A (en) 2021-08-17 2021-08-17 Data processing method, device, equipment and readable storage medium

Country Status (1)

Country Link
CN (1) CN113626512A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114547171A (en) * 2022-02-22 2022-05-27 广州品推科技有限公司 Business data processing method and system based on big data analysis
CN116719866A (en) * 2023-05-09 2023-09-08 上海银满仓数字科技有限公司 Multi-format data self-adaptive distribution method and system

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114547171A (en) * 2022-02-22 2022-05-27 广州品推科技有限公司 Business data processing method and system based on big data analysis
CN116719866A (en) * 2023-05-09 2023-09-08 上海银满仓数字科技有限公司 Multi-format data self-adaptive distribution method and system
CN116719866B (en) * 2023-05-09 2024-02-13 上海银满仓数字科技有限公司 Multi-format data self-adaptive distribution method and system

Similar Documents

Publication Publication Date Title
CN111414407A (en) Data query method and device of database, computer equipment and storage medium
CN110647316B (en) Method and device for generating universal business object, computer equipment and storage medium
CN111683066A (en) Heterogeneous system integration method and device, computer equipment and storage medium
CN113626512A (en) Data processing method, device, equipment and readable storage medium
CN112702228B (en) Service flow limit response method, device, electronic equipment and readable storage medium
CN111507439A (en) User information checking method and system based on information code
CN114416877A (en) Data processing method, device and equipment and readable storage medium
CN109062906B (en) Translation method and device for program language resources
CN112653579A (en) OpenResty-based gray scale publishing method and related equipment
CN108153896B (en) Processing method and device for input data and output data
CN116360769A (en) Code generation method, device, equipment and storage medium
CN113709187B (en) Multi-server data request method, device, equipment and storage medium
CN115016754A (en) Method and device for synchronously displaying pages among devices, electronic device and medium
CN111488286B (en) Method and device for independently developing Android modules
CN114896161A (en) File construction method and device based on artificial intelligence, computer equipment and medium
CN113704120A (en) Data transmission method, device, equipment and storage medium
CN115061785A (en) Information issuing method and device, storage medium and server
CN113836428A (en) Business pushing method and device, computer equipment and storage medium
CN110851446B (en) Data table generation method and device, computer equipment and storage medium
CN110334142B (en) Intelligent data acquisition method, terminal, server and interaction system
CN113590352A (en) Data calling method, device, equipment and readable storage medium
CN112612427A (en) Vehicle stop data processing method and device, storage medium and terminal
CN112445811A (en) Data service method, device, storage medium and component based on SQL configuration
CN111581578A (en) Interface request processing method and device
CN111158661A (en) System interface docking method, device, medium and electronic equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination