CN113971192A - Data processing method and device, readable medium and electronic equipment - Google Patents

Data processing method and device, readable medium and electronic equipment Download PDF

Info

Publication number
CN113971192A
CN113971192A CN202111234197.1A CN202111234197A CN113971192A CN 113971192 A CN113971192 A CN 113971192A CN 202111234197 A CN202111234197 A CN 202111234197A CN 113971192 A CN113971192 A CN 113971192A
Authority
CN
China
Prior art keywords
target
data
function
application
application program
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202111234197.1A
Other languages
Chinese (zh)
Inventor
龙跃
高官涛
郭俊
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Volcano Engine Technology Co Ltd
Original Assignee
Beijing Volcano Engine Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Volcano Engine Technology Co Ltd filed Critical Beijing Volcano Engine Technology Co Ltd
Priority to CN202111234197.1A priority Critical patent/CN113971192A/en
Publication of CN113971192A publication Critical patent/CN113971192A/en
Priority to PCT/CN2022/120087 priority patent/WO2023065937A1/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/25Integrating or interfacing systems involving database management systems
    • G06F16/258Data format conversion from or to a database

Abstract

The present disclosure relates to a data processing method, an apparatus, a readable medium and an electronic device, wherein the method is applied to a first application program and includes: responding to a received data processing instruction, and acquiring first target data to be processed; determining whether a first application is capable of executing the target processing policy on first target data; under the condition that the first application program is determined not to be capable of executing the target processing strategy on the first target data, determining a second application program capable of executing the target processing strategy, wherein the second application program is coded in a mode different from that of the first application program; converting the first target data into second target data according to a first data format which can be processed by the second application program; executing, by the second application, the target processing policy on the second target data. Therefore, the first application program and the second application program can be mutually universal without redefining the framework, and the implementation difficulty is reduced.

Description

Data processing method and device, readable medium and electronic equipment
Technical Field
The present disclosure relates to the field of computer technologies, and in particular, to a data processing method, an apparatus, a readable medium, and an electronic device.
Background
With the development of big data technology, a variety of big data based query engines have emerged. For example, the query engine Hive, the query engine Presto, the query engine Spark, and the query engine Impala. These query engines have powerful query functions and provide rich UDFs (User Defined functions). However, different query engines support UDFs differently and are difficult to use commonly with each other.
In the related art, by defining a new UDF framework, a user rewrites a set of UDFs in the new framework to meet the grammatical requirements of different query engines. However, this method is more labor-intensive, resulting in greater implementation difficulty.
Disclosure of Invention
This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the detailed description. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.
In a first aspect, the present disclosure provides a data processing method applied to a first application, the method including:
in response to receiving a data processing instruction, acquiring first target data to be processed, wherein the data processing instruction is used for indicating that a target processing strategy is executed on the first target data;
determining whether the first application is capable of executing the target processing policy on the first target data;
under the condition that the first application program is determined not to be capable of executing the target processing strategy on the first target data, determining a second application program capable of executing the target processing strategy, wherein the second application program is coded in a mode different from that of the first application program;
converting the first target data into second target data according to a first data format which can be processed by the second application program;
executing, by the second application, the target processing policy on the second target data.
In a second aspect, the present disclosure provides a data processing apparatus applied to a first application, the apparatus comprising:
the data processing device comprises a data acquisition module, a data processing module and a data processing module, wherein the data acquisition module is used for responding to a received data processing instruction and acquiring first target data to be processed, and the data processing instruction is used for indicating that a target processing strategy is executed on the first target data;
an execution determination module to determine whether the first application is capable of executing the target processing policy on the first target data;
an application determination module, configured to determine, when it is determined that the first application cannot execute the target processing policy on the first target data, a second application that can execute the target processing policy, where the second application is encoded in a different manner from the first application;
the first format conversion module is used for converting the first target data into second target data according to a first data format which can be processed by the second application program;
and the strategy execution module is used for executing the target processing strategy on the second target data through the second application program.
In a third aspect, the present disclosure provides a computer readable medium having stored thereon a computer program which, when executed by a processing apparatus, performs the steps of the method of the first aspect of the present disclosure.
In a fourth aspect, the present disclosure provides an electronic device comprising:
a storage device having a computer program stored thereon;
processing means for executing the computer program in the storage means to implement the steps of the method of the first aspect of the present disclosure.
According to the technical scheme, in response to receiving a data processing instruction, first target data to be processed is obtained, wherein the data processing instruction is used for indicating that a target processing strategy is executed on the first target data; determining whether the first application is capable of executing the target processing policy on the first target data; under the condition that the first application program is determined not to be capable of executing the target processing strategy on the first target data, determining a second application program capable of executing the target processing strategy, wherein the second application program is coded in a mode different from that of the first application program; converting the first target data into second target data according to a first data format which can be processed by the second application program; executing, by the second application, the target processing policy on the second target data. That is to say, under the condition that the first application program cannot execute the target processing policy on the first target data, the first target data is subjected to format conversion according to the first data format which can be processed by the second application program to obtain the second target data, and the target processing policy is executed on the second target data by the second application program.
Additional features and advantages of the disclosure will be set forth in the detailed description which follows.
Drawings
The above and other features, advantages and aspects of various embodiments of the present disclosure will become more apparent by referring to the following detailed description when taken in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numbers refer to the same or similar elements. It should be understood that the drawings are schematic and that elements and features are not necessarily drawn to scale. In the drawings:
FIG. 1 is a flow diagram illustrating a method of data processing in accordance with an exemplary embodiment;
FIG. 2 is a flow diagram illustrating another data processing method in accordance with an exemplary embodiment;
FIG. 3 is a block diagram illustrating a data processing apparatus in accordance with an exemplary embodiment;
FIG. 4 is a block diagram illustrating a second type of data processing apparatus in accordance with an exemplary embodiment;
FIG. 5 is a block diagram illustrating a third type of data processing apparatus in accordance with an exemplary embodiment;
FIG. 6 is a block diagram illustrating a fourth data processing apparatus in accordance with an exemplary embodiment;
FIG. 7 is a block diagram illustrating an electronic device in accordance with an example embodiment.
Detailed Description
Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it is to be understood that the present disclosure may be embodied in various forms and should not be construed as limited to the embodiments set forth herein, but rather are provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the disclosure are for illustration purposes only and are not intended to limit the scope of the disclosure.
It should be understood that the various steps recited in the method embodiments of the present disclosure may be performed in a different order, and/or performed in parallel. Moreover, method embodiments may include additional steps and/or omit performing the illustrated steps. The scope of the present disclosure is not limited in this respect.
The term "include" and variations thereof as used herein are open-ended, i.e., "including but not limited to". The term "based on" is "based, at least in part, on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Relevant definitions for other terms will be given in the following description.
It should be noted that the terms "first", "second", and the like in the present disclosure are only used for distinguishing different devices, modules or units, and are not used for limiting the order or interdependence relationship of the functions performed by the devices, modules or units.
It is noted that references to "a", "an", and "the" modifications in this disclosure are intended to be illustrative rather than limiting, and that those skilled in the art will recognize that "one or more" may be used unless the context clearly dictates otherwise.
The names of messages or information exchanged between devices in the embodiments of the present disclosure are for illustrative purposes only, and are not intended to limit the scope of the messages or information.
First, an application scenario of the present disclosure will be explained. The UDF function can enable a user to define own functions in query instead of only using the functions preset by the system, thereby greatly improving the query experience of the user and expanding the query function of the query engine. However, different query engines support different UDFs and are difficult to be used universally. This non-commonality makes it difficult for users to multiplex their queries among multiple query engines, even requiring multiple versions of UDFs to be maintained. While some query engines support the ability to run other query engines UDFs (e.g., Spark supports running Hive UDFs), this is facilitated by their close syntax, Presto's UDF is a comment type definition, much the same as Hive UDF.
For historical reasons, Hive is born early and widely applied, most of industry companies use Hive in large quantity, and a large amount of Hive UDFs play an important role in production environment. Therefore, how to support Hive UDF in Presto is a technical problem, and the expansibility and the universality of the Presto query engine are also greatly influenced.
In the related art, by defining a new UDF framework, a user rewrites a set of UDFs in the new framework to meet the grammatical requirements of different query engines. However, this method is heavy in workload, and a user needs to be familiar with the new framework grammar and rewrite the existing UDF into the UDF under the new framework, which results in high implementation difficulty and poor feasibility.
In order to solve the above technical problem, the present disclosure provides a data processing method, an apparatus, a readable medium, and an electronic device, where, under a condition that a first application cannot execute a target processing policy on first target data, format conversion is performed on the first target data according to a first data format that a second application can process to obtain second target data, and the second application executes the target processing policy on the second target data, so that a framework does not need to be redefined, mutual commonality between the first application and the second application can be achieved, and implementation difficulty is reduced.
The present disclosure is described below with reference to specific examples.
Fig. 1 is a flowchart illustrating a data processing method applied to a first application program, which may be a data query tool, such as a Presto query engine, or a data processing tool, according to an exemplary embodiment, which is not limited by the present disclosure. In the embodiment of the present disclosure, the first application is taken as an example of a data query tool for explanation; the method can comprise the following steps:
s101, responding to the received data processing instruction, and acquiring first target data to be processed.
Wherein the data processing instruction may be used to instruct to execute a target processing policy on the first target data, and the data processing instruction may be an SQL (Structured Query Language) statement, for example.
In this step, after receiving a data processing instruction triggered by a user, the first target data to be processed may be acquired according to the data processing instruction. For example, if the data processing instruction is an SQL statement, the first target data may be obtained from a preset database according to the SQL statement, for example, in a case that the SQL statement is "select avg (col) from table," the first target data may be each line of data in the table.
S102, determining whether the first application program can execute the target processing strategy on the first target data.
In this step, after the first target data is obtained, a target processing policy to be executed on the first target data may be determined according to the data processing instruction, and for example, if the data processing instruction continues to be described as "select avg (col) from table", the target processing policy is averaging. Further, after determining the target processing policy, it may be determined whether the first application is capable of executing the target processing policy on the first target data.
In one possible implementation, it may be determined whether the first application includes a first objective function, the first objective function being used to execute the target processing policy, and in a case where it is determined that the first application includes the first objective function, it is determined that the first application is capable of executing the target processing policy; determining that the first application cannot execute the target processing policy if it is determined that the first application does not include the first objective function. Continuing with the example where the data processing instruction is "select avg (col) from table," the first objective function may be a function to obtain an average value, in the case where it is determined that the function to obtain an average value is included in the first application, it may be determined that the first application is capable of executing the target processing policy, and in the case where it is determined that the function to obtain an average value is not included in the first application, it may be determined that the first application is not capable of executing the target processing policy.
S103, under the condition that the first application program is determined not to be capable of executing the target processing strategy on the first target data, a second application program capable of executing the target processing strategy is determined.
The second application may be encoded in a different manner from the first application, for example, if the first application is a Presto query engine, the second application may be a Hive query engine.
In this step, in a case that it is determined that the first application cannot execute the target processing policy on the first target data, a second application capable of executing the target processing policy may be determined from a plurality of predetermined applications, where the plurality of predetermined applications may be applications having the same function as the first application but different coding modes, and for example, if the first application is a Presto query engine, the predetermined application may be another query engine in the prior art. It should be noted that, in the case that the preset application includes one application, the preset application may be used as the second application.
And S104, converting the first target data into second target data according to a first data format which can be processed by the second application program.
In this step, the second application program and the first application program have different encoding manners, and the target processing policy cannot be directly executed on the first target data by the second application program, and after the second application program capable of executing the target processing policy is determined, the first data format that the second application program can process may be obtained.
And S105, executing the target processing strategy on the second target data through the second application program.
In this step, after the first target data is converted into the second target data, a corresponding function in the second application program may be called, and the target processing policy may be executed on the second target data.
Further, after the target processing policy is executed on the second target data by the second application program, the first processing result may be converted into a second processing result according to a second data format that can be processed by the first application program, and the second processing result is output, where the first processing result is a result of executing the target processing policy on the second target data by the second application program.
By adopting the method, under the condition that the first application program cannot execute the target processing strategy on the first target data, the first target data is subjected to format conversion according to the first data format which can be processed by the second application program to obtain the second target data, and the target processing strategy is executed on the second target data through the second application program, so that the mutual universality between the first application program and the second application program can be realized without redefining a frame, and the realization difficulty is reduced.
FIG. 2 is a flow diagram illustrating another data processing method according to an example embodiment, which may include:
s201, responding to the received data processing instruction, and acquiring first target data to be processed.
Wherein the data processing instruction may be used to instruct to execute a target processing policy on the first target data, and the data processing instruction may be an SQL (Structured Query Language) statement, for example.
S202, determining whether the first application program comprises a first objective function, executing step S203 under the condition that the first application program comprises the first objective function, and executing step S204 to step S208 under the condition that the first application program does not comprise the first objective function.
Wherein the first objective function may be used to execute the objective processing policy.
In this step, after the first target data is obtained, a target processing policy to be executed on the first target data may be determined according to the data processing instruction, for example, if the first application program is a Presto query engine and the data processing instruction is an SQL statement, the SQL statement may be parsed to determine a first target function corresponding to the SQL statement, where the first target function may include multiple first target functions, and then the first target function may be searched for in a built-in function of the Presto query engine. For example, if the SQL statement is "select avg (col) from table", the first objective function may be determined as a function for obtaining an average value, and then the function for obtaining an average value may be searched for in the built-in function of the Presto query engine.
If the first objective function is found in the built-in function of the Presto query engine, it indicates that the Presto query engine can execute the objective processing policy on the first objective data, and if the first objective function is not found in the built-in function of the Presto query engine, it indicates that the Presto query engine cannot execute the objective processing policy on the first objective data.
S203, determining that the first application program can execute the target processing strategy on the first target data, and executing the target processing strategy on the first target data through the first application program.
S204, determining that the first application program cannot execute the target processing strategy on the first target data.
And S205, taking the preset application program including the second objective function in the plurality of preset application programs as the second application program.
The plurality of predetermined applications may be applications with the same function as the first application but different encoding manners, for example, if the first application is a Presto query engine, the predetermined application may be another query engine in the prior art; the second objective function is used to execute the objective processing policy.
In this step, in a case where it is determined that the first application cannot execute the target processing policy on the first target data, a second application that can execute the target processing policy on the first target data may be searched from a plurality of preset applications. In a possible implementation manner, a second objective function may be sequentially searched in a plurality of preset application programs, and a first preset application program including the second objective function is taken as the second application program.
It should be noted that, because the encoding modes of different preset applications are different, the second objective functions corresponding to different preset applications may also be different.
S206, converting the first target data into the second target data according to a first data format which can be processed by a second target function of the second application program.
Wherein the first data format may be a parameter type of the second objective function.
In this step, after determining the second application program, a parameter type of a second objective function of the second application program may be determined, and the first objective function may be converted into the second objective function according to the parameter type.
In a possible implementation manner, before converting the first target data into the second target data according to a first data format that can be processed by a second target function of the second application, it may be determined whether the first application caches a target resource packet corresponding to the second target function of the second application, and in a case where it is determined that the first application caches a target resource packet corresponding to the second target function of the second application, the first target data may be converted into the second target data according to the first data format that can be processed by the second target function of the second application.
For example, whether the second objective function is cached in the local server corresponding to the first application program may be determined according to preset resource information corresponding to the second objective function, and in a case that the second objective function is cached in the local server corresponding to the first application program, the first objective data may be directly converted into the second objective data according to a first data format that can be processed by the second objective function of the second application program. The resource information may include a resource address, a class name, a parameter type, a return type, a creation time, a modification time, and the like.
It should be noted that, it may also be determined whether the process of the first application program has loaded the target resource package corresponding to the second target function, and in a case that the process of the first application program has loaded the target resource package corresponding to the second target function, it indicates that the first application program caches the target resource package corresponding to the second target function.
When it is determined that the first application program does not cache the target resource package corresponding to the second target function of the second application program, the target resource package may be obtained from the server corresponding to the second application program, the target resource package is loaded, and after the target resource package is loaded, the first target data is converted into the second target data according to a first data format that can be processed by the second target function of the second application program.
For example, the target resource package may be obtained from the server corresponding to the second application program according to a preset resource address corresponding to the second target function, the target resource package is downloaded to the local server corresponding to the first application program, then, the target resource package may be loaded through the process of the first application program, and after the target resource package is loaded, the first target data may be converted into the second target data according to the parameter type corresponding to the second target function.
It should be noted that, after the target resource package is loaded, the resource information and the version information of the target resource package may be stored, so that, when it is determined that the first application program caches the second target function, it may be further determined whether a new target resource package needs to be acquired and loaded according to the resource information and the version information of the target resource package corresponding to the second target function.
In addition, before the first target data is converted into the second target data according to the first data format that can be processed by the second target function of the second application program, an initialization method corresponding to the second target function in the second application program may be executed to obtain all data types that can be supported by the second target function, and then, it may be determined whether the first application program can perform format conversion on the first target data according to the first data format. If the first application program can perform format conversion on the first target data according to the first data format, converting the first target data into the second target data according to a first data format which can be processed by a second target function of the second application program; and if the first application program can not perform format conversion on the first target data according to the first data format, outputting exception prompt information.
And S207, executing the target processing strategy on the second target data through the second target function.
The second target data may include a plurality of data, for example, if the data processing instruction is "select avg (col) from table", the first target data may be each line of data in the table, and the second target data includes data obtained by performing format conversion on each line of data in the table according to the first data format. The second objective Function may include a first Function, a second Function, and a third Function, and may be, for example, a UDAF (User Defined aggregation Function).
In this step, after the first target data is converted into the second target data, the first function may be called through a first preset function of the first application program, so as to process a plurality of second target data through the first function, respectively, to obtain a plurality of first data; calling the second function through a second preset function of the first application program so as to aggregate a plurality of first data through the second function to obtain a plurality of second data; and calling the third function through a third preset function of the first application program so as to process a plurality of second data through the third function.
For example, the first function may be an iterate function, the second function may be a merge function, the third function may be a terminate function, a call to the first function may be added in advance to the first preset function, a call to the second function may be added to the second preset function, and a call to the third function may be added to the third preset function. The iterative function may include a plurality of iterative functions, different iterative functions may be run on different servers, and the obtained plurality of first data are also located on different servers. When the first application program calls the iterative function, the second target data is used as an input parameter of the iterative function, and the second target data is processed through a plurality of iterative functions respectively to obtain a plurality of first data.
Further, after a plurality of first data are obtained, the merge function may be called through a second preset function, the first data are used as input parameters of the merge function, and the plurality of first data are aggregated through the merge function, that is, the first data are sent to the same server, so as to obtain a plurality of second data. The merge function may include a plurality of merge functions, and different merge functions may run on different servers.
And finally, calling the terminate function through the third preset function, taking a plurality of second data as input parameters of the terminate function, and executing a final processing flow on the plurality of second data through the terminate function.
It should be noted that the first function, the second function, and the third function are only examples, and the first function, the second function, and the third function may also be functions that execute other policies, which is not limited in this disclosure. The first preset function, the second preset function and the third preset function may be existing functions in the first application program, or may be newly added functions in the first application program, which is not limited in this disclosure.
And S208, converting the first processing result into a second processing result according to a second data format which can be processed by the first application program, and outputting the second processing result.
And executing the result of the target processing strategy on the second target data through a second target function of the second application program when the first processing result can be obtained.
In this step, after the first processing result is obtained, the type of the return value of the first preset function may be used as the second data format, the first processing result may be converted into the second processing result, and the second processing result may be output. It should be noted that, after the first processing result is converted into the second processing result, a subsequent processing flow may also be executed according to the second processing result, which is not limited in this disclosure.
By adopting the method, under the condition that the first application program cannot execute the target processing strategy on the first target data, the first target data is subjected to format conversion according to the first data format which can be processed by the second application program to obtain the second target data, and the target processing strategy is executed on the second target data through the second application program, so that the mutual universality between the first application program and the second application program can be realized without redefining a frame, and the realization difficulty is reduced; further, when the second target data includes a plurality of second target data, the second target data can be aggregated through a plurality of functions, so that mutual universality of the UDAF functions in the first application program and the second application program can be realized, the use scenes of the first application program and the second application program are enlarged, and the user experience is improved.
Fig. 3 is a block diagram illustrating a data processing apparatus applied to a first application according to an exemplary embodiment, as shown in fig. 3, including:
a data obtaining module 301, configured to obtain, in response to receiving a data processing instruction, first target data to be processed, where the data processing instruction is used to instruct to execute a target processing policy on the first target data;
an execution determination module 302 for determining whether the first application is capable of executing the target processing policy on the first target data;
an application determining module 303, configured to determine, when it is determined that the first application cannot execute the target processing policy on the first target data, a second application that can execute the target processing policy, where the second application is encoded in a different manner from the first application;
a first format conversion module 304, configured to convert the first target data into second target data according to a first data format that can be processed by the second application;
a policy executing module 305, configured to execute the target processing policy on the second target data through the second application.
Optionally, the execution determination module 302 is further configured to:
determining whether the first application includes a first objective function, the first objective function being for executing the target processing policy;
determining that the first application is capable of executing the target processing policy on the first target data if it is determined that the first application includes the first target function;
in an instance in which it is determined that the first application does not include the first objective function, it is determined that the first application is unable to execute the target processing policy on the first target data.
Optionally, the application determining module 303 is further configured to:
and taking a preset application program comprising a second objective function in the plurality of preset application programs as the second application program, wherein the second objective function is used for executing the target processing strategy.
Optionally, the first format conversion module 304 is further configured to:
converting the first target data into the second target data according to a first data format which can be processed by a second target function of the second application program;
the executing the target processing policy on the second target data by the second application program includes:
and executing the target processing strategy on the second target data through the second target function.
Optionally, fig. 4 is a block diagram illustrating a second data processing apparatus according to an exemplary embodiment, as shown in fig. 4, the apparatus further includes:
a cache determining module 306, configured to determine whether the first application caches a target resource packet corresponding to a second target function of the second application;
the first format conversion module 304 is further configured to:
and under the condition that the first application program caches the target resource packet corresponding to the second target function of the second application program, converting the first target data into the second target data according to a first data format which can be processed by the second target function of the second application program.
Optionally, fig. 5 is a block diagram illustrating a third data processing apparatus according to an exemplary embodiment, and as shown in fig. 5, the apparatus further includes:
a resource package obtaining module 307, configured to obtain a target resource package corresponding to a second target function of the second application from a server corresponding to the second application when it is determined that the target resource package is not cached by the first application;
a resource package loading module 308, configured to load the target resource package;
the first format conversion module 304 is further configured to, after the target resource package is loaded, convert the first target data into the second target data according to a first data format that can be processed by a second target function of the second application program.
Optionally, the second target data includes a plurality of second target functions, and the second target function includes a first function, a second function, and a third function; the policy enforcement module 305 is further configured to:
calling the first function through a first preset function of the first application program so as to process a plurality of second target data through the first function respectively and obtain a plurality of first data;
calling the second function through a second preset function of the first application program so as to aggregate the first data through the second function to obtain second data;
and calling the third function through a third preset function of the first application program so as to process a plurality of second data through the third function.
Alternatively, fig. 6 is a block diagram illustrating a fourth data processing apparatus according to an exemplary embodiment, as shown in fig. 6, the apparatus further includes:
a second format conversion module 309, configured to convert the first processing result into a second processing result according to a second data format that can be processed by the first application, and output the second processing result, where the first processing result is a result of executing the target processing policy on the second target data through the second application.
By the aid of the device, under the condition that the first application program cannot execute the target processing strategy on the first target data, the first target data is subjected to format conversion according to the first data format which can be processed by the second application program to obtain the second target data, and the target processing strategy is executed on the second target data by the second application program.
Referring now to FIG. 7, shown is a schematic diagram of an electronic device 700 suitable for use in implementing embodiments of the present disclosure. The terminal device in the embodiments of the present disclosure may include, but is not limited to, a mobile terminal such as a mobile phone, a notebook computer, a digital broadcast receiver, a PDA (personal digital assistant), a PAD (tablet computer), a PMP (portable multimedia player), a vehicle terminal (e.g., a car navigation terminal), and the like, and a stationary terminal such as a digital TV, a desktop computer, and the like. The electronic device shown in fig. 7 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present disclosure.
As shown in fig. 7, electronic device 700 may include a processing means (e.g., central processing unit, graphics processor, etc.) 701 that may perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM)702 or a program loaded from storage 708 into a Random Access Memory (RAM) 703. In the RAM 703, various programs and data necessary for the operation of the electronic apparatus 700 are also stored. The processing device 701, the ROM 702, and the RAM 703 are connected to each other by a bus 704. An input/output (I/O) interface 705 is also connected to bus 704.
Generally, the following devices may be connected to the I/O interface 705: input devices 706 including, for example, a touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; an output device 707 including, for example, a Liquid Crystal Display (LCD), a speaker, a vibrator, and the like; storage 708 including, for example, magnetic tape, hard disk, etc.; and a communication device 709. The communication means 709 may allow the electronic device 700 to communicate wirelessly or by wire with other devices to exchange data. While fig. 7 illustrates an electronic device 700 having various means, it is to be understood that not all illustrated means are required to be implemented or provided. More or fewer devices may alternatively be implemented or provided.
In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a non-transitory computer readable medium, the computer program containing program code for performing the method illustrated by the flow chart. In such embodiments, the computer program may be downloaded and installed from a network via the communication means 709, or may be installed from the storage means 708, or may be installed from the ROM 702. The computer program, when executed by the processing device 701, performs the above-described functions defined in the methods of the embodiments of the present disclosure.
It should be noted that the computer readable medium in the present disclosure can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In contrast, in the present disclosure, a computer readable signal medium may comprise a propagated data signal with computer readable program code embodied therein, either in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, optical cables, RF (radio frequency), etc., or any suitable combination of the foregoing.
In some embodiments, the clients, servers may communicate using any currently known or future developed network Protocol, such as HTTP (HyperText Transfer Protocol), and may interconnect with any form or medium of digital data communication (e.g., a communications network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), the Internet (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed network.
The computer readable medium may be embodied in the electronic device; or may exist separately without being assembled into the electronic device.
The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: in response to receiving a data processing instruction, acquiring first target data to be processed, wherein the data processing instruction is used for indicating that a target processing strategy is executed on the first target data; determining whether the first application is capable of executing the target processing policy on the first target data; under the condition that the first application program is determined not to be capable of executing the target processing strategy on the first target data, determining a second application program capable of executing the target processing strategy, wherein the second application program is coded in a mode different from that of the first application program; converting the first target data into second target data according to a first data format which can be processed by the second application program; executing, by the second application, the target processing policy on the second target data.
Computer program code for carrying out operations for the present disclosure may be written in any combination of one or more programming languages, including but not limited to an object oriented programming language such as Java, Smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The modules described in the embodiments of the present disclosure may be implemented by software or hardware. The name of the module does not in some cases constitute a limitation of the module itself, and for example, the data acquisition module may also be described as a "module that acquires first target data to be processed".
The functions described herein above may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), systems on a chip (SOCs), Complex Programmable Logic Devices (CPLDs), and the like.
In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
Example 1 provides, in accordance with one or more embodiments of the present disclosure, a data processing method applied to a first application, the method including: in response to receiving a data processing instruction, acquiring first target data to be processed, wherein the data processing instruction is used for indicating that a target processing strategy is executed on the first target data; determining whether the first application is capable of executing the target processing policy on the first target data; under the condition that the first application program is determined not to be capable of executing the target processing strategy on the first target data, determining a second application program capable of executing the target processing strategy, wherein the second application program is coded in a mode different from that of the first application program; converting the first target data into second target data according to a first data format which can be processed by the second application program; executing, by the second application, the target processing policy on the second target data.
Example 2 provides the method of example 1, the determining whether the first application is capable of executing the target processing policy on the first target data comprising: determining whether the first application includes a first objective function, the first objective function to implement the target processing policy; determining that the first application is capable of executing the target processing policy on the first target data if it is determined that the first application includes the first target function; determining that the first application cannot execute the target processing policy on the first target data if it is determined that the first application does not include the first target function.
Example 3 provides the method of example 1, the determining a second application capable of executing the target processing policy comprising: and taking a preset application program comprising a second objective function in the plurality of preset application programs as the second application program, wherein the second objective function is used for executing the target processing strategy.
Example 4 provides the method of example 3, wherein converting the first target data into second target data in a first data format that the second application is capable of processing comprises: converting the first target data into the second target data according to a first data format which can be processed by a second target function of the second application program; the executing, by the second application, the target processing policy on the second target data comprises: executing the target processing policy on the second target data through the second target function.
Example 5 provides the method of example 4, before converting the first target data into the second target data in the first data format that can be processed by the second target function of the second application, according to one or more embodiments of the present disclosure, the method further including: determining whether the first application program caches a target resource packet corresponding to a second target function of the second application program; the converting the first target data into the second target data according to the first data format that can be processed by the second target function of the second application program includes: and under the condition that the first application program caches a target resource packet corresponding to a second target function of the second application program, converting the first target data into the second target data according to a first data format which can be processed by the second target function of the second application program.
Example 6 provides the method of example 5, further comprising, in accordance with one or more embodiments of the present disclosure: under the condition that the first application program does not cache a target resource packet corresponding to a second target function of the second application program, acquiring the target resource packet from a server corresponding to the second application program; loading the target resource packet; and after the target resource package is loaded, converting the first target data into the second target data according to a first data format which can be processed by a second target function of the second application program.
Example 7 provides the method of example 4, the second objective data comprising a plurality, the second objective function comprising a first function, a second function, and a third function; the executing the target processing policy on the second target data by the second target function comprises: calling the first function through a first preset function of the first application program so as to process a plurality of second target data through the first function respectively and obtain a plurality of first data; calling the second function through a second preset function of the first application program so as to aggregate the plurality of first data through the second function to obtain a plurality of second data; and calling the third function through a third preset function of the first application program so as to process a plurality of second data through the third function.
Example 8 provides the method of any one of examples 1 to 7, further comprising, in accordance with one or more embodiments of the present disclosure: and converting a first processing result into a second processing result according to a second data format which can be processed by the first application program, and outputting the second processing result, wherein the first processing result is a result of executing the target processing strategy on the second target data through the second application program.
Example 9 provides, in accordance with one or more embodiments of the present disclosure, a data processing apparatus to be applied to a first application, the apparatus comprising: the data processing device comprises a data acquisition module, a data processing module and a data processing module, wherein the data acquisition module is used for responding to a received data processing instruction and acquiring first target data to be processed, and the data processing instruction is used for indicating that a target processing strategy is executed on the first target data; an execution determination module to determine whether the first application is capable of executing the target processing policy on the first target data; an application determination module, configured to determine, when it is determined that the first application cannot execute the target processing policy on the first target data, a second application that can execute the target processing policy, where the second application is encoded in a different manner from the first application; the first format conversion module is used for converting the first target data into second target data according to a first data format which can be processed by the second application program; and the strategy execution module is used for executing the target processing strategy on the second target data through the second application program.
Example 10 provides a computer-readable medium having stored thereon a computer program that, when executed by a processing device, performs the steps of the method of any of examples 1-8, in accordance with one or more embodiments of the present disclosure.
Example 11 provides, in accordance with one or more embodiments of the present disclosure, an electronic device, comprising: a storage device having a computer program stored thereon; processing means for executing said computer program in said storage means to carry out the steps of the method of any of examples 1-8.
The foregoing description is only exemplary of the preferred embodiments of the disclosure and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the disclosure herein is not limited to the particular combination of features described above, but also encompasses other embodiments in which any combination of the features described above or their equivalents does not depart from the spirit of the disclosure. For example, the above features and (but not limited to) the features disclosed in this disclosure having similar functions are replaced with each other to form the technical solution.
Further, while operations are depicted in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Likewise, while several specific implementation details are included in the above discussion, these should not be construed as limitations on the scope of the disclosure. Certain features that are described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination.
Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims. With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.

Claims (11)

1. A data processing method, applied to a first application, the method comprising:
in response to receiving a data processing instruction, acquiring first target data to be processed, wherein the data processing instruction is used for indicating that a target processing strategy is executed on the first target data;
determining whether the first application is capable of executing the target processing policy on the first target data;
under the condition that the first application program is determined not to be capable of executing the target processing strategy on the first target data, determining a second application program capable of executing the target processing strategy, wherein the second application program is coded in a mode different from that of the first application program;
converting the first target data into second target data according to a first data format which can be processed by the second application program;
executing, by the second application, the target processing policy on the second target data.
2. The method of claim 1, wherein the determining whether the first application is capable of executing the target processing policy on the first target data comprises:
determining whether the first application includes a first objective function, the first objective function to implement the target processing policy;
determining that the first application is capable of executing the target processing policy on the first target data if it is determined that the first application includes the first target function;
determining that the first application cannot execute the target processing policy on the first target data if it is determined that the first application does not include the first target function.
3. The method of claim 1, wherein determining a second application that can execute the target processing policy comprises:
and taking a preset application program comprising a second objective function in the plurality of preset application programs as the second application program, wherein the second objective function is used for executing the target processing strategy.
4. The method of claim 3, wherein converting the first target data into second target data in a first data format that the second application is capable of handling comprises:
converting the first target data into the second target data according to a first data format which can be processed by a second target function of the second application program;
the executing, by the second application, the target processing policy on the second target data comprises:
executing the target processing policy on the second target data through the second target function.
5. The method of claim 4, wherein prior to converting the first target data into the second target data in the first data format that is processable according to the second target function of the second application, the method further comprises:
determining whether the first application program caches a target resource packet corresponding to a second target function of the second application program;
the converting the first target data into the second target data according to the first data format that can be processed by the second target function of the second application program includes:
and under the condition that the first application program caches a target resource packet corresponding to a second target function of the second application program, converting the first target data into the second target data according to a first data format which can be processed by the second target function of the second application program.
6. The method of claim 5, further comprising:
under the condition that the first application program does not cache a target resource packet corresponding to a second target function of the second application program, acquiring the target resource packet from a server corresponding to the second application program;
loading the target resource packet;
and after the target resource package is loaded, converting the first target data into the second target data according to a first data format which can be processed by a second target function of the second application program.
7. The method of claim 4, wherein the second objective data comprises a plurality, and the second objective function comprises a first function, a second function, and a third function; the executing the target processing policy on the second target data by the second target function comprises:
calling the first function through a first preset function of the first application program so as to process a plurality of second target data through the first function respectively and obtain a plurality of first data;
calling the second function through a second preset function of the first application program so as to aggregate the plurality of first data through the second function to obtain a plurality of second data;
and calling the third function through a third preset function of the first application program so as to process a plurality of second data through the third function.
8. The method according to any one of claims 1-7, further comprising:
and converting a first processing result into a second processing result according to a second data format which can be processed by the first application program, and outputting the second processing result, wherein the first processing result is a result of executing the target processing strategy on the second target data through the second application program.
9. A data processing apparatus, applied to a first application, the apparatus comprising:
the data processing device comprises a data acquisition module, a data processing module and a data processing module, wherein the data acquisition module is used for responding to a received data processing instruction and acquiring first target data to be processed, and the data processing instruction is used for indicating that a target processing strategy is executed on the first target data;
an execution determination module to determine whether the first application is capable of executing the target processing policy on the first target data;
an application determination module, configured to determine, when it is determined that the first application cannot execute the target processing policy on the first target data, a second application that can execute the target processing policy, where the second application is encoded in a different manner from the first application;
the first format conversion module is used for converting the first target data into second target data according to a first data format which can be processed by the second application program;
and the strategy execution module is used for executing the target processing strategy on the second target data through the second application program.
10. A computer-readable medium, on which a computer program is stored, characterized in that the program, when being executed by processing means, carries out the steps of the method of any one of claims 1 to 8.
11. An electronic device, comprising:
a storage device having a computer program stored thereon;
processing means for executing the computer program in the storage means to carry out the steps of the method according to any one of claims 1 to 8.
CN202111234197.1A 2021-10-22 2021-10-22 Data processing method and device, readable medium and electronic equipment Pending CN113971192A (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN202111234197.1A CN113971192A (en) 2021-10-22 2021-10-22 Data processing method and device, readable medium and electronic equipment
PCT/CN2022/120087 WO2023065937A1 (en) 2021-10-22 2022-09-21 Data processing method and apparatus, and readable medium and electronic device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111234197.1A CN113971192A (en) 2021-10-22 2021-10-22 Data processing method and device, readable medium and electronic equipment

Publications (1)

Publication Number Publication Date
CN113971192A true CN113971192A (en) 2022-01-25

Family

ID=79587969

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111234197.1A Pending CN113971192A (en) 2021-10-22 2021-10-22 Data processing method and device, readable medium and electronic equipment

Country Status (2)

Country Link
CN (1) CN113971192A (en)
WO (1) WO2023065937A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023065937A1 (en) * 2021-10-22 2023-04-27 北京火山引擎科技有限公司 Data processing method and apparatus, and readable medium and electronic device

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109033123A (en) * 2018-05-31 2018-12-18 康键信息技术(深圳)有限公司 Querying method, device, computer equipment and storage medium based on big data
CN109426571A (en) * 2017-08-28 2019-03-05 阿里巴巴集团控股有限公司 Function call and the method for data access, system, storage medium, processor and device
CN111611221A (en) * 2019-02-26 2020-09-01 北京京东尚科信息技术有限公司 Hybrid computing system, data processing method and device
CN111897525A (en) * 2020-07-15 2020-11-06 深圳市腾讯网域计算机网络有限公司 Big data processing method and system
CN112860798A (en) * 2021-02-10 2021-05-28 挂号网(杭州)科技有限公司 Data processing method and device, electronic equipment and storage medium

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108415786B (en) * 2018-03-13 2022-05-31 联想(北京)有限公司 Information processing method and device, electronic equipment and computer readable medium
CN111124713B (en) * 2019-12-24 2024-03-26 北京安兔兔科技有限公司 Equipment system function calling method, device, terminal equipment and storage medium
CN113971192A (en) * 2021-10-22 2022-01-25 北京火山引擎科技有限公司 Data processing method and device, readable medium and electronic equipment

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109426571A (en) * 2017-08-28 2019-03-05 阿里巴巴集团控股有限公司 Function call and the method for data access, system, storage medium, processor and device
CN109033123A (en) * 2018-05-31 2018-12-18 康键信息技术(深圳)有限公司 Querying method, device, computer equipment and storage medium based on big data
CN111611221A (en) * 2019-02-26 2020-09-01 北京京东尚科信息技术有限公司 Hybrid computing system, data processing method and device
CN111897525A (en) * 2020-07-15 2020-11-06 深圳市腾讯网域计算机网络有限公司 Big data processing method and system
CN112860798A (en) * 2021-02-10 2021-05-28 挂号网(杭州)科技有限公司 Data processing method and device, electronic equipment and storage medium

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023065937A1 (en) * 2021-10-22 2023-04-27 北京火山引擎科技有限公司 Data processing method and apparatus, and readable medium and electronic device

Also Published As

Publication number Publication date
WO2023065937A1 (en) 2023-04-27

Similar Documents

Publication Publication Date Title
CN111581563B (en) Page response method and device, storage medium and electronic equipment
CN110753089B (en) Method, device, medium and electronic equipment for managing client
CN111930534A (en) Data calling method and device and electronic equipment
CN110300165B (en) Method, device and equipment for calling functional program and storage medium
CN112035529A (en) Caching method and device, electronic equipment and computer readable storage medium
CN113886336A (en) Flow detection method, device, equipment and medium
CN111309304B (en) Method, device, medium and electronic equipment for generating IDL file
CN112099982A (en) Collapse information positioning method, device, medium and electronic equipment
WO2023065937A1 (en) Data processing method and apparatus, and readable medium and electronic device
CN113704291A (en) Data query method and device, storage medium and electronic equipment
CN111241137A (en) Data processing method and device, electronic equipment and storage medium
CN109614089B (en) Automatic generation method, device, equipment and storage medium of data access code
CN114741686A (en) Method and device for detecting program white list and related equipment
CN113360348A (en) Exception request processing method and device, electronic equipment and storage medium
CN111538717A (en) Data processing method and device, electronic equipment and computer readable medium
CN111796865A (en) Byte code file modification method and device, terminal equipment and medium
CN111580890A (en) Method, apparatus, electronic device, and computer-readable medium for processing features
CN113391860A (en) Service request processing method and device, electronic equipment and computer storage medium
CN111240738A (en) File processing method, device, equipment and medium
CN111447298A (en) Method, device, equipment and medium for acquiring network address in application program
CN112445820A (en) Data conversion method and device
CN111209042B (en) Method, device, medium and electronic equipment for establishing function stack
CN111562913B (en) Method, device and equipment for pre-creating view component and computer readable medium
CN116680316A (en) Method, device, medium and electronic equipment for acquiring dimension data
CN111831655B (en) Data processing method, device, medium and electronic equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20220125

RJ01 Rejection of invention patent application after publication