CN113742364B - Data access method, device, electronic equipment, storage medium and program product - Google Patents

Data access method, device, electronic equipment, storage medium and program product Download PDF

Info

Publication number
CN113742364B
CN113742364B CN202111064251.2A CN202111064251A CN113742364B CN 113742364 B CN113742364 B CN 113742364B CN 202111064251 A CN202111064251 A CN 202111064251A CN 113742364 B CN113742364 B CN 113742364B
Authority
CN
China
Prior art keywords
user
request
data
defined interface
data source
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202111064251.2A
Other languages
Chinese (zh)
Other versions
CN113742364A (en
Inventor
关振宇
朱家强
郑为锋
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Lakala Payment Co ltd
Original Assignee
Lakala Payment Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Lakala Payment Co ltd filed Critical Lakala Payment Co ltd
Priority to CN202111064251.2A priority Critical patent/CN113742364B/en
Publication of CN113742364A publication Critical patent/CN113742364A/en
Application granted granted Critical
Publication of CN113742364B publication Critical patent/CN113742364B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/242—Query formulation
    • G06F16/2433—Query languages
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/25—Integrating or interfacing systems involving database management systems

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Computational Linguistics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The embodiment of the disclosure discloses a data access method, a device, an electronic device, a storage medium and a program product, wherein the method comprises the following steps: acquiring a data access request; when the data access request requests an external data source, an asynchronous data request is sent to the external data source by calling a first user-defined interface; asynchronously acquiring request data returned by the external data source from a background storage medium by calling a second user-defined interface; after the first user-defined interface sends an asynchronous data request to the external data source, a background thread receives the request data from the external data source and stores the request data to the background storage medium. The technical scheme can improve the data access efficiency of the large database management system to the external data source, and further improve the data processing efficiency of the large data processing system on the whole.

Description

Data access method, device, electronic equipment, storage medium and program product
Technical Field
The embodiment of the disclosure relates to the technical field of big data, in particular to a data access method, a device, electronic equipment, a storage medium and a program product.
Background
In recent years, with the advent of the big data age, the data is explosively increased, the information scale is increasingly huge, the information data provides a solid information foundation for the decision of enterprises, the generation of massive data is unprecedented for the impact and transformation brought by society, and for enterprises, how to extract useful values from the data rapidly and efficiently becomes a new challenge. The traditional data is mainly structured data, but the data from social websites, electronic commerce and Internet of things are basically unstructured and semi-structured data, and the management system of the relational database for the traditional data can realize effective management and use, and the data greatly exceeds the capability range of the software tool of the traditional database due to the large quantity, rapidness and complexity. Thus, big data processing engines have evolved.
However, data in large data processing engines typically has one or more data sources, which may include homogeneous/heterogeneous (large) databases, file systems, service interfaces, and the like. In the prior art, the efficiency of multi-source data processing is low, the accuracy is poor, the interface is not friendly enough, and the processing of multi-source data by workers is inconvenient.
Therefore, how to improve the access efficiency of multiple data sources in a big data system is one of the main problems that needs to be solved currently.
Disclosure of Invention
The embodiment of the disclosure provides a data access method, a data access device, electronic equipment, a storage medium and a program product.
In a first aspect, an embodiment of the present disclosure provides a data access method, including:
acquiring a data access request;
when the data access request requests an external data source, an asynchronous data request is sent to the external data source by calling a first user-defined interface;
asynchronously acquiring request data returned by the external data source from a background storage medium by calling a second user-defined interface; after the first user-defined interface sends an asynchronous data request to the external data source, a background thread receives the request data from the external data source and stores the request data to the background storage medium.
Further, the method further comprises:
a first algorithm for receiving a request for asynchronous data sent to an external data source and input by a user;
registering the first algorithm as the first user-defined interface.
Further, the method further comprises:
Receiving a second algorithm input by a user and asynchronously acquiring request data from the background storage medium;
registering the second algorithm as the second user-defined interface.
Further, the data access request includes an SQL statement.
Further, when the data access request requests an external data source, sending an asynchronous data request to the external data source by calling the first user-defined interface, including:
analyzing the SQL statement;
when the SQL statement comprises a data request for an external data source, the first user-defined interface is called to send an asynchronous data request to the external data source.
Further, asynchronously acquiring the request data returned by the external data source from the background storage medium by calling a second user-defined interface, including:
determining whether request data returned by the external data source has been received;
and when the request data returned by the external data source is received, calling the second user-defined interface to asynchronously acquire the request data returned by the external data source from the background storage medium.
Further, asynchronously acquiring the request data returned by the external data source from the background storage medium by calling a second user-defined interface, including:
And periodically calling the second user-defined interface so as to periodically access the background storage medium, and acquiring the request data when the request data exists in the background storage medium.
Further, when the data access request requests an external data source, sending an asynchronous data request to the external data source by calling the first user-defined interface, including:
analyzing the SQL statement;
when the SQL statement comprises a parameter command which is transmitted into the first user-defined interface, the first user-defined interface is called to send an asynchronous data request to the external data source.
Further, asynchronously acquiring the request data returned by the external data source from the background storage medium by calling a second user-defined interface, including:
receiving SQL sentences input by a user after the data access request;
analyzing the SQL statement;
and when the SQL statement comprises a parameter command which is transmitted into the second user-defined interface, the second user-defined interface is called to asynchronously acquire the request data returned by the external data source from the background storage medium.
In a second aspect, an embodiment of the present disclosure provides a data access method, including:
the data request end sends a data access request to the database management system;
when the data access request requests an external data source, the database management system sends an asynchronous data request to the external data source by calling a first user-defined interface;
the database management system asynchronously acquires request data returned by the external data source from a background storage medium by calling a second user-defined interface; after the first user-defined interface sends an asynchronous data request to the external data source, a background thread receives the request data from the external data source and stores the request data to the background storage medium;
and the database management system returns the request data to the data request terminal.
Further, the method further comprises:
and after the external data source receives the asynchronous data request, the request data is returned to the database management system.
Further, the method further comprises:
the client receives a first algorithm input by a user and used for sending an asynchronous data request to an external data source;
The client sends the first algorithm to the database management system;
the database management system registers the first algorithm as the first user-defined interface.
Further, the method further comprises:
the client receives a second algorithm input by a user and asynchronously acquiring request data from the background storage medium;
the client sends the second algorithm to the database management system;
the database management system registers the second algorithm as the second user-defined interface.
Further, the data access request includes an SQL statement.
Further, when the external data source is requested by the data access request, the database management system sends an asynchronous data request to the external data source by calling the first user-defined interface, and the method comprises the following steps:
the database management system analyzes the SQL statement;
and when the SQL sentence comprises a data request for an external data source, the database management system calls the first user-defined interface to send an asynchronous data request to the external data source.
Further, the database management system asynchronously acquires the request data returned by the external data source from the background storage medium by calling a second user-defined interface, and the method comprises the following steps:
The database management system determines whether request data returned by the external data source is received;
and when the database management system receives the request data returned by the external data source, the second user-defined interface is called to asynchronously acquire the request data returned by the external data source from the background storage medium.
Further, the database management system asynchronously acquires the request data returned by the external data source from the background storage medium by calling a second user-defined interface, and the method comprises the following steps:
and the database management system periodically calls the second user-defined interface so as to periodically access the background storage medium, and when the request data exists in the background storage medium, the request data is acquired.
Further, when the external data source is requested by the data access request, the database management system sends an asynchronous data request to the external data source by calling the first user-defined interface, and the method comprises the following steps:
the database management system analyzes the SQL statement;
and when the SQL sentence comprises a parameter command which is transmitted into the first user-defined interface, the database management system calls the first user-defined interface to send an asynchronous data request to the external data source.
Further, the database management system asynchronously acquires the request data returned by the external data source from the background storage medium by calling a second user-defined interface, and the method comprises the following steps:
the database management system receives SQL sentences input by a user after the data access request;
the database management system analyzes the SQL statement;
and when the SQL sentence comprises a parameter command which is transmitted into the second user-defined interface, the database management system calls the second user-defined interface to asynchronously acquire the request data returned by the external data source from the background storage medium.
In a third aspect, in an embodiment of the present disclosure, there is provided a data access apparatus, including:
the first acquisition module is configured to acquire a data access request;
the first calling module is configured to send an asynchronous data request to an external data source by calling a first user-defined interface when the data access request requests the external data source;
the second calling module is configured to asynchronously acquire the request data returned by the external data source from the background storage medium by calling a second user-defined interface; after the first user-defined interface sends an asynchronous data request to the external data source, a background thread receives the request data from the external data source and stores the request data to the background storage medium.
In a fourth aspect, in an embodiment of the present disclosure, there is provided a data access system, including: a data request end and a database management system;
the data request end sends a data access request to a database management system;
when the data access request requests an external data source, the database management system sends an asynchronous data request to the external data source by calling a first user-defined interface;
the database management system asynchronously acquires request data returned by the external data source from a background storage medium by calling a second user-defined interface; after the first user-defined interface sends an asynchronous data request to the external data source, a background thread receives the request data from the external data source and stores the request data to the background storage medium;
and the database management system returns the request data to the data request terminal.
The functions may be implemented by hardware, or may be implemented by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the functions described above.
In one possible design, the structure of the above apparatus includes a memory for storing one or more computer instructions for supporting the above apparatus to perform the corresponding method, and a processor configured to execute the computer instructions stored in the memory. The apparatus may further comprise a communication interface for the apparatus to communicate with other devices or a communication network.
In a fifth aspect, embodiments of the present disclosure provide an electronic device comprising a memory for storing one or more computer instructions supporting any of the apparatus for performing the corresponding method described above, and a processor configured to execute the computer instructions stored in the memory. Any of the above-described apparatuses may further include a communication interface for communicating with other devices or a communication network.
In a sixth aspect, embodiments of the present disclosure provide a computer-readable storage medium storing computer instructions for use by any one of the above-described apparatuses, comprising computer instructions for performing any one of the above-described methods.
In a seventh aspect, embodiments of the present disclosure provide a computer program product comprising computer instructions for implementing the steps of the method of any one of the above aspects when executed by a processor.
The technical scheme provided by the embodiment of the disclosure can comprise the following beneficial effects:
in the embodiment of the disclosure, in a big data access system, after a database management system receives a data access request, for the data access request of an external data source, an asynchronous data acquisition request is sent to the external data source by calling a first user-defined interface, a background thread is started to monitor request data returned by the external data source, the request data received from the external data source is stored in a background storage medium, and then the database management system can take out the request data from the background storage medium by calling a second user-defined interface and return the request data to a data request end. In this way, the data access efficiency of the large database management system to the external data source can be improved, and the data processing efficiency of the large data processing system can be further improved as a whole.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of embodiments of the disclosure.
Drawings
Other features, objects and advantages of the embodiments of the present disclosure will become more apparent from the following detailed description of non-limiting embodiments, taken in conjunction with the accompanying drawings. In the drawings:
FIG. 1 illustrates a flow chart of a data access method according to an embodiment of the present disclosure;
FIG. 2 illustrates a flow chart of a data access method according to another embodiment of the present disclosure;
FIG. 3 illustrates an application flow diagram of a data access method according to an embodiment of the present disclosure;
FIG. 4 shows a block diagram of a data access apparatus according to an embodiment of the present disclosure;
FIG. 5 shows a block diagram of a data access system according to an embodiment of the present disclosure;
fig. 6 is a schematic diagram of a computer system suitable for use in implementing a data access method according to an embodiment of the present disclosure.
Detailed Description
Hereinafter, exemplary implementations of the embodiments of the present disclosure will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily implement them. In addition, for the sake of clarity, portions irrelevant to description of the exemplary embodiments are omitted in the drawings.
In the presently disclosed embodiments, it is to be understood that the terms such as "comprises" or "comprising" and the like are intended to indicate the presence of features, numbers, steps, acts, components, portions, or combinations thereof disclosed in the present specification, and are not intended to exclude the possibility of one or more other features, numbers, steps, acts, components, portions, or combinations thereof being present or added.
In addition, it should be noted that, without conflict, the embodiments of the present disclosure and features of the embodiments may be combined with each other. Embodiments of the present disclosure will be described in detail below with reference to the attached drawings in conjunction with the embodiments.
Fig. 1 shows a flowchart of a data access method according to an embodiment of the present disclosure, as shown in fig. 1, including the steps of:
in step S101, a data access request is acquired;
in step S102, when the data access request requests an external data source, an asynchronous data request is sent to the external data source by calling a first user-defined interface;
in step S103, asynchronously acquiring the request data returned by the external data source from the background storage medium by calling a second user-defined interface; after the first user-defined interface sends an asynchronous data request to the external data source, a background thread receives the request data from the external data source and stores the request data to the background storage medium.
As mentioned above, in recent years, with the advent of the big data age, the data has been explosively increased, the information scale has been increasingly huge, the information data provides a solid information foundation for the decision of enterprises, the impact and transformation brought by the generation of massive data to society are unprecedented, and how to extract useful values from data quickly and efficiently has become a new challenge for enterprises. The traditional data is mainly structured data, but the data from social websites, electronic commerce and Internet of things are basically unstructured and semi-structured data, and the management system of the relational database for the traditional data can realize effective management and use, and the data greatly exceeds the capability range of the software tool of the traditional database due to the large quantity, rapidness and complexity. Thus, big data processing engines have evolved.
However, data in large data processing engines typically has one or more data sources, which may include homogeneous/heterogeneous (large) databases, file systems, service interfaces, and the like. In the prior art, the efficiency of multi-source data processing is low, the accuracy is poor, the interface is not friendly enough, and the processing of multi-source data by workers is inconvenient.
Therefore, how to improve the access efficiency of multiple data sources in a big data system is one of the main problems that needs to be solved currently.
In one embodiment of the present disclosure, the data access method may be adapted to be executed on a database management system.
In an embodiment of the present disclosure, the data access request may be sent by the data request end to the database management system, and after the database management system receives the data access request, the database management system may analyze the data access request first, and determine, according to an analysis result, whether the request of the data access request is an external data source. It should be noted that, the database management system includes a database, and the external data source may be other data sources besides the database managed by the database management system, for example, may be heterogeneous external databases related to the database management system, or may be isomorphic external databases.
In a general process flow of the database management system, a data access request is sent to an external data source according to a data access request of the external data source, and the external data source waits for data to be returned and then carries out the next process. However, in this case, if the response of the external data source is slow, the data access efficiency is reduced, especially, the data processing efficiency is greatly reduced for the continuous multiple data accesses of the external data source, which further affects the overall processing efficiency of the database.
In order to solve the above technical problem, the embodiments of the present disclosure send an asynchronous data request to an external data source by calling a first user-defined interface for a data access request of the external data source, and immediately return to perform the next processing after the asynchronous data request is sent, instead of waiting for a response of the external data source. And then asynchronously acquiring the request data requested from the external data source from the background storage medium by calling the second user-defined interface. That is, for an external data request, the database management system may acquire the request data from the external data source by calling the first user-defined interface and the second user-defined interface in a manner of asynchronously acquiring the data, and during a period of response of the external data source, the database management system continues to perform the next processing, instead of waiting for the external data source to return the request data and then performing the next processing, which can improve the access efficiency of the external data source.
The background thread may be started after the first user-defined interface is called, and the database management system may monitor the request data returned by the external data source at the communication interface with the external data source, and store the received request data in the background storage medium, so that after the second user-defined interface is called by the database management system, the second user-defined interface may request to obtain the request data stored in the background storage medium from the background thread, and after the second user-defined interface obtains the request data from the background storage medium, the database management system returns the request data to the client. Of course, it will be understood that the data access request may be received from a data analysis processing end instead of the client, or may even be received by a data processing analysis program running in the database management system, where the request data may be returned to the data analysis processing end or the data processing analysis program, and specifically, the request data may be returned to the data request end according to the actual situation.
In the embodiment of the disclosure, in a big data access system, after a database management system receives a data access request, for the data access request of an external data source, an asynchronous data acquisition request is sent to the external data source by calling a first user-defined interface, a background thread is started to monitor request data returned by the external data source, the request data received from the external data source is stored in a background storage medium, and then the database management system can take out the request data from the background storage medium by calling a second user-defined interface and return the request data to a data request end. In this way, the data access efficiency of the large database management system to the external data source can be improved, and the data processing efficiency of the large data processing system can be further improved as a whole.
In an embodiment of the present disclosure, the method may further include the steps of:
a first algorithm for receiving a request for asynchronous data sent to an external data source and input by a user;
registering the first algorithm as the first user-defined interface.
In this alternative implementation, the first custom interface may be a registered user-written algorithm, rather than a self-contained interface of the database management system. The user writes a first algorithm for asynchronously sending a data request to an external data source according to the access flow of the external data source, inputs the first algorithm to a database management system, registers the first algorithm in the database management system after receiving the first algorithm, registers the first algorithm as a first user-defined interface, and after successful registration, the user can use the first user-defined interface by writing SQL sentences.
In an embodiment of the present disclosure, the method may further include the steps of:
receiving a second algorithm input by a user and asynchronously acquiring request data from the background storage medium;
registering the second algorithm as the second user-defined interface.
In this alternative implementation, the second custom interface may be a user-written algorithm registered, rather than a database management system self-contained interface. The user writes a second algorithm for asynchronously acquiring the request data from the background storage medium according to the access flow of the external data source, the second algorithm is input into the database management system, the database management system registers the second algorithm in the database management system after receiving the second algorithm, the second algorithm is registered as a second user-defined interface, and after the registration is successful, the user can use the second user-defined interface by writing SQL sentences.
In one embodiment of the disclosure, the data access request includes an SQL statement.
In this alternative implementation, the data access request may be a SQL statement entered by the user. Of course, it is understood that the SQL statement may be an SQL statement written in a procedure of performing data statistics analysis or the like.
In an embodiment of the present disclosure, step S102, that is, when the external data source is requested by the data access request, may further include the following steps of:
analyzing the SQL statement;
when the SQL statement comprises a data request for an external data source, the first user-defined interface is called to send an asynchronous data request to the external data source.
In this alternative implementation manner, after the SQL statement is obtained, the SQL statement may be parsed, if the parsing result finds that the SQL statement needs to access the external data source, the database management system may call the first user-defined interface to send an asynchronous data request to the external data source, instead of using the default access manner of the database, that is, directly send the data request to the external data source, and wait for the external data source to return the requested data for the next processing. In this embodiment, after the asynchronous data request is sent to the external data source by calling the first user-defined interface, the next process may be continued, instead of waiting for the external data source to return the requested data.
In an embodiment of the present disclosure, step S103, that is, a step of asynchronously acquiring, by calling a second user-defined interface, the request data returned by the external data source from the background storage medium, may further include the following steps:
determining whether request data returned by the external data source has been received;
and when the request data returned by the external data source is received, calling the second user-defined interface to asynchronously acquire the request data returned by the external data source from the background storage medium.
In this alternative implementation, the database management system may first determine whether the background thread has received the data returned by the external data source, and if so, may invoke the second user-defined interface to obtain the request data from the background storage medium, and further process the request data. In some embodiments, the background thread may send an acknowledgement to the database management system after receiving the request data returned by the external data source, so that the database management system can determine whether the request data has been received. In other embodiments, the background storage medium may store the request data after the background storage medium while recording the storage identifier, and the database management system may determine whether the request data returned by the external data source has been received by looking at whether there is a corresponding storage identifier.
In some embodiments, the request data may be returned to the requestor. Of course, in some other embodiments, statistical analysis may be performed on the request data, which is specific to the actual application scenario, and is not limited herein. It will be appreciated that by invoking the first user-defined interface to send an asynchronous data request to the external data source and invoking the second user-defined interface to retrieve the requested data from the background storage medium, which is equivalent to directly retrieving the requested data from the external data source using a default interface in the database management system, except that the database management system performs other processing during the period from sending the asynchronous data request to retrieving the requested data from the background storage medium, while waiting for the external data source to return the requested data, and no other processing is performed. In the mode of the embodiment of the disclosure, the data access efficiency can be remarkably improved under the scene of continuously requesting data from an external data source.
In an embodiment of the present disclosure, step S103, that is, a step of asynchronously acquiring, by calling a second user-defined interface, the request data returned by the external data source from the background storage medium, may further include the following steps:
And periodically calling the second user-defined interface so as to periodically access the background storage medium, and acquiring the request data when the request data exists in the background storage medium.
In this alternative implementation manner, the database management system may also periodically call the second user-defined interface, so that after the background thread stores the request data returned by the external data source into the background storage medium, the request data can be obtained from the background storage medium and further processed.
In an embodiment of the present disclosure, step S102, that is, when the external data source is requested by the data access request, may further include the following steps of:
analyzing the SQL statement;
when the SQL statement comprises a parameter command which is transmitted into the first user-defined interface, the first user-defined interface is called to send an asynchronous data request to the external data source.
In this alternative implementation manner, the user may directly request to obtain the request data from the external data source by calling the first user-defined interface when writing the SQL statement. That is, when the user writes the SQL sentence, the user-defined interface is directly used, after the database management system analyzes the SQL sentence, if the identification of the first user-defined interface is analyzed, the first user-defined interface is directly called and executed based on the parameter command in the SQL sentence, so that the asynchronous data request is sent to the external data source.
In an embodiment of the present disclosure, step S103, that is, a step of asynchronously acquiring, by calling a second user-defined interface, the request data returned by the external data source from the background storage medium, may further include the following steps:
receiving SQL sentences input by a user after the data access request;
analyzing the SQL statement;
and when the SQL statement comprises a parameter command which is transmitted into the second user-defined interface, the second user-defined interface is called to asynchronously acquire the request data returned by the external data source from the background storage medium.
In this optional implementation manner, the user may write and call the second user-defined interface in the data access procedure, and acquire the SQL statement of the request data returned by the external data source from the background storage medium, that is, when the user writes the SQL statement corresponding to the data access procedure, may write, at the location where the external data source needs to be called, the parameter command that is transmitted to the first user-defined interface to send the parameter command of the asynchronous data request to the external data source, and then write, at the later time, the parameter command that is transmitted to the second user-defined interface to call the second user-defined interface to acquire, from the background storage medium, the parameter command of the request data returned by the external data source, so as to finally achieve the purpose of asynchronously acquiring the request data from the external data source by using the user-defined interface. It will be appreciated that the SQL statement that invokes the second user-defined interface may occur in pairs after the SQL statement that invokes the first user-defined interface. In addition, the appearance position of the SQL statement for calling the second user-defined interface can be determined based on actual needs, for example, the SQL statement for calling the first user-defined interface can be immediately after the SQL statement for calling the first user-defined interface, or can be at a position far away from the SQL statement for calling the first user-defined interface, but before the SQL statement for requesting data is used.
Fig. 2 shows a flowchart of a data access method according to another embodiment of the present disclosure, as shown in fig. 2, including the steps of:
in step S201, the data request end sends a data access request to the database management system;
in step S202, when the external data source is requested by the data access request, the database management system sends an asynchronous data request to the external data source by calling a first user-defined interface;
in step S203, the database management system asynchronously acquires the request data returned by the external data source from the background storage medium by calling a second user-defined interface; after the first user-defined interface sends an asynchronous data request to the external data source, a background thread receives the request data from the external data source and stores the request data to the background storage medium;
in step S204, the database management system returns the request data to the data request terminal.
In one embodiment of the present disclosure, the data access method may be adapted for execution in a data access system comprising a data requesting end and a database management system.
In an embodiment of the present disclosure, the data access request may be sent by the data request end to the database management system, and after the database management system receives the data access request, the database management system may analyze the data access request first, and determine, according to an analysis result, whether the request of the data access request is an external data source. It should be noted that, the database management system includes a database, and the external data source may be other data sources besides the database managed by the database management system, for example, may be heterogeneous external databases related to the database management system, or may be isomorphic external databases.
In a general process flow of the database management system, a data access request is sent to an external data source according to a data access request of the external data source, and the external data source waits for data to be returned and then carries out the next process. However, in this case, if the response of the external data source is slow, the data access efficiency is reduced, especially, the data processing efficiency is greatly reduced for the continuous multiple data accesses of the external data source, which further affects the overall processing efficiency of the database.
In order to solve the above technical problem, a database management system sends an asynchronous data request to an external data source by calling a first user-defined interface, and returns to perform the next processing immediately after the asynchronous data request is sent, instead of waiting for a response of the external data source. The database management system then asynchronously retrieves the requested data from the external data source from the background storage medium by invoking the second user-defined interface. That is, for an external data request, the database management system may acquire the request data from the external data source by calling the first user-defined interface and the second user-defined interface in a manner of asynchronously acquiring the data, and during a period of response of the external data source, the database management system continues to perform the next processing, instead of waiting for the external data source to return the request data and then performing the next processing, which can improve the access efficiency of the external data source.
The background thread may be started after the first user-defined interface is called, and the database management system may monitor the request data returned by the external data source at the communication interface with the external data source, and store the received request data in the background storage medium, so that after the second user-defined interface is called by the database management system, the second user-defined interface may request to obtain the request data stored in the background storage medium from the background thread, and after the second user-defined interface obtains the request data from the background storage medium, the database management system returns the request data to the client. Of course, it will be understood that the data access request may be received from a data analysis processing end instead of the client, or may even be received by a data processing analysis program running in the database management system, where the request data may be returned to the data analysis processing end or the data processing analysis program, and specifically, the request data may be returned to the data request end according to the actual situation.
In the embodiment of the disclosure, in a big data access system, after a database management system receives a data access request, for the data access request of an external data source, an asynchronous data acquisition request is sent to the external data source by calling a first user-defined interface, a background thread is started to monitor request data returned by the external data source, the request data received from the external data source is stored in a background storage medium, and then the database management system can take out the request data from the background storage medium by calling a second user-defined interface and return the request data to a data request end. In this way, the data access efficiency of the large database management system to the external data source can be improved, and the data processing efficiency of the large data processing system can be further improved as a whole.
In an embodiment of the present disclosure, the method may further include the steps of:
and after the external data source receives the asynchronous data request, the request data is returned to the database management system.
In this alternative implementation, the external data source may prepare the requested request data after receiving the asynchronous data request of the database management system and return the prepared request data in the manner indicated in the digital asynchronous data request. The manner indicated in the asynchronous data request may include an interface for asynchronous data return, such as an interface for a background thread in a database management system to snoop data.
In an embodiment of the present disclosure, the method may further include the steps of:
the client receives a first algorithm input by a user and used for sending an asynchronous data request to an external data source;
the client sends the first algorithm to the database management system;
the database management system registers the first algorithm as the first user-defined interface.
In this alternative implementation, the first custom interface may be a registered user-written algorithm, rather than a self-contained interface of the database management system. A user writes a first algorithm for asynchronously sending a data request to an external data source on a client aiming at an access flow of the external data source, the client sends the first algorithm to a database management system, the database management system registers the first algorithm in the database management system after receiving the first algorithm, the first algorithm is registered as a first user-defined interface, and after successful registration, the user can use the first user-defined interface by writing SQL sentences.
In an embodiment of the present disclosure, the method may further include the steps of:
the client receives a second algorithm input by a user and asynchronously acquiring request data from the background storage medium;
The client sends the second algorithm to the database management system;
the database management system registers the second algorithm as the second user-defined interface.
In this alternative implementation, the second custom interface may be a user-written algorithm registered, rather than a database management system self-contained interface. And writing a second algorithm for asynchronously acquiring the request data from the background storage medium on the client side aiming at the access flow of the external data source by the user, sending the second algorithm to the database management system by the client side, registering the second algorithm in the database management system after the database management system receives the second algorithm, registering the second algorithm as a second user-defined interface, and writing SQL sentences by the user to use the second user-defined interface after the registration is successful.
In one embodiment of the disclosure, the data access request includes an SQL statement.
In this alternative implementation, the data access request may be a SQL statement entered by the user. Of course, it is understood that the SQL statement may be an SQL statement written in a procedure of performing data statistics analysis or the like.
In an embodiment of the present disclosure, step S102, that is, when the external data source is requested by the data access request, the database management system may further include the following steps of:
the database management system analyzes the SQL statement;
and when the SQL sentence comprises a data request for an external data source, the database management system calls the first user-defined interface to send an asynchronous data request to the external data source.
In this alternative implementation manner, the database management system may parse the SQL statement after the SQL statement is obtained, and if the parsing result finds that the SQL statement needs to access the external data source, the database management system may call the first user-defined interface to send an asynchronous data request to the external data source, instead of using the default access manner of the database, that is, directly send the data request to the external data source, and wait for the external data source to return the requested data for further processing. In this embodiment, after the database management system sends an asynchronous data request to the external data source by calling the first user-defined interface, the next process may be continued, instead of waiting for the external data source to return the requested data.
In an embodiment of the present disclosure, step S103, that is, the step of asynchronously obtaining, by the database management system, the request data returned by the external data source from the background storage medium by calling the second user-defined interface, may further include the following steps:
the database management system determines whether request data returned by the external data source is received;
and when the database management system receives the request data returned by the external data source, the second user-defined interface is called to asynchronously acquire the request data returned by the external data source from the background storage medium.
In this alternative implementation, the database management system may first determine whether the background thread has received the data returned by the external data source, and if so, may invoke the second user-defined interface to obtain the request data from the background storage medium, and further process the request data. In some embodiments, the background thread may send an acknowledgement to the database management system after receiving the request data returned by the external data source, so that the database management system can determine whether the request data has been received. In other embodiments, the background storage medium may store the request data after the background storage medium while recording the storage identifier, and the database management system may determine whether the request data returned by the external data source has been received by looking at whether there is a corresponding storage identifier.
In some embodiments, the database management system may return the requested data to the data requesting end. Of course, in some other embodiments, statistical analysis may be performed on the request data, which is specific to the actual application scenario, and is not limited herein. It will be appreciated that the database management system sends an asynchronous data request to the external data source by invoking the first user-defined interface and retrieves the requested data from the background storage medium by invoking the second user-defined interface, which is equivalent to retrieving the requested data from the external data source directly using the default interface in the database management system, except that the database management system performs other processing during the period from the sending of the asynchronous data request to the retrieval of the requested data from the background storage medium, while the database management system waits for the external data source to return the requested data, and does not perform other processing. In the mode of the embodiment of the disclosure, the data access efficiency can be remarkably improved under the scene of continuously requesting data from an external data source.
In an embodiment of the present disclosure, step S103, that is, the step of asynchronously obtaining, by the database management system, the request data returned by the external data source from the background storage medium by calling the second user-defined interface, may further include the following steps:
And the database management system periodically calls the second user-defined interface so as to periodically access the background storage medium, and when the request data exists in the background storage medium, the request data is acquired.
In this alternative implementation manner, the database management system may also periodically call the second user-defined interface, so that after the background thread stores the request data returned by the external data source into the background storage medium, the request data can be obtained from the background storage medium and further processed.
In an embodiment of the present disclosure, step S102, that is, when the external data source is requested by the data access request, the database management system may further include the following steps of:
the database management system analyzes the SQL statement;
and when the SQL sentence comprises a parameter command which is transmitted into the first user-defined interface, the database management system calls the first user-defined interface to send an asynchronous data request to the external data source.
In this alternative implementation manner, the user may directly request to obtain the request data from the external data source by calling the first user-defined interface when writing the SQL statement. That is, when the user writes the SQL sentence, the user-defined interface is directly used, after the database management system analyzes the SQL sentence, if the identification of the first user-defined interface is analyzed, the first user-defined interface is directly called and executed based on the parameter command in the SQL sentence, so that the asynchronous data request is sent to the external data source.
In an embodiment of the present disclosure, step S103, that is, the step of asynchronously obtaining, by the database management system, the request data returned by the external data source from the background storage medium by calling the second user-defined interface, may further include the following steps:
the database management system receives SQL sentences input by a user after the data access request;
the database management system analyzes the SQL statement;
and when the SQL sentence comprises a parameter command which is transmitted into the second user-defined interface, the database management system calls the second user-defined interface to asynchronously acquire the request data returned by the external data source from the background storage medium.
In this optional implementation manner, the user may write and call the second user-defined interface in the data access procedure, and acquire the SQL statement of the request data returned by the external data source from the background storage medium, that is, when the user writes the SQL statement corresponding to the data access procedure, may write, at the location where the external data source needs to be called, the parameter command that is transmitted to the first user-defined interface to send the parameter command of the asynchronous data request to the external data source, and then write, at the later time, the parameter command that is transmitted to the second user-defined interface to call the second user-defined interface to acquire, from the background storage medium, the parameter command of the request data returned by the external data source, so as to finally achieve the purpose of asynchronously acquiring the request data from the external data source by using the user-defined interface. It will be appreciated that the SQL statement that invokes the second user-defined interface may occur in pairs after the SQL statement that invokes the first user-defined interface. In addition, the appearance position of the SQL statement for calling the second user-defined interface can be determined based on actual needs, for example, the SQL statement for calling the first user-defined interface can be immediately after the SQL statement for calling the first user-defined interface, or can be at a position far away from the SQL statement for calling the first user-defined interface, but before the SQL statement for requesting data is used.
The technical terms and features related to the embodiment shown in fig. 2 are the same as or similar to those mentioned in the embodiment shown in fig. 1, and the explanation and description of the technical terms and features related to the embodiment shown in fig. 2 may refer to the explanation of the embodiment shown in fig. 1 and described herein, which is not repeated.
Fig. 3 shows an application flow diagram of a data access method according to an embodiment of the present disclosure. As shown in fig. 3, the data request end sends a data access request to the database management system, the database management system analyzes the data access request, and when the data access request relates to an access request of an external data source, invokes the first user-defined interface to send an asynchronous data request to the external data source, the external data source prepares the requested data after receiving the asynchronous data request, and returns the prepared request data to the background thread, and the background thread stores the request data to the background storage medium after monitoring the data sent by the external data source. The database management system also calls a second user-defined interface, acquires request data from the background storage medium, and returns the request data to the data request terminal.
The user also sends the first algorithm and the second algorithm to the database management system through the client so that the database management system registers the first algorithm and the second algorithm as a first user custom interface and a second user custom interface.
The following are device embodiments of the present disclosure that may be used to perform method embodiments of the present disclosure.
Fig. 4 shows a block diagram of a data access apparatus according to an embodiment of the present disclosure, which may be implemented as part or all of an electronic device by software, hardware, or a combination of both. As shown in fig. 4, the data access device includes:
a first acquisition module 401 configured to acquire a data access request;
a first calling module 402 configured to send an asynchronous data request to an external data source by calling a first user-defined interface when the data access request requests the external data source;
a second calling module 403, configured to asynchronously acquire the request data returned by the external data source from the background storage medium by calling a second user-defined interface; after the first user-defined interface sends an asynchronous data request to the external data source, a background thread receives the request data from the external data source and stores the request data to the background storage medium.
In an embodiment of the disclosure, the apparatus further comprises:
a first receiving module configured to receive a first algorithm input by a user to send an asynchronous data request to an external data source;
a first registration module configured to register the first algorithm as the first user-defined interface.
In an embodiment of the disclosure, the apparatus further comprises:
a second receiving module configured to receive a second algorithm for asynchronously acquiring request data from the background storage medium, which is input by a user;
and a second registration module configured to register the second algorithm as the second user-defined interface.
In one embodiment of the disclosure, the data access request includes an SQL statement.
In an embodiment of the disclosure, the first calling module includes:
a first parsing sub-module configured to parse the SQL statement;
and the first calling sub-module is configured to call the first user-defined interface to send an asynchronous data request to the external data source when the data request to the external data source is included in the SQL statement.
In an embodiment of the disclosure, the second calling module includes:
A first determination submodule configured to determine whether request data returned by the external data source has been received;
and the second calling sub-module is configured to call the second user-defined interface to asynchronously acquire the request data returned by the external data source from the background storage medium when the request data returned by the external data source is received.
In an embodiment of the disclosure, the second calling module includes:
and the third calling sub-module is configured to periodically call the second user-defined interface so as to periodically access the background storage medium, and acquire the request data when the request data exists in the background storage medium.
In an embodiment of the disclosure, the first calling module includes:
a second parsing sub-module configured to parse the SQL statement;
and the fourth calling sub-module is configured to call the first user-defined interface to send an asynchronous data request to the external data source when the parameter command input into the first user-defined interface is included in the SQL sentence.
In an embodiment of the disclosure, the second calling module includes:
The first receiving submodule is configured to receive SQL sentences input by a user after the data access request;
a second parsing sub-module configured to parse the SQL statement;
and the fifth calling sub-module is configured to call the second user-defined interface when the parameter command transmitted into the second user-defined interface is included in the SQL sentence, so as to asynchronously acquire the request data returned by the external data source from the background storage medium.
Fig. 5 shows a block diagram of a data access system according to an embodiment of the present disclosure, which may be implemented as part or all of an electronic device by software, hardware, or a combination of both. As shown in fig. 5, the data access system includes: a data request terminal 501 and a database management system 502;
the data request terminal 501 sends a data access request to the database management system 502;
when the data access request requests an external data source, the database management system 502 sends an asynchronous data request to the external data source by calling a first user-defined interface;
the database management system 502 asynchronously acquires the request data returned by the external data source from the background storage medium by calling a second user-defined interface; after the first user-defined interface sends an asynchronous data request to the external data source, a background thread receives the request data from the external data source and stores the request data to the background storage medium;
The database management system 502 returns the requested data to the data requesting end.
In an embodiment of the present disclosure, the system further comprises: an external data source; wherein,
and after the external data source receives the asynchronous data request, the request data is returned to the database management system.
In an embodiment of the present disclosure, the system further comprises: a client; wherein,
the client receives a first algorithm input by a user and used for sending an asynchronous data request to an external data source;
the client sends the first algorithm to the database management system;
the database management system registers the first algorithm as the first user-defined interface.
In an embodiment of the present disclosure, the system further comprises: a client; wherein,
the client receives a second algorithm input by a user and asynchronously acquiring request data from the background storage medium;
the client sends the second algorithm to the database management system;
the database management system registers the second algorithm as the second user-defined interface.
In one embodiment of the disclosure, the data access request includes an SQL statement.
In one embodiment of the disclosure, the database management system parses the SQL statement; and when the SQL sentence comprises a data request for an external data source, the database management system calls the first user-defined interface to send an asynchronous data request to the external data source.
In one embodiment of the present disclosure, the database management system determines whether the request data returned by the external data source has been received;
and when the database management system receives the request data returned by the external data source, the second user-defined interface is called to asynchronously acquire the request data returned by the external data source from the background storage medium.
In an embodiment of the present disclosure, the database management system periodically invokes the second user-defined interface, so as to periodically access the background storage medium, and when the request data exists in the background storage medium, acquire the request data.
In one embodiment of the disclosure, the database management system parses the SQL statement;
and when the SQL sentence comprises a parameter command which is transmitted into the first user-defined interface, the database management system calls the first user-defined interface to send an asynchronous data request to the external data source.
In one embodiment of the present disclosure, the database management system receives the SQL statement input by the user after the data access request;
the database management system analyzes the SQL statement;
and when the SQL sentence comprises a parameter command which is transmitted into the second user-defined interface, the database management system calls the second user-defined interface to asynchronously acquire the request data returned by the external data source from the background storage medium.
The technical features and the corresponding explanations and descriptions related to the above-mentioned apparatus embodiments are the same, corresponding or similar to the technical features and the corresponding explanations and descriptions related to the above-mentioned method embodiments, and reference may be made to the technical features and the corresponding explanations and descriptions related to the above-mentioned method embodiments for the technical features and the corresponding explanations and descriptions related to the above-mentioned apparatus embodiments, which are not repeated herein.
The embodiment of the disclosure also discloses an electronic device, which comprises a memory and a processor; wherein,
the memory is used to store one or more computer instructions that are executed by the processor to perform any of the method steps described above.
FIG. 6 is a schematic diagram of a computer system suitable for use in implementing a method according to an embodiment of the present disclosure.
As shown in fig. 6, the computer system 600 includes a processing unit 601, which can execute various processes in the above-described embodiments according to a program stored in a Read Only Memory (ROM) 602 or a program loaded from a storage section 608 into a Random Access Memory (RAM) 603. In the RAM603, various programs and data required for the operation of the computer system 600 are also stored. The processing unit 601, the ROM602, and the RAM603 are connected to each other through a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.
The following components are connected to the I/O interface 605: an input portion 606 including a keyboard, mouse, etc.; an output portion 607 including a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, a speaker, and the like; a storage section 608 including a hard disk and the like; and a communication section 609 including a network interface card such as a LAN card, a modem, or the like. The communication section 609 performs communication processing via a network such as the internet. The drive 610 is also connected to the I/O interface 605 as needed. Removable media 611 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is installed as needed on drive 610 so that a computer program read therefrom is installed as needed into storage section 608. The processing unit 601 may be implemented as a processing unit such as CPU, GPU, TPU, FPGA, NPU.
In particular, according to embodiments of the present disclosure, the methods described above may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program tangibly embodied on a machine-readable medium, the computer program comprising program code for performing the data transmission method. In such an embodiment, the computer program can be downloaded and installed from a network through the communication portion 609, and/or installed from the removable medium 611.
The disclosed embodiments also disclose a computer program product comprising a computer program/instructions which, when executed by a processor, implement any of the method steps described above.
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units or modules described in the embodiments of the present disclosure may be implemented by software, or may be implemented by hardware. The units or modules described may also be provided in a processor, the names of which in some cases do not constitute a limitation of the unit or module itself.
As another aspect, the embodiments of the present disclosure also provide a computer-readable storage medium, which may be a computer-readable storage medium included in the apparatus described in the above-described embodiment; or may be a computer-readable storage medium, alone, that is not assembled into a device. The computer-readable storage medium stores one or more programs for use by one or more processors in performing the methods described in the embodiments of the present disclosure.
The foregoing description is only of the preferred embodiments of the present disclosure and description of the principles of the technology being employed. It will be appreciated by those skilled in the art that the scope of the invention in the embodiments of the present disclosure is not limited to the specific combination of the above technical features, but encompasses other technical features formed by any combination of the above technical features or their equivalents without departing from the inventive concept. Such as the technical solution formed by mutually replacing the above-mentioned features and the technical features with similar functions (but not limited to) disclosed in the embodiments of the present disclosure.

Claims (15)

1. A data access method, comprising:
acquiring SQL sentences;
analyzing the SQL statement;
when the SQL sentence comprises a parameter command which is transmitted into a first user-defined interface, the first user-defined interface is called to send an asynchronous data request to an external data source;
receiving SQL sentences input by a user after the data access request;
analyzing the SQL statement;
when the SQL statement comprises a parameter command transmitted into a second user-defined interface, the second user-defined interface is called to asynchronously acquire request data returned by the external data source from a background storage medium; after the first user-defined interface sends an asynchronous data request to the external data source, a background thread receives the request data from the external data source and stores the request data to the background storage medium; the first user-defined interface and the second user-defined interface are obtained by registering an algorithm written by a user, and are not interfaces of a database management system; after the first user-defined interface and the second user-defined interface are successfully registered, the first user-defined interface and the second user-defined interface are used by writing SQL sentences.
2. The method of claim 1, wherein the method further comprises:
a first algorithm for receiving a request for asynchronous data sent to an external data source and input by a user;
registering the first algorithm as the first user-defined interface.
3. The method according to claim 1 or 2, wherein the method further comprises:
receiving a second algorithm input by a user and asynchronously acquiring request data from the background storage medium;
registering the second algorithm as the second user-defined interface.
4. The method of claim 1, wherein asynchronously obtaining the request data returned by the external data source from the background storage medium by invoking a second user-defined interface comprises:
determining whether request data returned by the external data source has been received;
and when the request data returned by the external data source is received, calling the second user-defined interface to asynchronously acquire the request data returned by the external data source from the background storage medium.
5. The method of claim 1, wherein asynchronously obtaining the request data returned by the external data source from the background storage medium by invoking a second user-defined interface comprises:
And periodically calling the second user-defined interface so as to periodically access the background storage medium, and acquiring the request data when the request data exists in the background storage medium.
6. A data access method, comprising:
the data request end sends SQL sentences to the database management system;
the database management system analyzes the SQL sentence, and when the SQL sentence comprises a parameter command which is transmitted into a first user-defined interface, the first user-defined interface is called to send an asynchronous data request to an external data source;
the database management system receives an SQL statement input by a user after the data access request, analyzes the SQL statement, and calls a second user-defined interface when the SQL statement comprises a parameter command transmitted into the second user-defined interface so as to asynchronously acquire request data returned by the external data source from a background storage medium; after the first user-defined interface sends an asynchronous data request to the external data source, a background thread receives the request data from the external data source and stores the request data to the background storage medium; the first user-defined interface and the second user-defined interface are obtained by registering an algorithm written by a user, and are not self-contained interfaces of the database management system; after the first user-defined interface and the second user-defined interface are successfully registered, the first user-defined interface and the second user-defined interface are used by writing SQL sentences;
And the database management system returns the request data to the data request terminal.
7. The method of claim 6, wherein the method further comprises:
and after the external data source receives the asynchronous data request, the request data is returned to the database management system.
8. The method according to claim 6 or 7, wherein the method further comprises:
the client receives a first algorithm input by a user and used for sending an asynchronous data request to an external data source;
the client sends the first algorithm to the database management system;
the database management system registers the first algorithm as the first user-defined interface.
9. The method according to claim 6 or 7, wherein the method further comprises:
the client receives a second algorithm input by a user and asynchronously acquiring request data from the background storage medium;
the client sends the second algorithm to the database management system;
the database management system registers the second algorithm as the second user-defined interface.
10. The method of claim 6, wherein the invoking the second user-defined interface to asynchronously obtain the request data returned by the external data source from the background storage medium comprises:
The database management system determines whether request data returned by the external data source is received;
and when the database management system receives the request data returned by the external data source, the second user-defined interface is called to asynchronously acquire the request data returned by the external data source from the background storage medium.
11. The method of claim 6, wherein the invoking the second user-defined interface to asynchronously obtain the request data returned by the external data source from the background storage medium comprises:
and the database management system periodically calls the second user-defined interface so as to periodically access the background storage medium, and when the request data exists in the background storage medium, the request data is acquired.
12. A data access apparatus comprising:
the first acquisition module is configured to acquire an SQL sentence and analyze the SQL sentence;
the first calling module is configured to call the first user-defined interface when the parameter command input into the first user-defined interface is included in the SQL sentence, so as to send an asynchronous data request to an external data source;
The second calling module is configured to receive an SQL statement input by a user after the data access request, analyze the SQL statement, and call the second user-defined interface when the SQL statement comprises a parameter command transmitted into the second user-defined interface so as to asynchronously acquire request data returned by the external data source from a background storage medium; after the first user-defined interface sends an asynchronous data request to the external data source, a background thread receives the request data from the external data source and stores the request data to the background storage medium; the first user-defined interface and the second user-defined interface are obtained by registering an algorithm written by a user, and are not interfaces of a database management system; after the first user-defined interface and the second user-defined interface are successfully registered, the first user-defined interface and the second user-defined interface are used by writing SQL sentences.
13. A data access system, comprising: a data request end and a database management system;
the data request end sends SQL sentences to a database management system;
The database management system analyzes the SQL sentence, and when the SQL sentence comprises a parameter command which is transmitted into a first user-defined interface, the first user-defined interface is called to send an asynchronous data request to an external data source;
the database management system receives an SQL statement input by a user after the data access request, analyzes the SQL statement, and calls a second user-defined interface when the SQL statement comprises a parameter command transmitted into the second user-defined interface so as to asynchronously acquire request data returned by the external data source from a background storage medium; after the first user-defined interface sends an asynchronous data request to the external data source, a background thread receives the request data from the external data source and stores the request data to the background storage medium; the first user-defined interface and the second user-defined interface are obtained by registering an algorithm written by a user, and are not self-contained interfaces of the database management system; after the first user-defined interface and the second user-defined interface are successfully registered, the first user-defined interface and the second user-defined interface are used by writing SQL sentences;
And the database management system returns the request data to the data request terminal.
14. An electronic device includes a memory and a processor; wherein,
the memory is for storing one or more computer instructions, wherein the one or more computer instructions are executable by the processor to implement the steps of the method of any one of claims 1-11.
15. A computer readable storage medium having stored thereon computer instructions, wherein the computer instructions, when executed by a processor, implement the steps of the method of any of claims 1-11.
CN202111064251.2A 2021-09-10 2021-09-10 Data access method, device, electronic equipment, storage medium and program product Active CN113742364B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111064251.2A CN113742364B (en) 2021-09-10 2021-09-10 Data access method, device, electronic equipment, storage medium and program product

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111064251.2A CN113742364B (en) 2021-09-10 2021-09-10 Data access method, device, electronic equipment, storage medium and program product

Publications (2)

Publication Number Publication Date
CN113742364A CN113742364A (en) 2021-12-03
CN113742364B true CN113742364B (en) 2023-12-26

Family

ID=78738104

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111064251.2A Active CN113742364B (en) 2021-09-10 2021-09-10 Data access method, device, electronic equipment, storage medium and program product

Country Status (1)

Country Link
CN (1) CN113742364B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114637777A (en) * 2022-02-17 2022-06-17 深圳市双合电气股份有限公司 Method and system for real-time exchange processing of foreground and background system data

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6023706A (en) * 1997-07-11 2000-02-08 International Business Machines Corporation Parallel file system and method for multiple node file access
US7523094B1 (en) * 2008-03-16 2009-04-21 International Business Machines Corporation Asynchronous task for energy cost aware database query optimization
CN106230795A (en) * 2016-07-22 2016-12-14 北京近颐科技有限公司 A kind of network safety system under mobile internet environment
CN111259047A (en) * 2018-12-03 2020-06-09 顺丰科技有限公司 Data loading method, device, equipment and storage medium thereof
CN111414391A (en) * 2020-03-25 2020-07-14 平安资产管理有限责任公司 Method and system for accessing multiple data sources
CN111858097A (en) * 2020-07-22 2020-10-30 安徽华典大数据科技有限公司 Distributed database system and database access method
CN112734349A (en) * 2019-10-14 2021-04-30 珠海格力电器股份有限公司 Interface generation method, data calling method, device and electronic equipment
CN112783954A (en) * 2019-11-06 2021-05-11 北京金山云网络技术有限公司 Data access method and device and server

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7503052B2 (en) * 2004-04-14 2009-03-10 Microsoft Corporation Asynchronous database API
GB201704710D0 (en) * 2017-03-24 2017-05-10 Pixit Media Ltd A data management system and method
US20210141920A1 (en) * 2019-11-08 2021-05-13 Okera, Inc. Dynamic view for implementing data access control policies

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6023706A (en) * 1997-07-11 2000-02-08 International Business Machines Corporation Parallel file system and method for multiple node file access
US7523094B1 (en) * 2008-03-16 2009-04-21 International Business Machines Corporation Asynchronous task for energy cost aware database query optimization
CN106230795A (en) * 2016-07-22 2016-12-14 北京近颐科技有限公司 A kind of network safety system under mobile internet environment
CN111259047A (en) * 2018-12-03 2020-06-09 顺丰科技有限公司 Data loading method, device, equipment and storage medium thereof
CN112734349A (en) * 2019-10-14 2021-04-30 珠海格力电器股份有限公司 Interface generation method, data calling method, device and electronic equipment
CN112783954A (en) * 2019-11-06 2021-05-11 北京金山云网络技术有限公司 Data access method and device and server
CN111414391A (en) * 2020-03-25 2020-07-14 平安资产管理有限责任公司 Method and system for accessing multiple data sources
CN111858097A (en) * 2020-07-22 2020-10-30 安徽华典大数据科技有限公司 Distributed database system and database access method

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
网格环境下的数据库访问和集成;石柯;;计算机工程(08);第66-68页 *

Also Published As

Publication number Publication date
CN113742364A (en) 2021-12-03

Similar Documents

Publication Publication Date Title
US7051034B1 (en) Dynamic optimization for processing a restartable sub-tree of a query execution plan
US8949220B2 (en) Techniques for managing XML data associated with multiple execution units
EP3816815A1 (en) Target data obtaining method and apparatus
US20170262531A1 (en) Data Visualization Method and Apparatus, and Database Server
US8185546B2 (en) Enhanced control to users to populate a cache in a database system
US8412721B2 (en) Efficient data extraction by a remote application
US20160034582A1 (en) Computing device and method for executing database operation command
CN111309760A (en) Data retrieval method, system, device and storage medium
CN111581234B (en) RAC multi-node database query method, device and system
US11263542B2 (en) Technologies for auto discover and connect to a rest interface
US11132363B2 (en) Distributed computing framework and distributed computing method
CN111666279B (en) Query data processing method and device, electronic equipment and computer storage medium
CN110929128A (en) Data crawling method, device, equipment and medium
CN113742364B (en) Data access method, device, electronic equipment, storage medium and program product
CN108363741B (en) Big data unified interface method, device, equipment and storage medium
CN111737564A (en) Information query method, device, equipment and medium
US6374257B1 (en) Method and system for removing ambiguities in a shared database command
CN111752920A (en) Method, system, and storage medium for managing metadata
CN113254519B (en) Access method, device, equipment and storage medium of multi-source heterogeneous database
US10866960B2 (en) Dynamic execution of ETL jobs without metadata repository
CN112487025A (en) Data query method and device, electronic equipment and storage medium
CN112579632A (en) Data verification method, device, equipment and medium
CN115905239A (en) Method and device for realizing highly-multiplexed performance index data retrieval
CN110806967A (en) Unit testing method and device
CN112069179A (en) Hbase query performance optimization-based device and method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant