CN113079046A

CN113079046A - Data access method and device, electronic equipment and medium

Info

Publication number: CN113079046A
Application number: CN202110328568.6A
Authority: CN
Inventors: 冯玉祥
Original assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Current assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Priority date: 2021-03-26
Filing date: 2021-03-26
Publication date: 2021-07-06

Abstract

The present disclosure provides a data access method, an apparatus, an electronic device, a computer-readable storage medium, and a computer program product, which relate to the field of data transmission technology, and in particular, to the field of data access. The implementation scheme is as follows: acquiring configuration information including information related to a data provider of data to be accessed and task information for accessing the data, wherein the configuration information and the task information are input by a user through a user interface; generating a corresponding task based on the configuration information and the task information; performing a task to access data from a data provider; determining target data corresponding to the accessed data in a data receiver according to a preset data mapping relation; and storing the accessed data and the corresponding target data in a correlation manner.

Description

Data access method and device, electronic equipment and medium

Technical Field

The present disclosure relates to the field of data transmission technologies, and in particular, to the field of data access, and in particular, to a data access method, a data access apparatus, an electronic device, and a computer-readable storage medium.

Background

Cloud computing (cloud computing) refers to a technology architecture that accesses a flexibly extensible shared physical or virtual resource pool through a network, where resources may include servers, operating systems, networks, software, applications, storage devices, and the like, and may be deployed and managed in an on-demand, self-service manner. Through the cloud computing technology, high-efficiency and strong data processing capacity can be provided for technical application and model training of artificial intelligence, block chains and the like.

With the advent of the big data age, companies and cooperation between companies are close, especially technical companies, and technical cooperation between both parties is indispensable to carry out interaction between data. Currently, data interfaces of partners are typically interfaced in accordance with the technical documentation of the data provider. During access, a whole set of docking system is re-developed, and an access layer, a service layer and a logic layer are not decoupled and pulled out. When a large amount of data is accessed, the tedious and repeated docking work can cause time and labor consumption of the data access work.

The approaches described in this section are not necessarily approaches that have been previously conceived or pursued. Unless otherwise indicated, it should not be assumed that any of the approaches described in this section qualify as prior art merely by virtue of their inclusion in this section. Similarly, unless otherwise indicated, the problems mentioned in this section should not be considered as having been acknowledged in any prior art.

Disclosure of Invention

The present disclosure provides a data access method, including: acquiring configuration information including information related to a data provider of data to be accessed and task information for accessing the data, wherein the configuration information and the task information are input by a user through a user interface; generating a corresponding task based on the configuration information and the task information; performing a task to access data from a data provider; determining target data corresponding to the accessed data in a data receiver according to a preset data mapping relation; and storing the accessed data and the corresponding target data in a correlation manner.

According to an aspect of the present disclosure, there is provided a data access apparatus, including: an acquisition unit configured to acquire configuration information including information on a data provider of data to be accessed and task information for accessing the data, wherein the configuration information and the task information are input by a user through a user interface; the task generating unit is configured to generate a corresponding task based on the configuration information and the task information; a data access unit configured to perform a task to access data from a data provider; the mapping unit is configured to determine target data corresponding to the accessed data in a data receiver according to a preset data mapping relation; and the storage unit is configured to store the accessed data and the corresponding target data in an associated manner.

According to another aspect of the present disclosure, there is provided an electronic device including: at least one processor; and a memory communicatively coupled to the at least one processor; the memory stores instructions executable by the at least one processor to cause the at least one processor to perform the methods described in the present disclosure.

According to another aspect of the present disclosure, there is provided a non-transitory computer readable storage medium storing computer instructions for causing a computer to perform the method described in the present disclosure.

According to another aspect of the disclosure, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the method described in the disclosure.

According to the data access method, the information of the data providers is configured on the user interface, a universal data access scheme can be achieved, a set of data access system does not need to be developed for each data provider, mapping storage of the access data and the target data can be automatically achieved, accordingly, code coupling is reduced, and access efficiency is improved.

It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present disclosure, nor do they limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate exemplary embodiments of the embodiments and, together with the description, serve to explain the exemplary implementations of the embodiments. The illustrated embodiments are for purposes of illustration only and do not limit the scope of the claims. Throughout the drawings, identical reference numbers designate similar, but not necessarily identical, elements.

FIG. 1 illustrates a schematic diagram of an exemplary system in which various methods described herein may be implemented, according to an embodiment of the present disclosure;

fig. 2 shows a flow diagram of a data access method according to an embodiment of the present disclosure;

FIG. 3 shows a schematic diagram of a user interface receiving configuration information and task information, according to an embodiment of the present disclosure;

FIG. 4 illustrates a flowchart of a method of performing a task to access data from a data provider, according to an embodiment of the disclosure;

fig. 5 shows a block diagram of a data access arrangement according to an embodiment of the present disclosure; and

FIG. 6 illustrates a block diagram of an exemplary electronic device that can be used to implement embodiments of the present disclosure.

Detailed Description

Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, in which various details of the embodiments of the disclosure are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

In the present disclosure, unless otherwise specified, the use of the terms "first", "second", etc. to describe various elements is not intended to limit the positional relationship, the timing relationship, or the importance relationship of the elements, and such terms are used only to distinguish one element from another. In some examples, a first element and a second element may refer to the same instance of the element, and in some cases, based on the context, they may also refer to different instances.

The terminology used in the description of the various described examples in this disclosure is for the purpose of describing particular examples only and is not intended to be limiting. Unless the context clearly indicates otherwise, if the number of elements is not specifically limited, the elements may be one or more. Furthermore, the term "and/or" as used in this disclosure is intended to encompass any and all possible combinations of the listed items.

Embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

Fig. 1 illustrates a schematic diagram of an exemplary system 100 in which various methods and apparatus described herein may be implemented in accordance with embodiments of the present disclosure. Referring to fig. 1, the system 100 includes one or

more client devices

101, 102, 103, 104, 105, and 106, a server 120, and one or more communication networks 110 coupling the one or more client devices to the server 120.

Client devices

101, 102, 103, 104, 105, and 106 may be configured to execute one or more applications.

In embodiments of the present disclosure, the server 120 may run one or more services or software applications that enable the execution of data access methods.

In some embodiments, the server 120 may also provide other services or software applications that may include non-virtual environments and virtual environments. In certain embodiments, these services may be provided as web-based services or cloud services, for example, provided to users of

client devices

101, 102, 103, 104, 105, and/or 106 under a software as a service (SaaS) model.

In the configuration shown in fig. 1, server 120 may include one or more components that implement the functions performed by server 120. These components may include software components, hardware components, or a combination thereof, which may be executed by one or more processors. A user operating a

client device

101, 102, 103, 104, 105, and/or 106 may, in turn, utilize one or more client applications to interact with the server 120 to take advantage of the services provided by these components. It should be understood that a variety of different system configurations are possible, which may differ from system 100. Accordingly, fig. 1 is one example of a system for implementing the various methods described herein and is not intended to be limiting.

Client devices

101, 102, 103, 104, 105, and/or 106 may be used to provide input interfaces and the like to a user. The client device may provide an interface that enables a user of the client device to interact with the client device. The client device may also output information to the user via the interface. Although fig. 1 depicts only six client devices, those skilled in the art will appreciate that any number of client devices may be supported by the present disclosure.

Client devices

101, 102, 103, 104, 105, and/or 106 may include various types of computer devices, such as portable handheld devices, general purpose computers (such as personal computers and laptop computers), workstation computers, wearable devices, gaming systems, thin clients, various messaging devices, sensors or other sensing devices, and so forth. These computer devices may run various types and versions of software applications and operating systems, such as Microsoft Windows, Apple iOS, UNIX-like operating systems, Linux, or Linux-like operating systems (e.g., Google Chrome OS); or include various Mobile operating systems, such as Microsoft Windows Mobile OS, iOS, Windows Phone, Android. Portable handheld devices may include cellular telephones, smart phones, tablets, Personal Digital Assistants (PDAs), and the like. Wearable devices may include head mounted displays and other devices. The gaming system may include a variety of handheld gaming devices, internet-enabled gaming devices, and the like. The client device is capable of executing a variety of different applications, such as various Internet-related applications, communication applications (e.g., email applications), Short Message Service (SMS) applications, and may use a variety of communication protocols.

Network 110 may be any type of network known to those skilled in the art that may support data communications using any of a variety of available protocols, including but not limited to TCP/IP, SNA, IPX, etc. By way of example only, one or more networks 110 may be a Local Area Network (LAN), an ethernet-based network, a token ring, a Wide Area Network (WAN), the internet, a virtual network, a Virtual Private Network (VPN), an intranet, an extranet, a Public Switched Telephone Network (PSTN), an infrared network, a wireless network (e.g., bluetooth, WIFI), and/or any combination of these and/or other networks.

The server 120 may include one or more general purpose computers, special purpose server computers (e.g., PC (personal computer) servers, UNIX servers, mid-end servers), blade servers, mainframe computers, server clusters, or any other suitable arrangement and/or combination. The server 120 may include one or more virtual machines running a virtual operating system, or other computing architecture involving virtualization (e.g., one or more flexible pools of logical storage that may be virtualized to maintain virtual storage for the server). In various embodiments, the server 120 may run one or more services or software applications that provide the functionality described below.

The computing units in server 120 may run one or more operating systems including any of the operating systems described above, as well as any commercially available server operating systems. The server 120 may also run any of a variety of additional server applications and/or middle tier applications, including HTTP servers, FTP servers, CGI servers, JAVA servers, database servers, and the like.

In some implementations, the server 120 may include one or more applications to analyze and consolidate data feeds and/or event updates received from users of the

client devices

101, 102, 103, 104, 105, and 106. Server 120 may also include one or more applications to display data feeds and/or real-time events via one or more display devices of

client devices

101, 102, 103, 104, 105, and 106.

In some embodiments, the server 120 may be a server of a distributed system, or a server incorporating a blockchain. The server 120 may also be a cloud server, or a smart cloud computing server or a smart cloud host with artificial intelligence technology. The cloud Server is a host product in a cloud computing service system, and is used for solving the defects of high management difficulty and weak service expansibility in the traditional physical host and Virtual Private Server (VPS) service.

The system 100 may also include one or more databases 130. In some embodiments, these databases may be used to store data and other information. The data store 130 may reside in various locations. For example, the data store used by the server 120 may be local to the server 120, or may be remote from the server 120 and may communicate with the server 120 via a network-based or dedicated connection. The data store 130 may be of different types. In certain embodiments, the data store used by the server 120 may be a database, such as a relational database. One or more of these databases may store, update, and retrieve data to and from the database in response to the command.

In some embodiments, one or more of the databases 130 may also be used by applications to store application data. The databases used by the application may be different types of databases, such as key-value stores, object stores, or conventional stores supported by a file system.

The system 100 of fig. 1 may be configured and operated in various ways to enable application of the various methods and apparatus described in accordance with the present disclosure.

At present, an access layer, a service layer and a logic layer are not decoupled and extracted in a data access system, a whole set of docking systems need to be developed respectively aiming at different cooperative data providers, and the complex and repeated docking work can cause time and labor consumption of data access work and overhigh personnel input cost.

Therefore, it would be necessary to develop a universal, scalable method of data access to allow decoupling of the business code from the logic code to allow fast and stable access to data from the data provider.

There is therefore provided, in accordance with an exemplary disclosed embodiment, a data access method 200, which, as shown in fig. 2, may include: acquiring configuration information including information related to a data provider of data to be accessed and task information for accessing the data, wherein the configuration information and the task information are input by a user through a user interface; step S203, generating a corresponding task based on the configuration information and the task information; step S204, executing a task to access data from a data provider; step S205, determining target data corresponding to the accessed data in a data receiver according to a preset data mapping relation; and step S206, the accessed data and the corresponding target data are stored in a correlation mode.

Therefore, by configuring the information of the data providers on the user interface, a universal data access scheme can be realized, a set of data access system does not need to be developed for each data provider, and mapping storage of the access data and target data (data receivers) can be automatically realized, so that the coupling of codes is reduced, and the access efficiency is improved.

In some embodiments, it may further include: a user interface is provided to receive user input to obtain configuration information for information relating to a data provider for data to be accessed and task information for accessing the data via user input on the user interface. The number of data providers may be one or more, i.e. the data in one or more data providers may be conveniently accessed by configuring the information of the data providers in the user interface.

In some embodiments, the configuration information may include at least one of: identification of the data provider, name of the data provider, and link address of the data provider. Thus, metadata of the corresponding data provider is obtained and stored, so that the data access operation is facilitated.

FIG. 3 shows a schematic diagram of a user interface receiving configuration information and task information, according to an embodiment of the disclosure. As shown in fig. 3, the partner identification is an identification of the data provider, the partner name is a name of the data provider, and the partner connection is a link address of the data provider. By configuring the information, the corresponding data provider can be uniquely determined to access the data. Still further, it is also possible to select a specific one or more data providers among the plurality of data providers.

In some embodiments, the data provider identification may be, but is not limited to, a latin letter as the identification, facilitating later expansion of the use of this field for index queries.

In some embodiments, the metadata of the data provider may support Multiple Character sets, wherein the Multiple Character sets may include, but are not limited to, American Standard Code for Information Interchange (ASCII), Universal Multi-eight Code Character Set (Unicode) including, but not limited to, UTF-8 (8-bit, 8-bit Unicode Transformation Format), UTF-16 (16-bit, 16-bit Unicode Transformation Format), and the like.

In some embodiments, as shown in fig. 3, the configuration information further includes parameter information for data transmission, wherein the parameter information includes at least one of: JSON (JavaScript Object Notation) format, XML (eXtensible Markup Language) format, and RSS (Simple Syndication) format. Therefore, the interface supports multiple data formats to realize data interaction, and the expansibility of the interface is enhanced.

In some examples, as shown in fig. 3, links for multiple data providers may be configured in the user interface. Or in some examples each data provider may also be configured with multiple data links to retrieve data from different memory areas of that data provider. The data link may include, but is not limited to, a URL (Uniform Resource Locator).

In some embodiments, as shown in FIG. 3, task information for accessing data may also be configured on the user interface. The task information may include: execution period, number of retries, and retry interval. Optionally, the task information may also include a task name. The task name may be, but is not limited to, in latin letters to facilitate extended use of this field for index queries. Based on the configuration information about the data provider information and the task information, a corresponding task can be generated to capture and convert the partner data through the task.

Fig. 4 illustrates a flowchart of a method of performing a task to access data from a data provider, according to an embodiment of the disclosure. In this embodiment, as shown in fig. 4, the performing of the task to access the data from the data provider at step S204 may include: step S401, executing tasks according to the execution cycle to access data from a data provider; step S402, responding to the situation that the task does not acquire correct data from a data provider during execution, and re-executing the task according to the retry times and the retry interval number; step S403, and generating alarm information in response to the number of times of re-executing the task reaching the number of retries. Therefore, the task retry can be carried out when the task fails to acquire data by setting the task retry times so as to eliminate accidental faults; and can give an alarm in time after the task retry is finished so as to manually remove the fault.

For example, the number of retries may be set to r times, and the retry interval may be set to n seconds; when a task has an error (for example, data cannot be acquired), the service is automatically retried every n seconds, and when the number of retries of the task operation reaches r times, the error indicating the task needs to be manually checked. Meanwhile, excessive dirty data is prevented from being stored in a database, so that the service is influenced.

In some examples, as shown in connection with fig. 3, the task information may also include a scheduling period, i.e., an execution period of the task. The scheduling period may include, but is not limited to, a basic unit of every minute or an hour. For example, the scheduling period may be set to an optional form to be selected by the user from a preset set of optional periods according to the user's needs to achieve flexible configuration of the data access operation.

In some embodiments, as shown in fig. 3, the error alarm modes may also be configured through the user interface, and may include, but are not limited to, sms, email, and database records. Therefore, the method and the device can remind the user of paying attention to the error in the data access process in real time, further ensure the data access efficiency and improve the data access accuracy.

Due to the problems of inconsistent naming rules of data storage of each platform or differences in data classification and storage, after data of a data provider is accessed, the accessed data needs to be converted according to a preset data mapping relation so as to realize the associated storage with target data of the platform.

According to some embodiments, the preset data mapping relationship is input by the user through the user interface based on the field name of the data to be accessed and the field name of the target data. Therefore, the method can be configured according to the requirement, and the universality of the data access method is further improved.

Illustratively, the data mapping may be implemented based on field names of the data. For example, the mapping relationship between the field names of the accessed data and the field names of the target data in the data receiver can be configured through the user interface. To implement associative storage based on the mapping relationship. Of course, the mapping relationship between the data is preset based on other attributes, including but not limited to time, user information, and the like, and is not limited herein. According to some embodiments, before the step S206, storing the accessed data in association with the corresponding target data, the method may further include: and cleaning the accessed data according to a preset rule. The preset rules are input by the user through the user interface. The data processing means is used for filtering and sorting the accessed data.

Optionally, filtering and sorting the accessed data may include, but is not limited to, the following:

where the values of certain columns are filtered.

Regex regular expression filtering is performed on the specified columns.

Replace the values of the specified columns, e.g., replace all ANDROID with ANDROID.

Regex _ replace regular expression replacement.

In indicates belonging, e.g., to the specified os listed in (other, symbian).

A not _ in can mean not, as opposed to an in, as a given os column is not in (other, symbian).

Kstrip, removing a certain character string.

In some embodiments, data is re-accessed from a data provider in response to a failure in applying the stored access data. Therefore, intelligent data error correction and data backtracking are realized.

Alternatively, when an error occurs during data consumption, the data may be retrieved from the corresponding data provider based on the attribute information of the data. Alternatively, the error data may be periodically sent to the data provider, and the error may be reported to the data provider to help the data provider repair the data. In some examples, the error data may be identified by way of a set tag to prevent it from being invoked by other applications. Alternatively, other attribute information of the error data may be acquired based on the tag to further retrieve the data from the corresponding data provider based on the attribute information.

Fig. 5 shows a block diagram of a data access device 500 according to an embodiment of the present disclosure. As shown in fig. 5, the data access apparatus may include: an obtaining unit 501 configured to obtain configuration information including information on a data provider of data to be accessed and task information for accessing the data, wherein the configuration information and the task information are input by a user through a user interface; a task generating unit 502 configured to generate a corresponding task based on the configuration information and the task information; an access unit 503 configured to perform a task to access data from a data provider; a mapping unit 504 configured to determine, according to a preset data mapping relationship, target data corresponding to the accessed data in a data receiver; and a storage unit 505 configured to store the accessed data in association with the corresponding target data.

In some embodiments, the data access device may further include an input unit 506 configured to provide a user interface to receive user input. The obtaining unit 501 may thus obtain the configuration information and the task information through user input on the user interface.

Here, the operations of the above units 501 to 505 of the data access apparatus 500 are similar to the operations of the steps 201 to 205 described above, and are not described herein again.

There is also provided, in accordance with an exemplary embodiment of the present disclosure, an electronic device, including: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the data access method described above.

There is also provided, in accordance with an exemplary embodiment of the present disclosure, a non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to execute the above-described data access method.

There is also provided, in accordance with an exemplary embodiment of the present disclosure, a computer program product, comprising a computer program, wherein the computer program, when executed by a processor, implements the above-mentioned data access method.

Referring to fig. 6, a block diagram of a structure of an electronic device 600, which may be a server or a client of the present disclosure, which is an example of a hardware device that may be applied to aspects of the present disclosure, will now be described. Electronic device is intended to represent various forms of digital electronic computer devices, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the disclosure described and/or claimed herein.

As shown in fig. 6, the apparatus 600 includes a computing unit 601, which can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM)602 or a computer program loaded from a storage unit 608 into a Random Access Memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the device 600 can also be stored. The calculation unit 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.

A number of components in the device 600 are connected to the I/O interface 605, including: an input unit 606, an output unit 607, a storage unit 608, and a communication unit 609. The input unit 606 may be any type of device capable of inputting information to the device 600, and the input unit 606 may receive input numeric or character information and generate key signal inputs related to user settings and/or function controls of the electronic device, and may include, but is not limited to, a mouse, a keyboard, a touch screen, a track pad, a track ball, a joystick, a microphone, and/or a remote control. Output unit 607 may be any type of device capable of presenting information and may include, but is not limited to, a display, speakers, a video/audio output terminal, a vibrator, and/or a printer. The storage unit 608 may include, but is not limited to, a magnetic disk, an optical disk. The communication unit 609 allows the device 600 to exchange information/data with other devices via a computer network, such as the internet, and/or various telecommunications networks, and may include, but is not limited to, a modem, a network card, an infrared communication device, a wireless communication transceiver, and/or a chipset, e.g., a bluetooth (TM) device, an 1302.11 device, a WiFi device, a WiMax device, a cellular communication device, and/or the like.

The computing unit 601 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various dedicated Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, and so forth. The computing unit 601 performs the various methods and processes described above, such as the data access method. For example, in some embodiments, the data access method may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as storage unit 608. In some embodiments, part or all of the computer program may be loaded and/or installed onto the device 600 via the ROM 602 and/or the communication unit 609. When the computer program is loaded into RAM 603 and executed by the computing unit 601, one or more steps of the data access method described above may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to perform the data access method by any other suitable means (e.g., by means of firmware).

Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), system on a chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.

Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions/operations specified in the flowchart and/or block diagram to be performed. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), and the Internet.

The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.

It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present disclosure may be performed in parallel, sequentially or in different orders, and are not limited herein as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved.

Although embodiments or examples of the present disclosure have been described with reference to the accompanying drawings, it is to be understood that the above-described methods, systems and apparatus are merely exemplary embodiments or examples and that the scope of the present invention is not limited by these embodiments or examples, but only by the claims as issued and their equivalents. Various elements in the embodiments or examples may be omitted or may be replaced with equivalents thereof. Further, the steps may be performed in an order different from that described in the present disclosure. Further, various elements in the embodiments or examples may be combined in various ways. It is important that as technology evolves, many of the elements described herein may be replaced with equivalent elements that appear after the present disclosure.

Claims

1. A data access method, comprising:

acquiring configuration information including information related to a data provider of data to be accessed and task information for accessing the data, wherein the configuration information and the task information are input by a user through a user interface;

generating a corresponding task based on the configuration information and the task information;

performing the task to access data from the data provider;

determining target data corresponding to the accessed data in a data receiver according to a preset data mapping relation; and

and storing the accessed data and the corresponding target data in a correlation manner.

2. The method of claim 1, wherein the task information comprises: an execution period, a number of retries, and a retry interval, and executing the task to access data from the data provider comprises:

executing the task to access data from the data provider according to the execution period;

in response to the task not acquiring correct data from the data provider during execution, re-executing the task according to the retry times and the retry interval number; and

generating alarm information in response to the number of times the task is re-executed reaching the number of retries.

3. The method of claim 1, wherein the preset data mapping relationship is input by a user through the user interface based on a field name of the data to be accessed and a field name of the target data.

4. The method of any one of claims 1-3, wherein prior to storing the accessed data in association with the corresponding target data, further comprising:

the accessed data is cleaned according to a preset rule,

wherein the preset rule is input by the user through the user interface.

5. The method of claim 1, further comprising: in response to a failure in applying the stored access data, re-accessing the data from the data provider.

6. The method of claim 1, wherein the configuration information comprises at least one of: an identification of the data provider, a name of the data provider, and a link address of the data provider.

7. The method of claim 6, wherein the configuration information further comprises parameter information for data transmission, wherein the parameter information comprises at least one of: JSON format, XML format, and RSS format.

8. A data access apparatus, comprising:

an acquisition unit configured to acquire configuration information including information on a data provider of data to be accessed and task information for accessing the data, wherein the configuration information and the task information are input by a user through a user interface;

the task generating unit is configured to generate a corresponding task based on the configuration information and the task information;

a data access unit configured to perform the task to access data from the data provider;

the mapping unit is configured to determine target data corresponding to the accessed data in a data receiver according to a preset data mapping relation; and

and the storage unit is configured to store the accessed data and the corresponding target data in an associated manner.

9. The apparatus of claim 8, wherein the task information comprises: an execution period, a number of retries, and a retry interval, and the access unit includes:

means for executing the task to access data from the data provider according to the execution period;

a unit that re-executes the task according to the retry number and the retry interval number in response to the task not acquiring correct data from the data provider at the time of execution; and

and a unit generating alarm information in response to the number of times of re-executing the task reaching the number of retries.

10. The apparatus of claim 8, wherein the preset data mapping relationship is input by a user through the user interface based on a field name of data to be accessed and a field name of the target data.

11. The apparatus of any one of claims 8-10, further comprising:

a unit for washing the accessed data according to a preset rule,

wherein the preset rule is input by the user through the user interface.

12. The apparatus of claim 8, further comprising:

in response to a failure in applying the stored access data, re-accessing the unit of data from the data provider.

13. The apparatus of claim 8, wherein the configuration information comprises at least one of: an identification of the data provider, a name of the data provider, and a link address of the data provider.

14. The apparatus of claim 13, wherein the configuration information further comprises parameter information for data transmission, wherein the parameter information comprises at least one of: JSON format, XML format, and RSS format.

15. An electronic device, comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein

The memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.

16. A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of any one of claims 1-7.

17. A computer program product comprising a computer program, wherein the computer program realizes the method of any one of claims 1-7 when executed by a processor.