CN111343269B - Data downloading method, device, computer equipment and storage medium - Google Patents
Data downloading method, device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN111343269B CN111343269B CN202010118332.5A CN202010118332A CN111343269B CN 111343269 B CN111343269 B CN 111343269B CN 202010118332 A CN202010118332 A CN 202010118332A CN 111343269 B CN111343269 B CN 111343269B
- Authority
- CN
- China
- Prior art keywords
- downloading
- task
- data
- target
- real
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/561—Adding application-functional data or data for application control, e.g. adding metadata
Abstract
The application discloses a data downloading method, a device, computer equipment and a storage medium, wherein the method comprises the following steps: receiving a downloading request, and acquiring a target task identifier carried in the downloading request; acquiring target task metadata from a pre-configured download task data table according to the target task identification; generating a corresponding downloading task according to the target task metadata and the downloading request; and executing the downloading task at fixed time, acquiring target data from the search engine, and importing the target data into a file in a preset format for downloading, wherein the data in the search engine comprises real-time data acquired from a service system. The application solves the problem of large data volume downloading, solves the timeliness of data, supports downloading after online real-time dynamic query, has high flexibility and strong expansibility, remarkably improves the efficiency of real-time downloading, improves the experience of users, shortens the development period and improves the development efficiency.
Description
Technical Field
The present application relates to the field of data processing technologies, and in particular, to a data downloading method, apparatus, computer device, and storage medium.
Background
In the actual logistics operation process, the logistics operators need to check the logistics orders of each logistics center or website operation in real time, and the data volume of the orders operated every day is usually relatively large, so that the sorting inquiry is inconvenient. Therefore, it is preferable to download the details of the order information into a form such as excel in real time, so that the filtering, viewing and analysis can be performed easily according to various conditions.
With the increase of real-time service data, the demand of real-time downloading is increasing. Due to the large number of people to download, the large amount of data to download, millions of individual downloads, and the like. The traditional synchronous queuing downloading mode has the problems of long waiting time, background memory overflow, system downtime and the like caused by high concurrency. In addition, the real-time downloading of different service demands is realized with similar functions, and the repeated coding causes low development efficiency.
The real-time data downloading methods commonly used in the industry currently include the following two methods,
1. synchronous download directly from relational database (oracle, mysql, postgresql): the method is most used in industry, and the data is most convenient to directly download because of real time. However, when the data volume is large and the downloading is concurrent, the waiting time is long, the load pressure on the database is large, and the system downtime can be influenced under severe conditions;
2. the background downloads files into ftp at fixed time, and the foreground downloads files in real time: the method can solve the problem of queuing under the condition of large data volume and large download concurrency. However, the timeliness of the data is not high, the flexibility is not enough, and the downloading scene after the real-time query is not supported.
In addition, the two modes are personalized development according to specific service requirements, the functions are similar, the repeated coding causes low development efficiency and high maintenance cost.
Thus, the method is applicable to a variety of applications. A new data downloading method has been continuously proposed, which has solved the above-mentioned problems.
Disclosure of Invention
In order to solve the problems of the prior art, the embodiment of the application provides a data downloading method, which aims to solve the problems of long waiting time after downloading and untimely data updating and the like existing in the prior art in the fast query and downloading of real-time data under the big data environment.
In order to solve one or more of the technical problems, the application adopts the following technical scheme:
in a first aspect, a data downloading method is provided, the method comprising the steps of:
receiving a downloading request, and acquiring a target task identifier carried in the downloading request;
acquiring target task metadata from a pre-configured download task data table according to the target task identifier;
generating a corresponding downloading task according to the target task metadata and the downloading request;
and executing the downloading task at fixed time, acquiring target data from a search engine, and importing the target data into a file in a preset format for downloading, wherein the data in the search engine comprises real-time data acquired from a service system.
Further, the method further comprises:
and after generating a corresponding download task according to the target task metadata and the download request, storing the download task into a pre-constructed download task instance table.
Further, the method further comprises:
and analyzing the downloading task in the downloading task instance table to obtain an analysis result.
Further, the method also comprises a data writing process of the search engine, which comprises the following steps:
the real-time data acquisition system sends the real-time service data acquired by the service system to a message queue, and writes the real-time service data into the search engine through the message queue; and/or the number of the groups of groups,
and the offline drawing number acquisition system sends the offline service data extracted from the database of the service system to a hive, and the hive writes the offline service data into the search engine.
In a second aspect, there is provided a data downloading apparatus, the apparatus comprising:
the device comprises:
the identification acquisition module is used for receiving the downloading request and acquiring a target task identification carried in the downloading request;
the data matching module is used for acquiring target task metadata from a pre-configured download task data table according to the target task identification;
the task generating module is used for generating a corresponding downloading task according to the target task metadata and the downloading request;
the data acquisition module is used for executing the downloading task at fixed time, acquiring target data from a search engine and importing the target data into a file in a preset format for downloading, wherein the data in the search engine comprises real-time data acquired from a service system.
Further, the device further comprises:
and the task storage module is used for storing the download task into a pre-constructed download task instance table after generating the corresponding download task according to the target task metadata and the download request.
Further, the device further comprises:
and the task analysis module is used for analyzing the downloading task in the downloading task instance table and obtaining an analysis result.
Further, the device further comprises a data writing module, and the data writing module comprises:
the first writing-in unit is used for sending the real-time service data acquired from the service system to a message queue by the real-time data acquisition system and writing the real-time service data into the search engine through the message queue;
the second writing unit is used for sending the offline business data extracted from the database of the business system to the hive by the offline lottery collecting system, and writing the offline business data into the search engine by the hive.
In a third aspect, there is provided a computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of:
receiving a downloading request, and acquiring a target task identifier carried in the downloading request;
acquiring target task metadata from a pre-configured download task data table according to the target task identifier;
generating a corresponding downloading task according to the target task metadata and the downloading request;
and executing the downloading task at fixed time, acquiring target data from a search engine, and importing the target data into a file in a preset format for downloading, wherein the data in the search engine comprises real-time data acquired from a service system.
In a fourth aspect, there is provided a computer readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of:
receiving a downloading request, and acquiring a target task identifier carried in the downloading request;
acquiring target task metadata from a pre-configured download task data table according to the target task identifier;
generating a corresponding downloading task according to the target task metadata and the downloading request;
and executing the downloading task at fixed time, acquiring target data from a search engine, and importing the target data into a file in a preset format for downloading, wherein the data in the search engine comprises real-time data acquired from a service system.
The technical scheme provided by the embodiment of the application has the beneficial effects that:
1. according to the data downloading method, device, computer equipment and storage medium provided by the embodiment of the application, the target task identification carried in the downloading request is obtained by receiving the downloading request, the target task metadata is obtained from the pre-configured downloading task data table according to the target task identification, the corresponding downloading task is generated according to the target task metadata and the downloading request, the downloading task is executed at regular time, the target data is obtained from the search engine, and is imported into the file in the preset format for downloading, wherein the data in the search engine comprises real-time data obtained from the service system, timeliness of the data is solved while the downloading of a large amount of data is solved, and the downloading is supported after the online real-time dynamic query, and the flexibility is high and the expansibility is strong;
2. according to the data downloading method, the device, the computer equipment and the storage medium, the target task identification carried in the downloading request is obtained by receiving the downloading request, the target task metadata is obtained from the pre-configured downloading task data table according to the target task identification, the service requirements with higher similarity are abstracted into the same data model, the research and development period of the service requirements with higher similarity is greatly shortened, the service change requirements can be responded more rapidly, the research and development efficiency is improved, and meanwhile, the user experience of operators is improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present application, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a technical architecture diagram illustrating a data download method according to an exemplary embodiment;
FIG. 2 is a flow chart illustrating a method of data download according to an exemplary embodiment;
FIG. 3 is a schematic diagram of a data download device according to an exemplary embodiment;
fig. 4 is a schematic diagram showing an internal structure of a computer device according to an exemplary embodiment.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the present application more apparent, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application, and it is apparent that the described embodiments are only some embodiments of the present application, not all embodiments of the present application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
Fig. 1 is a technical architecture diagram of a data downloading method according to an exemplary embodiment, and referring to fig. 1, the data downloading method provided by the embodiment of the present application aims to implement rapid query and download of real-time data under the condition of large data, and solve the pain problems of long waiting time and untimely data update after user downloading. Meanwhile, by utilizing a self-defined configuration scheme, the research and development period of real-time downloading requirements is shortened, the innovative design of real-time asynchronous downloading is used, the requirement realization period of a business party is greatly shortened, the working efficiency of developers is improved, the user experience of logistics operators is improved, and the stability, the robustness and the expansibility of a system are improved.
With further reference to fig. 1, the application creatively proposes that the service requirement with higher similarity is abstracted into the same data model, the real-time asynchronous downloading function is designed into two tables, one is a downloading task table, the other is a downloading task instance table, the downloading task table is used for storing preconfigured task metadata, the downloading task instance table is used for storing the generated downloading task, after receiving the downloading request, the target task identifier carried in the downloading request is obtained, the corresponding target task metadata is obtained from the downloading task data table according to the target task identifier, and then the corresponding downloading task is generated according to the target task metadata and the downloading request and is stored in the downloading task instance table, so that the development period of the downloading requirement is greatly shortened, and the working efficiency of developers is improved.
Fig. 2 is a flow chart illustrating a data downloading method according to an exemplary embodiment, and referring to fig. 2, the method includes:
s1: and receiving a downloading request, and acquiring a target task identifier carried in the downloading request.
Specifically, in the embodiment of the present application, a download request triggering module may be set on the real-time data query interface, for example, a "export" button may be set on the real-time data query interface. When the service personnel real-time data query interface queries real-time service data, if the queried data needs to be downloaded, the 'export' button can be clicked, and then a downloading request can be automatically sent.
In specific implementation, the method can be configured that when a downloading request is sent, at least the target task identifier is carried in the downloading request. The target identification information may be any identification information of the downloaded task configured in advance by the developer, for example, information such as a downloaded task ID. In this embodiment of the present application, the target task identifier is not specifically limited, and a developer may set the target task identifier according to the actual situation.
S2: and acquiring target task metadata from a pre-configured download task data table according to the target task identification.
Specifically, in the embodiment of the application, in order to shorten the development period of the service requirements, the service requirements with higher similarity are abstracted into the same data model. In specific implementation, a developer may actively sort out the real-time downloading service requirement in advance, or receive the real-time downloading service requirement sent by the front-end service personnel, and configure task metadata of a corresponding downloading task according to the real-time downloading service requirement, where the task metadata includes, but is not limited to, information such as a downloaded task ID, a downloaded service name, a downloaded displayed file name, downloaded field information, a downloaded es index table, a downloaded file path, and the like.
After the target task identifier carried in the downloading request is obtained, inquiring is carried out in the downloading task data table according to the target task identifier, and after the corresponding identifier is inquired, task metadata corresponding to the identifier in the downloading task data table is obtained and used as target task metadata.
S3: and generating a corresponding downloading task according to the target task metadata and the downloading request.
Specifically, after receiving the download request, the related information required for downloading carried in the download request needs to be obtained, including information such as a downloaded service data table, a downloaded name, a downloaded query condition, a downloaded concurrency condition and the like. And then generating a corresponding downloading task according to the relevant information required by the downloading and the target task metadata acquired in the steps. In this embodiment of the present application, configuration of relevant parameters for the download task is supported, for example, information such as the concurrency number of the download, the specific task of the download, and the threshold value of the size task may be configured, which is not listed here.
S4: and executing the downloading task at fixed time, acquiring target data from a search engine, and importing the target data into a file in a preset format for downloading, wherein the data in the search engine comprises real-time data acquired from a service system.
Specifically, in the embodiment of the present application, a spark streaming timing task is pre-configured, then the downloading task generated by the above steps is regularly pulled and executed by using the spark streaming task, after calculation and analysis, target data corresponding to the downloading task is obtained from a search engine, and then the target data is imported into a file in a preset format, for example, an FTP file, etc., so that service personnel (i.e., a foreground user) can download the target data subsequently. In this embodiment of the present application, the specific number of spark streaming timing tasks is not limited, and a developer may set the number according to specific requirements. In addition, in the embodiment of the application, a plurality of spark streaming timing tasks can be configured for decoupling, so that the mutual influence among the park streaming timing tasks is avoided.
As a preferred implementation manner, in an embodiment of the present application, the method further includes:
and after generating a corresponding download task according to the target task metadata and the download request, storing the download task into a pre-constructed download task instance table.
Specifically, in the embodiment of the application, the real-time asynchronous downloading function is designed into two tables, one is a downloading task table, the other is a downloading task instance table, and the functions of the downloading task table mainly comprise configuration downloading task ID, downloading service name, downloading displayed file name, downloading field information, downloading es index table, downloading file path and the like. The function of the downloading task instance table is mainly to record information such as downloading conditions, downloading task, downloading time, downloading task state, operation starting time, ending time and the like which are clicked by downloading personnel to inquire in real time. After each download task is generated, the download task is stored in the download task instance table so as to analyze the download task later.
As a preferred implementation manner, in an embodiment of the present application, the method further includes:
and analyzing the downloading task in the downloading task instance table to obtain an analysis result.
Specifically, in the embodiment of the application, the downloading task in the downloading task instance table can be analyzed at regular time to obtain the analysis result so as to realize the monitoring of the downloading task. For example, including but not limited to analyzing information such as completion of the download task, download frequency, etc., based on the analysis results. In addition, configuration data of the download task may be adjusted according to the analysis result, for example, the processing priority level of the download task with a high download frequency may be increased.
As a preferred implementation manner, in the embodiment of the present application, the method further includes a data writing process of the search engine, including:
the real-time data acquisition system sends the real-time service data acquired by the service system to a message queue, and writes the real-time service data into the search engine through the message queue; and/or the number of the groups of groups,
and the offline drawing number acquisition system sends the offline service data extracted from the database of the service system to a hive, and the hive writes the offline service data into the search engine.
Specifically, in the embodiment of the present application, the search engine may employ an elastic search, where the data in the search engine includes real-time service data generated by a front-end service system (e.g., a logistics service system, including an order, warehouse, distribution, transportation, distribution, after-sales, and billing service system). The data acquisition system comprises a real-time data acquisition system (such as a Lodcs real-time data acquisition system) and an off-line decimation number acquisition system (such as an LBI off-line decimation number acquisition system). The real-time data acquisition system is used for acquiring real-time service data generated by the front-end service system, sending the acquired data to a message queue (such as kafka), and writing the data into the search engine through the message queue. The off-line drawing number acquisition system is used for extracting off-line service data from a database of the front-end service system.
Fig. 3 is a schematic structural view of a data downloading apparatus according to an exemplary embodiment, and referring to fig. 3, the apparatus includes:
the identification acquisition module is used for receiving the downloading request and acquiring a target task identification carried in the downloading request;
the data matching module is used for acquiring target task metadata from a pre-configured download task data table according to the target task identification;
the task generating module is used for generating a corresponding downloading task according to the target task metadata and the downloading request;
the data acquisition module is used for executing the downloading task at fixed time, acquiring target data from a search engine and importing the target data into a file in a preset format for downloading, wherein the data in the search engine comprises real-time data acquired from a service system.
As a preferred implementation manner, in an embodiment of the present application, the apparatus further includes:
and the task storage module is used for storing the download task into a pre-constructed download task instance table after generating the corresponding download task according to the target task metadata and the download request.
As a preferred implementation manner, in an embodiment of the present application, the apparatus further includes:
and the task analysis module is used for analyzing the downloading task in the downloading task instance table and obtaining an analysis result.
As a preferred implementation manner, in an embodiment of the present application, the apparatus further includes a data writing module, where the data writing module includes:
the first writing-in unit is used for sending the real-time service data acquired from the service system to a message queue by the real-time data acquisition system and writing the real-time service data into the search engine through the message queue;
the second writing unit is used for sending the offline business data extracted from the database of the business system to the hive by the offline lottery collecting system, and writing the offline business data into the search engine by the hive.
Fig. 4 is a schematic diagram showing an internal structure of a computer device including a processor, a memory, and a network interface connected through a system bus, as shown with reference to fig. 4, according to an exemplary embodiment. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a method of optimizing an execution plan.
It will be appreciated by persons skilled in the art that the architecture shown in fig. 4 is merely a block diagram of some of the architecture relevant to the present inventive arrangements and is not limiting as to the computer device to which the present inventive arrangements are applicable, and that a particular computer device may include more or fewer components than shown, or may combine some of the components, or have a different arrangement of components.
As a preferred implementation manner, in an embodiment of the present application, a computer device includes a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor executes the computer program to implement the following steps:
receiving a downloading request, and acquiring a target task identifier carried in the downloading request;
acquiring target task metadata from a pre-configured download task data table according to the target task identifier;
generating a corresponding downloading task according to the target task metadata;
and executing the downloading task at fixed time, acquiring target data from a search engine, and importing the target data into a file in a preset format for downloading, wherein the data in the search engine comprises real-time data acquired from a service system.
In a preferred embodiment of the present application, the processor executes the computer program to further implement the following steps:
and after generating a corresponding downloading task according to the target task metadata, storing the downloading task into a pre-constructed downloading task instance table.
In a preferred embodiment of the present application, the processor executes the computer program to further implement the following steps:
and analyzing the downloading task in the downloading task instance table to obtain an analysis result.
In a preferred embodiment of the present application, the processor further implements a data writing step of the search engine when executing the computer program, including:
the real-time data acquisition system sends the real-time service data acquired by the service system to a message queue, and writes the real-time service data into the search engine through the message queue; and/or the number of the groups of groups,
and the offline drawing number acquisition system sends the offline service data extracted from the database of the service system to a hive, and the hive writes the offline service data into the search engine.
In an embodiment of the present application, there is also provided a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of:
receiving a downloading request, and acquiring a target task identifier carried in the downloading request;
acquiring target task metadata from a pre-configured download task data table according to the target task identifier;
generating a corresponding downloading task according to the target task metadata;
and executing the downloading task at fixed time, acquiring target data from a search engine, and importing the target data into a file in a preset format for downloading, wherein the data in the search engine comprises real-time data acquired from a service system.
As a preferred implementation manner, in the embodiment of the present application, when the computer program is executed by the processor, the following steps are further implemented:
and after generating a corresponding downloading task according to the target task metadata, storing the downloading task into a pre-constructed downloading task instance table.
As a preferred implementation manner, in the embodiment of the present application, when the computer program is executed by the processor, the following steps are further implemented:
and analyzing the downloading task in the downloading task instance table to obtain an analysis result.
In a preferred embodiment of the present application, when the computer program is executed by the processor, the data writing step of the search engine is further implemented, including:
the real-time data acquisition system sends the real-time service data acquired by the service system to a message queue, and writes the real-time service data into the search engine through the message queue; and/or the number of the groups of groups,
and the offline drawing number acquisition system sends the offline service data extracted from the database of the service system to a hive, and the hive writes the offline service data into the search engine.
In summary, the technical solution provided by the embodiment of the present application has the following beneficial effects:
it should be noted that: the data downloading device provided in the above embodiment only uses the division of the above functional modules to illustrate when triggering the data downloading service, and in practical application, the above functional allocation may be performed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the data downloading device and the data downloading method provided in the foregoing embodiments belong to the same concept, that is, the device is based on the data downloading method, and the specific implementation process of the device is detailed in the method embodiment, which is not described herein again.
It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program for instructing relevant hardware, where the program may be stored in a computer readable storage medium, and the storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.
The foregoing description of the preferred embodiments of the application is not intended to limit the application to the precise form disclosed, and any such modifications, equivalents, and alternatives falling within the spirit and scope of the application are intended to be included within the scope of the application.
Claims (4)
1. A method of data download, the method comprising the steps of:
receiving a downloading request, and acquiring a target task identifier carried in the downloading request;
acquiring target task metadata from a pre-configured download task data table according to the target task identifier;
generating a corresponding downloading task according to the target task metadata and the downloading request;
the downloading task is executed at fixed time, target data are obtained from a search engine, and the target data are imported into a file with a preset format for downloading, wherein the data in the search engine comprise real-time data obtained from a service system;
the method also includes a data writing process of the search engine, including:
the real-time data acquisition system sends the real-time service data acquired by the service system to a message queue, and the real-time service data is written into the search engine through the message queue; and/or the number of the groups of groups,
the offline drawing number acquisition system sends the offline service data extracted from the database of the service system to a hive, and the hive writes the offline service data into the search engine;
the method further comprises the steps of:
after generating a corresponding download task according to the target task metadata and the download request, storing the download task into a pre-constructed download task instance table;
analyzing the downloading task in the downloading task instance table, obtaining an analysis result to realize monitoring of the downloading task, and adjusting configuration data of the downloading task according to the analysis result.
2. A data downloading apparatus, the apparatus comprising:
the identification acquisition module is used for receiving the downloading request and acquiring a target task identification carried in the downloading request;
the data matching module is used for acquiring target task metadata from a pre-configured download task data table according to the target task identification;
the task generating module is used for generating a corresponding downloading task according to the target task metadata and the downloading request;
the data acquisition module is used for executing the downloading task at regular time, acquiring target data from a search engine and importing the target data into a file in a preset format for downloading, wherein the data in the search engine comprises real-time data acquired from a service system;
the apparatus further includes a data writing module including:
the first writing-in unit is used for sending the real-time service data acquired from the service system to a message queue by the real-time data acquisition system, and writing the real-time service data into the search engine through the message queue;
the second writing unit is used for sending the offline service data extracted from the database of the service system to the hive by the offline lottery acquisition system, and writing the offline service data into the search engine by the hive;
the apparatus further comprises:
the task storage module is used for storing the downloading task into a pre-constructed downloading task instance table after generating the corresponding downloading task according to the target task metadata and the downloading request;
the task analysis module is used for analyzing the downloading task in the downloading task instance table, acquiring an analysis result so as to realize monitoring of the downloading task and adjusting configuration data of the downloading task according to the analysis result.
3. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the method of claim 1 when executing the computer program.
4. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method of claim 1.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010118332.5A CN111343269B (en) | 2020-02-25 | 2020-02-25 | Data downloading method, device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010118332.5A CN111343269B (en) | 2020-02-25 | 2020-02-25 | Data downloading method, device, computer equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111343269A CN111343269A (en) | 2020-06-26 |
| CN111343269B true CN111343269B (en) | 2023-09-08 |
Family
ID=71185615
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010118332.5A Active CN111343269B (en) | 2020-02-25 | 2020-02-25 | Data downloading method, device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111343269B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112579321A (en) * | 2020-12-23 | 2021-03-30 | 京东数字科技控股股份有限公司 | Method, device and equipment for downloading service data |
| CN112507672A (en) * | 2020-12-23 | 2021-03-16 | 安徽航天信息有限公司 | Payroll data acquisition method, device, system and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103207785A (en) * | 2013-04-23 | 2013-07-17 | 北京奇虎科技有限公司 | Method, device, and system for processing data download request |
| CN106020944A (en) * | 2016-05-04 | 2016-10-12 | 北京思特奇信息技术股份有限公司 | Background database configuration based data downloading method and system |
| CN106919685A (en) * | 2017-03-02 | 2017-07-04 | 浪潮软件集团有限公司 | Mass data file processing method |
| CN107071036A (en) * | 2017-04-21 | 2017-08-18 | 广东浪潮大数据研究有限公司 | A kind of method, the webserver and system for downloading driving file |
| CN108920659A (en) * | 2018-07-03 | 2018-11-30 | 广州唯品会信息科技有限公司 | Data processing system and its data processing method, computer readable storage medium |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101399729B (en) * | 2007-09-28 | 2010-12-08 | 鸿富锦精密工业（深圳）有限公司 | Management system and method for file download and upload |
-
2020
- 2020-02-25 CN CN202010118332.5A patent/CN111343269B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103207785A (en) * | 2013-04-23 | 2013-07-17 | 北京奇虎科技有限公司 | Method, device, and system for processing data download request |
| CN106020944A (en) * | 2016-05-04 | 2016-10-12 | 北京思特奇信息技术股份有限公司 | Background database configuration based data downloading method and system |
| CN106919685A (en) * | 2017-03-02 | 2017-07-04 | 浪潮软件集团有限公司 | Mass data file processing method |
| CN107071036A (en) * | 2017-04-21 | 2017-08-18 | 广东浪潮大数据研究有限公司 | A kind of method, the webserver and system for downloading driving file |
| CN108920659A (en) * | 2018-07-03 | 2018-11-30 | 广州唯品会信息科技有限公司 | Data processing system and its data processing method, computer readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111343269A (en) | 2020-06-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111597257A (en) | Database synchronization method and device, storage medium and terminal | |
| CN107247811B (en) | SQL statement performance optimization method and device based on Oracle database | |
| CN104599032A (en) | Distributed memory power grid construction method and system for resource management | |
| CN111343269B (en) | Data downloading method, device, computer equipment and storage medium | |
| CN113360554A (en) | Method and equipment for extracting, converting and loading ETL (extract transform load) data | |
| CN111400393B (en) | Data processing method and device based on multi-application platform and storage medium | |
| CN112988863A (en) | Elasticissearch-based efficient search engine method for heterogeneous multiple data sources | |
| CN113282611B (en) | Method, device, computer equipment and storage medium for synchronizing stream data | |
| CN111258978A (en) | Data storage method | |
| CN113407623A (en) | Data processing method and device and server | |
| CN114398520A (en) | Data retrieval method, system, device, electronic equipment and storage medium | |
| CN104881475A (en) | Method and system for randomly sampling big data | |
| CN110704442A (en) | Real-time acquisition method and device for big data | |
| CN107193749B (en) | Test method, device and equipment | |
| CN106940710B (en) | Information pushing method and device | |
| CN111046113B (en) | Data retrieval device and method for different types of data warehouses | |
| CN107357919A (en) | User behaviors log inquiry system and method | |
| CN110990472A (en) | Data deriving method and device based on Hbase | |
| CN115495463A (en) | Data processing method and device, intelligent equipment and storage medium | |
| CN115391286A (en) | Link tracking data management method, device, equipment and storage medium | |
| CN113468275A (en) | Data importing method and device of graph database, storage medium and electronic equipment | |
| CN111475505A (en) | Data acquisition method and equipment | |
| CN110990430A (en) | Large-scale data parallel processing system | |
| CN111654410B (en) | Gateway request monitoring method, device, equipment and medium | |
| CN113515494A (en) | Database processing method based on distributed file system and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |