CN108228644B

CN108228644B - Method and device for exporting report

Info

Publication number: CN108228644B
Application number: CN201611192913.3A
Authority: CN
Inventors: 战勇
Original assignee: Beijing Jingdong Century Trading Co Ltd; Beijing Jingdong Shangke Information Technology Co Ltd
Current assignee: Beijing Jingdong Century Trading Co Ltd; Beijing Jingdong Shangke Information Technology Co Ltd
Priority date: 2016-12-21
Filing date: 2016-12-21
Publication date: 2021-07-30
Anticipated expiration: 2036-12-21
Also published as: CN108228644A

Abstract

The application discloses a method and a device for exporting reports. The method comprises the following steps: acquiring export parameters and task information of a task, wherein the task information comprises user information; generating push information of the task through the export parameters according to an export strategy; generating an export file according to the push information and the export parameters; and updating the task information and recording the path of the export file. The method for exporting the report can improve user experience and resource utilization rate.

Description

Method and device for exporting report

Technical Field

The invention relates to the field of data processing, in particular to a method and a device for exporting reports.

Background

With the explosion of networks, the data volume of application systems is continuously increasing, and report-derived-like functions are almost spread over most business systems. The method is characterized in that a exporting function can be completed only by performing a series of processes of business processing, data cleaning, assembling and the like in a background due to the large data and complex business logic and the diversity of data storage and query, and a large amount of system resources are consumed in the exporting process, so that the situations of slow business processing, system false death, memory overflow and the like can be caused if the system is not strictly controlled.

At present, the report exporting function provided by the application system is to provide a query page of the report, a user can select or input a plurality of query conditions on the query page, the application system searches a data result meeting the conditions from the database according to the user parameters, and finally, a report file is generated and downloaded to the user. After the user executes export, under the condition of complex service or large data volume, the processing duration of the server is long, the export completion time cannot be estimated in advance due to factors such as system complexity and processing performance, and the like, the user blindly waits for the export result, in order to improve the user experience, most service systems add prompt information such as a light shield and the like to inform the user that the background is processing prompt for patience waiting and the like, and shield an export button and the like, so as to improve the user experience, but still cannot change the result of the blindness waiting of the user (for a B/S system, the export starts to execute due to the particularity of an HTTP protocol, the user must wait on the current page until the server returns the export result, and once the user refreshes or leaves the current page and the server returns the result, the client cannot be informed), some users are anxious to obtain the export result and refresh the page again, the same export operation is executed, the system starts a new export task, and after the operation is repeated, a plurality of export tasks under the same condition are generated, so that the performance of the system is seriously influenced, the normal service is influenced if the export tasks are slightly influenced, the memory overflow is caused if the export tasks are seriously influenced, the system is falsely dead, and the like.

Therefore, a new method and apparatus for report export are needed.

The above information disclosed in this background section is only for enhancement of understanding of the background of the invention and therefore it may contain information that does not constitute prior art that is already known to a person of ordinary skill in the art.

Disclosure of Invention

In view of this, the present invention provides a method and an apparatus for exporting a report, which can improve user experience and resource utilization.

Additional features and advantages of the invention will be set forth in the detailed description which follows, or may be learned by practice of the invention.

According to an aspect of the present invention, a method for report export is provided, the method comprising: acquiring export parameters and task information of a task, wherein the task information comprises user information; generating push information of the task through the derived parameters according to the derived strategy; generating an export file through the push information and the export parameters; and updating the task information and recording the path of the export file.

In an exemplary embodiment of the present disclosure, acquiring derived parameters and task information of a task includes: judging whether tasks with the same export parameters from the same user exist or not according to the export parameters and the task information; and if no task with the same derived parameter from the same user exists, storing the task information to a first preset position and storing the derived parameter to a second preset position.

In an exemplary embodiment of the present disclosure, the first predetermined location is a cache cloud, and the second predetermined location is a data cloud.

In an exemplary embodiment of the present disclosure, further comprising: and if the tasks with the same derived parameters from the same user exist, pushing the corresponding message to the client according to the state of the tasks.

In an exemplary embodiment of the present disclosure, the state of the task includes: ready and in preparation.

In an exemplary embodiment of the present disclosure, generating push information of a task by deriving a parameter according to a derivation policy includes: generating push information in real time through the derived parameters according to the derived strategy; and generating push information in a delayed mode through the derived parameters according to the derived strategy.

In an exemplary embodiment of the present disclosure, updating the task information and recording a path of the export file includes: and storing the task information to a download task prompt list.

In an exemplary embodiment of the present disclosure, further comprising: inquiring task information in a downloading task prompt table at regular time, and judging whether a user corresponding to the task information is online or not; and when the user is on line, pushing the corresponding message to the client.

According to an aspect of the present invention, an apparatus for report export is provided, the apparatus comprising: the receiving module is used for acquiring the export parameters and the task information of the task, and the task information comprises user information; the pushing module is used for generating pushing information of the task through the derived parameters according to the derived strategy; the task module is used for generating an export file through the push information and the export parameters; and the file module is used for updating the task information and recording the path of the export file.

In an exemplary embodiment of the present disclosure, further comprising: the judging module is used for judging whether tasks with the same export parameters from the same user exist or not according to the export parameters and the task information; and the storage module is used for storing the task information to a first preset position and storing the derived parameters to a second preset position when the tasks with the same derived parameters from the same user do not exist.

According to the method and the device for exporting the report, the user experience and the resource utilization rate can be improved.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention, as claimed.

Drawings

The above and other objects, features and advantages of the present invention will become more apparent by describing in detail exemplary embodiments thereof with reference to the attached drawings. The drawings described below are only some embodiments of the invention and other drawings may be derived from those drawings by a person skilled in the art without inventive effort.

FIG. 1 is a flow diagram illustrating a method for report export according to an exemplary embodiment.

FIG. 2 is a flow diagram illustrating a method for report export according to another exemplary embodiment.

FIG. 3 is a flowchart illustrating a method for report export, according to another exemplary embodiment.

FIG. 4 is a flowchart illustrating a method for report export, according to another exemplary embodiment.

FIG. 5 is a flowchart illustrating a method for report export, according to another exemplary embodiment.

FIG. 6 is a block diagram illustrating an apparatus for report export according to another exemplary embodiment.

DETAILED DESCRIPTION OF EMBODIMENT (S) OF INVENTION

Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art. The same reference numerals denote the same or similar parts in the drawings, and thus, a repetitive description thereof will be omitted.

Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to provide a thorough understanding of embodiments of the invention. One skilled in the relevant art will recognize, however, that the invention may be practiced without one or more of the specific details, or with other methods, components, devices, steps, and so forth. In other instances, well-known methods, devices, implementations or operations have not been shown or described in detail to avoid obscuring aspects of the invention.

The block diagrams shown in the figures are functional entities only and do not necessarily correspond to physically separate entities. I.e. these functional entities may be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor means and/or microcontroller means.

The flow charts shown in the drawings are merely illustrative and do not necessarily include all of the contents and operations/steps, nor do they necessarily have to be performed in the order described. For example, some operations/steps may be decomposed, and some operations/steps may be combined or partially combined, so that the actual execution sequence may be changed according to the actual situation.

It will be understood that, although the terms first, second, third, etc. may be used herein to describe various components, these components should not be limited by these terms. These terms are used to distinguish one element from another. Thus, a first component discussed below may be termed a second component without departing from the teachings of the disclosed concept. As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items.

It will be appreciated by those skilled in the art that the drawings are merely schematic representations of exemplary embodiments, and that the blocks or flow charts in the drawings are not necessarily required to practice the present invention and are, therefore, not intended to limit the scope of the present invention.

As shown in fig. 1, in S102, the derived parameters and task information of the task are acquired, and the task information includes user information. Monitoring the update condition of task data, and obtaining export parameters and task information of a task, where the task may be, for example, a report export task, and the task information may include, for example, user information, and may also include, for example, report information, request time, status information, and the like, and the invention is not limited thereto.

In S104, push information of the task is generated by deriving the parameters according to the derivation policy. The export strategy may be, for example, real-time export or delayed export; the push information may be generated in real time by deriving parameters, e.g., according to a derivation strategy; and generating push information in a delayed mode through the derived parameters according to the derived strategy. For example, the push information may include an export policy, and the real-time push information may be sent to the execution module, and the execution module may execute the task in real time or delay the execution of the task according to the export policy included in the export information. The invention is not limited thereto.

In S106, an export file is generated from the push information and the export parameter. Export files may be generated, for example, by export jobs, which may include, for example, querying a database, data cleansing, data processing, etc., through which export jobs are generated.

In S108, the job information is updated, and the path of the export file is recorded. After the export file is generated, the related information of the task is updated according to the content of the file, and meanwhile, the storage path of the export file is recorded.

According to the method for exporting the report, the limit that the traditional export of the WEB page needs to wait on the original page is broken through the mode of asynchronously exporting the report, and the experience of a report export user can be improved.

It should be clearly understood that the present disclosure describes how to make and use particular examples, but the principles of the present disclosure are not limited to any details of these examples. Rather, these principles can be applied to many other embodiments based on the teachings of the present disclosure.

In an exemplary embodiment of the present disclosure, acquiring derived parameters and task information of a task includes: judging whether tasks with the same export parameters from the same user exist or not according to the export parameters and the task information; and if no task with the same derived parameter from the same user exists, storing the task information to a first preset position and storing the derived parameter to a second preset position. And if the tasks with the same derived parameters from the same user exist, pushing the corresponding message to the client according to the state of the tasks. The first predetermined location is a cache cloud and the second predetermined location is a data cloud. The states of the tasks include: ready and in preparation. The cache cloud includes, for example: redis, Mongdb, and Hbase.

redis is a key-value storage system. It supports relatively many stored value types, including string, list, set, zset and hash. These data types all support push/pop, add/remove, and intersect union and difference, and richer operations, and these operations are all atomic. On this basis, redis supports various different ways of ordering. To ensure efficiency, data is cached in memory. The redis periodically writes updated data into a disk or writes modification operation into an additional recording file, and realizes master-slave synchronization on the basis of the updated data or the modification operation.

Mongodb is a product between a relational database and a non-relational database, and the non-relational database has the most abundant functions and is most similar to the relational database. The data structure supported by the user is very loose and is in a json-like bson format, so that more complex data types can be stored. The biggest characteristic of Mongo is that the query language supported by Mongo is very strong, the syntax of Mongo is similar to the object-oriented query language, most functions of single-table query of similar relational databases can be almost realized, and index establishment of data is supported.

HBase is a distributed, column-oriented open source database, and provides Bigtable-like capabilities over Hadoop. Hadoop is a distributed system infrastructure developed by the Apache Foundation. A user can develop a distributed program without knowing the distributed underlying details. The power of the cluster is fully utilized to carry out high-speed operation and storage. HBase is a sub-item of the Hadoop item of Apache. HBase is different from a general relational database, and is a database suitable for unstructured data storage. Another difference is that HBase is based on a column rather than a row based pattern.

According to the method for exporting the report, the additional system overhead caused by the multi-task execution of the same user and the same condition at the same time is avoided by judging whether the report exporting tasks of the same user and the same condition exist at the same time, and the resource utilization rate can be improved.

As shown in fig. 2, in this embodiment, after a user submits a export request, a system for processing a task first queries whether there is an export task that is not executed by the user under the same condition in a cache cloud, if so, a prompt message such as "XXXX query condition export task is executing and please pay attention to push information later" may be returned, otherwise, key data (for example, information about user information, name of a service module, task creation time, export status, and the like) is stored in the cache cloud, and an export parameter is stored in a data cloud; monitoring data changes in the data cloud in real time, and pushing a export task to a task execution module in real time/in a delayed manner according to an export strategy when a new export task is generated; and the task execution module is responsible for exporting the query database to generate an export file, storing the file in a file system, and updating and recording the task state and the file path in the cache cloud. Monitoring the data updating condition in the cache cloud in real time, actively pushing a export completion message to a prompt information table when a new task is executed, polling the prompt information table at regular time, judging whether a user to which the task belongs is online, if so, informing the user that the downloading task is ready, and starting downloading.

As shown in fig. 3, in this embodiment, after the user selects/enters the export parameters in the service module, the system receives the export parameters, first determines whether an export task with the same conditions as the current user exists in the cache cloud, if so, determines the status of the record, if the status is "in preparation", returns an export task such as "xxxxxx time xxxxxx export parameter is being executed, and please pay attention to the prompt information of the system push information" later, so as to prevent the same export task from being created, and if the status is "ready", prompts the user whether the task is ready for downloading. Otherwise, storing the information such as user information, module names, report information, request time, states and the like in the cache cloud, storing the derived parameter information in the data cloud (for subsequent processes), and returning to 'success in creating the derived task and please follow the system push information'.

1) And the user selects/enters the export parameters in the export page and submits an export request.

2) And the server receives the export request of the user and verifies whether the export task of the current user and the same condition exists in the cache cloud. If yes, judging the task recording state, if the state is 'preparation', returning to 'the condition that the export task with XXXXXX time and XXXX is being executed, and then paying attention to the push information'; and if the state is 'ready', inquiring information by combining with a 'download task prompt information table', and prompting a user whether the task is ready to be downloaded or not. Otherwise, the report related information (user information, module name, report information, request time, state and the like) is stored in the cache cloud, and the derived parameter information is stored in the data cloud.

3) The user is informed that "export task creation was successful, export results will be pushed later".

As shown in fig. 4, in this embodiment, the task pushing module monitors whether there is a newly added export parameter in the data cloud, pushes the export parameter to the execution module, and the task execution module queries the service database according to the condition, generates an export file, stores the export file in the file system, and updates the task state and the file path in the cache cloud.

1) The task pushing module monitors data change in the data cloud, and when new data (export parameters) are generated, a pushing task is generated and sent to the export task pushing module.

2) And after receiving the message, the task pushing module pushes the derived parameters to the execution module in real time or in a delayed mode according to the strategy.

3) And after receiving the export instruction sent by the push module, the task execution module starts to execute formal export work such as database query, data processing and the like, and generates an export file to be stored in a file system.

4) And updating the state of the task in the cache cloud to be ready and recording an export file path.

As shown in fig. 5, in this embodiment, a data update situation in the cache cloud is monitored, and when there is new "ready" data in the cache cloud, a user is prompted to download a file.

1) Monitoring the data updating condition in the cache cloud, and when new ready downloading task data exists, storing the task data to a downloading task prompt information table of the database, wherein the status is 'not downloaded'.

2) And regularly inquiring the task with the status of 'not downloaded' in the 'download task prompt information table', verifying whether the user to which the current task belongs is online, and if so, pushing a message to the client.

3) The client side obtains a task message pushed by the server side and prompts a user that a derived task created in XXXX time is ready and whether the derived task is downloaded or not. And if the user clicks 'ignore and delete the downloading task', the server deletes the cache cloud and corresponding data in the prompt information table of the downloading task.

4) And the task downloading module acquires a file input stream from the file system according to the current task information, stores the file input stream to a user disk, updates a downloading task prompt information table, and finally deletes the current task record in the cache cloud, wherein the current task state is 'downloaded'.

According to the method for exporting the report, the user experience can be comprehensively improved, the resource utilization rate is effectively improved, and the system reliability is improved in a mode of asynchronously exporting the report and synchronously pushing the report exporting information.

Those skilled in the art will appreciate that all or part of the steps implementing the above embodiments are implemented as computer programs executed by a CPU. The computer program, when executed by the CPU, performs the functions defined by the method provided by the present invention. The program may be stored in a computer readable storage medium, which may be a read-only memory, a magnetic or optical disk, or the like.

Furthermore, it should be noted that the above-mentioned figures are only schematic illustrations of the processes involved in the method according to exemplary embodiments of the invention, and are not intended to be limiting. It will be readily understood that the processes shown in the above figures are not intended to indicate or limit the chronological order of the processes. In addition, it is also readily understood that these processes may be performed synchronously or asynchronously, e.g., in multiple modules.

The following are embodiments of the apparatus of the present invention that may be used to perform embodiments of the method of the present invention. For details which are not disclosed in the embodiments of the apparatus of the present invention, reference is made to the embodiments of the method of the present invention.

The receiving module 602 is configured to obtain a derived parameter of a task and task information, where the task information includes user information.

The pushing module 604 is configured to generate pushing information of the task according to the export policy and the export parameter.

The task module 606 is used to generate an export file by pushing information and export parameters.

The file module 608 is used to update the task information and record the path of the export file.

The determining module 610 is configured to determine whether there is a task with the same derived parameter from the same user according to the derived parameter and the task information.

The storage module 612 is configured to store the task information to a first predetermined location and store the derived parameter to a second predetermined location when there is no task with the same derived parameter from the same user.

According to the device for exporting the report, the limit that the traditional export of the WEB page needs to wait on the original page is broken through in a mode of asynchronously exporting the report, and the experience of a report export user can be improved.

Those skilled in the art will appreciate that the modules described above may be distributed in the apparatus according to the description of the embodiments, or may be modified accordingly in one or more apparatuses unique from the embodiments. The modules of the above embodiments may be combined into one module, or further split into multiple sub-modules.

Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein may be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solution according to the embodiment of the present invention can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a usb disk, a removable hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) to execute the method according to the embodiment of the present invention.

Those skilled in the art will readily appreciate from the foregoing detailed description that the method and apparatus for report export according to embodiments of the present invention have one or more of the following advantages.

According to some embodiments, the method for exporting the report breaks through the limitation that the traditional export of the WEB page needs to wait on the original page in a mode of asynchronously exporting the report, and can improve the experience of a report export user.

According to other embodiments, the method for exporting the report avoids extra system overhead caused by the execution of multiple tasks under the same condition of the same user at the same time by judging whether the report export tasks under the same condition of the same user exist at the same time, and can improve the resource utilization rate.

According to other embodiments, the method for exporting the report can comprehensively improve the user experience, effectively improve the resource utilization rate and improve the system reliability by adopting the modes of asynchronously exporting the report and synchronously pushing the report exporting information.

Exemplary embodiments of the present invention are specifically illustrated and described above. It is to be understood that the invention is not limited to the precise construction, arrangements, or instrumentalities described herein; on the contrary, the invention is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

In addition, the structures, the proportions, the sizes, and the like shown in the drawings of the present specification are only used for matching with the contents disclosed in the specification, so as to be understood and read by those skilled in the art, and are not used for limiting the limit conditions which the present disclosure can implement, so that the present disclosure has no technical essence, and any modification of the structures, the change of the proportion relation, or the adjustment of the sizes, should still fall within the scope which the technical contents disclosed in the present disclosure can cover without affecting the technical effects which the present disclosure can produce and the purposes which can be achieved. In addition, the terms "above", "first", "second" and "first" used in the present specification are for the sake of clarity only, and are not intended to limit the scope of the present disclosure, and changes or adjustments of the relative relationship thereof may be considered as the scope of the present invention without substantial changes in the technical content.

Claims

1. A method for report export, comprising:

acquiring export parameters and task information of a task, wherein the task information comprises user information, request time information and state information;

generating push information of the task through the export parameters according to an export strategy, wherein the export strategy is used for determining whether the push information is generated in real time;

generating an export file according to the push information and the export parameters; and

updating state information in the task information and recording the path of the export file;

the updating of the state information in the task information and the recording of the path of the export file include:

storing the task information after updating the state information to a download task prompt table;

inquiring the state information of the task information in the download task prompt table at regular time, and judging whether a user corresponding to the task information is online or not;

and when the user is on line, pushing the message corresponding to the state information to a client.

2. The method of claim 1, wherein the obtaining of the derived parameters and the task information of the task comprises:

judging whether the tasks with the same export parameters from the same user exist or not according to the export parameters and the task information;

and if the task with the same derived parameter from the same user does not exist, storing the task information to a first preset position, and storing the derived parameter to a second preset position.

3. The method of claim 2, wherein the first predetermined location is a cache cloud and the second predetermined location is a data cloud.

4. The method of claim 2, further comprising:

and if the tasks with the same derived parameters from the same user exist, pushing corresponding messages to the client according to the state of the tasks.

5. The method of claim 4, wherein the state of the task comprises: ready and in preparation.

6. The method of claim 1, wherein generating push information for the task via the derived parameters according to a derived policy comprises:

generating the push information in real time through the derived parameters according to a derived strategy; and

and generating the push information through the delay of the derived parameters according to the derived strategy.

7. An apparatus for report export, comprising:

the system comprises a receiving module, a processing module and a processing module, wherein the receiving module is used for acquiring export parameters and task information of a task, and the task information comprises user information, request time information and state information;

the pushing module is used for generating pushing information of the task through the export parameters according to an export strategy, wherein the export strategy is used for determining whether the pushing information is generated in real time;

the task module is used for generating an export file through the push information and the export parameter; and

the file module is used for updating the state information in the task information and recording the path of the export file;

wherein, the updating the state information in the task information and recording the path of the export file includes:

8. The apparatus of claim 7, further comprising:

the judging module is used for judging whether the tasks with the same export parameters from the same user exist or not according to the export parameters and the task information;

and the storage module is used for storing the task information to a first preset position and storing the derived parameters to a second preset position when the tasks with the same derived parameters from the same user do not exist.

9. An electronic device, comprising:

one or more processors;

storage means for storing one or more programs;

when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-6.

10. A computer-readable medium, on which a computer program is stored, which, when being executed by a processor, carries out the method according to any one of claims 1-6.