CN113760177A - Data reporting method and device - Google Patents

Data reporting method and device Download PDF

Info

Publication number
CN113760177A
CN113760177A CN202110003276.5A CN202110003276A CN113760177A CN 113760177 A CN113760177 A CN 113760177A CN 202110003276 A CN202110003276 A CN 202110003276A CN 113760177 A CN113760177 A CN 113760177A
Authority
CN
China
Prior art keywords
data
reporting
real
reported
mode
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110003276.5A
Other languages
Chinese (zh)
Inventor
吴飞
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Jingdong Century Trading Co Ltd
Beijing Wodong Tianjun Information Technology Co Ltd
Original Assignee
Beijing Jingdong Century Trading Co Ltd
Beijing Wodong Tianjun Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Jingdong Century Trading Co Ltd, Beijing Wodong Tianjun Information Technology Co Ltd filed Critical Beijing Jingdong Century Trading Co Ltd
Priority to CN202110003276.5A priority Critical patent/CN113760177A/en
Publication of CN113760177A publication Critical patent/CN113760177A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0602Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0608Saving storage space on storage systems
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0602Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0614Improving the reliability of storage systems
    • G06F3/0619Improving the reliability of storage systems in relation to data integrity, e.g. data losses, bit errors
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0628Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0655Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
    • G06F3/0659Command handling arrangements, e.g. command buffers, queues, command scheduling
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0668Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/0671In-line storage system
    • G06F3/0683Plurality of storage devices

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Security & Cryptography (AREA)
  • Mobile Radio Communication Systems (AREA)

Abstract

The invention discloses a data reporting method and device, and relates to the technical field of computers. One embodiment of the method comprises: acquiring data to be reported, and determining a data reporting mode according to a reporting strategy; the data reporting mode comprises a real-time reporting mode and a non-real-time reporting mode, wherein the data to be reported corresponding to the real-time reporting mode is first data, and the data to be reported corresponding to the non-real-time reporting mode is second data; reporting the first data; and storing the second data, and reporting the second data according to the reporting condition. The implementation mode can flexibly set the data reporting mode according to specific requirements, thereby not only ensuring the timeliness of data reporting, but also avoiding the loss condition of the data to be reported, improving the accuracy and efficiency of data reporting, and improving the user experience.

Description

Data reporting method and device
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a method and an apparatus for reporting data.
Background
With the development of the internet, more and more enterprises develop client application products to expand user groups and promote self development. Meanwhile, in order to provide better user experience, data such as application products, even clients, for example, data such as power, upload/download speed, or usage data such as server failure response codes, etc. need to be reported. The existing data reporting modes mainly include two types: real-time reporting and non-real-time reporting.
The prior art has at least the following problems:
the existing data reporting method cannot flexibly set a data reporting mode according to specific requirements, easily causes the situations of data loss, reporting delay and the like to be reported, and has the technical problems of low data reporting accuracy, low reporting efficiency and poor user experience.
Disclosure of Invention
In view of this, embodiments of the present invention provide a data reporting method and apparatus, which can flexibly set a data reporting mode according to specific requirements, thereby ensuring timeliness of data reporting, avoiding a loss situation of data to be reported, improving accuracy and efficiency of data reporting, and improving user experience.
In order to achieve the above object, according to a first aspect of the embodiments of the present invention, a method for reporting data is provided, including:
acquiring data to be reported, and determining a data reporting mode according to a reporting strategy; the data reporting mode comprises a real-time reporting mode and a non-real-time reporting mode, wherein the data to be reported corresponding to the real-time reporting mode is first data, and the data to be reported corresponding to the non-real-time reporting mode is second data;
reporting the first data;
and storing the second data, and reporting the second data according to the reporting condition.
Further, storing the second data includes:
generating a write operation corresponding to the second data, and placing the write operation into a serial queue;
and storing the second data according to the execution sequence of the data operation and the write operation in the serial queue.
Further, the reporting condition includes a maximum reporting number and/or a reporting frequency; and reporting the second data according to the reporting condition, wherein the reporting process comprises the following steps:
generating a reading operation corresponding to the second data, and placing the reading operation into a serial queue;
and determining target second data from the stored second data according to the execution sequence and the reading operation of the data operation in the serial queue and the maximum reporting quantity and/or the reporting frequency, and reporting the target second data.
Further, the method further comprises:
generating a deletion operation corresponding to the target second data, and placing the deletion operation after the reading operation in the serial queue;
and after the target second data is reported successfully, deleting the target second data from the stored second data according to the execution sequence and the deletion operation of the data operation in the serial queue.
Further, data numbers are sequentially set for the stored second data; determining target second data from the stored second data according to the maximum reporting number, further comprising:
and determining target second data from the stored second data according to the maximum reporting quantity and the data number.
Further, for the first data with failed reporting processing, the method further includes:
judging whether the processing failure times exceed a time threshold value or not;
if not, continuing to report the first data which fails in reporting;
if so, storing the first data which fails in the reporting processing.
Further, reporting the first data in an asynchronous mode; and reporting the stored data in a synchronous mode.
According to a second aspect of the embodiments of the present invention, there is provided a device for reporting data, including:
the data reporting mode determining module is used for acquiring data to be reported and determining a data reporting mode according to a reporting strategy; the data reporting mode comprises a real-time reporting mode and a non-real-time reporting mode, wherein the data to be reported corresponding to the real-time reporting mode is first data, and the data to be reported corresponding to the non-real-time reporting mode is second data;
the first reporting module is used for reporting the first data;
and the second reporting module is used for storing the second data and reporting the second data according to the reporting condition.
Further, the second reporting module is further configured to:
generating a write operation corresponding to the second data, and placing the write operation into a serial queue;
and storing the second data according to the execution sequence of the data operation and the write operation in the serial queue.
Further, the second reporting module is further configured to:
generating a reading operation corresponding to the second data, and placing the reading operation into a serial queue;
and determining target second data from the stored second data according to the execution sequence and the reading operation of the data operation in the serial queue and the maximum reporting quantity and/or the reporting frequency, and reporting the target second data.
According to a third aspect of embodiments of the present invention, there is provided an electronic apparatus, including:
one or more processors;
a storage device for storing one or more programs,
when the one or more programs are executed by the one or more processors, the one or more processors implement any of the above methods for reporting data.
According to a fourth aspect of the embodiments of the present invention, there is provided a computer-readable medium, on which a computer program is stored, where the computer program, when executed by a processor, implements any one of the above-mentioned data reporting methods.
One embodiment of the above invention has the following advantages or benefits: the data to be reported is obtained, and the data reporting mode is determined according to the reporting strategy; the data reporting mode comprises a real-time reporting mode and a non-real-time reporting mode, wherein the data to be reported corresponding to the real-time reporting mode is first data, and the data to be reported corresponding to the non-real-time reporting mode is second data; reporting the first data; the technical means of storing the second data and reporting the second data according to the reporting condition overcome the technical problems of low data reporting accuracy, low reporting efficiency and poor user experience in the existing data reporting method, and further achieve the purpose of flexibly setting a data reporting mode according to specific requirements, thereby not only ensuring the timeliness of data reporting, but also avoiding the loss condition of the data to be reported, improving the accuracy and efficiency of data reporting, and improving the technical effect of user experience.
Further effects of the above-mentioned non-conventional alternatives will be described below in connection with the embodiments.
Drawings
The drawings are included to provide a better understanding of the invention and are not to be construed as unduly limiting the invention. Wherein:
fig. 1 is a schematic diagram illustrating a main flow of a reporting method of data according to a first embodiment of the present invention;
fig. 2a is a schematic diagram of a main flow of a method for reporting data according to a second embodiment of the present invention;
fig. 2b is a schematic diagram of a main flow of dividing reporting modes of data in the method shown in fig. 2 a;
fig. 2c is a schematic diagram of a main flow of data reporting in the method shown in fig. 2 a;
FIG. 2d is a schematic diagram of the main flow of data manipulation into a serial queue in the method of FIG. 2 a;
fig. 3 is a schematic diagram of main modules of an apparatus for reporting data according to an embodiment of the present invention;
FIG. 4 is an exemplary system architecture diagram in which embodiments of the present invention may be employed;
fig. 5 is a schematic block diagram of a computer system suitable for use in implementing a terminal device or server of an embodiment of the invention.
Detailed Description
Exemplary embodiments of the present invention are described below with reference to the accompanying drawings, in which various details of embodiments of the invention are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
Fig. 1 is a schematic diagram illustrating a main flow of a reporting method of data according to a first embodiment of the present invention; as shown in fig. 1, a method for reporting data provided in the embodiment of the present invention mainly includes:
step S101, acquiring data to be reported, and determining a data reporting mode according to a reporting strategy; the data reporting mode comprises a real-time reporting mode and a non-real-time reporting mode, the data to be reported corresponding to the real-time reporting mode is first data, and the data to be reported corresponding to the non-real-time reporting mode is second data.
Specifically, according to the embodiment of the present invention, the reporting policy is preconfigured, the reporting policy may be written into the terminal in advance, and the terminal may also download the reporting policy from the server. The reporting policy indicates a reporting mode of specific data, for example, for performance data such as electric quantity and uploading/downloading speed, the reporting mode is non-real-time reporting; for the service condition data such as the server failure response code, the reporting mode is real-time reporting.
Further, according to the embodiment of the present invention, the step of determining the data reporting mode according to the reporting policy may be: and setting the data type to be reported in real time and the data type to be reported in non-real time according to the reporting strategy, and further determining the corresponding data reporting mode according to the acquired data type.
According to a real-time specific embodiment of the present invention, an identification field may be added to the data, and a data reporting mode corresponding to the data may be determined according to a field value of the identification field.
It should be noted that the present invention mainly provides a method for flexibly setting data reporting in real time to improve the accuracy and efficiency of data reporting, so that the situation of data not to be reported is not mentioned, and in practical application, the type of data not to be reported can be determined according to specific requirements, and for the data not to be reported, only persistent processing is required.
Preferably, according to an embodiment of the present invention, after acquiring the data, the method further includes: adding application parameters to the acquired data, wherein the application parameters comprise at least one of the following parameters: appID (application number), user pin (user identification code), application version number, environmental parameters (offline, pre-sent, online), etc.
Through the setting, the corresponding application product can be updated and maintained quickly aiming at the reported data subsequently, so that the user experience is improved.
And step S102, reporting the first data.
The first data determined according to the reporting strategy is data which needs to be reported in real time, and the reporting priority of the data is the highest, so that the network module is directly called to carry out reporting processing. Specifically, according to a specific implementation manner of the embodiment of the present invention, the current network state may be determined according to the network module, and the batch reporting processing may be performed on the first data according to the current network state, so as to further improve the reporting efficiency.
According to the embodiment of the invention, the first data can be reported by adopting a plurality of background threads, so that the reporting of a plurality of first data is realized, and the reporting efficiency is improved.
And step S103, storing the second data, and reporting the second data according to the reporting condition.
The reporting priority of the second data determined according to the reporting policy is low, real-time reporting processing is not needed, and the specific reporting time of the second data needs to be determined according to the reporting condition.
Specifically, according to an embodiment of the present invention, the storing the second data includes:
generating a write operation corresponding to the second data, and placing the write operation into a serial queue;
and storing the second data according to the execution sequence of the data operation and the write operation in the serial queue.
Through the arrangement, data operations (writing operation, reading operation and deleting operation) on the storage devices such as the database and the memory are placed in the serial queue, and the serial queue executes data operation tasks in sequence according to the sequence of the data operations placed in the serial queue, so that the situation of data loss is avoided.
Further, according to the embodiment of the present invention, the reporting condition includes a maximum reporting number and/or a reporting frequency; the step of reporting the second data according to the reporting condition comprises:
generating a reading operation corresponding to the second data, and placing the reading operation into a serial queue;
and determining target second data from the stored second data according to the execution sequence and the reading operation of the data operation in the serial queue and the maximum reporting quantity and/or the reporting frequency, and reporting the target second data.
Specifically, the reporting condition may be set according to an actual situation, and may be adjusted according to a network type (such as a cellular network, a WIFI network, and the like) adopted for reporting, and a current network condition (bandwidth, whether network congestion exists, and the like). According to the embodiment of the present invention, it is also possible to determine the target second data that needs to be reported, generate a read operation for the target second data, place the read operation in the serial queue, execute the corresponding read operation according to the execution sequence of the data operations in the serial queue, and report the read target second data.
The serial queue ensures the execution sequence of each data operation task, is favorable for avoiding deadlock, and improves the reporting efficiency of data. According to the embodiment of the invention, the serial queue corresponds to a certain background thread, and the thread is not fixed, so that UI refreshing and user response of the main thread are not influenced.
Preferably, according to the embodiment of the present invention, data numbers are sequentially set for the stored second data; determining target second data from the stored second data according to the maximum reporting number, further comprising:
and determining target second data from the stored second data according to the maximum reporting quantity and the data number.
According to the embodiment of the invention, in the process of storing the data, the incremental data numbers are sequentially generated, and the numbers are used as the reporting priority of the stored data. Under the condition that the reporting conditions are met (maximum reporting quantity and reporting frequency), the target data are determined according to the maximum reporting quantity and the data number, data loss can be effectively avoided, the reporting accuracy of the data is improved, data reporting delay can be avoided, and user experience is improved.
Further, according to an embodiment of the present invention, the method further comprises:
generating a deletion operation corresponding to the target second data, and placing the deletion operation after the reading operation in the serial queue;
and after the target second data is reported successfully, deleting the target second data from the stored second data according to the execution sequence and the deletion operation of the data operation in the serial queue.
And deleting the successfully reported data in the storage devices such as a database, a memory and the like in the storage device, so that the storage space is released.
Further, according to an embodiment of the present invention, for the first data with processing failure being reported, the method further includes:
judging whether the processing failure times exceed a time threshold value or not;
if not, continuing to report the first data which fails in reporting;
if so, storing the first data which fails in the reporting processing.
Through the arrangement, a bottom-of-pocket strategy is provided for the reporting processing process of the first data, and under the condition that the reporting of the first data is still unsuccessful for a certain number of times (which may be that the availability of the current network or system resources is not high), the first data which is failed in the reporting processing is stored and is taken as the data of a non-real-time reporting mode to be subjected to subsequent reporting processing, so that the influence of repeated reporting retries on other normal functional services is avoided, and the overall efficiency of data reporting is improved.
Preferably, according to the embodiment of the present invention, the first data is reported in an asynchronous manner; and reporting the stored data in a synchronous mode.
Through the arrangement, the efficient reporting of the data is ensured, and the data is not lost after the reporting fails.
According to the technical scheme of the embodiment of the invention, the data to be reported is obtained, and the data reporting mode is determined according to the reporting strategy; the data reporting mode comprises a real-time reporting mode and a non-real-time reporting mode, wherein the data to be reported corresponding to the real-time reporting mode is first data, and the data to be reported corresponding to the non-real-time reporting mode is second data; reporting the first data; the technical means of storing the second data and reporting the second data according to the reporting condition overcome the technical problems of low data reporting accuracy, low reporting efficiency and poor user experience in the existing data reporting method, and further achieve the purpose of flexibly setting a data reporting mode according to specific requirements, thereby not only ensuring the timeliness of data reporting, but also avoiding the loss condition of the data to be reported, improving the accuracy and efficiency of data reporting, and improving the technical effect of user experience.
Fig. 2a is a schematic diagram of a main flow of a method for reporting data according to a second embodiment of the present invention; an application scenario of the embodiment of the present invention is reporting of performance data, and as shown in fig. 2a, the method for reporting data provided in the embodiment of the present invention mainly includes:
step S201, acquiring performance data, and adding an application parameter to the acquired performance data.
Specifically, according to an embodiment of the present invention, the performance data includes: starting time of the application, memory usage amount, CPU occupancy rate, application flash back frequency, application service abnormal data, electric quantity, uploading/downloading speed and the like. The application parameters comprise at least one of the following parameters: appID, user pin, application version number, environmental parameters (offline, pre-sent, online), etc.
Through the setting, the corresponding application product can be updated and maintained quickly aiming at the reported data subsequently, so that the user experience is improved.
Step S202, according to the reporting strategy, the performance data is divided into non-reporting performance data, real-time reporting performance data and non-real-time reporting performance data.
Specifically, according to the embodiment of the present invention, the step of dividing the data reporting modes (non-reporting, real-time reporting, and non-real-time reporting) according to the reporting policy may be: and setting the data type to be reported in real time, the data type to be reported in non-real time and the data type not to be reported according to the reporting strategy, and further determining the corresponding data reporting mode according to the type of the acquired data.
According to a real-time specific embodiment of the present invention, an identification field may be added to the data, and a data reporting mode corresponding to the data may be determined according to a field value of the identification field. For example, the division is performed according to a field rt (real time), where a field value corresponding to the rt is 1, which indicates real-time reporting; the field value corresponding to rt is 0, which indicates that no report is made; the field value corresponding to rt is any value except 1 and 0, which indicates non-real-time reporting.
It should be noted that the present invention mainly provides a method for flexibly setting data reporting in real time, so as to improve the accuracy and efficiency of data reporting, and may not relate to the situation of data that is not reported, and may also determine the type of data that is not reported according to specific requirements during actual application, and for the data that is not reported, only persistent processing is required.
As shown in fig. 2b, after the performance data is obtained, the non-reported data, the real-time reported data and the non-real-time reported data may be determined according to the service requirements (real-time reporting type, non-real-time reporting type) and other conditions (e.g., network conditions, reporting time interval, maximum reporting number, data reporting condition, etc.).
Step S203, storing the performance data which is not reported.
Specifically, according to the embodiment of the present invention, performance data that is not reported may also be discarded according to the actual situation, and does not need to be stored.
Step S204, reporting the real-time reported performance data, and judging whether the report is successful. If yes, representing the processing result of the reporting flow of the real-time reporting performance data; if not, go to step S205.
The reporting priority of the real-time reporting performance data determined according to the service requirements is the highest, so that the network module is directly called to carry out reporting processing. Specifically, according to a specific implementation manner of the embodiment of the present invention, the current network state may be determined according to the network module, and the real-time reporting performance data is subjected to batch reporting processing according to the current network state, so as to further improve the reporting efficiency.
According to the embodiment of the invention, the real-time reporting performance data can be reported by adopting a plurality of background threads, so that the reporting processing of the real-time reporting performance data is realized, and the reporting efficiency is improved. As shown in fig. 2c, real-time reporting can be performed by multiple threads, but for non-real-time reporting of performance data, since the performance data is sequentially executed according to data operations in the serial queue, there is only one thread, but the thread allows batch reporting of data in multiple storage devices.
In step S205, it is determined whether the number of times of processing failure exceeds the number threshold. If yes, i.e. the number of processing failures exceeds the threshold number, go to step S206; if not, that is, the number of times of processing failure does not exceed the number threshold, the process goes to step S204.
Specifically, the number threshold may be set according to actual conditions.
Step S206, storing the real-time reporting performance data failed in the reporting process. And then goes to step S208.
Through the arrangement, a bottom-of-pocket strategy is provided for the reporting processing process of reporting the performance data in real time, and under the condition that the reporting of the performance data in real time is still unsuccessful for a certain number of times (which may be that the availability of the current network or system resources is not high), the performance data reported in real time and failed in the reporting processing is stored, for example, the performance data reported in real time is stored in a database, a memory, a hard disk and the like, so that the performance data reported without reporting can be prevented from being lost, and the accuracy of reporting the performance data can be improved.
Step S207, generating a write operation corresponding to the non-real-time reported performance data, and placing the write operation into a serial queue; and storing the non-real-time reported performance data according to the execution sequence and the write-in operation of the data operation in the serial queue.
The reporting priority of the non-real-time reporting performance data determined according to the reporting strategy specified by the specific requirement is low, real-time reporting processing is not needed, the specific reporting time of the non-real-time reporting performance data is determined according to the reporting condition, and the non-real-time reporting performance data is subjected to persistent processing through the setting, so that the unreported performance data can be prevented from being lost, and the accuracy of reporting the performance data is improved.
Through the arrangement, data operations (writing operation, reading operation and deleting operation) on the storage devices such as the database and the memory are placed in the serial queue, and the serial queue executes data operation tasks in sequence according to the sequence of the data operations placed in the serial queue, so that the situation of performance data loss is avoided.
Step S208, judging whether the stored performance data meets the reporting condition. If yes, the stored performance data meets the reporting condition, and step S209 is executed; if not, the stored performance data does not accord with the reporting condition, and other performance data are continuously stored.
Specifically, the reporting condition includes a reporting number and a reporting frequency, and the reporting condition may be set according to an actual situation, and may be adjusted according to a network type (such as a cellular network, a WIFI network, and the like) adopted for reporting, and a current network condition (bandwidth, whether network congestion exists, and the like).
Step S209 is to generate a read operation and a delete operation corresponding to the stored performance data, and place the read operation and the delete operation in a serial queue, where the delete operation in the serial queue is located after the read operation.
According to the embodiment of the present invention, after the target performance data that needs to be reported is determined, a read operation may be generated for the target performance data, the read operation may be placed in the serial queue, the read operation that responds may be executed according to the execution sequence of the data operations in the serial queue, and the read target data may be reported.
The serial queue ensures the execution sequence of each data operation task, is favorable for avoiding deadlock, and improves the reporting efficiency of performance data. According to the embodiment of the invention, the serial queue corresponds to a certain background thread, and the thread is not fixed, so that UI refreshing and user response of the main thread are not influenced.
As shown in fig. 2d, the generated data operations (write operation, read operation, delete operation) are placed in the serial queue, and then the corresponding operations are sequentially completed according to the execution sequence of each data operation in the serial queue.
Preferably, according to the embodiment of the present invention, data numbers are sequentially set for the stored performance data; determining target performance data from the stored performance data according to the maximum reported number, further comprising:
and determining target performance data from the stored performance data according to the maximum reporting quantity and the data number.
For example, reporting that the data is n, and determining the first n pieces of performance data as target performance data from the stored performance data according to the ascending sequence of the data numbers.
Step S210, determining target performance data from the stored performance data according to the execution sequence and the reading operation related to the data operation in the serial queue, and according to the maximum reporting number and/or the reporting frequency, and reporting the target performance data.
According to the embodiment of the invention, in the process of storing the data, the incremental data numbers are sequentially generated, and the numbers are used as the reporting priority of the stored data. Under the condition that the reporting conditions are met (maximum reporting quantity and reporting frequency), the target data are determined according to the maximum reporting quantity and the data number, data loss can be effectively avoided, the reporting accuracy of the data is improved, data reporting delay can be avoided, and user experience is improved.
Step S211, after the report processing of the target performance data is successful, delete the target performance data in the storage device according to the deletion operation in the serial queue.
And deleting the successfully reported performance data in the storage devices such as the database, the memory and the like in the storage device, so that the storage space is released.
Preferably, according to the embodiment of the present invention, the reporting processing is performed on the real-time reported performance data in an asynchronous manner; and reporting the stored performance data (including non-real-time reported performance data and real-time reported performance data with failed report) in a synchronous mode.
Through the arrangement, the efficient reporting of the performance data is ensured, and the performance data is not lost after the reporting fails.
According to the technical scheme of the embodiment of the invention, the data to be reported is obtained, and the data reporting mode is determined according to the reporting strategy; the data reporting mode comprises a real-time reporting mode and a non-real-time reporting mode, wherein the data to be reported corresponding to the real-time reporting mode is first data, and the data to be reported corresponding to the non-real-time reporting mode is second data; reporting the first data; the technical means of storing the second data and reporting the second data according to the reporting condition overcome the technical problems of low data reporting accuracy, low reporting efficiency and poor user experience in the existing data reporting method, and further achieve the purpose of flexibly setting a data reporting mode according to specific requirements, thereby not only ensuring the timeliness of data reporting, but also avoiding the loss condition of the data to be reported, improving the accuracy and efficiency of data reporting, and improving the technical effect of user experience.
Fig. 3 is a schematic diagram of main modules of an apparatus for reporting data according to an embodiment of the present invention; as shown in fig. 3, a device 300 for reporting data provided in the embodiment of the present invention mainly includes:
a data reporting mode determining module 301, configured to obtain data to be reported, and determine a data reporting mode according to a reporting policy; the data reporting mode comprises a real-time reporting mode and a non-real-time reporting mode, the data to be reported corresponding to the real-time reporting mode is first data, and the data to be reported corresponding to the non-real-time reporting mode is second data.
Specifically, according to the embodiment of the present invention, the reporting policy is preconfigured, the reporting policy may be written into the terminal in advance, and the terminal may also download the reporting policy from the server. The reporting policy indicates a reporting mode of specific data, for example, for performance data such as electric quantity and uploading/downloading speed, the reporting mode is non-real-time reporting; for the service condition data such as the server failure response code, the reporting mode is real-time reporting.
Further, according to the embodiment of the present invention, the data reporting mode determining module 301 is further configured to:
and setting the data type to be reported in real time and the data type to be reported in non-real time according to the reporting strategy, and further determining the corresponding data reporting mode according to the acquired data type.
According to a real-time specific embodiment of the present invention, an identification field may be added to the data, and a data reporting mode corresponding to the data may be determined according to a field value of the identification field.
It should be noted that the present invention mainly provides a flexible data reporting device in real time to improve the data reporting accuracy and efficiency, and therefore, the situation of data not to be reported is not mentioned, and in practical application, the data type not to be reported can be determined according to the service requirement, and for the data not to be reported, only persistent processing is needed.
Preferably, according to the embodiment of the present invention, after the data is acquired, the device 300 for reporting the data further includes an application parameter adding module, configured to add an application parameter to the data
Adding application parameters to the acquired data, wherein the application parameters comprise at least one of the following parameters: appID (application number), user pin (user identification code), application version number, environmental parameters (offline, pre-sent, online), etc.
Through the setting, the corresponding application product can be updated and maintained quickly aiming at the reported data subsequently, so that the user experience is improved.
The first reporting module 302 is configured to report the first data.
The reporting priority corresponding to the first data is highest, so that the network module is directly called to carry out reporting processing. Specifically, according to a specific implementation manner of the embodiment of the present invention, the first reporting module 302 is further configured to: and determining the current network state according to the network module, and carrying out batch reporting processing on the first data according to the current network state so as to further improve the reporting efficiency.
According to the embodiment of the invention, the first data can be reported by adopting a plurality of background threads, so that the reporting of a plurality of first data is realized, and the reporting efficiency is improved.
The second reporting module 303 is configured to store the second data, and perform reporting processing on the second data according to a reporting condition.
The reporting priority of the second data determined according to the reporting policy is low, real-time reporting processing is not needed, and the specific reporting time of the second data needs to be determined according to the reporting condition.
Specifically, according to the embodiment of the present invention, the second reporting module 303 is further configured to:
generating a write operation corresponding to the second data, and placing the write operation into a serial queue;
and storing the second data according to the execution sequence of the data operation and the write operation in the serial queue.
Through the arrangement, data operations (writing operation, reading operation and deleting operation) on the storage devices such as the database and the memory are placed in the serial queue, and the serial queue executes data operation tasks in sequence according to the sequence of the data operations placed in the serial queue, so that the situation of data loss is avoided.
Further, according to the embodiment of the present invention, the second reporting module 303 is further configured to:
generating a reading operation corresponding to the second data, and placing the reading operation into a serial queue;
and determining target second data from the stored second data according to the execution sequence and the reading operation of the data operation in the serial queue and the maximum reporting quantity and/or the reporting frequency, and reporting the target second data.
Specifically, the reporting condition may be set according to an actual situation, and may be adjusted according to a network type (such as a cellular network, a WIFI network, and the like) adopted for reporting, and a current network condition (bandwidth, whether network congestion exists, and the like). According to the embodiment of the present invention, it is also possible to determine the target second data that needs to be reported, generate a read operation for the target second data, place the read operation in the serial queue, execute the corresponding read operation according to the execution sequence of the data operations in the serial queue, and report the read target second data.
The serial queue ensures the execution sequence of each data operation task, is favorable for avoiding deadlock, and improves the reporting efficiency of data. According to the embodiment of the invention, the serial queue corresponds to a certain background thread, and the thread is not fixed, so that UI refreshing and user response of the main thread are not influenced.
Preferably, according to the embodiment of the present invention, data numbers are sequentially set for the stored second data; the second reporting module 303 is further configured to:
and determining target second data from the stored second data according to the maximum reporting quantity and the data number.
According to the embodiment of the invention, in the process of storing the data, the incremental data numbers are sequentially generated, and the numbers are used as the reporting priority of the stored data. Under the condition that the reporting conditions are met (maximum reporting quantity and reporting frequency), the target data are determined according to the maximum reporting quantity and the data number, data loss can be effectively avoided, the reporting accuracy of the data is improved, data reporting delay can be avoided, and user experience is improved.
Further, according to the embodiment of the present invention, the device for reporting data 300 further includes a deleting module, and after the step of generating the read operation corresponding to the target data, the deleting module is further configured to:
generating a deletion operation corresponding to the target second data, and placing the deletion operation after the reading operation in the serial queue;
and after the target second data is reported successfully, deleting the target second data from the stored second data according to the execution sequence and the deletion operation of the data operation in the serial queue.
And deleting the successfully reported data in the storage devices such as a database, a memory and the like in the storage device, so that the storage space is released.
Further, according to the embodiment of the present invention, the apparatus 300 for reporting data further includes a first data storage module, and for the first data that fails in reporting processing, the first data persistence module is configured to:
judging whether the processing failure times exceed a time threshold value or not;
if not, continuing to report the first data which fails in reporting;
if so, storing the first data which fails in the reporting processing.
Through the arrangement, a bottom-of-pocket strategy is provided for the reporting processing process of the first data, and under the condition that the reporting of the first data is still unsuccessful for a certain number of times (which may be that the availability of the current network or system resources is not high), the first data which is failed in the reporting processing is stored and is taken as the data of a non-real-time reporting mode to be subjected to subsequent reporting processing, so that the influence of repeated reporting retries on other normal functional services is avoided, and the overall efficiency of data reporting is improved.
Preferably, according to the embodiment of the present invention, the first data is reported in an asynchronous manner; and reporting the stored data in a synchronous mode.
Through the arrangement, the efficient reporting of the data is ensured, and the data is not lost after the reporting fails.
According to the technical scheme of the embodiment of the invention, the data to be reported is obtained, and the data reporting mode is determined according to the reporting strategy; the data reporting mode comprises a real-time reporting mode and a non-real-time reporting mode, wherein the data to be reported corresponding to the real-time reporting mode is first data, and the data to be reported corresponding to the non-real-time reporting mode is second data; reporting the first data; the technical means of storing the second data and reporting the second data according to the reporting condition overcome the technical problems of low data reporting accuracy, low reporting efficiency and poor user experience in the existing data reporting method, and further achieve the purpose of flexibly setting a data reporting mode according to specific requirements, thereby not only ensuring the timeliness of data reporting, but also avoiding the loss condition of the data to be reported, improving the accuracy and efficiency of data reporting, and improving the technical effect of user experience.
Fig. 4 shows an exemplary system architecture 400 of a reporting method or a reporting apparatus for data to which the present invention may be applied.
As shown in fig. 4, the system architecture 400 may include terminal devices 401, 402, 403, a network 404, and a server 405 (this architecture is merely an example, and the components included in a particular architecture may be adapted according to application specific circumstances). The network 404 serves as a medium for providing communication links between the terminal devices 401, 402, 403 and the server 405. Network 404 may include various types of connections, such as wire, wireless communication links, or fiber optic cables, to name a few.
A user may use terminal devices 401, 402, 403 to interact with a server 405 over a network 404 to receive or send messages or the like. The terminal devices 401, 402, 403 may have installed thereon various communication client applications, such as a data report application, a data collection application, a search application, an instant messaging tool, a mailbox client, social platform software, etc. (by way of example only).
The terminal devices 401, 402, 403 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop portable computers, desktop computers, and the like.
The server 405 may be a server providing various services, such as a background management server (for example only) providing support for shopping websites browsed by users using the terminal devices 401, 402, 403. The backend management server may analyze and perform other processing on the received data, and feed back a processing result (for example, the first data and the second data — just an example) to the terminal device.
It should be noted that the method for reporting data provided in the embodiment of the present invention is generally executed by the server 405, and accordingly, the device for reporting data is generally disposed in the server 405.
It should be understood that the number of terminal devices, networks, and servers in fig. 4 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
Referring now to FIG. 5, a block diagram of a computer system 500 suitable for use with a terminal device or server implementing an embodiment of the invention is shown. The terminal device or the server shown in fig. 5 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present invention.
As shown in fig. 5, the computer system 500 includes a Central Processing Unit (CPU)501 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM)502 or a program loaded from a storage section 508 into a Random Access Memory (RAM) 503. In the RAM 503, various programs and data necessary for the operation of the system 500 are also stored. The CPU 501, ROM 502, and RAM 503 are connected to each other via a bus 504. An input/output (I/O) interface 505 is also connected to bus 504.
The following components are connected to the I/O interface 505: an input portion 506 including a keyboard, a mouse, and the like; an output portion 507 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage portion 508 including a hard disk and the like; and a communication section 509 including a network interface card such as a LAN card, a modem, or the like. The communication section 509 performs communication processing via a network such as the internet. The driver 510 is also connected to the I/O interface 505 as necessary. A removable medium 511 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 510 as necessary, so that a computer program read out therefrom is mounted into the storage section 508 as necessary.
In particular, according to the embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 509, and/or installed from the removable medium 511. The computer program performs the above-described functions defined in the system of the present invention when executed by the Central Processing Unit (CPU) 501.
It should be noted that the computer readable medium shown in the present invention can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present invention, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present invention, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The modules described in the embodiments of the present invention may be implemented by software or hardware. The described modules may also be provided in a processor, which may be described as: a processor comprises a data reporting mode determining module, a first reporting module and a second reporting module. The names of the modules do not limit the modules themselves under certain conditions, for example, the data reporting mode determining module may also be described as "for acquiring data to be reported and determining a data reporting mode according to a reporting policy; the data reporting mode comprises a real-time reporting mode and a non-real-time reporting mode, wherein the data to be reported corresponding to the real-time reporting mode is a module of first data, and the data to be reported corresponding to the non-real-time reporting mode is second data.
As another aspect, the present invention also provides a computer-readable medium that may be contained in the apparatus described in the above embodiments; or may be separate and not incorporated into the device. The computer readable medium carries one or more programs which, when executed by a device, cause the device to comprise: acquiring data to be reported, and determining a data reporting mode according to a reporting strategy; the data reporting mode comprises a real-time reporting mode and a non-real-time reporting mode, wherein the data to be reported corresponding to the real-time reporting mode is first data, and the data to be reported corresponding to the non-real-time reporting mode is second data; reporting the first data; and storing the second data, and reporting the second data according to the reporting condition.
According to the technical scheme of the embodiment of the invention, the data to be reported is obtained, and the data reporting mode is determined according to the reporting strategy; the data reporting mode comprises a real-time reporting mode and a non-real-time reporting mode, wherein the data to be reported corresponding to the real-time reporting mode is first data, and the data to be reported corresponding to the non-real-time reporting mode is second data; reporting the first data; the technical means of storing the second data and reporting the second data according to the reporting condition overcome the technical problems of low data reporting accuracy, low reporting efficiency and poor user experience in the existing data reporting method, and further achieve the purpose of flexibly setting a data reporting mode according to specific requirements, thereby not only ensuring the timeliness of data reporting, but also avoiding the loss condition of the data to be reported, improving the accuracy and efficiency of data reporting, and improving the technical effect of user experience.
The above-described embodiments should not be construed as limiting the scope of the invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions can occur, depending on design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (12)

1. A method for reporting data is characterized by comprising the following steps:
acquiring data to be reported, and determining a data reporting mode according to a reporting strategy; the data reporting mode comprises a real-time reporting mode and a non-real-time reporting mode, wherein the data to be reported corresponding to the real-time reporting mode is first data, and the data to be reported corresponding to the non-real-time reporting mode is second data;
reporting the first data;
and storing the second data, and reporting the second data according to a reporting condition.
2. The method for reporting data according to claim 1, wherein the storing the second data includes:
generating a write operation corresponding to the second data, and placing the write operation into a serial queue;
and storing the second data according to the execution sequence of the data operation in the serial queue and the write operation.
3. The method according to claim 2, wherein the reporting condition includes a maximum reporting number and/or a reporting frequency; the reporting the second data according to the reporting condition includes:
generating a reading operation corresponding to the second data, and placing the reading operation into the serial queue;
and determining target second data from the stored second data according to the execution sequence and the reading operation of the data operation in the serial queue, the maximum reporting number and/or the reporting frequency, and reporting the target second data.
4. The method for reporting data as claimed in claim 3, wherein the method further comprises:
generating a deletion operation corresponding to the target second data, and placing the deletion operation after the reading operation in the serial queue;
and after the target second data is reported successfully, deleting the target second data from the stored second data according to the execution sequence of the data operations in the serial queue and the deletion operation.
5. A method for reporting data as claimed in claim 3, wherein the data numbers are sequentially set for the stored second data; the determining the target second data from the stored second data according to the maximum reporting number further includes:
and determining the target second data from the stored second data according to the maximum reporting quantity and the data number.
6. The method according to claim 1, wherein for the first data with failed reporting process, the method further comprises:
judging whether the processing failure times exceed a time threshold value or not;
if not, continuing to report the first data failed in the reporting process;
and if so, storing the first data which fails in the reporting processing.
7. The method according to claim 1, wherein the first data is reported asynchronously; and reporting the stored data in a synchronous mode.
8. An apparatus for reporting data, comprising:
the data reporting mode determining module is used for acquiring data to be reported and determining a data reporting mode according to a reporting strategy; the data reporting mode comprises a real-time reporting mode and a non-real-time reporting mode, wherein the data to be reported corresponding to the real-time reporting mode is first data, and the data to be reported corresponding to the non-real-time reporting mode is second data;
a first reporting module, configured to perform reporting processing on the first data;
and the second reporting module is used for storing the second data and reporting the second data according to a reporting condition.
9. The apparatus for reporting data of claim 8, wherein the second reporting module is further configured to:
generating a write operation corresponding to the second data, and placing the write operation into a serial queue;
and storing the second data according to the execution sequence of the data operation in the serial queue and the write operation.
10. The apparatus for reporting data of claim 9, wherein the second reporting module is further configured to:
generating a reading operation corresponding to the second data, and placing the reading operation into the serial queue;
and determining target second data from the stored second data according to the execution sequence and the reading operation of the data operation in the serial queue, the maximum reporting number and/or the reporting frequency, and reporting the target second data.
11. An electronic device, comprising:
one or more processors;
a storage device for storing one or more programs,
when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-7.
12. A computer-readable medium, on which a computer program is stored, which, when being executed by a processor, carries out the method according to any one of claims 1-7.
CN202110003276.5A 2021-01-04 2021-01-04 Data reporting method and device Pending CN113760177A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110003276.5A CN113760177A (en) 2021-01-04 2021-01-04 Data reporting method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110003276.5A CN113760177A (en) 2021-01-04 2021-01-04 Data reporting method and device

Publications (1)

Publication Number Publication Date
CN113760177A true CN113760177A (en) 2021-12-07

Family

ID=78786258

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110003276.5A Pending CN113760177A (en) 2021-01-04 2021-01-04 Data reporting method and device

Country Status (1)

Country Link
CN (1) CN113760177A (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107171856A (en) * 2017-06-21 2017-09-15 珠海信达九州科技有限公司 A kind of method that instant communicating system organizational structure quickly updates
CN107885590A (en) * 2017-11-30 2018-04-06 百度在线网络技术(北京)有限公司 Task processing method and device for smart machine
CN109618331A (en) * 2019-02-01 2019-04-12 深圳乐信软件技术有限公司 Data reporting method, device, equipment and storage medium
CN111510395A (en) * 2020-06-16 2020-08-07 腾讯科技(深圳)有限公司 Service message reporting method, device, equipment and medium
CN112000541A (en) * 2020-07-23 2020-11-27 招联消费金融有限公司 Buried point data reporting method and device, computer equipment and storage medium

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107171856A (en) * 2017-06-21 2017-09-15 珠海信达九州科技有限公司 A kind of method that instant communicating system organizational structure quickly updates
CN107885590A (en) * 2017-11-30 2018-04-06 百度在线网络技术(北京)有限公司 Task processing method and device for smart machine
CN109618331A (en) * 2019-02-01 2019-04-12 深圳乐信软件技术有限公司 Data reporting method, device, equipment and storage medium
CN111510395A (en) * 2020-06-16 2020-08-07 腾讯科技(深圳)有限公司 Service message reporting method, device, equipment and medium
CN112000541A (en) * 2020-07-23 2020-11-27 招联消费金融有限公司 Buried point data reporting method and device, computer equipment and storage medium

Similar Documents

Publication Publication Date Title
CN109905286B (en) Method and system for monitoring running state of equipment
CN112596920A (en) Message processing method and device, electronic equipment and storage medium
CN112398669A (en) Hadoop deployment method and device
CN113742389A (en) Service processing method and device
CN110795328A (en) Interface testing method and device
CN113760982A (en) Data processing method and device
CN113360815A (en) Request retry method and device
CN111767126A (en) System and method for distributed batch processing
CN112685481A (en) Data processing method and device
CN112667368A (en) Task data processing method and device
CN112395337A (en) Data export method and device
CN113760177A (en) Data reporting method and device
CN112688982B (en) User request processing method and device
CN109213815B (en) Method, device, server terminal and readable medium for controlling execution times
CN113779122A (en) Method and apparatus for exporting data
CN113535020A (en) Method, apparatus, device, medium and product for generating application icons
CN113742376A (en) Data synchronization method, first server and data synchronization system
CN113722193A (en) Method and device for detecting page abnormity
CN112463616A (en) Chaos testing method and device for Kubernetes container platform
CN110019671B (en) Method and system for processing real-time message
CN113760487A (en) Service processing method and device
CN112306371A (en) Method, apparatus and computer program product for storage management
CN113468053B (en) Application system testing method and device
CN113766437B (en) Short message sending method and device
CN112953810B (en) Processing method and device of network request

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination