CN112541101A - Subscription data pushing method and device, electronic equipment and computer storage medium - Google Patents

Subscription data pushing method and device, electronic equipment and computer storage medium Download PDF

Info

Publication number
CN112541101A
CN112541101A CN202011464802.XA CN202011464802A CN112541101A CN 112541101 A CN112541101 A CN 112541101A CN 202011464802 A CN202011464802 A CN 202011464802A CN 112541101 A CN112541101 A CN 112541101A
Authority
CN
China
Prior art keywords
period
time
data
sub
subscription
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202011464802.XA
Other languages
Chinese (zh)
Other versions
CN112541101B (en
Inventor
袁沅祥
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Wuhan Kuangshi Jinzhi Technology Co ltd
Beijing Megvii Technology Co Ltd
Original Assignee
Wuhan Kuangshi Jinzhi Technology Co ltd
Beijing Megvii Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Wuhan Kuangshi Jinzhi Technology Co ltd, Beijing Megvii Technology Co Ltd filed Critical Wuhan Kuangshi Jinzhi Technology Co ltd
Priority to CN202011464802.XA priority Critical patent/CN112541101B/en
Publication of CN112541101A publication Critical patent/CN112541101A/en
Application granted granted Critical
Publication of CN112541101B publication Critical patent/CN112541101B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90Details of database functions independent of the retrieved data types
    • G06F16/903Querying
    • G06F16/90335Query processing
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90Details of database functions independent of the retrieved data types
    • G06F16/903Querying
    • G06F16/9032Query formulation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90Details of database functions independent of the retrieved data types
    • G06F16/95Retrieval from the web
    • G06F16/957Browsing optimisation, e.g. caching or content distillation
    • G06F16/9574Browsing optimisation, e.g. caching or content distillation of access to content, e.g. by caching
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02DCLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • Mathematical Physics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention relates to a subscription data pushing method and device, electronic equipment and a computer storage medium, and belongs to the field of data pushing. The method comprises the following steps: acquiring a subscription period, wherein the subscription period comprises a starting time and a terminating time, and data with the generation time in the subscription period is taken as subscription data; pushing a task processing time and the starting time, or determining whether the subscription data comprises future data according to the pushing task processing time, the starting time and the terminating time; when the subscription data comprises future data, reading the future data contained in the subscription data from the cache and pushing the data from the earliest generation time of the future data contained in the subscription data; the future data is data of which the data generation time is later than the push task processing time, and the future data is written into the cache in real time after being generated. By the method, the number of times of querying the historical data storage unit is reduced, and therefore the query pressure of the historical data storage unit is reduced.

Description

Subscription data pushing method and device, electronic equipment and computer storage medium
Technical Field
The application belongs to the field of data pushing, and particularly relates to a subscription data pushing method and device, electronic equipment and a computer storage medium.
Background
The upper database may subscribe to the lower database for content (for example, may include video, image, information, and the like), and at this time, a service (sub, subscription push service) of the lower database is responsible for pushing the subscription data to the upper database.
In the prior art, in order to ensure the speed of acquiring the subscription content by the higher-level database, after the sub is started, the sub acquires the subscription task which is in charge of the sub and is under subscription, and then continuously polls the historical data storage unit of the electronic device in which the sub is located at a fixed time interval, so as to acquire the subscription data corresponding to the subscription task from the historical data storage unit and push the subscription data to the higher-level database. In the process, even if the subscription data corresponding to the subscription task is not stored in the historical data storage unit, the sub waits for the data to be stored in the historical data storage unit, and then reads the data from the historical data storage unit in a mode of continuously polling the historical data storage unit at a fixed time interval and pushes the data.
Therefore, in the prior art, the query of the historical data storage unit is frequent, so that the query pressure of the historical data storage unit is high.
Disclosure of Invention
In view of this, an object of the present application is to provide a method, an apparatus, an electronic device, and a computer storage medium for pushing subscription data, which are beneficial to reducing the number of queries to a historical data storage unit, so as to reduce the query pressure of the historical data storage unit.
The embodiment of the application is realized as follows:
in a first aspect, an embodiment of the present application provides a subscription data pushing method, where the method includes: acquiring a subscription period, wherein the subscription period comprises a starting time and a terminating time, and data with the generation time in the subscription period is taken as subscription data; determining whether the subscription data comprises future data according to a push task processing time and at least one of the starting time and the ending time; when the subscription data comprises future data, reading the future data comprised by the subscription data from a cache and pushing the future data from the earliest generation moment of the future data comprised by the subscription data; the future data is data of which the data generation time is later than the push task processing time, and the future data is written into the cache in real time after being generated. Compared with the scheme in the prior art that the future data is read from the historical data storage unit after being stored in the historical data storage unit, by adopting the scheme, the inquiry to the historical data storage unit when the sub reads the future data can be at least reduced, so that the inquiry frequency to the historical data storage unit can be reduced, and the inquiry pressure of the historical data storage unit is reduced.
With reference to the embodiment of the first aspect, in a possible implementation manner, the determining whether the subscription data includes future data includes: when the push task processing time is earlier than the starting time, determining that all the subscription data are future data; when the push task processing time is between the starting time and the ending time, determining that part of the subscription data is future data and part of the subscription data is historical data; when the processing time of the pushing task is later than the termination time, determining that all the subscription data are historical data; the historical data is data of which the generation time of the data is earlier than the processing time of the push task.
With reference to the embodiment of the first aspect, in one possible implementation manner, the method further includes: when the subscription data comprises historical data, reading the historical data contained in the subscription data from a historical data storage unit and pushing the historical data; the historical data is data of which the data generation time is earlier than the push task processing time; the historical data storage unit is a storage unit different from the cache.
With reference to the embodiment of the first aspect, in one possible implementation manner, the method further includes: dividing the subscription period into at least one sub-period according to the push task processing time; adding each sub-period to a period list; correspondingly, the reading and pushing future data included in the subscription data includes: starting a first process and/or a second process to read and push data generated in each sub-period in the period list; correspondingly, the reading and pushing the historical data included in the subscription data includes: and starting the first process and/or the second process to read and push the data generated in each sub-period in the period list.
With reference to the embodiment of the first aspect, in a possible implementation manner, each sub-period has a sub-start time and a sub-end time, and the dividing the subscription period into at least one sub-period according to the push task processing time includes: if the processing time of the push task does not fall into the subscription period, taking the subscription period as a sub-period, wherein the sub-starting time of the sub-period is the starting time, and the sub-terminating time of the sub-period is the terminating time; if the push task processing time falls into the subscription period, the subscription period is divided into a first sub-period and a second sub-period, the sub-starting time of the first sub-period is the starting time, the sub-ending time of the first sub-period is the push task processing time, the sub-starting time of the second sub-period is the push task processing time, and the sub-ending time of the second sub-period is the ending time.
With reference to the embodiment of the first aspect, in a possible implementation manner, the starting the first process and/or the second process to read and push data generated in each sub-period in the period list includes: and reading and pushing the data generated in the last sub-period in the period list through the second process until the pushing is finished.
With reference to the first aspect, in a possible implementation manner, if the data generated in the last sub-period is future data, the step of reading and pushing the data generated in the last sub-period in the period list starts from a sub-start time of the last sub-period, and reads the data from the cache; and/or if the data generated in the last sub-period is history data, reading the data generated in the last sub-period in the period list and pushing the data, and reading the data from the history data storage unit from the processing time of the pushing task.
With reference to the embodiment of the first aspect, in a possible implementation manner, the starting the first process and/or the second process to read and push data generated in each sub-period in the period list further includes: reading and pushing data generated in other sub-periods in the period list through the first process until pushing is finished; the other sub-periods are sub-periods in the period list except the last sub-period.
With reference to the embodiment of the first aspect, in a possible implementation manner, the reading and pushing data generated in other sub-periods in the period list includes: and reading the data generated in the other sub-periods from the historical data storage unit from the push task processing time and pushing.
With reference to the embodiment of the first aspect, in a possible implementation manner, the starting the first process and/or the second process to read and push data generated in each sub-period in the period list includes: determining a current time slice according to the sub-period to be pushed; and reading and pushing data generated in the current time slice, and updating the sub starting time of the sub time period to be pushed in the time period list to the slice ending time of the current time slice until the data in the sub time period to be pushed is pushed.
With reference to the embodiment of the first aspect, in a possible implementation manner, the determining a current time slice according to a sub-period to be pushed includes: and taking the sub-starting time of the sub-period to be pushed as the slicing starting time of the current time slice, and taking the smaller value of the sum of the slicing starting time of the current time slice and the preset duration granularity and the sub-ending time of the sub-period to be pushed as the slicing ending time of the current time slice.
With reference to the embodiment of the first aspect, in one possible implementation manner, the method further includes: when the abnormal restart exists, acquiring the sub-period included by the period list at the restart time; according to the restarting time, the sub-time intervals included in the time interval list at the restarting time are divided one by one; updating the time period list by using the divided sub time period and/or grand time period; and starting the first process and/or the second process to read and push data generated in each sub-period and/or grand period in the period list.
With reference to the embodiment of the first aspect, in a possible implementation manner, each grandchild period has a grandchild start time and a grandchild end time, and the dividing the child periods included in the period list at the restart time one by one includes: if the restarting time does not fall into the sub-time period to be divided, keeping the sub-time period to be divided unchanged; if the restarting time falls into the period of the children to be segmented, the period of the children to be segmented is segmented into a first grandchild period and a second grandchild period, the grandchild starting time of the first grandchild period is the child starting time of the period of the children to be segmented, the grandchild ending time of the first grandchild period is the restarting time, the grandchild starting time of the second grandchild period is the restarting time, and the grandchild ending time of the second grandchild period is the child ending time of the period of the children to be segmented.
In a second aspect, an embodiment of the present application provides an apparatus for subscribing to data, where the apparatus includes: the device comprises an acquisition module, a judgment module and a pushing module. The device comprises an acquisition module, a subscription period acquisition module and a subscription processing module, wherein the subscription period comprises a starting time and a terminating time, and data with a generation time in the subscription period is taken as subscription data; the judging module is used for determining whether the subscription data comprises future data according to the push task processing time and at least one of the starting time and the ending time; the pushing module is used for reading the future data included by the subscription data from a cache and pushing the future data from the earliest generation moment of the future data included by the subscription data when the subscription data includes the future data; the future data is data of which the data generation time is later than the push task processing time, and the future data is written into the cache in real time after being generated.
With reference to the second aspect, in a possible implementation manner, the determining module is configured to determine that all the subscription data are future data when the push task processing time is earlier than the start time; when the push task processing time is between the starting time and the ending time, determining that part of the subscription data is future data and part of the subscription data is historical data; when the processing time of the pushing task is later than the termination time, determining that all the subscription data are historical data; the historical data is data of which the generation time of the data is earlier than the processing time of the push task.
With reference to the second aspect, in a possible implementation manner, the pushing module is further configured to, when the subscription data includes history data, read the history data included in the subscription data from a history data storage unit and push the history data; the historical data is data of which the data generation time is earlier than the push task processing time; the historical data storage unit is a storage unit different from the cache.
With reference to the embodiment of the second aspect, in a possible implementation manner, the apparatus further includes a dividing module, configured to divide the subscription period into at least one sub-period according to the push task processing time; adding each sub-period to a period list; correspondingly, the pushing module is used for starting the first process and/or the second process to read and push the data generated in each sub-period in the period list; and the system is also used for starting the first process and/or the second process to read and push data generated in each sub-period in the period list.
With reference to the second aspect, in a possible implementation manner, each sub-period has a sub-start time and a sub-end time, and the dividing module is configured to take the subscription period as the sub-period if the push task processing time does not fall into the subscription period, where the sub-start time of the sub-period is the start time, and the sub-end time of the sub-period is the end time; if the push task processing time falls into the subscription period, the subscription period is divided into a first sub-period and a second sub-period, the sub-starting time of the first sub-period is the starting time, the sub-ending time of the first sub-period is the push task processing time, the sub-starting time of the second sub-period is the push task processing time, and the sub-ending time of the second sub-period is the ending time.
With reference to the second aspect embodiment, in a possible implementation manner, the pushing module is configured to read, by the second process, data generated in a last sub-period in the period list and perform pushing until pushing is completed.
With reference to the second aspect embodiment, in a possible implementation manner, if the data generated in the last sub-period is future data, when the pushing module reads and pushes the data generated in the last sub-period in the period list, the pushing module reads the data from the cache from a sub-start time of the last sub-period; and/or if the data generated in the last sub-period is history data, when the pushing module reads the data generated in the last sub-period in the period list and pushes the data, the pushing module reads the data from the history data storage unit from the processing time of the pushing task.
With reference to the second aspect, in a possible implementation manner, the pushing module is configured to read, by the first process, data generated in other sub-periods in the period list and perform pushing until pushing is completed; the other sub-periods are sub-periods in the period list except the last sub-period.
With reference to the second aspect, in a possible implementation manner, the pushing module is configured to read, from the historical data storage unit, data generated in the other sub-period from the push task processing time and push the data.
With reference to the second aspect, in a possible implementation manner, the pushing module is configured to determine a current time slice according to a sub-period to be pushed; and reading and pushing data generated in the current time slice, and updating the sub starting time of the sub time period to be pushed in the time period list to the slice ending time of the current time slice until the data in the sub time period to be pushed is pushed.
With reference to the second aspect, in a possible implementation manner, the pushing module is configured to use a sub-starting time of the sub-period to be pushed as a slice starting time of the current time slice, and use a smaller value of a sum of a slice starting time of the current time slice and a preset duration granularity and a sub-ending time of the sub-period to be pushed as a slice ending time of the current time slice.
With reference to the embodiment of the second aspect, in a possible implementation manner, the apparatus further includes an exception handling module, configured to, when it is determined that there is an abnormal restart, obtain a sub-period included in the period list at a restart time; according to the restarting time, the sub-time intervals included in the time interval list at the restarting time are divided one by one; updating the time period list by using the divided sub time period and/or grand time period; and starting the first process and/or the second process to read and push data generated in each sub-period and/or grand period in the period list.
With reference to the second aspect embodiment, in a possible implementation manner, each grandchild time period has a grandchild starting time and a grandchild ending time, and the exception handling module is configured to, if the restart time does not fall into a to-be-segmented child time period, keep the to-be-segmented child time period unchanged; and if the restarting time falls into the period of the to-be-divided children, dividing the period of the to-be-divided children into a first grandchild period and a second grandchild period, wherein a grandchild starting time of the first grandchild period is a child starting time of the to-be-divided children period, a grandchild ending time of the first grandchild period is the restarting time, a grandchild starting time of the second grandchild period is the restarting time, and a grandchild ending time of the second grandchild period is a child ending time of the to-be-divided children period.
In a third aspect, an embodiment of the present application further provides an electronic device, including: a memory and a processor, the memory and the processor connected; the memory is used for storing programs; the processor calls a program stored in the memory to perform the method of the first aspect embodiment and/or any possible implementation manner of the first aspect embodiment.
In a fourth aspect, the present application further provides a non-transitory computer-readable storage medium (hereinafter referred to as a computer storage medium), on which a computer program is stored, where the computer program is executed by a computer to perform the method in the foregoing first aspect and/or any possible implementation manner of the first aspect.
Additional features and advantages of the application will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by the practice of the embodiments of the application. The objectives and other advantages of the application may be realized and attained by the structure particularly pointed out in the written description and drawings.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings without creative efforts. The foregoing and other objects, features and advantages of the application will be apparent from the accompanying drawings. Like reference numerals refer to like parts throughout the drawings. The drawings are not intended to be to scale as practical, emphasis instead being placed upon illustrating the subject matter of the present application.
Fig. 1 shows a flowchart of a subscription data pushing method provided in an embodiment of the present application.
Fig. 2 shows a block diagram of a push apparatus for subscribing to data according to an embodiment of the present application.
Fig. 3 shows a schematic structural diagram of an electronic device provided in an embodiment of the present application.
Reference numbers: 100-an electronic device; 110-a processor; 120-a memory; 400-a push device of subscription data; 410-an obtaining module; 420-a judgment module; 430-push module.
Detailed Description
The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures. Meanwhile, relational terms such as "first," "second," and the like may be used solely in the description herein to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
Further, the term "and/or" in the present application is only one kind of association relationship describing the associated object, and means that three kinds of relationships may exist, for example, a and/or B may mean: a exists alone, A and B exist simultaneously, and B exists alone.
In addition, the defects (query of the historical data storage unit is more frequent, so that the query pressure of the historical data storage unit is greater) existing when the sub of the lower level database pushes the subscription data corresponding to the subscription task to the upper level database in the prior art are the results obtained after the applicant has practiced and studied carefully, and therefore, the discovery process of the defects and the solutions proposed in the embodiments of the present application in the following description for the defects should be considered as contributions of the applicant to the present application.
In order to solve the above problem, embodiments of the present application provide a method and an apparatus for pushing subscription data, an electronic device, and a computer storage medium, which are beneficial to reducing the number of times of querying a historical data storage unit, so as to reduce the query pressure of the historical data storage unit.
The technology can be realized by adopting corresponding software, hardware and a combination of software and hardware. The following describes embodiments of the present application in detail.
First, a concept related to the embodiments of the present application will be described.
A subscription period: refers to a certain time range, e.g., [ a, b ] (a < ═ b), [ a, b) (a < b). Where a is referred to as the start time of the subscription period and b is referred to as the end time of the subscription period. In general, the subscription data required to be obtained by the subscription task is data within a time range included in a subscription period formed from one time to another time, that is, the subscription data required to be obtained by the subscription task is for one subscription period.
Temporal resolution: refers to the minimum duration granularity of the time recorded when the data is stored to the historical data storage unit. For example, when the data is stored, the recorded time is accurate to seconds, and the time resolution is 1 second; if the time recorded is accurate to a few minutes, the time resolution is 1 minute.
Preset duration granularity (time increment) inc: when the sub acquires the subscription data in one subscription period from the history data storage unit or the cache in a plurality of times (time slices), the sub represents the time width between the earliest generation time and the latest generation time of the subscription data acquired in one time (one time slice) in the conventional case. For example, the subscription period is [10,20), inc ═ 5, the first time slice is [10, 15), and the second time slice is [15, 20). The subscription period converges: in the embodiment of the present application, for the subscription period [ a, b), the sub is always enabled to obtain the subscription data from the subscription period [ a, b) in multiple times from the starting time a. Each time the subscription data is acquired, the current time slice corresponds to one current time slice, and each current time slice has a slice start time and a slice end time. After the subscription data in the current time slice is pushed, updating the value of a in the subscription period [ a, b) to be the slice termination time of the current time slice, then updating the current time slice, and repeating the above operations. With the progress of the sub pushing task, the value of a in the subscription period [ a, b) is continuously updated to be larger, and the time range included in the subscription period is smaller and smaller until a is larger than or equal to b, and [ a, b) is 0. After this, [ a, b) is in a converged state.
The slice start time a of the first current time slice (P0) is the start time a of the subscription period [ a, b), the slice end time of the first current time slice (P0) is the sum of the slice start times a and inc, i.e., a + inc, and a + inc needs to be less than b, and if a + inc is greater than or equal to b, the slice end time of the first current time slice (P0) is b.
After the subscription data in the first current time slice (P0) is pushed, the slice start time a of the next current time slice (P1) is the slice end time a + inc of the previous current time slice (P0), and the slice end time of the next current time slice (P1) is the smaller value of the sum of the slice start time of the current time slice (P1) and the preset duration granularity inc and b, that is, the smaller value of a + inc + inc and b.
The following describes the convergence process of the subscription period [ a, b) as an example.
Assuming that there is a temporal resolution p, a preset duration granularity inc (the value of inc > is the value of p), the current time slice is [ a, a + inc). In the absence of an exception, when sub is started for the first time, let a be a, and the current time slice P0 be [ a, a + inc), that is, [ a, a + inc), sub acquires subscription data in the [ a, a + inc) time slice, and pushes the subscription data in the [ a, a + inc) time slice to an upper database. After the push is successful, the value of a in the subscription period [ a, b) is updated to be the slice ending time a + inc of the current time slice P0, the current time slice P1 is updated, the value of a is modified to be the starting time of the updated subscription period, i.e., the slice ending time a + inc of the last current time slice, and then the current time slice P0 is [ a + inc, a + inc + inc ]. sub acquires the subscription data in the [ a, a +2inc) time slice and pushes the subscription data in the [ a, a +2inc) time slice to a superior database. The process is then repeated until [ a, b) converges when a ≧ b. By the above example, in the whole process of acquiring data and pushing subscription data, a is a dynamic variable.
It is worth pointing out that the slice ending time of the next current time slice is the smaller value of the sum of the slice starting time of the current time slice and the preset duration granularity inc and b, that is, the smaller value of a +2inc and b, which means that the time range covered by the current time slice corresponding to the subscription data acquired each time sub cannot exceed the time range covered by the subscription duration corresponding to the most original subscription task, that is, a + inc < (b). If a + inc > b, it is necessary to truncate a + inc from the time point where the value of b is located, and modify the slice termination time of the current time slice from a + inc to b.
For example, assuming that the time resolution p is 1s, the preset duration granularity inc is 10, and the period corresponding to the data required for subscribing to the task is [10,36 ]. For this case, a is 10 and b is 36.
According to the above flow, after sub is started, a is 10, the current time slice [ a, a + inc) is [10,20), and the subscription data between [10,20) is acquired and pushed for the first time. When acquiring subscription data for the second time, modifying A to 20, setting the current time slice [ A, A + inc) to [20, 30), and acquiring and pushing subscription data between [20, 30) for the second time. When the subscription data is acquired for the third time, a is modified to 30, the current time slice [ a, a + inc) is [30, 40), the time span exceeds [10,36), and therefore, the current time slice [ a, a + inc) is modified to [ a, b), i.e., to [30, 36), and so subscription data between [30, 36) is acquired and pushed for the third time. When the subscription data is acquired for the fourth time, a is modified to 36, and at this time 36 ═ b, [10,36] converges, which indicates [10,36) that the subscription data of this subscription period completes pushing.
The following description will be made with respect to a push method provided in an embodiment of the present application.
The embodiment of the application provides a sub pushing method applied to an electronic device, which is used for pushing subscription data required by a subscription task in a local database of the electronic device to a superior database, wherein the superior database initiates the subscription task to the local database in advance, and the subscription task comprises a subscription period corresponding to the subscription data required to be acquired by the subscription task.
The steps involved will be described below with reference to fig. 1.
Step S110: and acquiring a subscription period, wherein the subscription period comprises a starting time and a terminating time, and data with the generation time in the subscription period is taken as subscription data.
Step S120: and determining whether the subscription data comprises future data according to a push task processing time and at least one of the starting time and the ending time.
Step S130: when the subscription data comprises future data, reading the future data comprised by the subscription data from a cache and pushing the future data from the earliest generation time of the future data comprised by the subscription data.
As mentioned above, the subscription data required to be obtained by the subscription task initiated by the upper level database is generally for one subscription period. Assume that the subscription period corresponding to the subscription data is [ a, b). Wherein, a is the starting time, and b is the ending time.
Furthermore, in the embodiments of the present application, the concept of caching is introduced. When new data is generated, the new data generated in real time is temporarily stored in the cache and then stored in the history data storage unit by the cache.
In this embodiment of the application, the state of the data covered by the subscription data may be determined according to the time sequence between the push task processing time now and the start time a and the end time b.
The state of the data can be divided into future data and historical data.
The future data is data whose generation time is later than the push task processing time now, that is, the generation time of the future data is after the push task processing time now, which indicates that the future data is about to be generated and temporarily stored in the cache.
The history data is data whose generation time is earlier than the push job processing time now, that is, the generation time of the history data is before the push job processing time now, and the history data is already stored in the history data storage unit. The history data storage unit is a storage unit that stores history data, and it is understood that the history data storage unit is not necessarily dedicated to storing history data, and may store other data. The history data storage unit is, for example, an elastic search (full text search engine, ES for short).
Optionally, when the push task processing time now is earlier than the starting time a of [ a, b "), all data in the representation subscription data are future data; when the processing time now of the pushing task is later than the termination time b of the [ a, b ], representing that all data in the subscription data are historical data; when the push task processing time now is between the starting time a and the ending time b, the data in the representation subscription data is partly history data and partly future data.
In the embodiment of the present application, the acquisition modes of the historical data and the future data included in the subscription data are different.
Wherein, for the history data portion, sub polls the history data storage unit a plurality of times to acquire history data according to a plurality of time slices in a manner of making the subscription period [ a, b) converge as mentioned above.
For the future data portion, sub reads the future data from the cache in real time in multiple times according to multiple time slices in the manner mentioned above to converge the subscription period [ a, b).
The following describes a process of acquiring and pushing subscription data for three cases, i.e., all future data, all historical data, part historical data, and part future data, of the data in the subscription data.
When all the data in the subscription data is future data (i.e. a > -now), the sub waits for the coming of the earliest generation time a of the future data, and reads and pushes the future data from the cache in real time from the earliest generation time (a time) of the future data until the subscription period [ a, b) converges.
When the data in the subscription data are all history data (i.e. b < ═ now), the sub successively acquires and pushes the history data in the period [ a, b) from the push task processing time now, until the period [ a, b) converges.
When the data part in the subscription data is history data and the part is future data (a < now > b), in order to reduce the number of times of querying the history data storage unit by the sub, in an optional implementation manner, the sub may read and push the subscription data as the future data (i.e., data generated during the time of [ now, b) from the cache from the now moment when the subscription data corresponding to the time period [ now, b) is temporarily stored in the cache, and after the time period [ now, b) converges, the sub reads and pushes the subscription data as the history data (i.e., data corresponding to the time period [ a, now ]) from the history data storage unit.
That is to say, in the embodiment of the present application, when there is future data in the subscription data generated in the subscription period [ a, b), sub directly reads the cache to obtain the future data and pushes the future data. Compared with the scheme that future data are read from the historical data storage unit after the future data are stored in the historical data storage unit in the prior art, in the embodiment of the application, the future data are read without inquiring the historical data storage unit, so that the inquiry frequency of the historical data storage unit can be at least reduced, and the inquiry pressure of the historical data storage unit is reduced.
The historical data storage unit is used as a data access channel, and when the query pressure is high, the reading and writing of data are affected, so that the query times of the historical data storage unit are reduced, the query pressure of the historical data storage unit is reduced, the data can be accessed into and out of the historical data storage unit, and the data pushing speed is increased.
It is noted that in the above embodiments, sub is based on a process, such as the first process or the second process, obtaining and pushing subscription data.
In another alternative embodiment, to improve the pushing efficiency of the subscription data, the sub may start two processes (e.g., a first process and a second process) to acquire and push the subscription data at the same time.
The first process is responsible for acquiring and pushing historical data generated in the subscription period [ a, b), and the second process is responsible for acquiring and pushing future data generated in the subscription period [ a, b); or the second process is responsible for acquiring and pushing the historical data generated by the subscription period [ a, b), and the first process is responsible for acquiring and pushing the future data generated by the subscription period [ a, b).
In this embodiment, the subscription period [ a, b) may be divided into at least one sub-period according to the push task processing time now. The data types of the data generated in each sub-period are the same, that is, the data generated in the same sub-period is either historical data or future data.
Specifically, each sub-period includes a corresponding sub-start time and a sub-end time. When the sub divides the subscription period [ a, b), the subscription period [ a, b) is divided according to the magnitude sequence among the now, a and b by taking the push task processing time now as a boundary, so as to obtain at least one sub-period.
When a > -now or b < -now indicates that now does not fall within the time range covered by the subscription period [ a, b), at this time, the sub-period obtained by dividing the subscription period [ a, b) is one, that is, the subscription period [ a, b) itself, and accordingly, the sub-start time of the obtained sub-period is a and the sub-end time is b.
When a < now < b, it indicates that now falls within the time range covered by the subscription period [ a, b), and then the subscription period [ a, b) is divided into a first sub-period [ a, now) and a second sub-period [ now, b) with the now non as a boundary.
The sub-start time of the first sub-period [ a, now) is the start time a of the subscription period [ a, b), the sub-end time of the first sub-period [ a, b) is the push task processing time now, the sub-start time of the second sub-period [ now, b) is the push task processing time now, and the sub-end time of the second sub-period [ now, b) is the end time b of the subscription period [ a, b).
After at least one sub-period is obtained, each sub-period is added into the period list L, and the first process and/or the second process acquire and push data generated in each sub-period in the period list L according to the type of the corresponding processing object.
In one embodiment, it is assumed that a first process is responsible for retrieving and pushing historical data and a second process is responsible for retrieving and pushing future data.
When the subscription period [ a, b) is divided, and the obtained sub-period is still [ a, b), L { [ a, b) }. If all the data generated in the [ a, b) is historical data, the first process is responsible for acquiring and pushing the historical data generated in the sub-period [ a, b), and the second process is in an idle state at the moment; if the data generated in [ a, b) is all future data, the second process is responsible for acquiring and pushing the future data generated in the sub-period [ a, b), and the first process is in an idle state.
When the subscription period [ a, b) is divided into sub-periods [ a, now) and [ now, b), L { [ a, now) { [ now), [ now, b) }. At this time, the first process is responsible for acquiring (inquiring the historical data storage unit) and pushing the historical data generated in the sub-period [ a, now), and the second process is responsible for acquiring (inquiring the cache) and pushing the future data generated in the sub-period [ now, b).
The two processes do not interfere with each other, and when the two sub-periods converge, the pushing of the subscription data in the subscription period [ a, b) is completed. Because the two processes can work simultaneously, the data pushing efficiency can be improved.
In addition, in an optional implementation manner, when the sub is started, on the premise that whether the subscription data includes future data is not distinguished, the sub may directly start a corresponding process to process the subscription data, which is as follows. For example, the subscription data generated in the last sub-period in the period list L may be read and pushed by the second process, and the subscription data generated in the sub-periods other than the last sub-period in the period list L may be read by the first process.
The sub still divides the subscription period into at least one sub-period in the manner of dividing the subscription period [ a, b) as described above, and adds the obtained sub-period to the period list L. When a plurality of sub-periods are obtained through segmentation, the sub-periods are sequentially added into the period list L according to the sequence of the time periods covered by the sub-periods. Thus, for the case of a > -now or b < -now, L { [ a, b) }, for the case of a < now < b, [ a, now) precedes [ now, b), [ L { [ a, now), [ now, b) }.
After the period list L is obtained, when the period list L comprises more than 2 sub-periods, the sub-starts a first process and a second process at the same time, then reads subscription data generated in the last sub-period in the period list L through the second process and pushes the subscription data until the pushing is finished, and reads subscription data generated in other sub-periods except the last sub-period in the period list L through the first process and pushes the subscription data until the pushing is finished.
Corresponding to the case of a > new or b < > new, the period list L contains only one sub-period, and sub reads and pushes subscription data corresponding to the last sub-period [ a, b) (i.e., [ a, b) is the last sub-period since there is only [ a, b ]) in L through the second process. At this time, the first process is in an idle waiting state.
It is worth pointing out that when a > now, since the subscription data is not generated yet, the second process needs to wait for the time a to arrive before reading and pushing the corresponding data from the cache at the time a. When a < now, since the subscription data is already generated, the second process may read and push the corresponding data from the history data storage unit from the time of now.
Corresponding to the situation of a < now < b >, the sub reads and pushes the subscription data corresponding to the last sub-period [ now, b) in the L through the second process, and reads and pushes the subscription data corresponding to other sub-periods [ a, now) in the L through the first process.
In this process, no matter what the size relationship between a, b and now is, for the first process and the second process, when the subscription data required to be read is history data, corresponding data is obtained from the history data storage unit, and when the subscription data required to be read is future data, corresponding data is obtained from the cache.
Referring to the description introduced above when the subscription period [ a, b) converges, when the subscription data in the subscription period [ a, b) is pushed, the sub is always made to acquire the subscription data from the subscription period [ a, b) in multiple times from the starting time a. Each time the subscription data is acquired, the current time slice corresponds to one current time slice, and each current time slice has a slice start time and a slice end time. After the subscription data in the current time slice is pushed, updating the value of a in the subscription period [ a, b) to be the slice termination time of the current time slice, then updating the current time slice, and repeating the above operations until the subscription period [ a, b) converges.
When at least one sub-period is obtained by dividing the subscription period [ a, b), in an optional implementation manner, for each sub-period, the subscription data in each sub-period may also be pushed in a similar manner as described above, and specifically, the following process may be referred to:
when each process processes the sub-period to be pushed which is responsible for processing, the current time slice is determined firstly, then the data generated in the current time slice is read and pushed, and after the data in the current time slice is pushed, the sub-starting time of the sub-period to be pushed in the period list is updated to the slice ending time of the current time slice until the data in the sub-period to be pushed is pushed. At this time, the sub-period to be pushed is deleted from the period list L.
The sub-starting time of the sub-period to be pushed is used as the slicing starting time of the current time slice, and the smaller value of the sum of the slicing starting time of the current time slice and the preset duration granularity inc and the sub-ending time of the sub-period to be pushed is used as the slicing ending time of the current time slice.
For the latest current time slice P0, the slice start time is the sub-start time of the sub-period to be pushed, and the slice end time is the smaller value of the sum of the slice start time of the current time slice P0 and the preset duration granularity inc and the sub-end time of the sub-period to be pushed.
Assuming that the sum of the slice start time and inc is less than the sub-termination time of the sub-period to be pushed, it indicates that the data in the sub-period to be pushed is not pushed completely after the data in the current time slice P0 is pushed completely. In this case, after the data in the current time slice P0 is pushed, the sub-start time of the to-be-pushed sub-period is updated to be the slice end time of the current time slice P0, the slice start time of the next current time slice P1 is updated to be the sub-start time of the current to-be-pushed sub-period (i.e., the slice end time of the previous current time slice P0), and the slice end time of the next current time slice P1 is updated to be the smaller value of the sum of the slice start time and inc of P and the sub-end time of the to-be-pushed sub-period.
Assuming that the sum of the slice start time and inc is greater than or equal to the sub-termination time of the sub-period to be pushed, it indicates that the data in the sub-period to be pushed is pushed after the data in the current time slice P0 is pushed.
As can be seen from the above process, when the first process and/or the second process acquires and pushes the subscription data in the sub-period for processing by the first process and/or the second process along with the processing time now of the push task, the sub may update the sub start time included in each sub-period in the L in real time according to the processing progress of the first process and/or the second process.
For example, the subscription period corresponding to the subscription data is [10,100 ], the time resolution is 1s, the preset duration granularity is 1, and the push task processing time now is 20, so that a is 10, b is 100, now is 20, p is 1, and inc is 1.
When sub starts at the push task processing time v, according to the size relationship between now, a and b, [10,100) is divided into [10,20) and [20,100), and L { [10,20), [20,100 } is obtained.
Starting a first process and a second process, and assuming that the second process is responsible for processing the subscription data corresponding to the sub-period [20,100), the first process is responsible for processing the subscription data corresponding to the sub-period [10, 20).
Assuming that after 5S of time, the first process acquires and pushes 8S of subscription data from the history data storage unit, and the second process acquires and pushes 5S of subscription data from the cache, at this time, the sub start time included in each sub period in L is updated in real time: the sub start time of [10,20) is updated from 10 to 18, and the sub start time of [20,100) is updated from 20 to 25. Up to this point, the updated L is L { [18,20), [25,100 }.
It is worth noting that there is no correlation between the speed of time lapse and the speed at which processes read and push data from the history data storage unit or from the cache, so that after 5S has elapsed here, the second process may push 8S of subscription data from the history data storage unit.
The sub push subscription data is introduced when the sub is in a normal state. When an exception occurs to a sub, the sub may stop the push service. When the sub is restarted again, the subscription data which is determined to be future data may be stored in the historical data storage unit as time passes, so that the subscription data is pushed in the previous mode, and the data loss problem exists.
In order to solve the above problem, when sub determines that the current start is an abnormal restart due to an abnormality, a restart time now' corresponding to the restart is acquired.
When sub is started, L may be queried first. If L is empty, it may be characterized that the current start is the first start, and the subscription period needs to be divided according to the above-mentioned flow, and added to L, to complete initialization of L, and to perform subsequent pushing. In addition, if L is empty, it may also represent that the subscription data has been pushed.
If L is not empty, the starting is represented as abnormal starting, and in order to avoid data loss, the existing sub-time period in L needs to be divided.
The restart time now' may be used as a boundary to re-segment the sub-periods existing in the current L one by one to obtain the grandchild period. As for the principle of dividing the sub period, the principle of dividing the subscription period is similar to that described above.
For each sub-period to be divided, if the restart time now' does not fall into the sub-period to be divided, the sub-period to be divided is kept unchanged.
If the restart time now' falls into the sub-period to be divided, dividing the sub-period to be divided into a first grandchild period and a second grandchild period. The grandchild starting time of the first grandchild period is the child starting time of the child period to be segmented, and the grandchild ending time of the first grandchild period is the restart time now'; the grandchild starting time of the second grandchild period is the restart time now', and the grandchild ending time of the second grandchild period is the child ending time of the child period to be divided.
After the respective sub periods within L are divided, the period list L is updated with the divided sub periods and/or grandchild periods. And then, reading and pushing subscription data corresponding to the last period (which may be a child period or a grandchild period) in the updated L by the sub through the second process, and reading and pushing subscription data corresponding to other child periods or grandchild periods except the last child period or grandchild period in the updated L through the first process.
Still, the above example is explained, that is, the subscription period corresponding to the subscription data is [10,100 ], a is 10, b is 100, now is 20, p is 1, and inc is 1.
As mentioned above, after 5S of time, the updated L is L { [18,20), [25,100 }. At this time, sub is abnormal and restarted after 10S.
sub, after the restart, a restart time is obtained, that is, a restart time now' is 20+5 (normal push over 5S) +10 is 35, the current L is obtained { [18,20), [25,100 }, and it is determined that L is not null. At this time, it is determined that the restart is abnormal, and the sub-segments each sub-period existing in the current L one by one again according to the now', a sub start time and a sub end time (corresponding to the sub-period [18,20 ], 18 and 20, respectively, and 25 and 100, respectively, corresponding to the sub-period [25, 100)), of each sub-period included in the current L, to obtain updated sub-periods and grandchild periods [18,20), [25,35), [35,100), and to obtain updated L { [18,20), [25,35), [35,100 }.
And sub starting a first process and a second process, acquiring and pushing subscription data corresponding to the last grandchild period [35,100) in the current L through the second process, and acquiring and pushing subscription data corresponding to other child periods [18,20) and other grandchild periods [25,35) in the current L through the first process.
In this process, regardless of the size relationship between the child start time and the child end time of each child period and the size relationship between the child end time and the now ', and regardless of the size relationship between the grandchild start time and the grandchild end time of each grandchild period and the now', for the first process and the second process, when the subscription data required to be read is history data, corresponding data is obtained from the history data storage unit, and when the subscription data required to be read is future data, corresponding data is obtained from the cache.
In the above example, the subscription data corresponding to the [25,35) sub-period, which should be read from the cache by the second process, is saved from the cache to the history data storage unit due to abnormal starting, so as to avoid loss of the subscription data. Therefore, after the sub restart, the part of the data needs to be read from the history data storage unit.
Of course, in the process of acquiring and pushing the subscription data corresponding to the respective sub-period and/or the grandchild period by the first process and the second process, the start time of each sub-period and the start time of a child or a grandchild included in the grandchild period in the current L may be updated in real time.
The scheme can still continue to push after the sub is restarted, and has robustness.
The method for pushing the subscription data includes the steps of obtaining a time period corresponding to the subscription data, wherein the time period includes a starting time and a terminating time; when the subscription data is determined to include future data according to the push task processing time and the starting time or according to the push task processing time, the starting time and the ending time, the future data is read from the cache in real time from the earliest generation time of the future data and pushed. Compared with the scheme that the future data are read from the historical data storage unit after the future data are stored in the historical data storage unit in the prior art, the sub in the scheme does not need to query the historical data storage unit when the future data are read, so that the query times of the historical data storage unit can be reduced, and the query pressure of the historical data storage unit is reduced.
Corresponding to fig. 1, please refer to fig. 2, an embodiment of the present application further provides a data subscription pushing apparatus 400, where the data subscription pushing apparatus 400 may include: an obtaining module 410, a judging module 420 and a pushing module 430.
The obtaining module 410 is configured to obtain a subscription period, where the subscription period includes a start time and an end time, and data of a generation time in the subscription period is used as subscription data;
a determining module 420, configured to determine whether the subscription data includes future data according to a push task processing time and at least one of the start time and the end time;
a pushing module 430, configured to, when the subscription data includes future data, start from an earliest generation time of the future data included in the subscription data, read the future data included in the subscription data from a cache and push the future data;
the future data is data of which the data generation time is later than the push task processing time, and the future data is written into the cache in real time after being generated.
In a possible implementation manner, the determining module 420 is configured to determine that all the subscription data are future data when the push task processing time is earlier than the starting time; when the push task processing time is between the starting time and the ending time, determining that part of the subscription data is future data and part of the subscription data is historical data; when the processing time of the pushing task is later than the termination time, determining that all the subscription data are historical data; the historical data is data of which the generation time of the data is earlier than the processing time of the push task.
In a possible implementation manner, the pushing module 430 is further configured to, when the subscription data includes history data, read the history data included in the subscription data from a history data storage unit and push the history data; the historical data is data of which the data generation time is earlier than the push task processing time; the historical data storage unit is a storage unit different from the cache.
In a possible implementation manner, the apparatus further includes a dividing module, configured to divide the subscription period into at least one sub-period according to the push task processing time; adding each sub-period to a period list; correspondingly, the pushing module 430 is configured to start the first process and/or the second process to read and push data generated in each sub-period in the period list; and the system is also used for starting the first process and/or the second process to read and push data generated in each sub-period in the period list.
In a possible implementation manner, each sub-period has a sub-start time and a sub-end time, and the partitioning module is configured to take the subscription period as the sub-period if the push task processing time does not fall into the subscription period, where the sub-start time of the sub-period is the start time, and the sub-end time of the sub-period is the end time; if the push task processing time falls into the subscription period, the subscription period is divided into a first sub-period and a second sub-period, the sub-starting time of the first sub-period is the starting time, the sub-ending time of the first sub-period is the push task processing time, the sub-starting time of the second sub-period is the push task processing time, and the sub-ending time of the second sub-period is the ending time.
In a possible implementation manner, the pushing module 430 is configured to read, by the second process, data generated in the last sub-period in the period list and push the data until pushing is completed.
In a possible implementation manner, if the data generated in the last sub-period is future data, when the pushing module reads and pushes the data generated in the last sub-period in the period list, the data is read from the cache from the sub-start time of the last sub-period; and/or if the data generated in the last sub-period is history data, when the pushing module reads the data generated in the last sub-period in the period list and pushes the data, the pushing module reads the data from the history data storage unit from the processing time of the pushing task.
In a possible implementation manner, the pushing module 430 is configured to read and push data generated in other sub-periods in the period list through the first process until pushing is completed; the other sub-periods are sub-periods in the period list except the last sub-period.
In a possible implementation manner, the pushing module 430 is configured to read, from the history data storage unit, data generated in the other sub-period from the time of the pushing task processing, and push the data.
In a possible implementation manner, the pushing module 430 is configured to determine a current time slice according to a sub-period to be pushed; and reading and pushing data generated in the current time slice, and updating the sub starting time of the sub time period to be pushed in the time period list to the slice ending time of the current time slice until the data in the sub time period to be pushed is pushed.
In a possible implementation manner, the pushing module 430 is configured to use the sub-start time of the sub-period to be pushed as the slice start time of the current time slice, and use a smaller value of a sum of the slice start time of the current time slice and a preset duration granularity and the sub-end time of the sub-period to be pushed as the slice end time of the current time slice.
In a possible implementation manner, the apparatus further includes an exception handling module, configured to, when it is determined that there is an abnormal restart, obtain a sub-period included in the period list at a restart time; according to the restarting time, the sub-time intervals included in the time interval list at the restarting time are divided one by one; updating the time period list by using the divided sub time period and/or grand time period; and starting the first process and/or the second process to read and push data generated in each sub-period and/or grand period in the period list.
In a possible implementation manner, each grandchild time interval has a grandchild starting time and a grandchild ending time, and the exception handling module is configured to, if the restart time does not fall into the child time interval to be segmented, keep the child time interval to be segmented unchanged; and if the restarting time falls into the period of the to-be-divided children, dividing the period of the to-be-divided children into a first grandchild period and a second grandchild period, wherein a grandchild starting time of the first grandchild period is a child starting time of the to-be-divided children period, a grandchild ending time of the first grandchild period is the restarting time, a grandchild starting time of the second grandchild period is the restarting time, and a grandchild ending time of the second grandchild period is a child ending time of the to-be-divided children period.
The implementation principle and the resulting technical effect of the push device 400 for subscription data provided in the embodiment of the present application are the same as those of the foregoing method embodiment, and for a brief description, reference may be made to corresponding contents in the foregoing method embodiment for a part of the embodiment of the device that is not mentioned.
In addition, an embodiment of the present application further provides a computer storage medium, where a computer program is stored on the computer storage medium, and when the computer program is executed by a computer, the steps included in the push method as described above are executed.
In addition, referring to fig. 3, an electronic device 100 for implementing the push method and the push apparatus described above is further provided in an embodiment of the present application, and includes a processor 110 and a memory 120.
It should be noted that the components and structure of electronic device 100 shown in FIG. 3 are exemplary only, and not limiting, and electronic device 100 may have other components and structures as desired.
The processor 110, memory 120, and other components that may be present in the electronic device 100 are electrically connected to each other, directly or indirectly, to enable the transfer or interaction of data. For example, the processor 110, the memory 120, and other components that may be present may be electrically coupled to each other via one or more communication buses or signal lines.
Part of the memory space in the memory 120 serves as a cache as mentioned in the embodiments of the present application. In addition, another part of the storage space in the memory 120 is also used for storing the database, the sub, and the program, for example, a program corresponding to the foregoing push method or the foregoing push apparatus is stored. Optionally, when the push device is stored in the memory 120, the push device includes at least one software function module that can be stored in the memory 120 in the form of software or firmware (firmware).
Optionally, the software function module included in the pushing device may also be solidified in an Operating System (OS) of the electronic device 100.
The processor 110 is adapted to execute an executable module stored in the memory 120, such as a software function module or a computer program comprised by the push device. When the processor 110 receives the execution instruction, it may execute the computer program, for example, to perform: acquiring a subscription period, wherein the subscription period comprises a starting time and a terminating time, and data with the generation time in the subscription period is taken as subscription data; determining whether the subscription data comprises future data according to a push task processing time and at least one of the starting time and the ending time; when the subscription data comprises future data, reading the future data comprised by the subscription data from a cache and pushing the future data from the earliest generation moment of the future data comprised by the subscription data; the future data is data of which the data generation time is later than the push task processing time, and the future data is written into the cache in real time after being generated.
Of course, the method disclosed in any of the embodiments of the present application can be applied to the processor 110, or implemented by the processor 110.
In summary, in the method, the apparatus, the electronic device, and the computer storage medium for pushing subscription data provided in the embodiments of the present invention, a time period corresponding to the subscription data is first obtained, where the time period includes a start time and an end time; when the subscription data is determined to include future data according to the push task processing time and the starting time or according to the push task processing time, the starting time and the ending time, the future data is read from the cache in real time from the earliest generation time of the future data and pushed. Compared with the scheme that the future data are read from the historical data storage unit after the future data are stored in the historical data storage unit in the prior art, the sub in the scheme does not need to query the historical data storage unit when the future data are read, so that the query times of the historical data storage unit can be reduced, and the query pressure of the historical data storage unit is reduced.
It should be noted that, in the present specification, the embodiments are all described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments may be referred to each other.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method can be implemented in other ways. The apparatus embodiments described above are merely illustrative, and for example, the flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
In addition, functional modules in the embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.
The functions, if implemented in the form of software functional modules and sold or used as a stand-alone product, may be stored in a computer storage medium. Based on such understanding, the technical solution of the present application or portions thereof that substantially contribute to the prior art may be embodied in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a notebook computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application.

Claims (16)

1. A method for pushing subscription data, the method comprising:
acquiring a subscription period, wherein the subscription period comprises a starting time and a terminating time, and data with the generation time in the subscription period is taken as subscription data;
determining whether the subscription data comprises future data according to a push task processing time and at least one of the starting time and the ending time;
when the subscription data comprises future data, reading the future data comprised by the subscription data from a cache and pushing the future data from the earliest generation moment of the future data comprised by the subscription data;
the future data is data of which the data generation time is later than the push task processing time, and the future data is written into the cache in real time after being generated.
2. The method of claim 1, wherein the determining whether the subscription data includes future data comprises:
when the push task processing time is earlier than the starting time, determining that all the subscription data are future data;
when the push task processing time is between the starting time and the ending time, determining that part of the subscription data is future data and part of the subscription data is historical data;
when the processing time of the pushing task is later than the termination time, determining that all the subscription data are historical data;
the historical data is data of which the generation time of the data is earlier than the processing time of the push task.
3. The method according to claim 1 or 2, characterized in that the method further comprises:
when the subscription data comprises historical data, reading the historical data contained in the subscription data from a historical data storage unit and pushing the historical data;
the historical data is data of which the data generation time is earlier than the push task processing time; the historical data storage unit is a storage unit different from the cache.
4. A method according to claim 2 or 3, characterized in that the method further comprises:
dividing the subscription period into at least one sub-period according to the push task processing time;
adding each sub-period to a period list;
correspondingly, the reading and pushing future data included in the subscription data includes:
starting a first process and/or a second process to read and push data generated in each sub-period in the period list;
correspondingly, the reading and pushing the historical data included in the subscription data includes:
and starting the first process and/or the second process to read and push the data generated in each sub-period in the period list.
5. The method of claim 4, wherein each sub-period has a sub-start time and a sub-end time, and wherein dividing the subscription period into at least one sub-period according to the push task processing time comprises:
if the processing time of the push task does not fall into the subscription period, taking the subscription period as a sub-period, wherein the sub-starting time of the sub-period is the starting time, and the sub-terminating time of the sub-period is the terminating time;
if the push task processing time falls into the subscription period, the subscription period is divided into a first sub-period and a second sub-period, the sub-starting time of the first sub-period is the starting time, the sub-ending time of the first sub-period is the push task processing time, the sub-starting time of the second sub-period is the push task processing time, and the sub-ending time of the second sub-period is the ending time.
6. The method of claim 5, wherein the initiating the first process and/or the second process to read and push data generated in each sub-period in the period list comprises:
and reading and pushing the data generated in the last sub-period in the period list through the second process until the pushing is finished.
7. The method of claim 6,
if the data generated in the last sub-period is future data, reading the data generated in the last sub-period in the period list and pushing the data, and reading the data from the cache from the sub-starting time of the last sub-period; and/or the presence of a gas in the gas,
and if the data generated in the last sub-period is historical data, reading the data generated in the last sub-period in the period list and pushing the data, and reading the data from a historical data storage unit from the moment of processing the pushing task.
8. The method according to claim 6 or 7, wherein the starting the first process and/or the second process reads and pushes data generated in each sub-period in the period list, further comprising:
reading and pushing data generated in other sub-periods in the period list through the first process until pushing is finished;
the other sub-periods are sub-periods in the period list except the last sub-period.
9. The method of claim 8, wherein reading and pushing data generated in other subintervals in the list of epochs comprises:
and reading the data generated in the other sub-periods from the historical data storage unit from the push task processing time and pushing.
10. The method according to any one of claims 5 to 9, wherein the initiating the first process and/or the second process to read and push data generated in each sub-period in the period list comprises:
determining a current time slice according to the sub-period to be pushed;
and reading and pushing data generated in the current time slice, and updating the sub starting time of the sub time period to be pushed in the time period list to the slice ending time of the current time slice until the data in the sub time period to be pushed is pushed.
11. The method of claim 10, wherein determining the current time slice according to the sub-period to be pushed comprises:
and taking the sub-starting time of the sub-period to be pushed as the slicing starting time of the current time slice, and taking the smaller value of the sum of the slicing starting time of the current time slice and the preset duration granularity and the sub-ending time of the sub-period to be pushed as the slicing ending time of the current time slice.
12. The method according to any one of claims 4-11, further comprising:
when the abnormal restart exists, acquiring the sub-period included by the period list at the restart time;
according to the restarting time, the sub-time intervals included in the time interval list at the restarting time are divided one by one;
updating the time period list by using the divided sub time period and/or grand time period;
and starting the first process and/or the second process to read and push data generated in each sub-period and/or grand period in the period list.
13. The method as claimed in claim 12, wherein each grandchild period has a grandchild start time and a grandchild end time, and the dividing the child periods included in the period list at the restart time comprises:
if the restarting time does not fall into the sub-time period to be divided, keeping the sub-time period to be divided unchanged;
if the restarting time falls into the period of the children to be segmented, the period of the children to be segmented is segmented into a first grandchild period and a second grandchild period, the grandchild starting time of the first grandchild period is the child starting time of the period of the children to be segmented, the grandchild ending time of the first grandchild period is the restarting time, the grandchild starting time of the second grandchild period is the restarting time, and the grandchild ending time of the second grandchild period is the child ending time of the period of the children to be segmented.
14. A device for pushing subscription data, the device comprising:
the device comprises an acquisition module, a subscription period acquisition module and a subscription processing module, wherein the subscription period comprises a starting time and a terminating time, and data with a generation time in the subscription period is taken as subscription data;
the judging module is used for determining whether the subscription data comprises future data according to the push task processing time and at least one of the starting time and the ending time;
the pushing module is used for reading the future data included by the subscription data from a cache and pushing the future data from the earliest generation moment of the future data included by the subscription data when the subscription data includes the future data;
the future data is data of which the data generation time is later than the push task processing time, and the future data is written into the cache in real time after being generated.
15. An electronic device, comprising: a memory and a processor, the memory and the processor connected;
the memory is used for storing programs;
the processor calls a program stored in the memory to perform the method of any of claims 1-13.
16. A computer storage medium, having stored thereon a computer program which, when executed by a computer, performs the method of any one of claims 1-13.
CN202011464802.XA 2020-12-11 2020-12-11 Subscription data pushing method and device, electronic equipment and computer storage medium Active CN112541101B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011464802.XA CN112541101B (en) 2020-12-11 2020-12-11 Subscription data pushing method and device, electronic equipment and computer storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011464802.XA CN112541101B (en) 2020-12-11 2020-12-11 Subscription data pushing method and device, electronic equipment and computer storage medium

Publications (2)

Publication Number Publication Date
CN112541101A true CN112541101A (en) 2021-03-23
CN112541101B CN112541101B (en) 2023-04-21

Family

ID=75018576

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011464802.XA Active CN112541101B (en) 2020-12-11 2020-12-11 Subscription data pushing method and device, electronic equipment and computer storage medium

Country Status (1)

Country Link
CN (1) CN112541101B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113254762A (en) * 2021-04-26 2021-08-13 北京旷视科技有限公司 Subscription data pushing method and device, electronic equipment and storage medium

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130031197A1 (en) * 2011-07-27 2013-01-31 Verizon Patent And Licensing Inc. Internet cache subscription for wireless mobile users
CN103886038A (en) * 2014-03-10 2014-06-25 中标软件有限公司 Data caching method and device
CN105471604A (en) * 2014-08-19 2016-04-06 中兴通讯股份有限公司 Custom event notification method, server and system
CN106612301A (en) * 2015-10-22 2017-05-03 招商局国际信息技术有限公司 Updated data pushing method and updated data pushing device
CN106934001A (en) * 2017-03-03 2017-07-07 广州天源迪科信息技术有限公司 Distributed quick inventory inquiry system and method
CN107528894A (en) * 2017-08-16 2017-12-29 郑州云海信息技术有限公司 A kind of storage system Real time data acquisition method and platform
CN111225069A (en) * 2020-03-13 2020-06-02 浙江书香荷马文化有限公司 Distributed market data processing system and method

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130031197A1 (en) * 2011-07-27 2013-01-31 Verizon Patent And Licensing Inc. Internet cache subscription for wireless mobile users
CN103886038A (en) * 2014-03-10 2014-06-25 中标软件有限公司 Data caching method and device
CN105471604A (en) * 2014-08-19 2016-04-06 中兴通讯股份有限公司 Custom event notification method, server and system
CN106612301A (en) * 2015-10-22 2017-05-03 招商局国际信息技术有限公司 Updated data pushing method and updated data pushing device
CN106934001A (en) * 2017-03-03 2017-07-07 广州天源迪科信息技术有限公司 Distributed quick inventory inquiry system and method
CN107528894A (en) * 2017-08-16 2017-12-29 郑州云海信息技术有限公司 A kind of storage system Real time data acquisition method and platform
CN111225069A (en) * 2020-03-13 2020-06-02 浙江书香荷马文化有限公司 Distributed market data processing system and method

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
吴霖等: "Redis在订阅推送系统中的应用", 《电脑知识与技术》 *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113254762A (en) * 2021-04-26 2021-08-13 北京旷视科技有限公司 Subscription data pushing method and device, electronic equipment and storage medium

Also Published As

Publication number Publication date
CN112541101B (en) 2023-04-21

Similar Documents

Publication Publication Date Title
CN107133234B (en) Method, device and system for updating cache data
US9438672B2 (en) Method for client specific database change propagation
US20220335011A1 (en) System and Method for Eliminating Full Rescan Synchronizations on Service Restarts
CN105488208A (en) Data search method and device
CN110688382A (en) Data storage query method and device, computer equipment and storage medium
CN111177159B (en) Data processing system and method and data updating equipment
CN112541101A (en) Subscription data pushing method and device, electronic equipment and computer storage medium
CN107181773A (en) Data storage and data managing method, the equipment of distributed memory system
EP2680168B1 (en) Columnwise range K-nearest neighbours search queries
CN114036226A (en) Data synchronization method, device, equipment and storage medium
CN113239012B (en) Database migration method and device, electronic equipment and storage medium
CN113254762A (en) Subscription data pushing method and device, electronic equipment and storage medium
CN111124650B (en) Stream data processing method and device
CN110457279B (en) Data offline scanning method and device, server and readable storage medium
CN108021562B (en) Disk storage method and device applied to distributed file system and distributed file system
CN113742420B (en) Data synchronization method and device
CN111681295A (en) Tile map slicing and service method and system based on super-resolution image reconstruction
WO2021163856A1 (en) Content pushing method and apparatus, and server and storage medium
CN108920722B (en) Parameter configuration method and device and computer storage medium
CN108345431B (en) Data reading method and device
CN113157716A (en) Data processing method, device, equipment and medium
CN112287649B (en) Report dynamic generation method, device and medium
CN117493276B (en) Reading method and device for Ceph file, server and storage medium
CN110297598B (en) Data synchronization method and storage system
CN116932589A (en) Metadata access method, device and system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant