CN108228624A - Data acquisition and update method, piece list service system and piece single system - Google Patents

Data acquisition and update method, piece list service system and piece single system Download PDF

Info

Publication number
CN108228624A
CN108228624A CN201611159578.7A CN201611159578A CN108228624A CN 108228624 A CN108228624 A CN 108228624A CN 201611159578 A CN201611159578 A CN 201611159578A CN 108228624 A CN108228624 A CN 108228624A
Authority
CN
China
Prior art keywords
data
piece
storage unit
updated
forms
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201611159578.7A
Other languages
Chinese (zh)
Inventor
田旭东
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Excellent Friends Bullock Media Development Co
Original Assignee
Shenzhen Excellent Friends Bullock Media Development Co
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Excellent Friends Bullock Media Development Co filed Critical Shenzhen Excellent Friends Bullock Media Development Co
Priority to CN201611159578.7A priority Critical patent/CN108228624A/en
Publication of CN108228624A publication Critical patent/CN108228624A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/23Updating
    • G06F16/2379Updates performed during online database operations; commit processing

Abstract

The invention discloses a kind of data acquisition and update method, piece list service system and piece single systems, belong to media asset management technical field, wherein, piece list service system includes:Piece single pair external tapping module, for sending access request to data access interface;Data access interface module is used to obtain target patch forms data from data buffer storage unit based on access request and return;And/or for updating the data the piece forms data in buffer unit based on the piece forms data new information received;Data monitoring unit updates messages to data access interface module for sending piece forms data in the piece forms data update for monitoring piece list storage unit;Wherein, described forms data new information includes:The unique attribute mark of piece forms data is updated in piece list storage unit.The embodiment of the present invention uses multi-level buffer scheme, increases data source, reduces the pressure that matchmaker provides database, increases data throughout, improves data loading and shows speed.

Description

Data acquisition and update method, piece list service system and piece single system
Technical field
The invention belongs to media asset management technical field, more particularly to a kind of data acquisition and update method, piece list service system System and piece single system.
Background technology
Matchmaker provide data, such as video frequency program list loading and show the experience that speed largely affects user, and Matchmaker provides the loading of data and shows the acquisition modes that speed directly depends on the handling capacity of server end and matchmaker provides data.
Current technology scheme has been done part and has been optimized, when matchmaker provides number to improve the loading of matchmaker's money data and show speed It changes according to library, piece single database is synchronized to by synchronization program, piece list interface server is obtained from independent piece single database Access evidence;This print list interface server does not access matchmaker directly and provides database, so as to alleviate the pressure that matchmaker provides database.
In the implementation of the present invention, still at least there are the following problems for inventor's discovery current technology scheme:By Database is remained stored in the key data source of piece list interface server, is accessed in the case that peak value or pressure are king-sized When, exist and access the problems such as data time is long, seriously affect systematic entirety energy and handling capacity, and then matchmaker is caused to provide adding for data It carries and to show speed slow.
Invention content
The purpose of the embodiment of the present invention is to provide a kind of data acquisition and update method, piece list service system and piece monosystem System;At least using multi-level buffer scheme, and a kind of side for accelerating data loading and showing speed is provided according to multi-level buffer scheme Method, and by newest data update to cachings at different levels by way of real-time update so that client can get newest Data, and due to use multi-level buffer scheme, increase data source, data acquiring mode is also with regard to corresponding increase, and then subtract The pressure of few server master library, increases data throughout, so as to improve data loading and show speed.
According to an aspect of the present invention, the embodiment provides a kind of single service system, including:
Piece single pair external tapping module, for when receiving target patch forms data inquiry request, being sent out to data access interface Send access request;
For being based on the access request, mesh is obtained from least one data buffer storage unit for data access interface module Standard film forms data, and the target patch forms data of acquisition is returned via described single pair external tapping module;And/or it is used for Based on the piece forms data in the piece forms data new information update at least data buffer storage unit received;
Piece list storage unit, for storing piece forms data;
Data monitoring unit, in the piece forms data update for monitoring piece list storage unit, sending piece forms data more New information is to the data access interface module;Wherein, described forms data new information includes:By more in piece list storage unit The unique attribute mark of new film forms data.
Optionally, at least one data buffer storage unit includes:Local cache unit;
The data access interface module for being based on access request, reads the data in local cache unit, if described There are the target patch forms datas for local cache unit, then obtain the target patch forms data from the local cache unit;With/ Or,
For being based on unique attribute mark, judge to whether there is in the local cache unit and be updated piece with described The corresponding stale data of forms data, if in the presence of the stale data is deleted;Wherein, the stale data is described by more Data of the new film forms data before not being updated.
Optionally, at least one data buffer storage unit further includes:Independent buffer unit;
The data access interface module, for the local cache unit be not present the target patch forms data when, Read the data in independent buffer unit, if the independent buffer unit there are the target patch forms data, from the independence Buffer unit obtains the target patch forms data;And/or
For being based on unique attribute mark, judge in the independent buffer unit with the presence or absence of being updated piece with described The corresponding stale data of forms data, if in the presence of the stale data is deleted.
Optionally, at least one data buffer storage unit includes:Independent buffer unit;
The data access interface module for being based on access request, reads the data in independent buffer unit, if described There are the target patch forms datas for independent buffer unit, then obtain the target patch forms data from the independent buffer unit;With/ Or,
For being based on unique attribute mark, judge in the independent buffer unit with the presence or absence of being updated piece with described The corresponding stale data of forms data, if in the presence of the stale data is deleted;Wherein, the stale data is described by more Data of the new film forms data before not being updated.
Optionally, the data access interface module is additionally operable to singular there is no the target patch in data buffer storage unit According to when, read the piece forms data in piece list storage unit, and the target patch forms data is obtained from described single storage unit.
Optionally, the data access interface module is additionally operable to obtaining the target patch from the independent buffer unit After forms data, the target patch forms data is updated in local cache unit.
Optionally, the data access interface module is additionally operable to obtain the target patch in described single storage unit After forms data, the target patch forms data is updated at least one data buffer storage unit.
Optionally, the data access interface module is additionally operable to identify based on preset rules and the unique attribute, judgement It is described to be updated whether piece forms data is frequently-used data;
If the piece forms data that is updated for frequently-used data, reads the data in described single storage unit with by described in It is updated piece forms data and is updated at least one data buffer storage unit;
Wherein, the preset rules are:Judge and the history quilt for being updated the corresponding stale data of piece forms data Whether access times are more than predetermined value, if more than predetermined value, then judge described in be updated piece forms data as frequently-used data;With/ Or be updated the unique attribute of piece forms data described in judging and identify whether to be present in preset frequently-used data table, if in the presence of, Piece forms data is updated described in judgement as frequently-used data.
Optionally, the data access interface module, be additionally operable to by it is described be updated piece forms data be updated to it is described extremely Before a few data buffer storage unit,
Read the memory configurations of at least one data buffer storage unit;With,
For judging whether the currently stored amount of at least one data buffer storage unit is not small based on the memory configurations In default memory;
If currently stored amount is not less than default memory, based on each odd number at least one data buffer storage unit According to history be accessed record, deleting in scheduled duration not accessed piece forms data or history, to be accessed number minimum Longest forms data of the time interval of time interval current time that piece forms data or last time are accessed is so that described at least one The currently stored amount of a data buffer storage unit is less than default memory;
If currently stored amount is less than default memory, the piece forms data that is updated is updated at least one data Buffer unit.
Another aspect according to embodiments of the present invention, the embodiment provides a kind of single system, including:More than The piece list service system.
Optionally, described single system further includes:
Matchmaker provides database, for storing piece single data source;
Whether data update module ETL, the piece single data source that database is provided for monitoring the matchmaker update;With, for When matchmaker provides the piece single data source update of database, piece single data source synchronized update after being updated to piece list storage unit.
Another aspect according to embodiments of the present invention, the embodiment provides a kind of forms data acquisition methods, Applied to piece list service system, including:
Piece single pair external tapping module sends to data access interface and accesses when receiving target patch forms data inquiry request Request;
Data access interface module is based on the access request, and target patch list is obtained from least one data buffer storage unit Data, and the target patch forms data of acquisition is returned via the outbound data interface.
Optionally, at least one data buffer storage unit includes:Local cache unit;
The data access interface module obtains the step of target patch forms data from least one data buffer storage unit and wraps It includes:
The data access interface module is based on access request, the data in local cache unit is read, if the local There are the target patch forms datas for buffer unit, then obtain the target patch forms data from the local cache unit.
Optionally, at least one data buffer storage unit further includes:Independent buffer unit;
The data access interface module obtains the step of target patch forms data from least one data buffer storage unit and wraps It includes:
If the target patch forms data is not present in the local cache unit, the data access interface module reads independent Data in buffer unit, if the independent buffer unit there are the target patch forms data, from the independent buffer unit Obtain the target patch forms data.
Optionally, at least one data buffer storage unit includes:Independent buffer unit;
The data access interface module obtains the step of target patch forms data from least one data buffer storage unit and wraps It includes:
Data access interface module reads the data in independent buffer unit based on access request, if the independent caching is single There are the target patch forms datas for member, then obtain the target patch forms data from the independent buffer unit.
Optionally, the method further includes:
If the target patch forms data is not present in data buffer storage unit, the data access interface module reads the storage of piece list Piece forms data in unit, and the target patch forms data is obtained from described single storage unit, wherein, described single storage Unit at least stores the target patch forms data.
Optionally, the data access interface module from the independent buffer unit obtain the target patch forms data it Afterwards, it further includes:The target patch forms data is updated in local cache unit.
Optionally, in the data access interface module target patch forms data is obtained from described single storage unit Later, it further includes:The target patch forms data is updated at least one data buffer storage unit.
Another aspect according to embodiments of the present invention, the embodiment provides a kind of forms data update method, Applied to piece list service system, including:
Whether the piece forms data of data monitoring unit detection lug list storage unit storage updates, and is monitoring the storage of piece list During the piece forms data update of unit, send piece forms data and update messages to the data access interface module;Wherein, described list Data update message includes:The unique attribute mark of piece forms data is updated in piece list storage unit;
Data access interface module updates the piece list in an at least data buffer storage unit based on piece forms data new information Data.
Optionally, the data access interface module is based on a piece forms data new information update at least data buffer storage list The step of piece forms data in member, includes:
Data access interface module based on the unique attribute identify, judge be at least one data buffer storage unit No presence is updated the corresponding stale data of piece forms data with described, if in the presence of the stale data is deleted;Wherein, institute It states stale data and is updated data of the piece forms data before not being updated to be described.
Optionally, the data access interface module is based on a piece forms data new information update at least data buffer storage list The also step of piece forms data in member includes:
Data access interface module is based on preset rules and the unique attribute identifies, and piece forms data is updated described in judgement Whether it is frequently-used data;
If the piece forms data that is updated for frequently-used data, reads the data in described single storage unit with by described in It is updated piece forms data and is updated at least one data buffer storage unit;
Wherein, the preset rules are:Judge and the history quilt for being updated the corresponding stale data of piece forms data Whether access times are more than predetermined value, if more than predetermined value, then judge described in be updated piece forms data as frequently-used data;With/ Or be updated the unique attribute of piece forms data described in judging and identify whether to be present in preset frequently-used data table, if in the presence of, Piece forms data is updated described in judgement as frequently-used data.
Optionally, by it is described be updated the step of piece forms data is updated at least one data buffer storage unit it Before, it further includes:
Data access interface module reads the memory configurations of at least one data buffer storage unit;
Data access interface module is used to judge working as at least one data buffer storage unit based on the memory configurations Whether preceding amount of storage is not less than default memory;
If currently stored amount is not less than default memory, data access interface module is based at least one data buffer storage The history of each forms data in unit is accessed record, deletes the piece forms data or history not being accessed in scheduled duration Longest forms data of the time interval of time interval current time that the minimum piece forms data of accessed number or last time are accessed So that the currently stored amount of at least one data buffer storage unit is less than default memory;
If currently stored amount is less than default memory, the piece forms data that is updated is updated at least one data Buffer unit.
Data acquisition provided in an embodiment of the present invention and update method, piece list service system and piece single system, using multistage Buffering scheme, and the method accelerated data loading and show speed is provided according to multi-level buffer scheme, and pass through real-time update Mode by newest data update to cachings at different levels so that client can get newest data, and due to using more Grade buffering scheme, increases data source, data acquiring mode also just corresponding increase, so reduce matchmaker provide the pressure of database with The pressure of data buffer storage unit, increases data throughout, so as to improve data loading and show speed.
Description of the drawings
Fig. 1 is the structure chart for the piece list service system that the first system embodiment of the present invention provides;
Fig. 2 is the structure chart for the piece list service system that second system embodiment of the present invention provides;
Fig. 3 is the structure chart for the piece list service system that third system embodiment of the present invention provides;
Fig. 4 is the structure chart of piece list service system that the 4th system embodiment of the invention provides;
Fig. 5 is the structure chart for the piece single system that the embodiment of the present invention provides;
Fig. 6 is the flow chart of piece forms data acquisition methods that the first method embodiment of the present invention provides;
Fig. 7 is the flow chart of piece forms data acquisition methods that the second method embodiment of the present invention provides;
Fig. 8 is the flow chart of piece forms data acquisition methods that the third method embodiment of the present invention provides;
Fig. 9 is the flow chart of piece forms data acquisition methods that the fourth method embodiment of the present invention provides.
Reference sign:1- piece single pair external tapping modules;2- data access interface modules;3- data buffer storage units;31- Local cache unit;32- independence buffer units;4- piece list storage units;5- data monitoring units;6- client modules;7- numbers According to update module ETL;8- matchmaker provides database.
Specific embodiment
Understand to make the object, technical solutions and advantages of the present invention clearer, With reference to embodiment and join According to attached drawing, the present invention is described in more detail.It should be understood that these descriptions are merely illustrative, and it is not intended to limit this hair Bright range.In addition, in the following description, the description to known features and technology is omitted, to avoid this is unnecessarily obscured The concept of invention.
The embodiment of the present invention provides a kind of data acquisition and update method, piece list service system and piece single system wherein, more The embodiment of the present invention, one application scenarios of following offers are understood well, but this application scenarios is not to the embodiment of the present invention It limits:When client modules get target patch forms data inquiry instruction input by user, client modules are to piece single system Target patch forms data inquiry request is sent, it is client modules that the piece single pair external tapping module 1 of piece single system, which is used as piece single system, The data-interface of offer is docked with client modules, and receives the target patch forms data inquiry request of client modules, at this time Piece single system can carry out target data inquiry according to this target data inquiry request.And the purpose of the present invention is exactly to improve piece monosystem The data of system load and show speed namely data query speed, while client modules is enabled to get newest number According to.
Referring to Fig. 1, Fig. 1 is the structure chart for the piece list service system that the first system embodiment of the present invention provides.
As shown in Figure 1, the first system embodiment of the present invention provides a kind of single service system, including:
Piece single pair external tapping module 1, for when receiving target patch forms data inquiry request, being sent out to data access interface Send access request;
Data access interface module 2 for being based on the access request, is obtained from least one data buffer storage unit 3 Target patch forms data, and the target patch forms data of acquisition is returned via described single pair external tapping module 1;And/or it uses In based on the piece forms data in the piece forms data new information received a update at least data buffer storage unit 3;
Piece list storage unit 4, for storing piece forms data;
Data monitoring unit 5, in the piece forms data update for monitoring piece list storage unit 4, sending piece forms data Update messages to the data access interface module 2;Wherein, described forms data new information includes:In piece list storage unit 4 It is updated the unique attribute mark of piece forms data.
Wherein, the data buffer storage unit 3 includes at least one of local cache unit 31, independent buffer unit 32, But this is not limited only to, other storage units or database can also be included;The number of data buffer storage unit can be according to different The number of business demand and system data amount is configured.Wherein, it should be noted that the access speed of local cache unit 31 It is most fast, is independent buffer unit 32 in fact, data access interface module 2 is carrying out data read access based on access request When, it can be preferential to access in data buffer storage unit 3 access speed most according to the speed of the access speed of data buffer storage unit 3 Fast buffer unit, that is, local cache unit 31.
Wherein, the effect of data access interface module 2 is carried for the access of different database or data buffer storage unit Unified interface is supplied;Different database or data buffer storage unit are able to access that by this data access interface module 2, because This allow piece single pair external tapping module 1 do not have to concern Back end data source, it is only necessary to pay close attention to the specification and safety of interface , different database or data buffer storage unit are being accessed so as to also simplify piece list service system to a certain extent When switching call operation, simplify the flow that switching is called, can also improve the number of piece list service system to a certain extent According to loading and showing speed, data access interface module 2 can be that ODBC (Open Database Connectivity) is opened Put Database Connectivity;Can also be that JDBC (Java Data Base Connectivity) i.e. java databases connect, either Other database interfaces.Wherein, front end buffer unit is further included as a kind of optionally embodiment, piece list service system, wherein, Front end buffer unit saves the history target patch list that client modules are returned by piece single pair external tapping module 1 in scheduled duration Data;The history target patch forms data is for this target patch forms data inquiry request, that is front end Data in buffer unit are last time or the preceding data that client modules are returned to by piece single pair external tapping module 1 several times, Before piece single pair external tapping module 1 receives target patch forms data inquiry request, front end buffer unit can receive target patch odd number During according to inquiry request, this target patch forms data asked can be whether there is in query history target patch forms data, if it does, Then target patch forms data is returned.Otherwise, then by target patch forms data inquiry request it is sent to piece single pair external tapping module 1.Its In, a length of scheduled duration when the data of front end buffer unit store, when reaching scheduled duration, front end buffer unit can delete it The data of storage, wherein, scheduled duration is preset value, can be adjusted.
Referring to Fig. 2, Fig. 2 is the structure chart for the piece list service system that second system embodiment of the present invention provides;
As shown in Figure 2, on the basis of the first system embodiment of the present invention, second system embodiment of the invention provides A kind of single service system, wherein, at least one data buffer storage unit 3 includes:Local cache unit 31;
The data access interface module 2 for being based on access request, reads the data in local cache unit 31, if There are the target patch forms datas for the local cache unit 31, then obtain the target patch list from the local cache unit 31 Data;And/or
For being based on unique attribute mark, judge in the local cache unit 31 with the presence or absence of being updated with described The corresponding stale data of piece forms data, if in the presence of the stale data is deleted;Wherein, the stale data is the quilt More data of the new film forms data before not being updated.Wherein, it should be noted that data access interface module 2 includes at least one It is a, before piece single pair external tapping module 1 sends access request to data access interface module 2, it can also be visited according to each data Ask that the number of the access request being currently received of interface module 2 carries out selection judgement, so that 1 energy of piece single pair external tapping module A minimum data access interface module 2 for current access request is enough linked into, by such mode, also can effectively be alleviated Data access interface module 2, and then can also improve the data loading of piece list service system and show speed namely data query Speed.
Referring to Fig. 3, Fig. 3 is the structure chart for the piece list service system that third system embodiment of the present invention provides;
As shown in Figure 3, on the basis of second system embodiment of the present invention, third system embodiment of the invention provides A kind of single service system, wherein, at least one data buffer storage unit 3 further includes:Independent buffer unit 32;Wherein, it needs It is noted that by the way that multi-level buffer unit is set to increase data source, data acquiring mode is also with regard to corresponding increase, and then subtract Few matchmaker provides the pressure of database, increases data throughout, so as to improve data loading and show speed.
The data access interface module 2, for the target patch forms data to be not present in the local cache unit 31 When, read the data in independent buffer unit 32, if the independent buffer unit 32 there are the target patch forms data, from institute It states independent buffer unit 32 and obtains the target patch forms data;And/or
For being based on unique attribute mark, judge to whether there is in the independent buffer unit 32 and be updated with described The corresponding stale data of piece forms data, if in the presence of the stale data is deleted.
By deleting stale data, enable to next client modules 6 that can be obtained in request target piece forms data Newest target patch forms data is prevented after being updated due to the target patch forms data in piece list storage unit 4, and data buffer storage Also there is the corresponding expired target patch forms data not timely updated also of target patch forms data with being updated in unit, lead It is not newest forms data to cause the piece forms data that client modules 6 obtain, and then improves the data updating capability of piece single system, Prevent the newer hysteresis quality of piece forms data.
Referring to Fig. 4, Fig. 4 is the structure chart of piece list service system that the 4th system embodiment of the invention provides,
As shown in Figure 4, on the basis of the first system embodiment of the present invention, the 4th system embodiment of the invention provides A kind of single service system, wherein, at least one data buffer storage unit 3 includes:Independent buffer unit 32;
The data access interface module 2 for being based on access request, reads the data in independent buffer unit 32, if There are the target patch forms datas for the independent buffer unit 32, then obtain the target patch list from the independent buffer unit 32 Data;And/or
For being based on unique attribute mark, judge to whether there is in the independent buffer unit 32 and be updated with described The corresponding stale data of piece forms data, if in the presence of the stale data is deleted;Wherein, the stale data is the quilt More data of the new film forms data before not being updated.
On the basis of first embodiment of the invention to fourth embodiment, the present invention also provides one kind to be preferably carried out Example, and in particular to a kind of single service system, wherein, the data access interface module 2 is additionally operable in data buffer storage unit 3 There is no during the target patch forms data, the piece forms data in piece list storage unit 4 is read, and from described single storage unit 4 It is middle to obtain the target patch forms data.
The present invention also provides a kind of preferred embodiments, and in particular to a kind of single service system, wherein, the data Access interface module 2 is additionally operable to after the target patch forms data is obtained from the independent buffer unit 32, by the target patch Forms data is updated in local cache unit 31.
The present invention also provides a kind of preferred embodiments, and in particular to a kind of single service system, wherein, the data Access interface module 2 is additionally operable to after obtaining the target patch forms data in described single storage unit 4, by the target patch Forms data is updated at least one data buffer storage unit 3.
The present invention also provides a kind of preferred embodiments, and in particular to a kind of single service system, wherein, the data Access interface module 2 is additionally operable to be identified based on preset rules and the unique attribute, whether piece forms data is updated described in judgement For frequently-used data;
If the piece forms data that is updated for frequently-used data, reads the data in described single storage unit 4 with by institute It states and is updated piece forms data and is updated at least one data buffer storage unit 3;
Wherein, the preset rules are:Judge and the history quilt for being updated the corresponding stale data of piece forms data Whether access times are more than predetermined value, if more than predetermined value, then judge described in be updated piece forms data as frequently-used data;With/ Or be updated the unique attribute of piece forms data described in judging and identify whether to be present in preset frequently-used data table, if in the presence of, Piece forms data is updated described in judgement as frequently-used data.
The present invention also provides a kind of preferred embodiments, and in particular to a kind of single service system, wherein, the data Access interface module 2, be additionally operable to by it is described be updated piece forms data be updated at least one data buffer storage unit 3 it Before,
Read the memory configurations of at least one data buffer storage unit 3;With,
For judging whether the currently stored amount of at least one data buffer storage unit 3 is not small based on the memory configurations In default memory;
If currently stored amount is not less than default memory, based on each list at least one data buffer storage unit 3 The history of data is accessed record, and deleting in scheduled duration not accessed piece forms data or history, to be accessed number minimum Piece forms data or longest forms data of the time interval of time interval current time for being accessed of last time so that it is described at least The currently stored amount of one data buffer storage unit 3 is less than default memory;If currently stored amount is less than default memory, by the quilt More new film forms data is updated at least one data buffer storage unit 3.
Namely be updated before piece forms data is updated at least one data buffer storage unit 3 by described, to data The current internal storage state of buffer unit is detected, and is not less than in the buffer memory for reaching the memory upper limit namely data buffer storage unit When the upper limit of the default internal memory quota, some data are deleted according to scheduled rule, so as to ensure data energy Data buffer storage unit is enough updated to, when preventing from reaching the upper limit due to memory so that the data being updated can not completely update Into data buffer storage unit, it is therefore prevented that the loss of data is updated caused by Out of Memory.It should be noted that in number It, also can be right before according to access interface module 2, the target patch forms data is updated at least one data buffer storage unit 3 Data buffer storage unit 3 carries out memory detection and judges to ensure that the target patch forms data can completely be updated to described at least one In a data buffer storage unit 3 (including local cache unit 31 and/or independent buffer unit 32).
Referring to Fig. 5, Fig. 5 is the structure chart for the piece single system that the embodiment of the present invention provides,
As shown in figure 5, on the basis of above-described embodiment, the embodiment provides a kind of single system, including:With Piece list service system described in upper embodiment.
As a kind of preferred embodiment, wherein, described single system further includes:
Matchmaker provides database 8, for storing piece single data source;
Whether data update module ETL7, the piece single data source that database 8 is provided for monitoring the matchmaker update;Be used for When matchmaker provides the piece single data source update of database 8, piece single data source synchronized update after being updated to piece list storage unit 4;
Client modules 6, for receiving target patch forms data inquiry instruction input by user, and based on target patch forms data Inquiry instruction generates target patch forms data inquiry request to be sent to piece single pair external tapping module 1.
Referring to Fig. 6, the flow chart of piece forms data acquisition methods that the first method embodiment that Fig. 6 is the present invention provides,
As shown in fig. 6, the first method embodiment of the present invention provides a kind of forms data acquisition methods, applied to piece list Service system, including:
Piece single pair external tapping module 1 sends to data access interface and visits when receiving target patch forms data inquiry request Ask request;
Data access interface module 2 is based on the access request, and target patch is obtained from least one data buffer storage unit 3 Forms data, and the target patch forms data of acquisition is returned via the outbound data interface.
Refering to Fig. 7, Fig. 7 is the flow chart of piece forms data acquisition methods that the second method embodiment of the present invention provides,
As shown in fig. 7, on the basis of first method embodiment of the present invention, second method embodiment of the invention provides A kind of forms data acquisition methods, wherein, at least one data buffer storage unit 3 includes:Local cache unit 31;
The data access interface module 2 obtains the step of target patch forms data from least one data buffer storage unit 3 Including:
The data access interface module 2 is based on access request, reads the data in local cache unit 31, if described There are the target patch forms datas for ground buffer unit 31, then obtain the target patch forms data from the local cache unit 31. Wherein, it should be noted that if the target patch forms data, the data access interface module is not present in local cache unit 31 2 read the piece forms data in piece list storage unit 4, and obtain the target patch forms data from described single storage unit 4, In, described single storage unit 4 at least stores the target patch forms data.
Refering to Fig. 8, Fig. 8 is the flow chart of piece forms data acquisition methods that the third method embodiment of the present invention provides,
As shown in figure 8, on the basis of second method embodiment of the present invention, third method embodiment of the invention provides A kind of forms data acquisition methods, wherein, at least one data buffer storage unit 3 further includes:Independent buffer unit 32;
The data access interface module 2 obtains the step of target patch forms data from least one data buffer storage unit 3 Including:
If the target patch forms data is not present in the local cache unit 31, the data access interface module 2 is read Data in independent buffer unit 32, if the independent buffer unit 32 there are the target patch forms data, from the independence Buffer unit 32 obtains the target patch forms data.
Wherein, it should be noted that if the target patch forms data, the data access is not present in independent buffer unit 32 Interface module 2 reads the piece forms data in piece list storage unit 4, and obtains the target patch from described single storage unit 4 Forms data, wherein, described single storage unit 4 at least stores the target patch forms data.
Refering to Fig. 9, Fig. 9 is the flow chart of piece forms data acquisition methods that the fourth method embodiment of the present invention provides,
As shown in figure 9, on the basis of first method embodiment of the invention, fourth method embodiment of the invention provides A kind of forms data acquisition methods, wherein, at least one data buffer storage unit 3 includes:Independent buffer unit 32;
The data access interface module 2 obtains the step of target patch forms data from least one data buffer storage unit 3 Including:
Data access interface module 2 reads the data in independent buffer unit 32 based on access request, if described independently delay There are the target patch forms datas for memory cell 32, then obtain the target patch forms data from the independent buffer unit 32.Wherein, It should be noted that if the target patch forms data is not present in independent buffer unit 32, the data access interface module 2 is read Piece forms data in piece list storage unit 4, and the target patch forms data is obtained from described single storage unit 4, wherein, institute It states piece list storage unit 4 and at least stores the target patch forms data.
As a kind of preferred embodiment, the method further includes:
If the target patch forms data is not present in data buffer storage unit 3, the data access interface module 2 reads piece list and deposits Piece forms data in storage unit 4, and the target patch forms data is obtained from described single storage unit 4, wherein, described list Storage unit 4 at least stores the target patch forms data.
As a kind of preferred embodiment, obtained in the data access interface module 2 from the independent buffer unit 32 After the target patch forms data, further include:The target patch forms data is updated in local cache unit 31.
As a kind of preferred embodiment, obtained from described single storage unit 4 in the data access interface module 2 After the target patch forms data, further include:The target patch forms data is updated at least one data buffer storage unit 3 In, wherein, it should be noted that the target patch forms data is updated at least one data buffer storage unit 3 and is included: The target patch forms data is updated to local cache unit 31 and/or independent buffer unit 32.
The embodiment of the present invention additionally provides a kind of forms data update method, applied to piece list service system, including:
Whether the piece forms data of 5 detection lug list storage unit 4 of data monitoring unit storage updates, and is monitoring that piece list deposits During the piece forms data update of storage unit 4, send piece forms data and update messages to the data access interface module 2;Wherein, it is described Piece forms data new information includes:The unique attribute mark of piece forms data is updated in piece list storage unit 4;
Data access interface module 2 updates the piece in an at least data buffer storage unit 3 based on piece forms data new information Forms data.
Wherein, the data access interface module 2 is based on a piece forms data new information update at least data buffer storage list Member 3 in piece forms data the step of include:
Data access interface module 2 is identified based on the unique attribute, is judged at least one data buffer storage unit 3 The corresponding stale data of piece forms data is updated with the presence or absence of with described, if in the presence of the stale data is deleted;Wherein, The stale data is updated data of the piece forms data before not being updated to be described.
Wherein, the data access interface module 2 is based on a piece forms data new information update at least data buffer storage list The also step of piece forms data in member 3 includes:
Data access interface module 2 is based on preset rules and the unique attribute identifies, and piece odd number is updated described in judgement According to whether being frequently-used data;
If the piece forms data that is updated for frequently-used data, reads the data in described single storage unit 4 with by institute It states and is updated piece forms data and is updated at least one data buffer storage unit 3;
Wherein, the preset rules are:Judge and the history quilt for being updated the corresponding stale data of piece forms data Whether access times are more than predetermined value, if more than predetermined value, then judge described in be updated piece forms data as frequently-used data;With/ Or be updated the unique attribute of piece forms data described in judging and identify whether to be present in preset frequently-used data table, if in the presence of, Piece forms data is updated described in judgement as frequently-used data.
Wherein, by it is described be updated the step of piece forms data is updated at least one data buffer storage unit 3 before, It further includes:
Data access interface module 2 reads the memory configurations of at least one data buffer storage unit 3;
Data access interface module 2 is used to judge at least one data buffer storage unit 3 based on the memory configurations Whether currently stored amount is not less than default memory;
If currently stored amount is not less than default memory, data access interface module 2 is delayed based at least one data The history of each forms data in memory cell 3 is accessed record, delete in scheduled duration not accessed piece forms data or Longest list of the time interval of time interval current time that history is accessed the minimum piece forms data of number or last time is accessed Data are so that the currently stored amount of at least one data buffer storage unit 3 is less than default memory;
If currently stored amount is less than default memory, the piece forms data that is updated is updated at least one data Buffer unit 3.
Wherein, it should be noted that the piece forms data that the embodiment of the present invention refers to can be video frequency program forms data and/ Or video playing file and/or audio list data and/or audio played file and/or graphic data file and/or word number According to file and/or other data.
The present invention is directed to protect a kind of data acquisition and update method, piece list service system and piece single system, the present invention is real Data acquisition and update method, the piece list service system and piece single system of example offer are applied, using multi-level buffer scheme, and according to more Grade buffering scheme provides the method accelerated data loading and show speed, and by newest data by way of real-time update Update cachings at different levels so that client can get newest data, and due to using multi-level buffer scheme, increase number According to source, data acquiring mode reduces matchmaker and provides the pressure of database and the pressure of data buffer storage unit also with regard to corresponding increase, Data throughout is increased, so as to improve data loading and show speed.
It should be understood that the above-mentioned specific embodiment of the present invention is used only for exemplary illustration or explains the present invention's Principle, without being construed as limiting the invention.Therefore, that is done without departing from the spirit and scope of the present invention is any Modification, equivalent replacement, improvement etc., should all be included in the protection scope of the present invention.In addition, appended claims purport of the present invention Covering the whole variations fallen into scope and boundary or this range and the equivalent form on boundary and repairing Change example.

Claims (22)

1. a kind of single service system, it is characterised in that, including:
Piece single pair external tapping module, for when receiving target patch forms data inquiry request, sending and visiting to data access interface Ask request;
For being based on the access request, target patch is obtained from least one data buffer storage unit for data access interface module Forms data, and the target patch forms data of acquisition is returned via described single pair external tapping module;And/or for being based on Piece forms data in the piece forms data new information update at least data buffer storage unit received;
Piece list storage unit, for storing piece forms data;
Data monitoring unit, in the piece forms data update for monitoring piece list storage unit, sending the update of piece forms data and disappearing It ceases to the data access interface module;Wherein, described forms data new information includes:Piece is updated in piece list storage unit The unique attribute mark of forms data.
2. according to claim 1 single service system, which is characterized in that at least one data buffer storage unit packet It includes:Local cache unit;
The data access interface module for being based on access request, reads the data in local cache unit, if the local There are the target patch forms datas for buffer unit, then obtain the target patch forms data from the local cache unit;And/or
For being based on the unique attribute mark, judge to be updated piece odd number with the presence or absence of with described in the local cache unit According to corresponding stale data, if in the presence of the stale data is deleted;Wherein, the stale data is updated piece to be described Data of the forms data before not being updated.
3. according to claim 2 single service system, which is characterized in that at least one data buffer storage unit also wraps It includes:Independent buffer unit;
The data access interface module, for when the target patch forms data is not present in the local cache unit, reading Data in independent buffer unit if there are the target patch forms datas for the independent buffer unit, are independently cached from described Unit obtains the target patch forms data;And/or
For being based on the unique attribute mark, judge to be updated piece odd number with the presence or absence of with described in the independent buffer unit According to corresponding stale data, if in the presence of the stale data is deleted.
4. according to claim 1 single service system, which is characterized in that at least one data buffer storage unit packet It includes:Independent buffer unit;
The data access interface module for being based on access request, reads the data in independent buffer unit, if the independence There are the target patch forms datas for buffer unit, then obtain the target patch forms data from the independent buffer unit;And/or
For being based on the unique attribute mark, judge to be updated piece odd number with the presence or absence of with described in the independent buffer unit According to corresponding stale data, if in the presence of the stale data is deleted;Wherein, the stale data is updated piece to be described Data of the forms data before not being updated.
5. according to any one of claims 1 to 4 single service system, which is characterized in that the data access interface mould Block is additionally operable to, when the target patch forms data is not present in data buffer storage unit, read the piece forms data in piece list storage unit, And obtain the target patch forms data from described single storage unit.
6. according to claim 3 single service system, which is characterized in that the data access interface module is additionally operable to After the target patch forms data is obtained from the independent buffer unit, the target patch forms data is updated to local cache list In member.
7. according to claim 5 single service system, which is characterized in that the data access interface module is additionally operable to After obtaining the target patch forms data in described single storage unit, the target patch forms data is updated to described at least one In a data buffer storage unit.
8. the piece list service system according to claims 1 or 2 or 3 or 4 or 6 or 7, it is characterised in that, the data access Interface module is additionally operable to be identified based on preset rules and the unique attribute, is updated whether piece forms data is normal described in judgement Use data;
If the piece forms data that is updated for frequently-used data, read the data in described single storage unit with by described by more New film forms data is updated at least one data buffer storage unit;
Wherein, the preset rules are:Judgement is accessed with the history for being updated the corresponding stale data of piece forms data Whether number is more than predetermined value, if more than predetermined value, then judge described in be updated piece forms data as frequently-used data;And/or sentence The disconnected unique attribute for being updated piece forms data identifies whether to be present in preset frequently-used data table, if in the presence of judging The piece forms data that is updated is frequently-used data.
9. according to claim 8 single service system, which is characterized in that the data access interface module is additionally operable to It is updated before piece forms data is updated at least one data buffer storage unit by described,
Read the memory configurations of at least one data buffer storage unit;With,
For judging the currently stored amount of at least one data buffer storage unit whether not less than pre- based on the memory configurations If memory;
If currently stored amount is not less than default memory, based on each forms data at least one data buffer storage unit History is accessed record, deletes the piece forms data or history not being accessed in scheduled duration and is accessed the minimum piece list of number Longest forms data of the time interval of time interval current time that data or last time are accessed is so that at least one number It is less than default memory according to the currently stored amount of buffer unit;
If currently stored amount is less than default memory, the piece forms data that is updated is updated at least one data buffer storage Unit.
10. a kind of single system, which is characterized in that including:Claim 1 to 9 any one of them piece list service system.
11. according to claim 10 single system, which is characterized in that described single system further includes:
Matchmaker provides database, for storing piece single data source;
Whether data update module ETL, the piece single data source that database is provided for monitoring the matchmaker update;With for being provided in matchmaker During the piece single data source update of database, piece single data source synchronized update after being updated to piece list storage unit.
12. a kind of forms data acquisition methods, applied to piece list service system, which is characterized in that including:
Piece single pair external tapping module sends to access to data access interface and ask when receiving target patch forms data inquiry request It asks;
Data access interface module is based on the access request, and target patch odd number is obtained from least one data buffer storage unit According to, and the target patch forms data of acquisition is returned via the outbound data interface.
13. according to the method for claim 12, which is characterized in that at least one data buffer storage unit includes:It is local Buffer unit;
The data access interface module obtains the step of target patch forms data from least one data buffer storage unit and includes:
The data access interface module is based on access request, the data in local cache unit is read, if the local cache There are the target patch forms datas for unit, then obtain the target patch forms data from the local cache unit.
14. according to the method for claim 13, which is characterized in that at least one data buffer storage unit further includes:Solely Vertical buffer unit;
The data access interface module obtains the step of target patch forms data from least one data buffer storage unit and includes:
If the target patch forms data is not present in the local cache unit, the data access interface module reads independent caching Data in unit if there are the target patch forms datas for the independent buffer unit, are obtained from the independent buffer unit The target patch forms data.
15. according to the method for claim 12, it is characterised in that, at least one data buffer storage unit includes:It is independent Buffer unit;
The data access interface module obtains the step of target patch forms data from least one data buffer storage unit and includes:
Data access interface module reads the data in independent buffer unit based on access request, if the independent buffer unit is deposited In the target patch forms data, then the target patch forms data is obtained from the independent buffer unit.
16. according to claim 12 to 15 any one of them method, it is characterised in that:The method further includes:
If the target patch forms data is not present in data buffer storage unit, the data access interface module reads piece list storage unit In piece forms data, and obtain the target patch forms data from described single storage unit, wherein, described single storage unit At least store the target patch forms data.
17. according to the method for claim 14, which is characterized in that independently delay from described in the data access interface module After memory cell obtains the target patch forms data, further include:The target patch forms data is updated in local cache unit.
18. according to the method for claim 16, which is characterized in that singly deposited from described in the data access interface module After the target patch forms data is obtained in storage unit, further include:The target patch forms data is updated to described at least one In data buffer storage unit.
19. a kind of forms data update method, applied to piece list service system, which is characterized in that including:
Whether the piece forms data of data monitoring unit detection lug list storage unit storage updates, and monitoring piece list storage unit Piece forms data update when, send piece forms data update messages to the data access interface module;Wherein, described forms data New information includes:The unique attribute mark of piece forms data is updated in piece list storage unit;
Data access interface module updates the piece forms data in an at least data buffer storage unit based on piece forms data new information.
20. according to claim 19 forms data update method, which is characterized in that the data access interface module base Include the step of piece forms data new information updates the piece forms data in an at least data buffer storage unit:
Data access interface module is identified based on the unique attribute, judges whether deposited at least one data buffer storage unit The corresponding stale data of piece forms data is being updated with described, if in the presence of the stale data is deleted;Wherein, the mistake Issue is updated data of the piece forms data before not being updated according to be described.
21. the piece forms data update method according to claim 19 or 20, which is characterized in that the data access interface mould The also step that block updates the piece forms data in an at least data buffer storage unit based on piece forms data new information includes:
Data access interface module is based on preset rules and the unique attribute identifies, and whether piece forms data is updated described in judgement For frequently-used data;
If the piece forms data that is updated for frequently-used data, read the data in described single storage unit with by described by more New film forms data is updated at least one data buffer storage unit;
Wherein, the preset rules are:Judgement is accessed with the history for being updated the corresponding stale data of piece forms data Whether number is more than predetermined value, if more than predetermined value, then judge described in be updated piece forms data as frequently-used data;And/or sentence The disconnected unique attribute for being updated piece forms data identifies whether to be present in preset frequently-used data table, if in the presence of judging The piece forms data that is updated is frequently-used data.
22. according to the method for claim 21, which is characterized in that by it is described be updated piece forms data be updated to it is described extremely Before the step of few data buffer storage unit, further include:
Data access interface module reads the memory configurations of at least one data buffer storage unit;
Data access interface module is used to judge that the current of at least one data buffer storage unit is deposited based on the memory configurations Whether reserves are not less than default memory;
If currently stored amount is not less than default memory, data access interface module is based at least one data buffer storage unit In the history of each forms data be accessed record, delete in scheduled duration that accessed piece forms data or history are not interviewed Ask the minimum piece forms data of number or longest forms data of the time interval of time interval current time that last time is accessed so that The currently stored amount for obtaining at least one data buffer storage unit is less than default memory;
If currently stored amount is less than default memory, the piece forms data that is updated is updated at least one data buffer storage Unit.
CN201611159578.7A 2016-12-15 2016-12-15 Data acquisition and update method, piece list service system and piece single system Pending CN108228624A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201611159578.7A CN108228624A (en) 2016-12-15 2016-12-15 Data acquisition and update method, piece list service system and piece single system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201611159578.7A CN108228624A (en) 2016-12-15 2016-12-15 Data acquisition and update method, piece list service system and piece single system

Publications (1)

Publication Number Publication Date
CN108228624A true CN108228624A (en) 2018-06-29

Family

ID=62650471

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201611159578.7A Pending CN108228624A (en) 2016-12-15 2016-12-15 Data acquisition and update method, piece list service system and piece single system

Country Status (1)

Country Link
CN (1) CN108228624A (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109815060A (en) * 2019-01-30 2019-05-28 北京百度网讯科技有限公司 Method and device for backup information
CN113419761A (en) * 2021-06-24 2021-09-21 广州欢网科技有限责任公司 Method and system for releasing sp/cp operator content injection work
CN113542326A (en) * 2020-04-17 2021-10-22 上海际链网络科技有限公司 Data caching method and device for distributed system, server and storage medium

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101692229A (en) * 2009-07-28 2010-04-07 武汉大学 Self-adaptive multilevel cache system for three-dimensional spatial data based on data content
CN102135985A (en) * 2011-01-28 2011-07-27 百度在线网络技术(北京)有限公司 Method and system for searching by calling search result of third-party search engine
CN102508854A (en) * 2011-09-29 2012-06-20 畅捷通信息技术股份有限公司 Data access device and method
CN102646072A (en) * 2012-02-28 2012-08-22 信雅达系统工程股份有限公司 Data management method
CN103024593A (en) * 2012-12-26 2013-04-03 人民搜索网络股份公司 Online VOD (video on demand) acceleration system and online VOD playing method
CN103226604A (en) * 2013-04-27 2013-07-31 上海先之决信息科技发展有限公司 SVG-based Web GIS system and relevant energy consumption monitoring system

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101692229A (en) * 2009-07-28 2010-04-07 武汉大学 Self-adaptive multilevel cache system for three-dimensional spatial data based on data content
CN102135985A (en) * 2011-01-28 2011-07-27 百度在线网络技术(北京)有限公司 Method and system for searching by calling search result of third-party search engine
CN102508854A (en) * 2011-09-29 2012-06-20 畅捷通信息技术股份有限公司 Data access device and method
CN102646072A (en) * 2012-02-28 2012-08-22 信雅达系统工程股份有限公司 Data management method
CN103024593A (en) * 2012-12-26 2013-04-03 人民搜索网络股份公司 Online VOD (video on demand) acceleration system and online VOD playing method
CN103226604A (en) * 2013-04-27 2013-07-31 上海先之决信息科技发展有限公司 SVG-based Web GIS system and relevant energy consumption monitoring system

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109815060A (en) * 2019-01-30 2019-05-28 北京百度网讯科技有限公司 Method and device for backup information
CN113542326A (en) * 2020-04-17 2021-10-22 上海际链网络科技有限公司 Data caching method and device for distributed system, server and storage medium
CN113542326B (en) * 2020-04-17 2023-04-25 上海际链网络科技有限公司 Data caching method and device of distributed system, server and storage medium
CN113419761A (en) * 2021-06-24 2021-09-21 广州欢网科技有限责任公司 Method and system for releasing sp/cp operator content injection work

Similar Documents

Publication Publication Date Title
CN106202112B (en) CACHE DIRECTORY method for refreshing and device
CN105872040B (en) A method of write performance is stored using gateway node cache optimization distributed block
CN105426321B (en) Using RDMA close friend's caching method of remote location information
CN105095313B (en) A kind of data access method and equipment
CN103607312B (en) Data request processing method and system for server system
CN100437590C (en) Method for prefetching object
CN108804234A (en) Data-storage system and its operating method
CN106326499B (en) A kind of data processing method and device
CN101226542B (en) Method for caching report
CN103488685B (en) Fragmented-file storage method based on distributed storage system
CN108228624A (en) Data acquisition and update method, piece list service system and piece single system
CN101673192A (en) Method for time-sequence data processing, device and system therefor
CN107623722A (en) A kind of remote data caching method, electronic equipment and storage medium
CN110096334A (en) Method for caching and processing, device, equipment and computer readable storage medium
CN110109958A (en) Method for caching and processing, device, equipment and computer readable storage medium
CN100416566C (en) Picture data storage and read method
CN107786623A (en) Message asynchronous processing method and device
CN101217449B (en) A remote call management procedure
CN103986779A (en) Application data transmission method, device and terminal equipment
CN100394404C (en) System and method for management of metadata
CN107704573A (en) A kind of intelligent buffer method coupled with business
CN103209397A (en) Method for controlling sending and receiving of short messages and server and terminal thereof
CN107888687A (en) A kind of agent client storage accelerated method and system based on distributed memory system
CN110245129A (en) Distributed global data deduplication method and device
CN107943958A (en) A kind of Individual Income Tax master data sharing method and system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
WD01 Invention patent application deemed withdrawn after publication

Application publication date: 20180629

WD01 Invention patent application deemed withdrawn after publication