CN108228624A - Data acquisition and update method, piece list service system and piece single system - Google Patents
Data acquisition and update method, piece list service system and piece single system Download PDFInfo
- Publication number
- CN108228624A CN108228624A CN201611159578.7A CN201611159578A CN108228624A CN 108228624 A CN108228624 A CN 108228624A CN 201611159578 A CN201611159578 A CN 201611159578A CN 108228624 A CN108228624 A CN 108228624A
- Authority
- CN
- China
- Prior art keywords
- data
- piece
- storage unit
- updated
- forms
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2379—Updates performed during online database operations; commit processing
Abstract
The invention discloses a kind of data acquisition and update method, piece list service system and piece single systems, belong to media asset management technical field, wherein, piece list service system includes:Piece single pair external tapping module, for sending access request to data access interface;Data access interface module is used to obtain target patch forms data from data buffer storage unit based on access request and return;And/or for updating the data the piece forms data in buffer unit based on the piece forms data new information received;Data monitoring unit updates messages to data access interface module for sending piece forms data in the piece forms data update for monitoring piece list storage unit;Wherein, described forms data new information includes:The unique attribute mark of piece forms data is updated in piece list storage unit.The embodiment of the present invention uses multi-level buffer scheme, increases data source, reduces the pressure that matchmaker provides database, increases data throughout, improves data loading and shows speed.
Description
Technical field
The invention belongs to media asset management technical field, more particularly to a kind of data acquisition and update method, piece list service system
System and piece single system.
Background technology
Matchmaker provide data, such as video frequency program list loading and show the experience that speed largely affects user, and
Matchmaker provides the loading of data and shows the acquisition modes that speed directly depends on the handling capacity of server end and matchmaker provides data.
Current technology scheme has been done part and has been optimized, when matchmaker provides number to improve the loading of matchmaker's money data and show speed
It changes according to library, piece single database is synchronized to by synchronization program, piece list interface server is obtained from independent piece single database
Access evidence;This print list interface server does not access matchmaker directly and provides database, so as to alleviate the pressure that matchmaker provides database.
In the implementation of the present invention, still at least there are the following problems for inventor's discovery current technology scheme:By
Database is remained stored in the key data source of piece list interface server, is accessed in the case that peak value or pressure are king-sized
When, exist and access the problems such as data time is long, seriously affect systematic entirety energy and handling capacity, and then matchmaker is caused to provide adding for data
It carries and to show speed slow.
Invention content
The purpose of the embodiment of the present invention is to provide a kind of data acquisition and update method, piece list service system and piece monosystem
System;At least using multi-level buffer scheme, and a kind of side for accelerating data loading and showing speed is provided according to multi-level buffer scheme
Method, and by newest data update to cachings at different levels by way of real-time update so that client can get newest
Data, and due to use multi-level buffer scheme, increase data source, data acquiring mode is also with regard to corresponding increase, and then subtract
The pressure of few server master library, increases data throughout, so as to improve data loading and show speed.
According to an aspect of the present invention, the embodiment provides a kind of single service system, including:
Piece single pair external tapping module, for when receiving target patch forms data inquiry request, being sent out to data access interface
Send access request;
For being based on the access request, mesh is obtained from least one data buffer storage unit for data access interface module
Standard film forms data, and the target patch forms data of acquisition is returned via described single pair external tapping module;And/or it is used for
Based on the piece forms data in the piece forms data new information update at least data buffer storage unit received;
Piece list storage unit, for storing piece forms data;
Data monitoring unit, in the piece forms data update for monitoring piece list storage unit, sending piece forms data more
New information is to the data access interface module;Wherein, described forms data new information includes:By more in piece list storage unit
The unique attribute mark of new film forms data.
Optionally, at least one data buffer storage unit includes:Local cache unit;
The data access interface module for being based on access request, reads the data in local cache unit, if described
There are the target patch forms datas for local cache unit, then obtain the target patch forms data from the local cache unit;With/
Or,
For being based on unique attribute mark, judge to whether there is in the local cache unit and be updated piece with described
The corresponding stale data of forms data, if in the presence of the stale data is deleted;Wherein, the stale data is described by more
Data of the new film forms data before not being updated.
Optionally, at least one data buffer storage unit further includes:Independent buffer unit;
The data access interface module, for the local cache unit be not present the target patch forms data when,
Read the data in independent buffer unit, if the independent buffer unit there are the target patch forms data, from the independence
Buffer unit obtains the target patch forms data;And/or
For being based on unique attribute mark, judge in the independent buffer unit with the presence or absence of being updated piece with described
The corresponding stale data of forms data, if in the presence of the stale data is deleted.
Optionally, at least one data buffer storage unit includes:Independent buffer unit;
The data access interface module for being based on access request, reads the data in independent buffer unit, if described
There are the target patch forms datas for independent buffer unit, then obtain the target patch forms data from the independent buffer unit;With/
Or,
For being based on unique attribute mark, judge in the independent buffer unit with the presence or absence of being updated piece with described
The corresponding stale data of forms data, if in the presence of the stale data is deleted;Wherein, the stale data is described by more
Data of the new film forms data before not being updated.
Optionally, the data access interface module is additionally operable to singular there is no the target patch in data buffer storage unit
According to when, read the piece forms data in piece list storage unit, and the target patch forms data is obtained from described single storage unit.
Optionally, the data access interface module is additionally operable to obtaining the target patch from the independent buffer unit
After forms data, the target patch forms data is updated in local cache unit.
Optionally, the data access interface module is additionally operable to obtain the target patch in described single storage unit
After forms data, the target patch forms data is updated at least one data buffer storage unit.
Optionally, the data access interface module is additionally operable to identify based on preset rules and the unique attribute, judgement
It is described to be updated whether piece forms data is frequently-used data;
If the piece forms data that is updated for frequently-used data, reads the data in described single storage unit with by described in
It is updated piece forms data and is updated at least one data buffer storage unit;
Wherein, the preset rules are:Judge and the history quilt for being updated the corresponding stale data of piece forms data
Whether access times are more than predetermined value, if more than predetermined value, then judge described in be updated piece forms data as frequently-used data;With/
Or be updated the unique attribute of piece forms data described in judging and identify whether to be present in preset frequently-used data table, if in the presence of,
Piece forms data is updated described in judgement as frequently-used data.
Optionally, the data access interface module, be additionally operable to by it is described be updated piece forms data be updated to it is described extremely
Before a few data buffer storage unit,
Read the memory configurations of at least one data buffer storage unit;With,
For judging whether the currently stored amount of at least one data buffer storage unit is not small based on the memory configurations
In default memory;
If currently stored amount is not less than default memory, based on each odd number at least one data buffer storage unit
According to history be accessed record, deleting in scheduled duration not accessed piece forms data or history, to be accessed number minimum
Longest forms data of the time interval of time interval current time that piece forms data or last time are accessed is so that described at least one
The currently stored amount of a data buffer storage unit is less than default memory;
If currently stored amount is less than default memory, the piece forms data that is updated is updated at least one data
Buffer unit.
Another aspect according to embodiments of the present invention, the embodiment provides a kind of single system, including:More than
The piece list service system.
Optionally, described single system further includes:
Matchmaker provides database, for storing piece single data source;
Whether data update module ETL, the piece single data source that database is provided for monitoring the matchmaker update;With, for
When matchmaker provides the piece single data source update of database, piece single data source synchronized update after being updated to piece list storage unit.
Another aspect according to embodiments of the present invention, the embodiment provides a kind of forms data acquisition methods,
Applied to piece list service system, including:
Piece single pair external tapping module sends to data access interface and accesses when receiving target patch forms data inquiry request
Request;
Data access interface module is based on the access request, and target patch list is obtained from least one data buffer storage unit
Data, and the target patch forms data of acquisition is returned via the outbound data interface.
Optionally, at least one data buffer storage unit includes:Local cache unit;
The data access interface module obtains the step of target patch forms data from least one data buffer storage unit and wraps
It includes:
The data access interface module is based on access request, the data in local cache unit is read, if the local
There are the target patch forms datas for buffer unit, then obtain the target patch forms data from the local cache unit.
Optionally, at least one data buffer storage unit further includes:Independent buffer unit;
The data access interface module obtains the step of target patch forms data from least one data buffer storage unit and wraps
It includes:
If the target patch forms data is not present in the local cache unit, the data access interface module reads independent
Data in buffer unit, if the independent buffer unit there are the target patch forms data, from the independent buffer unit
Obtain the target patch forms data.
Optionally, at least one data buffer storage unit includes:Independent buffer unit;
The data access interface module obtains the step of target patch forms data from least one data buffer storage unit and wraps
It includes:
Data access interface module reads the data in independent buffer unit based on access request, if the independent caching is single
There are the target patch forms datas for member, then obtain the target patch forms data from the independent buffer unit.
Optionally, the method further includes:
If the target patch forms data is not present in data buffer storage unit, the data access interface module reads the storage of piece list
Piece forms data in unit, and the target patch forms data is obtained from described single storage unit, wherein, described single storage
Unit at least stores the target patch forms data.
Optionally, the data access interface module from the independent buffer unit obtain the target patch forms data it
Afterwards, it further includes:The target patch forms data is updated in local cache unit.
Optionally, in the data access interface module target patch forms data is obtained from described single storage unit
Later, it further includes:The target patch forms data is updated at least one data buffer storage unit.
Another aspect according to embodiments of the present invention, the embodiment provides a kind of forms data update method,
Applied to piece list service system, including:
Whether the piece forms data of data monitoring unit detection lug list storage unit storage updates, and is monitoring the storage of piece list
During the piece forms data update of unit, send piece forms data and update messages to the data access interface module;Wherein, described list
Data update message includes:The unique attribute mark of piece forms data is updated in piece list storage unit;
Data access interface module updates the piece list in an at least data buffer storage unit based on piece forms data new information
Data.
Optionally, the data access interface module is based on a piece forms data new information update at least data buffer storage list
The step of piece forms data in member, includes:
Data access interface module based on the unique attribute identify, judge be at least one data buffer storage unit
No presence is updated the corresponding stale data of piece forms data with described, if in the presence of the stale data is deleted;Wherein, institute
It states stale data and is updated data of the piece forms data before not being updated to be described.
Optionally, the data access interface module is based on a piece forms data new information update at least data buffer storage list
The also step of piece forms data in member includes:
Data access interface module is based on preset rules and the unique attribute identifies, and piece forms data is updated described in judgement
Whether it is frequently-used data;
If the piece forms data that is updated for frequently-used data, reads the data in described single storage unit with by described in
It is updated piece forms data and is updated at least one data buffer storage unit;
Wherein, the preset rules are:Judge and the history quilt for being updated the corresponding stale data of piece forms data
Whether access times are more than predetermined value, if more than predetermined value, then judge described in be updated piece forms data as frequently-used data;With/
Or be updated the unique attribute of piece forms data described in judging and identify whether to be present in preset frequently-used data table, if in the presence of,
Piece forms data is updated described in judgement as frequently-used data.
Optionally, by it is described be updated the step of piece forms data is updated at least one data buffer storage unit it
Before, it further includes:
Data access interface module reads the memory configurations of at least one data buffer storage unit;
Data access interface module is used to judge working as at least one data buffer storage unit based on the memory configurations
Whether preceding amount of storage is not less than default memory;
If currently stored amount is not less than default memory, data access interface module is based at least one data buffer storage
The history of each forms data in unit is accessed record, deletes the piece forms data or history not being accessed in scheduled duration
Longest forms data of the time interval of time interval current time that the minimum piece forms data of accessed number or last time are accessed
So that the currently stored amount of at least one data buffer storage unit is less than default memory;
If currently stored amount is less than default memory, the piece forms data that is updated is updated at least one data
Buffer unit.
Data acquisition provided in an embodiment of the present invention and update method, piece list service system and piece single system, using multistage
Buffering scheme, and the method accelerated data loading and show speed is provided according to multi-level buffer scheme, and pass through real-time update
Mode by newest data update to cachings at different levels so that client can get newest data, and due to using more
Grade buffering scheme, increases data source, data acquiring mode also just corresponding increase, so reduce matchmaker provide the pressure of database with
The pressure of data buffer storage unit, increases data throughout, so as to improve data loading and show speed.
Description of the drawings
Fig. 1 is the structure chart for the piece list service system that the first system embodiment of the present invention provides;
Fig. 2 is the structure chart for the piece list service system that second system embodiment of the present invention provides;
Fig. 3 is the structure chart for the piece list service system that third system embodiment of the present invention provides;
Fig. 4 is the structure chart of piece list service system that the 4th system embodiment of the invention provides;
Fig. 5 is the structure chart for the piece single system that the embodiment of the present invention provides;
Fig. 6 is the flow chart of piece forms data acquisition methods that the first method embodiment of the present invention provides;
Fig. 7 is the flow chart of piece forms data acquisition methods that the second method embodiment of the present invention provides;
Fig. 8 is the flow chart of piece forms data acquisition methods that the third method embodiment of the present invention provides;
Fig. 9 is the flow chart of piece forms data acquisition methods that the fourth method embodiment of the present invention provides.
Reference sign:1- piece single pair external tapping modules;2- data access interface modules;3- data buffer storage units;31-
Local cache unit;32- independence buffer units;4- piece list storage units;5- data monitoring units;6- client modules;7- numbers
According to update module ETL;8- matchmaker provides database.
Specific embodiment
Understand to make the object, technical solutions and advantages of the present invention clearer, With reference to embodiment and join
According to attached drawing, the present invention is described in more detail.It should be understood that these descriptions are merely illustrative, and it is not intended to limit this hair
Bright range.In addition, in the following description, the description to known features and technology is omitted, to avoid this is unnecessarily obscured
The concept of invention.
The embodiment of the present invention provides a kind of data acquisition and update method, piece list service system and piece single system wherein, more
The embodiment of the present invention, one application scenarios of following offers are understood well, but this application scenarios is not to the embodiment of the present invention
It limits:When client modules get target patch forms data inquiry instruction input by user, client modules are to piece single system
Target patch forms data inquiry request is sent, it is client modules that the piece single pair external tapping module 1 of piece single system, which is used as piece single system,
The data-interface of offer is docked with client modules, and receives the target patch forms data inquiry request of client modules, at this time
Piece single system can carry out target data inquiry according to this target data inquiry request.And the purpose of the present invention is exactly to improve piece monosystem
The data of system load and show speed namely data query speed, while client modules is enabled to get newest number
According to.
Referring to Fig. 1, Fig. 1 is the structure chart for the piece list service system that the first system embodiment of the present invention provides.
As shown in Figure 1, the first system embodiment of the present invention provides a kind of single service system, including:
Piece single pair external tapping module 1, for when receiving target patch forms data inquiry request, being sent out to data access interface
Send access request;
Data access interface module 2 for being based on the access request, is obtained from least one data buffer storage unit 3
Target patch forms data, and the target patch forms data of acquisition is returned via described single pair external tapping module 1;And/or it uses
In based on the piece forms data in the piece forms data new information received a update at least data buffer storage unit 3;
Piece list storage unit 4, for storing piece forms data;
Data monitoring unit 5, in the piece forms data update for monitoring piece list storage unit 4, sending piece forms data
Update messages to the data access interface module 2;Wherein, described forms data new information includes:In piece list storage unit 4
It is updated the unique attribute mark of piece forms data.
Wherein, the data buffer storage unit 3 includes at least one of local cache unit 31, independent buffer unit 32,
But this is not limited only to, other storage units or database can also be included;The number of data buffer storage unit can be according to different
The number of business demand and system data amount is configured.Wherein, it should be noted that the access speed of local cache unit 31
It is most fast, is independent buffer unit 32 in fact, data access interface module 2 is carrying out data read access based on access request
When, it can be preferential to access in data buffer storage unit 3 access speed most according to the speed of the access speed of data buffer storage unit 3
Fast buffer unit, that is, local cache unit 31.
Wherein, the effect of data access interface module 2 is carried for the access of different database or data buffer storage unit
Unified interface is supplied;Different database or data buffer storage unit are able to access that by this data access interface module 2, because
This allow piece single pair external tapping module 1 do not have to concern Back end data source, it is only necessary to pay close attention to the specification and safety of interface
, different database or data buffer storage unit are being accessed so as to also simplify piece list service system to a certain extent
When switching call operation, simplify the flow that switching is called, can also improve the number of piece list service system to a certain extent
According to loading and showing speed, data access interface module 2 can be that ODBC (Open Database Connectivity) is opened
Put Database Connectivity;Can also be that JDBC (Java Data Base Connectivity) i.e. java databases connect, either
Other database interfaces.Wherein, front end buffer unit is further included as a kind of optionally embodiment, piece list service system, wherein,
Front end buffer unit saves the history target patch list that client modules are returned by piece single pair external tapping module 1 in scheduled duration
Data;The history target patch forms data is for this target patch forms data inquiry request, that is front end
Data in buffer unit are last time or the preceding data that client modules are returned to by piece single pair external tapping module 1 several times,
Before piece single pair external tapping module 1 receives target patch forms data inquiry request, front end buffer unit can receive target patch odd number
During according to inquiry request, this target patch forms data asked can be whether there is in query history target patch forms data, if it does,
Then target patch forms data is returned.Otherwise, then by target patch forms data inquiry request it is sent to piece single pair external tapping module 1.Its
In, a length of scheduled duration when the data of front end buffer unit store, when reaching scheduled duration, front end buffer unit can delete it
The data of storage, wherein, scheduled duration is preset value, can be adjusted.
Referring to Fig. 2, Fig. 2 is the structure chart for the piece list service system that second system embodiment of the present invention provides;
As shown in Figure 2, on the basis of the first system embodiment of the present invention, second system embodiment of the invention provides
A kind of single service system, wherein, at least one data buffer storage unit 3 includes:Local cache unit 31;
The data access interface module 2 for being based on access request, reads the data in local cache unit 31, if
There are the target patch forms datas for the local cache unit 31, then obtain the target patch list from the local cache unit 31
Data;And/or
For being based on unique attribute mark, judge in the local cache unit 31 with the presence or absence of being updated with described
The corresponding stale data of piece forms data, if in the presence of the stale data is deleted;Wherein, the stale data is the quilt
More data of the new film forms data before not being updated.Wherein, it should be noted that data access interface module 2 includes at least one
It is a, before piece single pair external tapping module 1 sends access request to data access interface module 2, it can also be visited according to each data
Ask that the number of the access request being currently received of interface module 2 carries out selection judgement, so that 1 energy of piece single pair external tapping module
A minimum data access interface module 2 for current access request is enough linked into, by such mode, also can effectively be alleviated
Data access interface module 2, and then can also improve the data loading of piece list service system and show speed namely data query
Speed.
Referring to Fig. 3, Fig. 3 is the structure chart for the piece list service system that third system embodiment of the present invention provides;
As shown in Figure 3, on the basis of second system embodiment of the present invention, third system embodiment of the invention provides
A kind of single service system, wherein, at least one data buffer storage unit 3 further includes:Independent buffer unit 32;Wherein, it needs
It is noted that by the way that multi-level buffer unit is set to increase data source, data acquiring mode is also with regard to corresponding increase, and then subtract
Few matchmaker provides the pressure of database, increases data throughout, so as to improve data loading and show speed.
The data access interface module 2, for the target patch forms data to be not present in the local cache unit 31
When, read the data in independent buffer unit 32, if the independent buffer unit 32 there are the target patch forms data, from institute
It states independent buffer unit 32 and obtains the target patch forms data;And/or
For being based on unique attribute mark, judge to whether there is in the independent buffer unit 32 and be updated with described
The corresponding stale data of piece forms data, if in the presence of the stale data is deleted.
By deleting stale data, enable to next client modules 6 that can be obtained in request target piece forms data
Newest target patch forms data is prevented after being updated due to the target patch forms data in piece list storage unit 4, and data buffer storage
Also there is the corresponding expired target patch forms data not timely updated also of target patch forms data with being updated in unit, lead
It is not newest forms data to cause the piece forms data that client modules 6 obtain, and then improves the data updating capability of piece single system,
Prevent the newer hysteresis quality of piece forms data.
Referring to Fig. 4, Fig. 4 is the structure chart of piece list service system that the 4th system embodiment of the invention provides,
As shown in Figure 4, on the basis of the first system embodiment of the present invention, the 4th system embodiment of the invention provides
A kind of single service system, wherein, at least one data buffer storage unit 3 includes:Independent buffer unit 32;
The data access interface module 2 for being based on access request, reads the data in independent buffer unit 32, if
There are the target patch forms datas for the independent buffer unit 32, then obtain the target patch list from the independent buffer unit 32
Data;And/or
For being based on unique attribute mark, judge to whether there is in the independent buffer unit 32 and be updated with described
The corresponding stale data of piece forms data, if in the presence of the stale data is deleted;Wherein, the stale data is the quilt
More data of the new film forms data before not being updated.
On the basis of first embodiment of the invention to fourth embodiment, the present invention also provides one kind to be preferably carried out
Example, and in particular to a kind of single service system, wherein, the data access interface module 2 is additionally operable in data buffer storage unit 3
There is no during the target patch forms data, the piece forms data in piece list storage unit 4 is read, and from described single storage unit 4
It is middle to obtain the target patch forms data.
The present invention also provides a kind of preferred embodiments, and in particular to a kind of single service system, wherein, the data
Access interface module 2 is additionally operable to after the target patch forms data is obtained from the independent buffer unit 32, by the target patch
Forms data is updated in local cache unit 31.
The present invention also provides a kind of preferred embodiments, and in particular to a kind of single service system, wherein, the data
Access interface module 2 is additionally operable to after obtaining the target patch forms data in described single storage unit 4, by the target patch
Forms data is updated at least one data buffer storage unit 3.
The present invention also provides a kind of preferred embodiments, and in particular to a kind of single service system, wherein, the data
Access interface module 2 is additionally operable to be identified based on preset rules and the unique attribute, whether piece forms data is updated described in judgement
For frequently-used data;
If the piece forms data that is updated for frequently-used data, reads the data in described single storage unit 4 with by institute
It states and is updated piece forms data and is updated at least one data buffer storage unit 3;
Wherein, the preset rules are:Judge and the history quilt for being updated the corresponding stale data of piece forms data
Whether access times are more than predetermined value, if more than predetermined value, then judge described in be updated piece forms data as frequently-used data;With/
Or be updated the unique attribute of piece forms data described in judging and identify whether to be present in preset frequently-used data table, if in the presence of,
Piece forms data is updated described in judgement as frequently-used data.
The present invention also provides a kind of preferred embodiments, and in particular to a kind of single service system, wherein, the data
Access interface module 2, be additionally operable to by it is described be updated piece forms data be updated at least one data buffer storage unit 3 it
Before,
Read the memory configurations of at least one data buffer storage unit 3;With,
For judging whether the currently stored amount of at least one data buffer storage unit 3 is not small based on the memory configurations
In default memory;
If currently stored amount is not less than default memory, based on each list at least one data buffer storage unit 3
The history of data is accessed record, and deleting in scheduled duration not accessed piece forms data or history, to be accessed number minimum
Piece forms data or longest forms data of the time interval of time interval current time for being accessed of last time so that it is described at least
The currently stored amount of one data buffer storage unit 3 is less than default memory;If currently stored amount is less than default memory, by the quilt
More new film forms data is updated at least one data buffer storage unit 3.
Namely be updated before piece forms data is updated at least one data buffer storage unit 3 by described, to data
The current internal storage state of buffer unit is detected, and is not less than in the buffer memory for reaching the memory upper limit namely data buffer storage unit
When the upper limit of the default internal memory quota, some data are deleted according to scheduled rule, so as to ensure data energy
Data buffer storage unit is enough updated to, when preventing from reaching the upper limit due to memory so that the data being updated can not completely update
Into data buffer storage unit, it is therefore prevented that the loss of data is updated caused by Out of Memory.It should be noted that in number
It, also can be right before according to access interface module 2, the target patch forms data is updated at least one data buffer storage unit 3
Data buffer storage unit 3 carries out memory detection and judges to ensure that the target patch forms data can completely be updated to described at least one
In a data buffer storage unit 3 (including local cache unit 31 and/or independent buffer unit 32).
Referring to Fig. 5, Fig. 5 is the structure chart for the piece single system that the embodiment of the present invention provides,
As shown in figure 5, on the basis of above-described embodiment, the embodiment provides a kind of single system, including:With
Piece list service system described in upper embodiment.
As a kind of preferred embodiment, wherein, described single system further includes:
Matchmaker provides database 8, for storing piece single data source;
Whether data update module ETL7, the piece single data source that database 8 is provided for monitoring the matchmaker update;Be used for
When matchmaker provides the piece single data source update of database 8, piece single data source synchronized update after being updated to piece list storage unit
4;
Client modules 6, for receiving target patch forms data inquiry instruction input by user, and based on target patch forms data
Inquiry instruction generates target patch forms data inquiry request to be sent to piece single pair external tapping module 1.
Referring to Fig. 6, the flow chart of piece forms data acquisition methods that the first method embodiment that Fig. 6 is the present invention provides,
As shown in fig. 6, the first method embodiment of the present invention provides a kind of forms data acquisition methods, applied to piece list
Service system, including:
Piece single pair external tapping module 1 sends to data access interface and visits when receiving target patch forms data inquiry request
Ask request;
Data access interface module 2 is based on the access request, and target patch is obtained from least one data buffer storage unit 3
Forms data, and the target patch forms data of acquisition is returned via the outbound data interface.
Refering to Fig. 7, Fig. 7 is the flow chart of piece forms data acquisition methods that the second method embodiment of the present invention provides,
As shown in fig. 7, on the basis of first method embodiment of the present invention, second method embodiment of the invention provides
A kind of forms data acquisition methods, wherein, at least one data buffer storage unit 3 includes:Local cache unit 31;
The data access interface module 2 obtains the step of target patch forms data from least one data buffer storage unit 3
Including:
The data access interface module 2 is based on access request, reads the data in local cache unit 31, if described
There are the target patch forms datas for ground buffer unit 31, then obtain the target patch forms data from the local cache unit 31.
Wherein, it should be noted that if the target patch forms data, the data access interface module is not present in local cache unit 31
2 read the piece forms data in piece list storage unit 4, and obtain the target patch forms data from described single storage unit 4,
In, described single storage unit 4 at least stores the target patch forms data.
Refering to Fig. 8, Fig. 8 is the flow chart of piece forms data acquisition methods that the third method embodiment of the present invention provides,
As shown in figure 8, on the basis of second method embodiment of the present invention, third method embodiment of the invention provides
A kind of forms data acquisition methods, wherein, at least one data buffer storage unit 3 further includes:Independent buffer unit 32;
The data access interface module 2 obtains the step of target patch forms data from least one data buffer storage unit 3
Including:
If the target patch forms data is not present in the local cache unit 31, the data access interface module 2 is read
Data in independent buffer unit 32, if the independent buffer unit 32 there are the target patch forms data, from the independence
Buffer unit 32 obtains the target patch forms data.
Wherein, it should be noted that if the target patch forms data, the data access is not present in independent buffer unit 32
Interface module 2 reads the piece forms data in piece list storage unit 4, and obtains the target patch from described single storage unit 4
Forms data, wherein, described single storage unit 4 at least stores the target patch forms data.
Refering to Fig. 9, Fig. 9 is the flow chart of piece forms data acquisition methods that the fourth method embodiment of the present invention provides,
As shown in figure 9, on the basis of first method embodiment of the invention, fourth method embodiment of the invention provides
A kind of forms data acquisition methods, wherein, at least one data buffer storage unit 3 includes:Independent buffer unit 32;
The data access interface module 2 obtains the step of target patch forms data from least one data buffer storage unit 3
Including:
Data access interface module 2 reads the data in independent buffer unit 32 based on access request, if described independently delay
There are the target patch forms datas for memory cell 32, then obtain the target patch forms data from the independent buffer unit 32.Wherein,
It should be noted that if the target patch forms data is not present in independent buffer unit 32, the data access interface module 2 is read
Piece forms data in piece list storage unit 4, and the target patch forms data is obtained from described single storage unit 4, wherein, institute
It states piece list storage unit 4 and at least stores the target patch forms data.
As a kind of preferred embodiment, the method further includes:
If the target patch forms data is not present in data buffer storage unit 3, the data access interface module 2 reads piece list and deposits
Piece forms data in storage unit 4, and the target patch forms data is obtained from described single storage unit 4, wherein, described list
Storage unit 4 at least stores the target patch forms data.
As a kind of preferred embodiment, obtained in the data access interface module 2 from the independent buffer unit 32
After the target patch forms data, further include:The target patch forms data is updated in local cache unit 31.
As a kind of preferred embodiment, obtained from described single storage unit 4 in the data access interface module 2
After the target patch forms data, further include:The target patch forms data is updated at least one data buffer storage unit 3
In, wherein, it should be noted that the target patch forms data is updated at least one data buffer storage unit 3 and is included:
The target patch forms data is updated to local cache unit 31 and/or independent buffer unit 32.
The embodiment of the present invention additionally provides a kind of forms data update method, applied to piece list service system, including:
Whether the piece forms data of 5 detection lug list storage unit 4 of data monitoring unit storage updates, and is monitoring that piece list deposits
During the piece forms data update of storage unit 4, send piece forms data and update messages to the data access interface module 2;Wherein, it is described
Piece forms data new information includes:The unique attribute mark of piece forms data is updated in piece list storage unit 4;
Data access interface module 2 updates the piece in an at least data buffer storage unit 3 based on piece forms data new information
Forms data.
Wherein, the data access interface module 2 is based on a piece forms data new information update at least data buffer storage list
Member 3 in piece forms data the step of include:
Data access interface module 2 is identified based on the unique attribute, is judged at least one data buffer storage unit 3
The corresponding stale data of piece forms data is updated with the presence or absence of with described, if in the presence of the stale data is deleted;Wherein,
The stale data is updated data of the piece forms data before not being updated to be described.
Wherein, the data access interface module 2 is based on a piece forms data new information update at least data buffer storage list
The also step of piece forms data in member 3 includes:
Data access interface module 2 is based on preset rules and the unique attribute identifies, and piece odd number is updated described in judgement
According to whether being frequently-used data;
If the piece forms data that is updated for frequently-used data, reads the data in described single storage unit 4 with by institute
It states and is updated piece forms data and is updated at least one data buffer storage unit 3;
Wherein, the preset rules are:Judge and the history quilt for being updated the corresponding stale data of piece forms data
Whether access times are more than predetermined value, if more than predetermined value, then judge described in be updated piece forms data as frequently-used data;With/
Or be updated the unique attribute of piece forms data described in judging and identify whether to be present in preset frequently-used data table, if in the presence of,
Piece forms data is updated described in judgement as frequently-used data.
Wherein, by it is described be updated the step of piece forms data is updated at least one data buffer storage unit 3 before,
It further includes:
Data access interface module 2 reads the memory configurations of at least one data buffer storage unit 3;
Data access interface module 2 is used to judge at least one data buffer storage unit 3 based on the memory configurations
Whether currently stored amount is not less than default memory;
If currently stored amount is not less than default memory, data access interface module 2 is delayed based at least one data
The history of each forms data in memory cell 3 is accessed record, delete in scheduled duration not accessed piece forms data or
Longest list of the time interval of time interval current time that history is accessed the minimum piece forms data of number or last time is accessed
Data are so that the currently stored amount of at least one data buffer storage unit 3 is less than default memory;
If currently stored amount is less than default memory, the piece forms data that is updated is updated at least one data
Buffer unit 3.
Wherein, it should be noted that the piece forms data that the embodiment of the present invention refers to can be video frequency program forms data and/
Or video playing file and/or audio list data and/or audio played file and/or graphic data file and/or word number
According to file and/or other data.
The present invention is directed to protect a kind of data acquisition and update method, piece list service system and piece single system, the present invention is real
Data acquisition and update method, the piece list service system and piece single system of example offer are applied, using multi-level buffer scheme, and according to more
Grade buffering scheme provides the method accelerated data loading and show speed, and by newest data by way of real-time update
Update cachings at different levels so that client can get newest data, and due to using multi-level buffer scheme, increase number
According to source, data acquiring mode reduces matchmaker and provides the pressure of database and the pressure of data buffer storage unit also with regard to corresponding increase,
Data throughout is increased, so as to improve data loading and show speed.
It should be understood that the above-mentioned specific embodiment of the present invention is used only for exemplary illustration or explains the present invention's
Principle, without being construed as limiting the invention.Therefore, that is done without departing from the spirit and scope of the present invention is any
Modification, equivalent replacement, improvement etc., should all be included in the protection scope of the present invention.In addition, appended claims purport of the present invention
Covering the whole variations fallen into scope and boundary or this range and the equivalent form on boundary and repairing
Change example.
Claims (22)
1. a kind of single service system, it is characterised in that, including:
Piece single pair external tapping module, for when receiving target patch forms data inquiry request, sending and visiting to data access interface
Ask request;
For being based on the access request, target patch is obtained from least one data buffer storage unit for data access interface module
Forms data, and the target patch forms data of acquisition is returned via described single pair external tapping module;And/or for being based on
Piece forms data in the piece forms data new information update at least data buffer storage unit received;
Piece list storage unit, for storing piece forms data;
Data monitoring unit, in the piece forms data update for monitoring piece list storage unit, sending the update of piece forms data and disappearing
It ceases to the data access interface module;Wherein, described forms data new information includes:Piece is updated in piece list storage unit
The unique attribute mark of forms data.
2. according to claim 1 single service system, which is characterized in that at least one data buffer storage unit packet
It includes:Local cache unit;
The data access interface module for being based on access request, reads the data in local cache unit, if the local
There are the target patch forms datas for buffer unit, then obtain the target patch forms data from the local cache unit;And/or
For being based on the unique attribute mark, judge to be updated piece odd number with the presence or absence of with described in the local cache unit
According to corresponding stale data, if in the presence of the stale data is deleted;Wherein, the stale data is updated piece to be described
Data of the forms data before not being updated.
3. according to claim 2 single service system, which is characterized in that at least one data buffer storage unit also wraps
It includes:Independent buffer unit;
The data access interface module, for when the target patch forms data is not present in the local cache unit, reading
Data in independent buffer unit if there are the target patch forms datas for the independent buffer unit, are independently cached from described
Unit obtains the target patch forms data;And/or
For being based on the unique attribute mark, judge to be updated piece odd number with the presence or absence of with described in the independent buffer unit
According to corresponding stale data, if in the presence of the stale data is deleted.
4. according to claim 1 single service system, which is characterized in that at least one data buffer storage unit packet
It includes:Independent buffer unit;
The data access interface module for being based on access request, reads the data in independent buffer unit, if the independence
There are the target patch forms datas for buffer unit, then obtain the target patch forms data from the independent buffer unit;And/or
For being based on the unique attribute mark, judge to be updated piece odd number with the presence or absence of with described in the independent buffer unit
According to corresponding stale data, if in the presence of the stale data is deleted;Wherein, the stale data is updated piece to be described
Data of the forms data before not being updated.
5. according to any one of claims 1 to 4 single service system, which is characterized in that the data access interface mould
Block is additionally operable to, when the target patch forms data is not present in data buffer storage unit, read the piece forms data in piece list storage unit,
And obtain the target patch forms data from described single storage unit.
6. according to claim 3 single service system, which is characterized in that the data access interface module is additionally operable to
After the target patch forms data is obtained from the independent buffer unit, the target patch forms data is updated to local cache list
In member.
7. according to claim 5 single service system, which is characterized in that the data access interface module is additionally operable to
After obtaining the target patch forms data in described single storage unit, the target patch forms data is updated to described at least one
In a data buffer storage unit.
8. the piece list service system according to claims 1 or 2 or 3 or 4 or 6 or 7, it is characterised in that, the data access
Interface module is additionally operable to be identified based on preset rules and the unique attribute, is updated whether piece forms data is normal described in judgement
Use data;
If the piece forms data that is updated for frequently-used data, read the data in described single storage unit with by described by more
New film forms data is updated at least one data buffer storage unit;
Wherein, the preset rules are:Judgement is accessed with the history for being updated the corresponding stale data of piece forms data
Whether number is more than predetermined value, if more than predetermined value, then judge described in be updated piece forms data as frequently-used data;And/or sentence
The disconnected unique attribute for being updated piece forms data identifies whether to be present in preset frequently-used data table, if in the presence of judging
The piece forms data that is updated is frequently-used data.
9. according to claim 8 single service system, which is characterized in that the data access interface module is additionally operable to
It is updated before piece forms data is updated at least one data buffer storage unit by described,
Read the memory configurations of at least one data buffer storage unit;With,
For judging the currently stored amount of at least one data buffer storage unit whether not less than pre- based on the memory configurations
If memory;
If currently stored amount is not less than default memory, based on each forms data at least one data buffer storage unit
History is accessed record, deletes the piece forms data or history not being accessed in scheduled duration and is accessed the minimum piece list of number
Longest forms data of the time interval of time interval current time that data or last time are accessed is so that at least one number
It is less than default memory according to the currently stored amount of buffer unit;
If currently stored amount is less than default memory, the piece forms data that is updated is updated at least one data buffer storage
Unit.
10. a kind of single system, which is characterized in that including:Claim 1 to 9 any one of them piece list service system.
11. according to claim 10 single system, which is characterized in that described single system further includes:
Matchmaker provides database, for storing piece single data source;
Whether data update module ETL, the piece single data source that database is provided for monitoring the matchmaker update;With for being provided in matchmaker
During the piece single data source update of database, piece single data source synchronized update after being updated to piece list storage unit.
12. a kind of forms data acquisition methods, applied to piece list service system, which is characterized in that including:
Piece single pair external tapping module sends to access to data access interface and ask when receiving target patch forms data inquiry request
It asks;
Data access interface module is based on the access request, and target patch odd number is obtained from least one data buffer storage unit
According to, and the target patch forms data of acquisition is returned via the outbound data interface.
13. according to the method for claim 12, which is characterized in that at least one data buffer storage unit includes:It is local
Buffer unit;
The data access interface module obtains the step of target patch forms data from least one data buffer storage unit and includes:
The data access interface module is based on access request, the data in local cache unit is read, if the local cache
There are the target patch forms datas for unit, then obtain the target patch forms data from the local cache unit.
14. according to the method for claim 13, which is characterized in that at least one data buffer storage unit further includes:Solely
Vertical buffer unit;
The data access interface module obtains the step of target patch forms data from least one data buffer storage unit and includes:
If the target patch forms data is not present in the local cache unit, the data access interface module reads independent caching
Data in unit if there are the target patch forms datas for the independent buffer unit, are obtained from the independent buffer unit
The target patch forms data.
15. according to the method for claim 12, it is characterised in that, at least one data buffer storage unit includes:It is independent
Buffer unit;
The data access interface module obtains the step of target patch forms data from least one data buffer storage unit and includes:
Data access interface module reads the data in independent buffer unit based on access request, if the independent buffer unit is deposited
In the target patch forms data, then the target patch forms data is obtained from the independent buffer unit.
16. according to claim 12 to 15 any one of them method, it is characterised in that:The method further includes:
If the target patch forms data is not present in data buffer storage unit, the data access interface module reads piece list storage unit
In piece forms data, and obtain the target patch forms data from described single storage unit, wherein, described single storage unit
At least store the target patch forms data.
17. according to the method for claim 14, which is characterized in that independently delay from described in the data access interface module
After memory cell obtains the target patch forms data, further include:The target patch forms data is updated in local cache unit.
18. according to the method for claim 16, which is characterized in that singly deposited from described in the data access interface module
After the target patch forms data is obtained in storage unit, further include:The target patch forms data is updated to described at least one
In data buffer storage unit.
19. a kind of forms data update method, applied to piece list service system, which is characterized in that including:
Whether the piece forms data of data monitoring unit detection lug list storage unit storage updates, and monitoring piece list storage unit
Piece forms data update when, send piece forms data update messages to the data access interface module;Wherein, described forms data
New information includes:The unique attribute mark of piece forms data is updated in piece list storage unit;
Data access interface module updates the piece forms data in an at least data buffer storage unit based on piece forms data new information.
20. according to claim 19 forms data update method, which is characterized in that the data access interface module base
Include the step of piece forms data new information updates the piece forms data in an at least data buffer storage unit:
Data access interface module is identified based on the unique attribute, judges whether deposited at least one data buffer storage unit
The corresponding stale data of piece forms data is being updated with described, if in the presence of the stale data is deleted;Wherein, the mistake
Issue is updated data of the piece forms data before not being updated according to be described.
21. the piece forms data update method according to claim 19 or 20, which is characterized in that the data access interface mould
The also step that block updates the piece forms data in an at least data buffer storage unit based on piece forms data new information includes:
Data access interface module is based on preset rules and the unique attribute identifies, and whether piece forms data is updated described in judgement
For frequently-used data;
If the piece forms data that is updated for frequently-used data, read the data in described single storage unit with by described by more
New film forms data is updated at least one data buffer storage unit;
Wherein, the preset rules are:Judgement is accessed with the history for being updated the corresponding stale data of piece forms data
Whether number is more than predetermined value, if more than predetermined value, then judge described in be updated piece forms data as frequently-used data;And/or sentence
The disconnected unique attribute for being updated piece forms data identifies whether to be present in preset frequently-used data table, if in the presence of judging
The piece forms data that is updated is frequently-used data.
22. according to the method for claim 21, which is characterized in that by it is described be updated piece forms data be updated to it is described extremely
Before the step of few data buffer storage unit, further include:
Data access interface module reads the memory configurations of at least one data buffer storage unit;
Data access interface module is used to judge that the current of at least one data buffer storage unit is deposited based on the memory configurations
Whether reserves are not less than default memory;
If currently stored amount is not less than default memory, data access interface module is based at least one data buffer storage unit
In the history of each forms data be accessed record, delete in scheduled duration that accessed piece forms data or history are not interviewed
Ask the minimum piece forms data of number or longest forms data of the time interval of time interval current time that last time is accessed so that
The currently stored amount for obtaining at least one data buffer storage unit is less than default memory;
If currently stored amount is less than default memory, the piece forms data that is updated is updated at least one data buffer storage
Unit.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201611159578.7A CN108228624A (en) | 2016-12-15 | 2016-12-15 | Data acquisition and update method, piece list service system and piece single system |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201611159578.7A CN108228624A (en) | 2016-12-15 | 2016-12-15 | Data acquisition and update method, piece list service system and piece single system |
Publications (1)
Publication Number | Publication Date |
---|---|
CN108228624A true CN108228624A (en) | 2018-06-29 |
Family
ID=62650471
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201611159578.7A Pending CN108228624A (en) | 2016-12-15 | 2016-12-15 | Data acquisition and update method, piece list service system and piece single system |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN108228624A (en) |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN109815060A (en) * | 2019-01-30 | 2019-05-28 | 北京百度网讯科技有限公司 | Method and device for backup information |
CN113419761A (en) * | 2021-06-24 | 2021-09-21 | 广州欢网科技有限责任公司 | Method and system for releasing sp/cp operator content injection work |
CN113542326A (en) * | 2020-04-17 | 2021-10-22 | 上海际链网络科技有限公司 | Data caching method and device for distributed system, server and storage medium |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101692229A (en) * | 2009-07-28 | 2010-04-07 | 武汉大学 | Self-adaptive multilevel cache system for three-dimensional spatial data based on data content |
CN102135985A (en) * | 2011-01-28 | 2011-07-27 | 百度在线网络技术(北京)有限公司 | Method and system for searching by calling search result of third-party search engine |
CN102508854A (en) * | 2011-09-29 | 2012-06-20 | 畅捷通信息技术股份有限公司 | Data access device and method |
CN102646072A (en) * | 2012-02-28 | 2012-08-22 | 信雅达系统工程股份有限公司 | Data management method |
CN103024593A (en) * | 2012-12-26 | 2013-04-03 | 人民搜索网络股份公司 | Online VOD (video on demand) acceleration system and online VOD playing method |
CN103226604A (en) * | 2013-04-27 | 2013-07-31 | 上海先之决信息科技发展有限公司 | SVG-based Web GIS system and relevant energy consumption monitoring system |
-
2016
- 2016-12-15 CN CN201611159578.7A patent/CN108228624A/en active Pending
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101692229A (en) * | 2009-07-28 | 2010-04-07 | 武汉大学 | Self-adaptive multilevel cache system for three-dimensional spatial data based on data content |
CN102135985A (en) * | 2011-01-28 | 2011-07-27 | 百度在线网络技术(北京)有限公司 | Method and system for searching by calling search result of third-party search engine |
CN102508854A (en) * | 2011-09-29 | 2012-06-20 | 畅捷通信息技术股份有限公司 | Data access device and method |
CN102646072A (en) * | 2012-02-28 | 2012-08-22 | 信雅达系统工程股份有限公司 | Data management method |
CN103024593A (en) * | 2012-12-26 | 2013-04-03 | 人民搜索网络股份公司 | Online VOD (video on demand) acceleration system and online VOD playing method |
CN103226604A (en) * | 2013-04-27 | 2013-07-31 | 上海先之决信息科技发展有限公司 | SVG-based Web GIS system and relevant energy consumption monitoring system |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN109815060A (en) * | 2019-01-30 | 2019-05-28 | 北京百度网讯科技有限公司 | Method and device for backup information |
CN113542326A (en) * | 2020-04-17 | 2021-10-22 | 上海际链网络科技有限公司 | Data caching method and device for distributed system, server and storage medium |
CN113542326B (en) * | 2020-04-17 | 2023-04-25 | 上海际链网络科技有限公司 | Data caching method and device of distributed system, server and storage medium |
CN113419761A (en) * | 2021-06-24 | 2021-09-21 | 广州欢网科技有限责任公司 | Method and system for releasing sp/cp operator content injection work |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN106202112B (en) | CACHE DIRECTORY method for refreshing and device | |
CN105872040B (en) | A method of write performance is stored using gateway node cache optimization distributed block | |
CN105426321B (en) | Using RDMA close friend's caching method of remote location information | |
CN105095313B (en) | A kind of data access method and equipment | |
CN103607312B (en) | Data request processing method and system for server system | |
CN100437590C (en) | Method for prefetching object | |
CN108804234A (en) | Data-storage system and its operating method | |
CN106326499B (en) | A kind of data processing method and device | |
CN101226542B (en) | Method for caching report | |
CN103488685B (en) | Fragmented-file storage method based on distributed storage system | |
CN108228624A (en) | Data acquisition and update method, piece list service system and piece single system | |
CN101673192A (en) | Method for time-sequence data processing, device and system therefor | |
CN107623722A (en) | A kind of remote data caching method, electronic equipment and storage medium | |
CN110096334A (en) | Method for caching and processing, device, equipment and computer readable storage medium | |
CN110109958A (en) | Method for caching and processing, device, equipment and computer readable storage medium | |
CN100416566C (en) | Picture data storage and read method | |
CN107786623A (en) | Message asynchronous processing method and device | |
CN101217449B (en) | A remote call management procedure | |
CN103986779A (en) | Application data transmission method, device and terminal equipment | |
CN100394404C (en) | System and method for management of metadata | |
CN107704573A (en) | A kind of intelligent buffer method coupled with business | |
CN103209397A (en) | Method for controlling sending and receiving of short messages and server and terminal thereof | |
CN107888687A (en) | A kind of agent client storage accelerated method and system based on distributed memory system | |
CN110245129A (en) | Distributed global data deduplication method and device | |
CN107943958A (en) | A kind of Individual Income Tax master data sharing method and system |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20180629 |
|
WD01 | Invention patent application deemed withdrawn after publication |