CN106776249A - A kind of processing method and system of the business diary for concurrently generating - Google Patents
A kind of processing method and system of the business diary for concurrently generating Download PDFInfo
- Publication number
- CN106776249A CN106776249A CN201611071496.7A CN201611071496A CN106776249A CN 106776249 A CN106776249 A CN 106776249A CN 201611071496 A CN201611071496 A CN 201611071496A CN 106776249 A CN106776249 A CN 106776249A
- Authority
- CN
- China
- Prior art keywords
- log area
- business diary
- buffering
- querying condition
- business
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
- G06F11/3466—Performance evaluation by tracing or monitoring
- G06F11/3476—Data logging
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
- G06F11/3404—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment for parallel or distributed programming
Abstract
The present invention passes through the processing method of the business diary to concurrently generating, and is separated with physical store by by buffering log area, reduces system pressure, and then reduces the slow problem of system response, improves insertion and the storage efficiency of business diary.Methods described includes：The business diary write-in buffering log area that will concurrently generate；Judge whether the state for buffering log area reaches pre-conditioned；When the state for buffering log area reaches pre-conditioned, during the business diary buffered in log area dumped into long-term storage media；According to existing querying condition in wake-up log area, the query resultses for waking up log area are updated, to realize the inquiry for business diary.
Description
Technical field
The present invention relates to Java Web fields, and more particularly, to a kind of business diary for concurrently generating
Processing method and system.
Background technology
In conventional traditional web projects, it is often necessary to all kinds of daily records outputs, be used to help administrative staff's orientation problem,
Monitor hazardous operation etc..General daily record is divided into two classes：
(1) system journal：Refer to the Debugging message of system-level and code level, mistake and program debugging are investigated for system；
(2) business diary：Refer to the associative operation record of business and security permission change, for monitoring the crucial behaviour of user
Make etc..
Existing frequently-used way is usually to add extra log code to customer data base business in Business Processing code
Log recording is inserted in log sheet, although this mode has coding to be easily understood under low concurrent scene, the advantages of real-time is high,
But it is unable to do what one wishes just to seem in the environment of high concurrent.In routine use scene, business diary can be called by high frequency, be used
Family amount reach it is a certain amount of after, data base read-write, request data source connection requirement can be frequently carried out, to application program and database
Very big burden is all caused, causes system to respond slow or even stuck, retardance business function is responded, seriously reduces Consumer's Experience.
The content of the invention
Present invention aim at the processing method and system for providing a kind of business diary for concurrently generating, solve high
The problems such as system response caused by the system pressure for concurrently triggering when lower insertion, inquiry business daily record is slow.
According to an aspect of the present invention, there is provided a kind of processing method of business diary for concurrently generating, the side
Method includes,
The business diary write-in buffering log area that will concurrently generate；
Judge whether the state for buffering log area reaches pre-conditioned；
When the state for buffering log area reaches pre-conditioned, the business diary buffered in log area is dumped to and is deposited for a long time
In storage media；And
According to existing querying condition in wake-up log area, the query resultses for waking up log area are updated, to realize being directed to
The inquiry of business diary.
Preferably, it is described pre-conditioned to be：Whether the size of data of the business diary cached in buffering log area reaches is faced
Dividing value.
Preferably, it is described pre-conditioned to be：Whether interval time meets the predetermined amount of time for synchronizing operation.
Preferably, when interval time meets synchronizes the predetermined amount of time of operation, if without number in buffering log area
According to then interrupting and synchronization and restart timing.
Preferably, business diary is written to buffering log area by calling insertion interface.
Preferably, the data of the wake-up log area storage are the corresponding relation of querying condition and query resultses, wherein
The querying condition does not include paging condition and time dimension.
Preferably, the inquiry for business diary is：
Meet the set R1 of the business diary of querying condition in acquisition buffering log area；
Inquiry whether there is the query resultses for meeting querying condition in waking up log area；
If there are the query resultses for meeting querying condition in waking up log area, the content according to query resultses is obtained
The set R2 of business diary；
If being looked into long-term storage media in the absence of the query resultses for meeting querying condition in waking up log area
The set R2 of the business diary for meeting querying condition is ask and obtained, and updates the query resultses for waking up log area；
Calculate the union R of R1 and R2-, and to R-The filtering of time dimension and paging condition is carried out, obtains all meeting inquiry
The set R of the business diary of condition.
Preferably, the filter method is to iterate over method.
Preferably, if not existing querying condition in waking up log area, the query resultses for waking up log area are not updated.
According to another aspect of the present invention, there is provided a kind of processing system of business diary for concurrently generating, including：
Daily record writing unit, for the business diary write-in buffering log area that will concurrently generate；
Condition adjudgement unit, for judging it is pre-conditioned whether the state for buffering log area reaches；
Unloading unit, for the business diary buffered in log area to be dumped to for the long-term of longer-term storage business diary
In storage medium；
Log query unit, for obtaining buffering log area according to querying condition, waking up log area and longer-term storage Jie
Business diary in matter simultaneously obtains Query Result；
Wherein described wake-up log area is used to store the corresponding relation of querying condition and query resultses.
Preferably, the buffering log area is cached in list forms.
Preferably, the wake-up log area is stored in map forms.
Preferably, enter when state of the unloading unit in condition adjudgement unit judges buffering log area reaches pre-conditioned
Row unloading.
Preferably, it is described pre-conditioned to be：Whether the size of data of the business diary cached in buffering log area reaches is faced
Dividing value.
Preferably, it is described pre-conditioned to be：Whether interval time meets the predetermined amount of time for synchronizing operation.
By buffering, log area will be inserted the present invention and inquiry business daily record is separated with the physical store of daily record, slow
Storage and inquiry velocity in the insertion rushed in log area and velocity contrast's file system of inquiry or Database Systems can be
Increase, and be to concentrate and multithreading when being synchronized to long-term storage media, the pressure brought to system is smaller, reduces system response
Slow problem.
Brief description of the drawings
By reference to the following drawings, illustrative embodiments of the invention can be more fully understood by：
Fig. 1 is the flow chart of the processing method according to the preferred embodiment of the present invention；
Fig. 2 is the querying method flow chart of the business diary according to the preferred embodiment of the present invention；
Fig. 3 is the structure chart of the processing system according to the preferred embodiment of the present invention.
Specific embodiment
With reference now to accompanying drawing, illustrative embodiments of the invention are introduced, however, the present invention can use many different shapes
Formula is implemented, and is not limited to embodiment described herein, there is provided these embodiments are to disclose at large and fully
The present invention, and fully pass on the scope of the present invention to person of ordinary skill in the field.For showing for being illustrated in the accompanying drawings
Term in example property implementation method is not limitation of the invention.In the accompanying drawings, identical cells/elements are attached using identical
Icon is remembered.
Unless otherwise indicated, term (including scientific and technical terminology) used herein has to person of ordinary skill in the field
It is common to understand implication.Further it will be understood that the term limited with usually used dictionary, is appreciated that and it
The linguistic context of association area has consistent implication, and is not construed as Utopian or excessively formal meaning.
Fig. 1 is the flow chart of the processing method according to the preferred embodiment of the present invention.As shown in figure 1, the business for concurrently generating
The processing method 100 of daily record is since step 101.In a step 101, the business diary that will concurrently generate is connect by calling insertion
Mouth is written in buffering log area and is cached.Preferably, log cache area can be local memory or independent buffer service
Device, because the speed that internal memory or independent caching server are written and read is very fast, the business diary generated under high concurrent is more
When, can be rapidly performed by the caching of business diary, reduce because frequently write into business diary and caused by system response it is slow etc.
Problem.
In a step 102, judge log cache area state whether reach it is pre-conditioned, if the state in log cache area is
Reach pre-conditioned, then carry out step 103；If the state in log cache area is not reaching to pre-conditioned, do not operated.It is excellent
Whether selection of land, the size of data of the pre-conditioned business diary for being, being cached in buffering log area reaches critical value；Can also
For whether interval time meets the predetermined amount of time for synchronizing operation, the pre- timing for synchronizing operation is met when interval time
Between section when, if there is no a data in buffering log area, interrupt synchronous and reclocking.
In step 103, longer-term storage is carried out in the business diary buffered in log area being dumped into long-term storage media.
Preferably, long-term storage media can be database or other can reach the storage medium of effect same, and relative in local
Deposit or independent caching server, the read or write speed of long-term storage media is slower, but the data storage of long-term safety can be carried out.
At step 104, according to existing querying condition in wake-up log area, the Query Result for waking up log area is updated
Collection, to realize the inquiry for business diary.Preferably, the data for waking up log area storage are querying condition and query resultses
Corresponding relation, i.e. key-value relation pairs, key is querying condition, and querying condition here do not include paging condition and when
Between dimension, value is query resultses, and because every business diary has unique numbering, value is and meets querying condition
The set of the numbering of all business diaries of key.When the renewal of log area wake up, according to waking up, log area is existing
Key, the numbering that will meet the new business diary for dumping to long-term storage media of querying condition is added in corresponding value.
Fig. 2 is the querying method flow chart of the business diary according to the preferred embodiment of the present invention.As shown in Fig. 2 business day
The querying method 200 of will is since step 201.In step 201, according to querying condition, symbol in buffering log area is obtained first
Close the set R1 of the business diary of querying condition.Preferably, cached in buffering log area and do not dump to long-term storage media also
In partial service daily record, when needing to carry out the inquiry of business diary because being all business diaries for having produced of inquiry,
There is a part of daily record in buffering log area and long-term storage media, so meeting inquiry bar in obtaining buffering log area first
The daily record of part.
In step 202., inquiry whether there is the query resultses for meeting querying condition in waking up log area, if in the presence of,
Then carry out step 203；If not existing, step 204 is carried out.Preferably, the query resultses for meeting querying condition here, be
Refer to wake up in log area there is a value, its corresponding key is completely the same with this querying condition, for example this is looked into
Inquiry condition is " the high-risk operation of the artificial Xiao Ming of operation ", if there is a key in waking up log area is the " height of the artificial Xiao Ming of operation
Danger operation ", then the business diary corresponding to the value in the corresponding value of the key is returned the result R2 inquired about as this.
There are the query resultses for meeting querying condition in log area is waken up, then carry out step 203.In step 203,
To wake up and meet the query resultses of querying condition in log area and return as R2.Preferably, by business diary from buffering day
When will area is transferred in long-term storage media, the renewal operation of log area is synchronously carried out waking up, if existing in waking up log area
One key is consistent with this querying condition, then the set one of the business diary corresponding to the value of its corresponding value is surely represented
The set of all business diaries for meeting querying condition, i.e. set R2 in long-term storage media.
In the absence of the query resultses for meeting querying condition in log area is waken up, then step 204 is carried out.In step 204
In, in the absence of the query resultses for meeting querying condition in wake-up log area, then inquire about eligible in long-term storage media
Business diary, and obtain the set R2 of Query Result.Preferably, long-term storage media is because amount of storage is big, read or write speed is relative
The reason such as slower, only when in waking up log area in the absence of the query resultses for meeting querying condition, in long-term storage media
The business diary of storage is inquired about.
When step 204 is finished, step 205 is performed.According to the business for meeting querying condition in long-term storage media
The set of daily record updates the data for waking up log area, increases new entry and preserves with this querying condition as key, with longer-term storage
Query Result in medium is the corresponding relation of value, is directly applied during so as to inquiry next time.
When step 203 or 205 be finished after, perform step 206.In step 206, the union R of R1 and R2 is calculated-,
And to R-The filtering of time dimension and paging condition is carried out, the set R of all business diaries for meeting querying condition is obtained.It is preferred that
Ground, the filter method is carried out with first carrying out the order of paging condition after time dimension when being filtered to iterate over method.
Fig. 3 is the structure chart of the processing system according to the preferred embodiment of the present invention.As shown in figure 3, processing system 300 is by day
Will writing unit 301, condition adjudgement unit 302, unloading unit 303, log query unit 304, buffering log area 305, wake-up
Log area 306 and long-term storage media 307 are constituted.
Preferably, daily record writing unit 301 is used for the business diary write-in buffering log area 305 that will concurrently generate, therefore day
Will writing unit can be to insert interface in Java Web, or can reach other interfaces of write-in effect.
Preferably, condition adjudgement unit 302 be used for judge buffering log area 305 state whether reach it is pre-conditioned, when
When the state for buffering log area 305 reaches pre-conditioned, unloading unit 303 will buffer all data conversion storages in log area 305
Into long-term storage media 307.Preferably, the data of the pre-conditioned business diary for being, being cached in buffering log area 305
Whether size reaches critical value；Whether the predetermined amount of time for synchronizing operation can also be met for interval time, when interval
Between meet when synchronizing the predetermined amount of time of operation, if there is no data in buffering log area 305, interrupt unloading operation and lay equal stress on
New timing.
Preferably, log query unit 304 is mainly used in obtaining buffering log area 305 in query process, wakes up log area
306 and long-term storage media 307 in business diary.In acquisition process, meet in acquisition buffering log area 305 look into first
The set R1 of the business diary of inquiry condition, secondly judges to whether there is the inquiry knot for meeting querying condition in wake-up log area 306
Fruit collects, if in the presence of, the query resultses correspondence business diary set R2 is obtained, if not existing, obtain longer-term storage and be situated between
Meet the business diary set R2 of querying condition in matter 307.The union of R1 and R2 is finally calculated, and carries out time dimension and paging
The filtering of condition, obtains the set R of all business diaries for meeting querying condition.
Preferably, the buffering log area 305 carries out the caching of business diary in list forms, the wake-up log area with
Map forms carry out the storage of key-value relation pairs.
The present invention is described by reference to a small amount of implementation method.However, it is known in those skilled in the art, as
What subsidiary Patent right requirement was limited, except the present invention other embodiments disclosed above equally fall of the invention
In the range of.
Normally, all terms for using in the claims are all solved according to them in the usual implication of technical field
Release, unless clearly defined in addition wherein.It is all of to be all opened ground with reference to " one/described/be somebody's turn to do [device, component etc.] "
At least one of described device, component etc. example is construed to, unless otherwise expressly specified.Any method disclosed herein
Step need not all be run with disclosed accurate order, unless explicitly stated otherwise.
Claims (15)
1. a kind of processing method of business diary for concurrently generating, methods described includes,
The business diary write-in buffering log area that will concurrently generate；
Judge whether the state for buffering log area reaches pre-conditioned；
When the state for buffering log area reaches pre-conditioned, the business diary buffered in log area is dumped into longer-term storage and is situated between
In matter；And
According to existing querying condition in wake-up log area, the query resultses for waking up log area are updated, to realize being directed to business
The inquiry of daily record.
2. method according to claim 1, it is characterised in that described pre-conditioned to be：The industry cached in buffering log area
Whether the size of data of business daily record reaches critical value.
3. method according to claim 1, it is characterised in that described pre-conditioned to be：Whether interval time meets is carried out
The predetermined amount of time of simultaneously operating.
4. method according to claim 3, when interval time meets synchronizes the predetermined amount of time of operation, if buffering
There is no data in log area, then interrupt synchronization and restart timing.
5. method according to claim 1, it is characterised in that business diary is written to buffering by calling insertion interface
Log area.
6. method according to claim 1, it is characterised in that the data of the wake-up log area storage for querying condition with
The corresponding relation of query resultses, wherein the querying condition does not include paging condition and time dimension.
7. method according to claim 1, it is characterised in that the inquiry for business diary is：
Meet the set R1 of the business diary of querying condition in acquisition buffering log area；
Inquiry whether there is the query resultses for meeting querying condition in waking up log area；
If there are the query resultses for meeting querying condition in waking up log area, the content according to query resultses obtains business
The set R2 of daily record；
If being inquired about simultaneously in long-term storage media in the absence of the query resultses for meeting querying condition in waking up log area
Acquisition meets the set R2 of the business diary of querying condition, and updates the query resultses for waking up log area；
Calculate the union R of R1 and R2-, and to R-The filtering of time dimension and paging condition is carried out, obtains all meeting querying condition
Business diary set R.
8. method according to claim 7, it is characterised in that the filter method is to iterate over method.
9. method according to claim 1, if not existing querying condition in waking up log area, does not update wake-up log area
Query resultses.
10. the processing system of a kind of business diary for concurrently generating, including：
Daily record writing unit, for the business diary write-in buffering log area that will concurrently generate；
Condition adjudgement unit, for judging it is pre-conditioned whether the state for buffering log area reaches；
Unloading unit, for the business diary buffered in log area to be dumped to the longer-term storage for longer-term storage business diary
In medium；
Log query unit, in obtaining buffering log area according to querying condition, waking up log area and long-term storage media
Business diary and obtain Query Result；
Wherein described wake-up log area is used to store the corresponding relation of querying condition and query resultses.
11. systems according to claim 10, it is characterised in that the buffering log area is cached in list forms.
12. systems according to claim 10, it is characterised in that the wake-up log area is stored in map forms.
13. systems according to claim 10, it is characterised in that the unloading unit is buffered in condition adjudgement unit judges
Unloading is carried out when the state of log area reaches pre-conditioned.
14. systems according to claim 10, it is characterised in that described pre-conditioned to be：Cached in buffering log area
Whether the size of data of business diary reaches critical value.
15. systems according to claim 10, it is characterised in that described pre-conditioned to be：Interval time whether meet into
The predetermined amount of time of row simultaneously operating.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611071496.7A CN106776249B (en) | 2016-11-28 | 2016-11-28 | Method and system for processing concurrently generated service logs |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611071496.7A CN106776249B (en) | 2016-11-28 | 2016-11-28 | Method and system for processing concurrently generated service logs |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106776249A true CN106776249A (en) | 2017-05-31 |
| CN106776249B CN106776249B (en) | 2020-07-28 |
Family
ID=58905360
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611071496.7A Active CN106776249B (en) | 2016-11-28 | 2016-11-28 | Method and system for processing concurrently generated service logs |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106776249B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108197233A (en) * | 2017-12-29 | 2018-06-22 | 飞狐信息技术（天津）有限公司 | A kind of data managing method, middleware and data management system |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102200972A (en) * | 2010-03-24 | 2011-09-28 | 腾讯科技（北京）有限公司 | Database maintenance method and database maintenance equipment |
| CN103164511A (en) * | 2013-02-21 | 2013-06-19 | 烽火通信科技股份有限公司 | Method for storage event log to automatically filter repeated jitter data |
| CN105488201A (en) * | 2015-12-08 | 2016-04-13 | 北京皮尔布莱尼软件有限公司 | Log inquiry method and system |
-
2016
- 2016-11-28 CN CN201611071496.7A patent/CN106776249B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102200972A (en) * | 2010-03-24 | 2011-09-28 | 腾讯科技（北京）有限公司 | Database maintenance method and database maintenance equipment |
| CN103164511A (en) * | 2013-02-21 | 2013-06-19 | 烽火通信科技股份有限公司 | Method for storage event log to automatically filter repeated jitter data |
| CN105488201A (en) * | 2015-12-08 | 2016-04-13 | 北京皮尔布莱尼软件有限公司 | Log inquiry method and system |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108197233A (en) * | 2017-12-29 | 2018-06-22 | 飞狐信息技术（天津）有限公司 | A kind of data managing method, middleware and data management system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106776249B (en) | 2020-07-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11113260B2 (en) | Transactional access to records on secondary storage in an in-memory database | |
| US7953710B2 (en) | Multi-master database synchronization without loss of convergence | |
| US10262002B2 (en) | Consistent execution of partial queries in hybrid DBMS | |
| EP3170106B1 (en) | High throughput data modifications using blind update operations | |
| CN104301360B (en) | A kind of method of logdata record, log server and system | |
| US8055613B1 (en) | Method and apparatus for efficiently detecting and logging file system changes | |
| KR101255391B1 (en) | Database corruption recovery systems and methods | |
| CN101183322B (en) | Method for deferred logging and apparatus thereof | |
| CN100583050C (en) | Method for protecting and recovering continuous data based on time stamp diary memory | |
| US8560500B2 (en) | Method and system for removing rows from directory tables | |
| US20060259517A1 (en) | Data synchronizer with failover facility | |
| US20060020616A1 (en) | Indexing operational logs in a distributed processing system | |
| KR102311032B1 (en) | Database Synchronization | |
| CN103475716B (en) | The method and system of data sharing is realized by shared storage | |
| CN105279285B (en) | A kind of synchronization system and method for relevant database and non-relational database | |
| CN108153804B (en) | Metadata log updating method for symmetric distributed file system | |
| CN106503158A (en) | Method of data synchronization and device | |
| CN111880993B (en) | Cluster operation and maintenance state diagnosis method, operation and maintenance monitoring system, terminal and storage medium | |
| CN111367991B (en) | MongoDB data real-time synchronization method and system based on message queue | |
| EP2901316A2 (en) | Method and system for memory efficient, update optimized, transactional full-text index view maintenance | |
| CN105550230B (en) | The method for detecting and device of distributed memory system node failure | |
| US8880478B2 (en) | Scan-free archiving | |
| CN106776249A (en) | A kind of processing method and system of the business diary for concurrently generating | |
| CN101853278A (en) | Application method of data storage system | |
| CN102597995A (en) | Synchronizing database and non-database resources |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |