CN108038018A - Expansible daily record data storage method and device - Google Patents
Expansible daily record data storage method and device Download PDFInfo
- Publication number
- CN108038018A CN108038018A CN201711416857.1A CN201711416857A CN108038018A CN 108038018 A CN108038018 A CN 108038018A CN 201711416857 A CN201711416857 A CN 201711416857A CN 108038018 A CN108038018 A CN 108038018A
- Authority
- CN
- China
- Prior art keywords
- field
- change rate
- dictionary
- daily record
- mode
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 69
- 238000013500 data storage Methods 0.000 title claims abstract description 34
- 230000008859 change Effects 0.000 claims abstract description 79
- 238000004590 computer program Methods 0.000 claims description 6
- 230000003068 static effect Effects 0.000 claims description 6
- 230000008569 process Effects 0.000 abstract description 14
- 238000005516 engineering process Methods 0.000 abstract description 4
- 230000006870 function Effects 0.000 description 8
- 238000010586 diagram Methods 0.000 description 5
- 230000009471 action Effects 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 239000002699 waste material Substances 0.000 description 2
- 241001269238 Data Species 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/0703—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation
- G06F11/0766—Error or fault reporting or storing
- G06F11/0787—Storage of error reports, e.g. persistent data storage, storage using memory protection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/0703—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation
- G06F11/0766—Error or fault reporting or storing
- G06F11/0775—Content or structure details of the error report, e.g. specific table structure, specific error fields
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3065—Monitoring arrangements determined by the means or processing involved in reporting the monitored data
- G06F11/3072—Monitoring arrangements determined by the means or processing involved in reporting the monitored data where the reporting involves data filtering, e.g. pattern matching, time or event triggered, adaptive or policy-based reporting
- G06F11/3082—Monitoring arrangements determined by the means or processing involved in reporting the monitored data where the reporting involves data filtering, e.g. pattern matching, time or event triggered, adaptive or policy-based reporting the data filtering being achieved by aggregating or compressing the monitored data
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Debugging And Monitoring (AREA)
Abstract
The present invention provides expansible daily record data storage method and device, it is related to field of computer technology, wherein, which includes：First, compare the change rate of each field and the size of standards change rate set in advance in daily record data, afterwards, when change rate is less than standards change rate, field is put into field dictionary and is stored, and key assignments different in each field is stored by Field ID, field contents store numerical value different in each field, when change rate is more than or equal to standards change rate, field is put into mode dictionary storage, and key assignments different in each field is stored by pattern ID, pattern ID stores key assignments different in each field, mode contents store numerical value different in each field, daily record data is divided into the small part of change rate and the big part of change rate, key assignments in field can be stored in the form of dictionary, so as to avoid in storing process, there are the problem of bulk information redundancy.
Description
Technical field
The present invention relates to field of computer technology, more particularly to expansible daily record data storage method and device.
Background technology
Daily record data (English full name is log data) is exactly the inherent meaning of a daily record, i.e. daily record data is exactly one
It is used for showing generate the information of daily record why in daily record.General daily record data include the time, client ip, server-side IP,
The information such as client host name, server-side host name, user, operation, daily record preserve equipment receive after daily record data by data into
Row slitting, or global storage are disk file, its specific storage organization is as shown in Figure 1.
But having the data largely repeated in daily record data, such as IP, port, host name, action type, are operating
During store one by one these daily record datas can cause this portions of log data repeat record, so as to take a large amount of disk spaces.
To sum up, on daily record data, there are the problem of substantial amounts of information redundancy, there is no effectively in storing process at present
Solution.
The content of the invention
In view of this, the purpose of the embodiment of the present invention is the provision of expansible daily record data storage method and device,
By setting field dictionary and mode dictionary etc., efficiently solve daily record data information bulk redundancy in storing process and ask
Topic.
In a first aspect, an embodiment of the present invention provides expansible daily record data storage method, including：Compare daily record data
In each field change rate and the size of standards change rate set in advance；
When change rate is less than standards change rate, field is put into field dictionary and is stored, wherein, in field dictionary
The storage format of data includes Field ID and field contents, and Field ID is used for storing key assignments different in each field, in field
Hold and be used for storing numerical value different in each field；
When change rate is more than or equal to standards change rate, field is put into mode dictionary and is stored, wherein, pattern
The storage format of data includes pattern ID and mode contents in dictionary, and pattern ID is used for storing key assignments different in each field,
Pattern ID is used for storing key assignments different in each field, and mode contents are used for storing numerical value different in each field.
With reference to first aspect, an embodiment of the present invention provides the first possible embodiment of first aspect, wherein, side
Method further includes：
The storage format of data further includes Field counter in field dictionary, when change rate is less than standards change rate, word
Segment counter is used for storing the number that each field occurs；
When counting the number that each field occurs, number corresponding with static fields is extracted in Field counter.
With reference to first aspect, an embodiment of the present invention provides second of possible embodiment of first aspect, wherein, side
Method further includes：
The storage format of data further includes mode counter in mode dictionary, when change rate is more than or equal to standards change
During rate, mode counter is used for storing the number that each field occurs；
When counting the number that each field occurs, number corresponding with static fields is extracted in mode counter.
With reference to first aspect, an embodiment of the present invention provides the third possible embodiment of first aspect, wherein, side
Method further includes：
Establish field dictionary and mode dictionary respectively according to priori value.
With reference to first aspect, an embodiment of the present invention provides the 4th kind of possible embodiment of first aspect, wherein, side
Method further includes：
Field dictionary and mode dictionary are established according to field dynamic.
With reference to first aspect, an embodiment of the present invention provides the 5th kind of possible embodiment of first aspect, wherein, side
Method further includes：
When daily record data is from different application programs or different equipment, connect according to open architecture and unified
Mouth carries out the foundation of field dictionary and mode dictionary.
With reference to first aspect, an embodiment of the present invention provides the 6th kind of possible embodiment of first aspect, wherein, word
Section ID and pattern ID is represented with binary code.
Second aspect, an embodiment of the present invention provides expansible log data storage, including：
Change rate comparison module, for comparing the change rate of each field and standards change set in advance in daily record data
The size of rate；
Field memory module, for when change rate being less than standards change rate, field being put into field dictionary and is deposited
Storage, wherein, the storage format of data includes Field ID and field contents in field dictionary, and Field ID is used for storing in each field
Different key assignments, field contents are used for storing numerical value different in each field；
Pattern memory module, for when change rate is more than or equal to standards change rate, field to be put into mode dictionary
Stored, wherein, the storage format of data includes pattern ID and mode contents in mode dictionary, and pattern ID is used for storing each
Different key assignments in field, pattern ID are used for storing key assignments different in each field, and mode contents are used for storing each field
Middle different numerical value.
The third aspect, the embodiment of the present invention also provide a kind of terminal, including memory and processor, and memory is used to deposit
Storage supports processor to perform the program for the expansible daily record data storage method that above-mentioned aspect provides, and processor is configured as using
In performing the program that is stored in memory.
Fourth aspect, the embodiment of the present invention also provide a kind of computer-readable recording medium, computer-readable recording medium
On be stored with computer program, when computer program is run by processor perform any of the above-described method the step of.
Expansible daily record data storage method provided in an embodiment of the present invention and device, wherein, the expansible daily record
Date storage method includes：It is to compare the change rate of each field and standards change rate set in advance in daily record data first
Size is, it is necessary to illustrate, and standards change rate set in advance is a reference value, and its numerical values recited can be according to reality
Border usage scenario is flexibly set, and after above-mentioned comparison procedure, when change rate is less than standards change rate, field is put into
Stored in field dictionary, wherein, the storage format of data includes Field ID and field contents in field dictionary, and Field ID is used
To store key assignments different in each field, correspondingly, identical key assignments is represented in field dictionary with same Field ID,
So as to save in field dictionary the space waste stored one by one caused by key assignments, field contents are used for storing in each field
Different numerical value, afterwards, when change rate is more than or equal to standards change rate, is put into mode dictionary by field and is stored,
In this application, the storage format of data includes pattern ID and mode contents in mode dictionary, also, pattern ID is used for storing respectively
Different key assignments in a field, correspondingly, identical key assignments is represented in mode dictionary with same pattern ID, so as to
The space waste caused by key assignments is stored in save mode dictionary one by one, pattern ID is used for storing key different in each field
Value, mode contents are used for storing numerical value different in each field, and the storage form pair of above-mentioned key assignments-numerical value is used in the application
Daily record data is stored, and in the prior art, the pattern that each daily record data is stored one by one is compared, and reduces daily record
Data largely repeat the process of record, effectively save the size of the shared disk space of daily record data storage, and then avoid
The bulk redundancy phenomenon of daily record data information in storing process.
Other features and advantages of the present invention will illustrate in the following description, also, partly become from specification
Obtain it is clear that or being understood by implementing the present invention.The purpose of the present invention and other advantages are in specification, claims
And specifically noted structure is realized and obtained in attached drawing.
To enable the above objects, features and advantages of the present invention to become apparent, preferred embodiment cited below particularly, and coordinate
Appended attached drawing, is described in detail below.
Brief description of the drawings
, below will be to specific in order to illustrate more clearly of the specific embodiment of the invention or technical solution of the prior art
Embodiment or attached drawing needed to be used in the description of the prior art are briefly described, it should be apparent that, in describing below
Attached drawing is some embodiments of the present invention, for those of ordinary skill in the art, before not making the creative labor
Put, other attached drawings can also be obtained according to these attached drawings.
Fig. 1 shows the structure diagram of daily record data storage in the prior art；
Fig. 2 shows the first pass figure for the expansible daily record data storage method that the embodiment of the present invention is provided；
Fig. 3 shows the second flow chart for the expansible daily record data storage method that the embodiment of the present invention is provided；
Fig. 4 shows the structure connection figure for the expansible log data storage that the embodiment of the present invention is provided.
Icon：11- change rate comparison modules；12- field memory modules；13- pattern memory modules.
Embodiment
Below in conjunction with attached drawing in the embodiment of the present invention, the technical solution in the embodiment of the present invention is carried out clear, complete
Ground describes, it is clear that described embodiment is only part of the embodiment of the present invention, instead of all the embodiments.Usually exist
The component of the embodiment of the present invention described and illustrated in attached drawing can be arranged and designed with a variety of configurations herein.Cause
This, the detailed description of the embodiment of the present invention to providing in the accompanying drawings is not intended to limit claimed invention below
Scope, but it is merely representative of the selected embodiment of the present invention.Based on the embodiment of the present invention, those skilled in the art are not doing
Go out all other embodiments obtained on the premise of creative work, belong to the scope of protection of the invention.
At present, it is necessary to by the time of daily record data, client ip, server-side IP, client in daily record data storing process
The information such as end main frame name, server-side host name, user, operation are put into memory space one by one to be stored, i.e. daily record is preserved and set
Data are carried out slitting after daily record data by standby receive, or global storage is disk file.In this way, the major part in daily record data
The data repeated, such as IP, port, host name, action type, will be repeatedly recorded in storing process, so as to take big
Disk space is measured, serious information redundancy is caused while a large amount of memory spaces are wasted.
Based on this, an embodiment of the present invention provides expansible daily record data storage method and device, below by implementation
Example is described.
Embodiment 1
Referring to Fig. 2 and Fig. 3, the expansible daily record data storage method that the present embodiment proposes specifically includes following steps：
Step S101：Compare the change rate of each field and the size of standards change rate set in advance in daily record data.
Need exist for illustrating, standards change rate set in advance is a priori value, and the size of the priori value and storage are empty
Between size, the requirement etc. of information redundance it is information-related.The size for comparing the change rate of field be in order to will in daily record data it is each
A field distinguishes.
Step S102：When change rate is less than standards change rate, field is put into field dictionary and is stored, i.e. is right
In the storage of the less field of change put it into field dictionary, need exist for illustrating, number in field dictionary
According to storage format include Field ID and field contents, in implementation process, Field ID be used for store it is different in each field
Key assignments, i.e., each different key assignments take a Field ID, and identical key assignments takes same Field ID, in this way, implementing
Cheng Zhong, can represent its title, so as to effectively save the sky of disk when key assignments is repeated several times by same Field ID
Between size, and each corresponding concrete numerical value of key assignments stores respectively, i.e. field contents are for storing number different in each field
Value, when carrying out the lookup of daily record data, can be combined by Field ID and corresponding field contents.
Step S103：When change rate is more than or equal to standards change rate, field is put into mode dictionary and is stored,
That is, the storage of the field for changing greatly is put it into field dictionary, needs exist for illustrating, mode dictionary
The storage format of middle data includes pattern ID and mode contents, and pattern ID is usually referred to as fixed part, and fixed part can
Represent that this partial data is smaller for specific system change, or Limited Number with a character string mode, therefore, Ke Yijian
A mode dictionary is found, pattern ID is used for storing key assignments different in each field, i.e., each different key assignments takes a mould
Formulas I D, identical key assignments take same pattern ID, in this way, in implementation process, can be by same when key assignments is repeated several times
A pattern ID represents its title, so as to effectively save the space size of disk, and the corresponding concrete numerical value of each key assignments
Store respectively, mode contents are used for storing numerical value different in each field.I.e. with a series of pattern ID and the number changed greatly
According to combination store this partial data.
In addition, expansible daily record data storage method further includes：
Step S001：The storage format of data further includes Field counter in field dictionary, i.e., in specific implementation process,
Each field in daily record data is represented in the form of key assignments-numerical value-reference count, for the storage of above two field
Pattern, when change rate is less than standards change rate, Field counter is used for storing the number that each field occurs, i.e. field count
The total number of each identical Field ID is stored in device, when it is implemented, the corresponding Field ID of each field counts respectively, when having
When the Field ID repeated occurs again, count and add one again.
Step S002：When counting the number that each field occurs, extracted in Field counter corresponding to static fields
Number.When the occurrence number of Field ID is wanted to call in other application end, the word with statistics is extracted directly from Field counter
The corresponding Field ID number of section, it is convenient and efficient.
In addition, it is necessary to illustrate, above-mentioned Field ID and pattern ID are represented with binary code.Need exist for into
Row explanation, in digital circuit, binary number refers to use binary notation system, i.e., with 2 for radix number representation system table
The numeral shown, is usually represented with two different symbols 0 and 1.Represent to be stored easy to computer using binary code
And calling.
In addition, in the prior art in order to by daily record data come the problem of checking, it is necessary to go out to a certain data in daily record
Occurrence number is counted, and is counted again during the occurrence number of a certain data, it is necessary to scan all data so in statistical log
Number, efficiency are very low.Further included in the expansible daily record data storage method of the application：
(1) storage format of data further includes mode counter in mode dictionary, i.e., in specific implementation process, by daily record
Each field in data is represented in the form of key assignments-numerical value-reference count, when change rate is more than or equal to standards change
During rate, mode counter is used for storing the number that each field occurs, i.e., each identical pattern ID is stored in mode counter
Total number, when it is implemented, the corresponding pattern ID of each field is counted respectively, when the pattern ID for having repetition occurs again,
Count and add one again, count very convenient.
(2) when counting the number that each field occurs, number corresponding with static fields is extracted in mode counter.
Similarly, the field with statistics is extracted when the occurrence number of invocation pattern ID is thought at other application end, in direct slave pattern counter
ID number of corresponding pattern, it is convenient and efficient.
In addition, expansible daily record data storage method further includes：Establish field dictionary and pattern respectively according to priori value
Dictionary.There is illustrated a kind of method that field dictionary and mode dictionary are established, that is, field dictionary and mode dictionary are pre-established,
The foundation of foundation be before daily record data in multiple fields i.e. be known as priori value.By priori value and its frequency of appearance come
Field dictionary and mode dictionary are established respectively, can be that consumption, convenient fortune are saved in the daily record data processing occurred in application operation
The daily record data occurred in row is stored after field is directly matched from field dictionary and mode dictionary.
In addition, expansible daily record data storage method further includes：Field dictionary and pattern-word are established according to field dynamic
Allusion quotation.There is illustrated another method that field dictionary and mode dictionary are established, i.e., by field dynamic come establish field dictionary and
Mode dictionary, the field of the daily record data occurred in application operation are dynamically added in field dictionary and mode dictionary,
Field dictionary and mode dictionary become huge with the generation of daily record data.Due to, in application operation, field dictionary and mould
Formula dictionary also needs to store the field of corresponding daily record data while foundation, therefore, this kind of implementation processing speed is slower.
In addition, expansible daily record data storage method further includes：When daily record data from different application programs or
During different equipment, the foundation of field dictionary and mode dictionary is carried out according to open architecture and unified interface.Need exist for
Illustrate, open architecture has mutual between each node on the portability and tailorability, network of application system
Strong operability, also, be easily obtained.Field dictionary and mode dictionary are established by open architecture and unified interface, just
In the transplanting between different application programs and operate.
In conclusion expansible daily record data storage method provided in this embodiment includes：Compare daily record data before this
In each field change rate and the size of standards change rate set in advance, afterwards when change rate is less than standards change rate,
Field is put into field dictionary and is stored, in the present embodiment, the storage format of data includes Field ID in field dictionary
And field contents, Field ID are used for storing key assignments different in each field, field contents are used for storing different in each field
Numerical value, and when change rate is more than or equal to standards change rate, field is put into mode dictionary and is stored, correspondingly,
The storage format of data includes pattern ID and mode contents in mode dictionary, and pattern ID is used for storing key different in each field
Value, pattern ID are used for storing key assignments different in each field, and mode contents are used for storing numerical value different in each field, lead to
Each field in daily record data is stored separately by the form for crossing above-mentioned dictionary, thus effectively prevent the field of repetition repeatedly into
The phenomenon of row storage, has saved substantial amounts of disk space, and largely reduce daily record data in storing process
Information redundancy, it is convenient and efficient.
Embodiment 2
Referring to Fig. 4, present embodiments providing expansible log data storage includes：Change rate comparison module 11,
For comparing the change rate of each field and the size of standards change rate set in advance in daily record data.
Field memory module 12, for when change rate being less than standards change rate, field being put into field dictionary and is carried out
Storage, wherein, the storage format of data includes Field ID and field contents in field dictionary, and Field ID is used for storing each field
Middle different key assignments, field contents are used for storing numerical value different in each field.
Pattern memory module 13, for when change rate is more than or equal to standards change rate, field to be put into pattern-word
Allusion quotation is stored, wherein, the storage format of data includes pattern ID and mode contents in mode dictionary, and pattern ID is used for storing each
Different key assignments in a field, pattern ID are used for storing key assignments different in each field, and mode contents are used for storing each word
Different numerical value in section.
Expansible log data storage provided in an embodiment of the present invention, provides expansible with above-described embodiment
Daily record data storage method has identical technical characteristic, so can also solve identical technical problem, reaches identical technology
Effect.
The embodiment of the present invention additionally provides a kind of terminal, including memory and processor, and memory, which is used to store, to be supported
Processor performs the program of above-described embodiment method, and processor is configurable for performing the program stored in memory.
The embodiment of the present invention also provides a kind of computer-readable recording medium, and meter is stored with computer-readable recording medium
Calculation machine program, when computer program is run by processor the step of the method for execution any of the above-described.
It should be noted that each embodiment in this specification is described by the way of progressive, each embodiment weight
Point explanation is all difference with other embodiment, between each embodiment identical similar part mutually referring to.
The technique effect of the expansible daily record data storage method and device that the embodiment of the present invention is provided, its realization principle and generation
Identical with preceding method embodiment, to briefly describe, device embodiment part does not refer to part, refers to preceding method embodiment
Middle corresponding contents.
In several embodiments provided herein, it should be understood that disclosed apparatus and method, can also pass through
Other modes are realized.Device embodiment described above is only schematical, for example, flow chart and block diagram in attached drawing
Show the devices of multiple embodiments according to the present invention, method and computer program product architectural framework in the cards,
Function and operation.At this point, each square frame in flow chart or block diagram can represent the one of a module, program segment or code
Part, a part for the module, program segment or code include one or more and are used for realization holding for defined logic function
Row instruction.It should also be noted that at some as in the implementation replaced, the function that is marked in square frame can also with different from
The order marked in attached drawing occurs.For example, two continuous square frames can essentially perform substantially in parallel, they are sometimes
It can perform in the opposite order, this is depending on involved function.It is it is also noted that every in block diagram and/or flow chart
The combination of a square frame and block diagram and/or the square frame in flow chart, can use function or the dedicated base of action as defined in performing
Realize, or can be realized with the combination of specialized hardware and computer instruction in the system of hardware.
In addition, each function module or unit in each embodiment of the present invention can integrate and to form an independence
Part or modules individualism, can also two or more modules be integrated to form an independent portion
Point.
If the function is realized in the form of software function module and is used as independent production marketing or in use, can be with
It is stored in a computer read/write memory medium.Based on such understanding, technical scheme is substantially in other words
The part to contribute to the prior art or the part of the technical solution can be embodied in the form of software product, the meter
Calculation machine software product is stored in a storage medium, including some instructions are used so that a computer equipment (can be
People's computer, server, or network equipment etc.) perform all or part of step of each embodiment the method for the present invention.
And foregoing storage medium includes：USB flash disk, mobile hard disk, read-only storage (ROM, Read-Only Memory), arbitrary access are deposited
Reservoir (RAM, Random Access Memory), magnetic disc or CD etc. are various can be with the medium of store program codes.
It should be noted that herein, relational terms such as first and second and the like are used merely to a reality
Body or operation are distinguished with another entity or operation, are deposited without necessarily requiring or implying between these entities or operation
In any this actual relation or order, can not be interpreted as indicating or implying relative importance.Moreover, term " bag
Include ", "comprising" or any other variant thereof is intended to cover non-exclusive inclusion so that the mistake including a series of elements
Journey, method, article or equipment not only include those key elements, but also including other elements that are not explicitly listed, either
Further include as elements inherent to such a process, method, article, or device.In the absence of more restrictions, by sentence
The key element that "including a ..." limits, it is not excluded that also deposit in the process, method, article or apparatus that includes the element
In other identical element.
The foregoing is only a preferred embodiment of the present invention, is not intended to limit the invention, for the skill of this area
For art personnel, the invention may be variously modified and varied.Within the spirit and principles of the invention, that is made any repaiies
Change, equivalent substitution, improvement etc., should all be included in the protection scope of the present invention.It should be noted that：Similar label and letter exists
Similar terms is represented in following attached drawing, therefore, once being defined in a certain Xiang Yi attached drawing, is then not required in subsequent attached drawing
It is further defined and is explained.
The above description is merely a specific embodiment, but protection scope of the present invention is not limited thereto, any
Those familiar with the art the invention discloses technical scope in, change or replacement can be readily occurred in, should all be contained
Cover within protection scope of the present invention.Therefore, protection scope of the present invention answers the scope of the claims of being subject to.
Claims (10)
1. expansible daily record data storage method, it is characterised in that including：
Compare the change rate of each field and the size of standards change rate set in advance in daily record data；
When the change rate is less than the standards change rate, the field is put into field dictionary and is stored, wherein, institute
Stating the storage format of data in field dictionary includes Field ID and field contents, and the Field ID is used for storing in each field not
Same key assignments, the field contents are used for storing numerical value different in each field；
When the change rate is more than or equal to the standards change rate, the field is put into mode dictionary and is stored,
Wherein, the storage format of data includes pattern ID and mode contents in the mode dictionary, and the pattern ID is used for storing each
Different key assignments in field, the pattern ID are used for storing key assignments different in each field, and the mode contents are used for storing
Different numerical value in each field.
2. expansible daily record data storage method according to claim 1, it is characterised in that the method further includes：
The storage format of data further includes Field counter in the field dictionary, when the change rate is less than the standards change
During rate, the Field counter is used for storing the number that each field occurs；
When counting the number that each field occurs, extracted in the Field counter corresponding with static fields secondary
Number.
3. expansible daily record data storage method according to claim 1, it is characterised in that the method further includes：
The storage format of data further includes mode counter in the mode dictionary, when the change rate is more than or equal to described
During standards change rate, the mode counter is used for storing the number that each field occurs；
When counting the number that each field occurs, number corresponding with static fields is extracted in the mode counter.
4. expansible daily record data storage method according to claim 1, it is characterised in that the method further includes：
Establish the field dictionary and the mode dictionary respectively according to priori value.
5. expansible daily record data storage method according to claim 1, it is characterised in that the method further includes：
The field dictionary and the mode dictionary are established according to field dynamic.
6. expansible daily record data storage method according to claim 1, it is characterised in that the method further includes：
When the daily record data is from different application programs or different equipment, connect according to open architecture and unified
Mouth carries out the foundation of the field dictionary and the mode dictionary.
7. expansible daily record data storage method according to claim 1, it is characterised in that the Field ID and described
Pattern ID is represented with binary code.
8. expansible log data storage, it is characterised in that including：
Change rate comparison module, for comparing the change rate of each field and standards change rate set in advance in daily record data
Size；
Field memory module, for when the change rate is less than the standards change rate, the field to be put into field dictionary
In stored, wherein, the storage format of data includes Field ID and field contents in the field dictionary, and the Field ID is used
To store key assignments different in each field, the field contents are used for storing numerical value different in each field；
Pattern memory module, for when the change rate is more than or equal to the standards change rate, the field to be put into
Mode dictionary is stored, wherein, the storage format of data includes pattern ID and mode contents, the mould in the mode dictionary
Formulas I D is used for storing key assignments different in each field, and the pattern ID is used for storing key assignments different in each field, described
Mode contents are used for storing numerical value different in each field.
9. a kind of terminal, it is characterised in that including memory and processor, the memory, which is used to store, supports processor to hold
The program of any one of row claim 1 to 7 the method, the processor are configurable for performing and are stored in the memory
Program.
10. a kind of computer-readable recording medium, computer program is stored with computer-readable recording medium, its feature exists
In when computer program is run by processor the step of any one of execution the claims 1 to 7 the method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711416857.1A CN108038018B (en) | 2017-12-22 | 2017-12-22 | Extensible log data storage method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711416857.1A CN108038018B (en) | 2017-12-22 | 2017-12-22 | Extensible log data storage method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108038018A true CN108038018A (en) | 2018-05-15 |
| CN108038018B CN108038018B (en) | 2020-09-29 |
Family
ID=62100835
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711416857.1A Active CN108038018B (en) | 2017-12-22 | 2017-12-22 | Extensible log data storage method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108038018B (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110474807A (en) * | 2019-08-16 | 2019-11-19 | 北京云中融信网络科技有限公司 | A kind of log processing method and device |
| CN110516122A (en) * | 2019-09-03 | 2019-11-29 | 网易（杭州）网络有限公司 | Embedded data merging method and device and electronic equipment |
| CN110797082A (en) * | 2019-10-24 | 2020-02-14 | 福建和瑞基因科技有限公司 | Method and system for storing and reading gene sequencing data |
| CN111107420A (en) * | 2018-10-25 | 2020-05-05 | 武汉斗鱼网络科技有限公司 | Medal loading method, storage medium, electronic device and system of live APP |
| CN111913922A (en) * | 2019-05-10 | 2020-11-10 | 北京百度网讯科技有限公司 | Binary structured log generation method, device, equipment and storage medium |
| CN112181765A (en) * | 2020-09-25 | 2021-01-05 | 武汉光庭信息技术股份有限公司 | Navigation map capacity monitoring method and system |
| CN115100757A (en) * | 2022-06-20 | 2022-09-23 | 重庆长安汽车股份有限公司 | Automobile data storage method and device, vehicle and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101539921A (en) * | 2008-03-19 | 2009-09-23 | 国际商业机器公司 | Optimal writing method, optimal writing equipment and data generating and writing equipment |
| CN104283723A (en) * | 2014-10-31 | 2015-01-14 | 北京蓝汛通信技术有限责任公司 | Network access log processing method and device |
| US9542430B1 (en) * | 2015-10-30 | 2017-01-10 | International Business Machines Corporation | Ensuring data maintenance within a LDAP environment |
| CN106354617A (en) * | 2016-08-29 | 2017-01-25 | 广州华多网络科技有限公司 | Program compression log file output method and device |
| CN107133151A (en) * | 2017-05-24 | 2017-09-05 | 努比亚技术有限公司 | A kind of daily record data processing method, equipment and computer-readable recording medium |
-
2017
- 2017-12-22 CN CN201711416857.1A patent/CN108038018B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101539921A (en) * | 2008-03-19 | 2009-09-23 | 国际商业机器公司 | Optimal writing method, optimal writing equipment and data generating and writing equipment |
| CN104283723A (en) * | 2014-10-31 | 2015-01-14 | 北京蓝汛通信技术有限责任公司 | Network access log processing method and device |
| US9542430B1 (en) * | 2015-10-30 | 2017-01-10 | International Business Machines Corporation | Ensuring data maintenance within a LDAP environment |
| CN106354617A (en) * | 2016-08-29 | 2017-01-25 | 广州华多网络科技有限公司 | Program compression log file output method and device |
| CN107133151A (en) * | 2017-05-24 | 2017-09-05 | 努比亚技术有限公司 | A kind of daily record data processing method, equipment and computer-readable recording medium |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111107420A (en) * | 2018-10-25 | 2020-05-05 | 武汉斗鱼网络科技有限公司 | Medal loading method, storage medium, electronic device and system of live APP |
| CN111913922A (en) * | 2019-05-10 | 2020-11-10 | 北京百度网讯科技有限公司 | Binary structured log generation method, device, equipment and storage medium |
| CN111913922B (en) * | 2019-05-10 | 2024-02-20 | 北京百度网讯科技有限公司 | Binary structured log generation method, device, equipment and storage medium |
| CN110474807A (en) * | 2019-08-16 | 2019-11-19 | 北京云中融信网络科技有限公司 | A kind of log processing method and device |
| CN110516122A (en) * | 2019-09-03 | 2019-11-29 | 网易（杭州）网络有限公司 | Embedded data merging method and device and electronic equipment |
| CN110516122B (en) * | 2019-09-03 | 2022-05-13 | 网易（杭州）网络有限公司 | Embedded data merging method and device and electronic equipment |
| CN110797082A (en) * | 2019-10-24 | 2020-02-14 | 福建和瑞基因科技有限公司 | Method and system for storing and reading gene sequencing data |
| CN112181765A (en) * | 2020-09-25 | 2021-01-05 | 武汉光庭信息技术股份有限公司 | Navigation map capacity monitoring method and system |
| CN115100757A (en) * | 2022-06-20 | 2022-09-23 | 重庆长安汽车股份有限公司 | Automobile data storage method and device, vehicle and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108038018B (en) | 2020-09-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108038018A (en) | Expansible daily record data storage method and device | |
| CN101651576B (en) | Alarm information processing method and system | |
| CN103902653A (en) | Method and device for creating data warehouse table blood relationship graph | |
| US20230143568A1 (en) | Intelligent table suggestion and conversion for text | |
| CN110083579A (en) | Incremental data synchronous method, apparatus, computer equipment and computer storage medium | |
| CN107678872A (en) | Processing method and device of electronic whiteboard | |
| CN107729063A (en) | Call method, device and the terminal of SDK | |
| CN106407226B (en) | A kind of data processing method, backup server and storage system | |
| CN106547888A (en) | A kind of building method and system of time series databases | |
| CN109684517B (en) | Historical data storage method, reading and writing method, storage device and equipment | |
| CN107943455A (en) | A kind of list rendering system based on JSON data formats | |
| CN110837511A (en) | Data processing method, system and related equipment | |
| CN116339643B (en) | Formatting method, formatting device, formatting equipment and formatting medium for disk array | |
| US9418055B2 (en) | Method for copying multiple content between applications | |
| CN117785262A (en) | Efficient data cleaning method and device supporting super-multiple cleaning rules and electronic equipment | |
| CN102799423A (en) | Method and device for implementing dynamic method in JSF (java service face) | |
| CN109068286A (en) | A kind of method, medium and the equipment of information parsing | |
| CN102184105A (en) | Data processing method based on modularity and data center system | |
| CN104346393B (en) | The modeling method of atomic scale data element model | |
| CN102567366A (en) | Method for realizing webpage resetting | |
| CN107491501A (en) | A kind of method and device for being grouped push | |
| CN107045554A (en) | Pipeline table data and Conversion of Spatial Data method and device | |
| CN107135258A (en) | Method and server that message is pushed | |
| CN109241208A (en) | Address location, address monitoring, information processing method and device | |
| JP2018109898A (en) | Data migration system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: 310000 Room 608, building 5, No. 998, Wenyi West Road, Wuchang Street, Yuhang District, Hangzhou City, Zhejiang Province Applicant after: Flash it Co.,Ltd. Address before: 310000 401A 5, 998 West Wen Yi Road, Wuchang Street, Yuhang District, Hangzhou, Zhejiang. Applicant before: HANGZHOU SECSMART INFORMATION TECHNOLOGY Co.,Ltd. |
|
| CB02 | Change of applicant information | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| EE01 | Entry into force of recordation of patent licensing contract |
Application publication date: 20180515 Assignee: Hangzhou Jintou Finance Leasing Co.,Ltd. Assignor: Flash it Co.,Ltd. Contract record no.: X2022980028282 Denomination of invention: Scalable log data storage method and device Granted publication date: 20200929 License type: Exclusive License Record date: 20230112 |
|
| EE01 | Entry into force of recordation of patent licensing contract | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right |
Denomination of invention: Scalable log data storage method and device Effective date of registration: 20230115 Granted publication date: 20200929 Pledgee: Hangzhou Jintou Finance Leasing Co.,Ltd. Pledgor: Flash it Co.,Ltd. Registration number: Y2023980031389 |
|
| PE01 | Entry into force of the registration of the contract for pledge of patent right | ||
| CP02 | Change in the address of a patent holder |
Address after: 311121 Room 101, Building 9, No. 998, Wenyi West Road, Wuchang Subdistrict, Yuhang District, Hangzhou City, Zhejiang Province Patentee after: Flash it Co.,Ltd. Address before: 310000 Room 608, Building No. 998 Wenyi West Road, Wuchang Street, Yuhang District, Hangzhou City, Zhejiang Province Patentee before: Flash it Co.,Ltd. |
|
| CP02 | Change in the address of a patent holder | ||
| EC01 | Cancellation of recordation of patent licensing contract |
Assignee: Hangzhou Jintou Finance Leasing Co.,Ltd. Assignor: Flash it Co.,Ltd. Contract record no.: X2022980028282 Date of cancellation: 20240327 |
|
| EC01 | Cancellation of recordation of patent licensing contract | ||
| PC01 | Cancellation of the registration of the contract for pledge of patent right |
Granted publication date: 20200929 Pledgee: Hangzhou Jintou Finance Leasing Co.,Ltd. Pledgor: Flash it Co.,Ltd. Registration number: Y2023980031389 |
|
| PC01 | Cancellation of the registration of the contract for pledge of patent right | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right |
Denomination of invention: Scalable method and device for storing log data Granted publication date: 20200929 Pledgee: The Bank of Hangzhou branch of Limited by Share Ltd. sea park Pledgor: Flash it Co.,Ltd. Registration number: Y2024980014339 |
|
| PE01 | Entry into force of the registration of the contract for pledge of patent right |