CN109284257A - A kind of log write-in method, apparatus, electronic equipment and storage medium - Google Patents
A kind of log write-in method, apparatus, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN109284257A CN109284257A CN201810912188.5A CN201810912188A CN109284257A CN 109284257 A CN109284257 A CN 109284257A CN 201810912188 A CN201810912188 A CN 201810912188A CN 109284257 A CN109284257 A CN 109284257A
- Authority
- CN
- China
- Prior art keywords
- log
- urgent
- rdma
- memory
- hbase
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Landscapes
- Debugging And Monitoring (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the present invention provides a kind of log write-in method, apparatus, electronic equipment and storage medium.The described method includes: obtaining remote direct data accesses RDMA system running log；Judge whether the RDMA system running log is urgent log；It, will be in the urgent log write-in memory database system Hbase if the RDMA system running log is urgent log；It, will be in the non-emergent log write-in disk distributed file system HDFS if the RDMA system running log is non-emergent log；If judgement knows that RDMA system is idle, the urgent log in the memory database system Hbase is written in the disk distributed file system HDFS.Log wiring method provided in an embodiment of the present invention improves system burst log extensive handling capacity oncurrent processing ability in real time, solves the orderly storage problem of a large amount of random burst logs.
Description
Technical field
The present embodiments relate to network technique fields, and in particular to a kind of log write-in method, apparatus, electronic equipment and
Storage medium.
Background technique
In data center field, data transmission has the characteristics that high bandwidth, low latency, performance test need to pay close attention to bandwidth,
The abnormal conditions probability of occurrence such as packet loss, delay, therefore need to test data transmission performance, it is lost with preventing from generating in transmission
The abnormal conditions such as packet, delay.Existing RDMA technology has become the calculating network in 25Gbps/100Gbps data center network
With the major networks interconnection technique of data network.RDMA technology full name is remote direct data access, is to solve network and pass
The delay of defeated middle servers' data processing and generate.RDMA provides the capacity of equipment of directly access remote host memory,
When executing this task, computer operating system and CPU interference are not needed, thus expense is small.RDMA technology provides height and handles up
Rate, the network of low latency.RDMA service application is extensive, meanwhile, the transmission of RDMA data, which depends on, has data center bridging
The interchanger of (Data Center Bridging, DCB) characteristic can be protected using the interchanger transmission data with DCB characteristic
Demonstrate,prove the transmission of RDMA data.In the big data platform under RDMA scene, need to carry out log monitoring to server, the network equipment,
In one event request of client subscription or unit exception failure, access log can be generated.Due to RDMA high bandwidth, low latency
Feature, data transmission and processing are more frequent, and the log number volume of generation is big, and RDMA network interface card working frequency generally exists
25Gpbs, under binding pattern, frequency may also reach up 50Gbps/100Gpbs, in this case, by access log file
When storing in RDMA system, since the quantity of low capacity file is very more, needs frequently to open file and read wherein content, from
And cause system crash.
In addition, the access log file generated when submitting event analysis request be mostly it is rambling, if directly
RDMA system is written into these mixed and disorderly journal files, then subsequent processing can be made very troublesome, meanwhile, also cause some important
Log information is not monitored in time and arrives, and causes event not handled in time, causes network interruption.
Summary of the invention
For the defects in the prior art, the embodiment of the invention provides a kind of logs, and method, apparatus, electronic equipment is written
And storage medium.
In a first aspect, the embodiment of the present invention provides a kind of log wiring method, comprising:
It obtains remote direct data and accesses RDMA system running log；
Judge whether the RDMA system running log is urgent log；
If the RDMA system running log is urgent log, memory database system is written into the urgent log
In Hbase；
If the RDMA system running log is non-emergent log, by the non-emergent distributed text of log write-in disk
In part system HDFS；
If judgement knows that RDMA system is idle, institute is written into the urgent log in the memory database system Hbase
It states in disk distributed file system HDFS.
It is optionally, described to judge whether the RDMA system running log is urgent log such as the above method, comprising:
According to predetermined level division principle, the urgency of the RDMA system running log is determined；
If the urgency is greater than pre-programmed emergency degree threshold value, it is determined that the RDMA system running log is urgent log；
Otherwise, it determines the RDMA system running log is non-emergent log.
Such as the above method, optionally, it is described judge whether the RDMA system running log is urgent log after, also wrap
It includes:
The corresponding daily record data of the RDMA system running log is determined according to default semi-structured data format, wherein
When the default semi-structured data format includes: the urgency of the RDMA system running log, Log Names, log generation
Between and log numerical value.
Such as the above method, optionally, if the RDMA system running log is urgent log, by the urgent day
Will is written in memory database system Hbase, comprising:
If the RDMA system running log is urgent log, by the corresponding daily record data of the urgent log according to tight
In anxious degree write-in memory database system Hbase.
It is optionally, described that memory is written into according to urgency in the corresponding daily record data of the urgent log such as the above method
In Database Systems Hbase, comprising:
The daily record data is cached according to its corresponding urgency into the first memory queue；
Daily record data in first memory queue is sequentially written in memory database system Hbase.
Such as the above method, optionally, the magnetic is written in the urgent log by the memory database system Hbase
In disk distributed file system HDFS, comprising:
Obtain the urgent journal queue in the memory database system Hbase；
The urgent journal queue is cached into the second memory queue；
The corresponding urgent log of daily record data in second memory queue is sequentially written in disk distributed field system
In system HDFS.
It is optionally, described by the non-emergent log write-in disk distributed file system HDFS such as the above method, it wraps
It includes:
The corresponding daily record data of the non-emergent log is cached according to its corresponding urgency into third memory queue；
The corresponding non-emergent log of daily record data in the third memory queue is sequentially written in disk distributed document
In system HDFS.
Second aspect, the embodiment of the present invention provide a kind of log writing station, comprising:
Module is obtained, for obtaining remote direct data access RDMA system running log；
Judgment module, for judging whether the RDMA system running log is urgent log；
Urgent log buffer module, if being urgent log for the RDMA system running log, by the urgent day
Will is written in memory database system Hbase；
Non-emergent log memory module will be described non-if being non-emergent log for the RDMA system running log
In urgent log write-in disk distributed file system HDFS；
Urgent log memory module, if for judging to know that RDMA system is idle, by the memory database system
Urgent log in Hbase is written in the disk distributed file system HDFS.
Such as above-mentioned apparatus, optionally, the judgment module is specifically used for:
According to predetermined level division principle, the urgency of the RDMA system running log is determined；
If the urgency is greater than pre-programmed emergency degree threshold value, it is determined that the RDMA system running log is urgent log；
Otherwise, it determines the RDMA system running log is non-emergent log.
Such as above-mentioned apparatus, optionally, further includes:
Preprocessing module, for determining that the RDMA system running log is corresponding according to default semi-structured data format
Daily record data, wherein the default semi-structured data format includes: the urgency of the RDMA system running log, log
Title, log generation time and log numerical value.
Such as above-mentioned apparatus, optionally, the urgent log buffer module is specifically used for:
If the RDMA system running log is urgent log, by the corresponding daily record data of the urgent log according to tight
In anxious degree write-in memory database system Hbase.
Such as above-mentioned apparatus, optionally, urgent log buffer module includes:
First cache unit, for caching the daily record data according to its corresponding urgency to the first memory queue
In；
First writing unit, for the daily record data in first memory queue to be sequentially written in memory database system
In Hbase.
Such as above-mentioned apparatus, optionally, the urgent log memory module includes:
Acquiring unit, for obtaining the urgent journal queue in the memory database system Hbase；
Second cache unit, for caching the urgent journal queue into the second memory queue；
Second writing unit, for the corresponding urgent log of daily record data in second memory queue to be sequentially written in
In disk distributed file system HDFS.
Such as above-mentioned apparatus, optionally, the non-emergent log memory module includes:
Third cache unit, for will the corresponding daily record data of the non-emergent log according to its corresponding urgency caching
Into third memory queue；
Third writing unit, for successively writing the corresponding non-emergent log of daily record data in the third memory queue
Enter in disk distributed file system HDFS.
The third aspect, the embodiment of the present invention provide a kind of electronic equipment, comprising:
Memory and processor, the processor and the memory complete mutual communication by bus；It is described to deposit
Reservoir is stored with the program instruction that can be executed by the processor, and it is as follows that the processor calls described program instruction to be able to carry out
Method: it obtains remote direct data and accesses RDMA system running log；Judge whether the RDMA system running log is urgent
Log；If the RDMA system running log is urgent log, memory database system Hbase is written into the urgent log
In；If the RDMA system running log is non-emergent log, disk distributed field system is written into the non-emergent log
In system HDFS；If judgement knows that RDMA system is idle, institute is written into the urgent log in the memory database system Hbase
It states in disk distributed file system HDFS.
Fourth aspect, the embodiment of the present invention provide a kind of storage medium, are stored thereon with computer program, the computer journey
Following method is realized when sequence is executed by processor: obtaining remote direct data access RDMA system running log；Described in judgement
Whether RDMA system running log is urgent log；It, will be described urgent if the RDMA system running log is urgent log
Log is written in memory database system Hbase；It, will be described non-tight if the RDMA system running log is non-emergent log
In anxious log write-in disk distributed file system HDFS；If judgement knows that RDMA system is idle, by the memory database
Urgent log in system Hbase is written in the disk distributed file system HDFS.
RDMA system running log is divided into urgent log and non-tight by log wiring method provided in an embodiment of the present invention
Anxious log stores urgent log into memory database system Hbase, the urgent log that will be stored in Hbase during idle time
It stores in disk distributed file system HDFS, improves system burst log extensive handling capacity concurrent processing energy in real time
Power, and urgent log information is sampled using high-frequency log, non-emergent log information is sampled using low frequency log, solution
It has determined the orderly storage problem of a large amount of random burst logs.
Detailed description of the invention
In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below
There is attached drawing needed in technical description to be briefly described, it should be apparent that, the accompanying drawings in the following description is this hair
Bright some embodiments for those of ordinary skill in the art without creative efforts, can be with root
Other attached drawings are obtained according to these attached drawings.
Fig. 1 is that method flow schematic diagram is written in log provided in an embodiment of the present invention；
Fig. 2 is the structural schematic diagram of log writing station provided in an embodiment of the present invention；
Fig. 3 is the structural schematic diagram of log writing system provided in an embodiment of the present invention；
Fig. 4 is the structural schematic diagram of the 2nd Flume system in log writing system provided in an embodiment of the present invention；
Fig. 5 is the structural schematic diagram of electronic equipment provided in an embodiment of the present invention.
Specific embodiment
In order to make the object, technical scheme and advantages of the embodiment of the invention clearer, below in conjunction with the embodiment of the present invention
In attached drawing, technical scheme in the embodiment of the invention is clearly and completely described, it is clear that described embodiment is
A part of the embodiment of the present invention, instead of all the embodiments.Based on the embodiments of the present invention, those of ordinary skill in the art
Every other embodiment obtained without making creative work, shall fall within the protection scope of the present invention.
Fig. 1 is log provided in an embodiment of the present invention write-in method flow schematic diagram, as shown in Figure 1, this method comprises:
Step S11, it obtains remote direct data and accesses RDMA system running log；
Specifically, in the big data platform under RDMA scene, data center uses the state of RoCE network rdma protocol
Log, wherein RoCE is that RDMA over Converged Ethernet writes a Chinese character in simplified form, and is a kind of allow through Ethernet using RDMA
Network protocol.Log writing station obtains server or the network equipment such as a plurality of remote direct data on two, three-tier switch
RDMA system running log is accessed, for example, obtaining RDMA system running log by Linux Shell script, running log can
With are as follows: register value when RDMA network interface card is run, the RDMA such as the attribute value or OFED Perftest of ethtool order output are surveyed
Try program operation result value.
Step S12, judge whether the RDMA system running log is urgent log；
Specifically, after getting RDMA system running log, judge whether the log is urgent log, for example, in advance
Different types of RDMA system running log is divided, RDMA system running log is divided into urgent log and non-emergent
Log two major classes for example, the important high-frequency log that will happen suddenly is as urgent log, or will control the corresponding RDMA system of message
Running log is divided into urgent log, and the corresponding RDMA system running log of data message is divided into non-emergent log etc..When obtaining
After getting RDMA system running log, according to RDMA system running log type, judge RDMA system running log whether be
Urgent log or non-emergent log.
If step S13, the described RDMA system running log is urgent log, internal storage data is written into the urgent log
In the system Hbase of library；
Specifically, generally in 25/10Gbps, the network interface card in the case where binding operating mode works frequently the working frequency of RDMA network interface card
Rate is up to 100/40Gpbs, and high-frequency daily record data needs to be written the distributed memory database system of high low latency of handling up
In Hbase.If it is judged that RDMA system running log is urgent log, then memory database system is written into urgent log
In Hbase, realize and sample to urgent log information using high-frequency log solve a large amount of random burst logs and orderly store
Problem.The data that wherein there is Hbase finally can be also written into disk distributed file system HDFS.
If step S14, the described RDMA system running log is non-emergent log, disk is written into the non-emergent log
In distributed file system HDFS；
It specifically, then can be directly by non-emergent log if it is judged that RDMA system running log is non-emergent log
Persistence in disk distributed file system HDFS is written, HDFS is Hadoop distributed file system, and HDFS can provide height and gulp down
Non-emergent log is written in HDFS for the data access for the amount of spitting, the application being very suitable on large-scale dataset, realizes to non-tight
Anxious log information is sampled using low frequency log, since HDFS sample frequency is lower, if directly write to all running logs
Enter in HDFS, will lead to the problem that burst handling capacity is inadequate in real time.
If step S15, judgement knows that RDMA system is idle, by the urgent day in the memory database system Hbase
Will is written in the disk distributed file system HDFS.
Specifically, when the RDAM system free time, need to be stored in the urgent log write in memory database system Hbase
Enter the persistence into disk distributed file system HDFS, realizes that final all RDMA system logs are all stored in disk distribution
In formula file system HDFS.For example, by monitoring system memory usage, when judging the system free time, by Hbase
The urgent log of storage is written in HDFS, the urgent log in Hbase is emptied, so as to subsequent acquisition RDMA system running log
When Hbase there is enough spaces to store urgent log.
RDMA system running log is divided into urgent log and non-tight by log wiring method provided in an embodiment of the present invention
Anxious log stores urgent log into memory database system Hbase, the urgent log that will be stored in Hbase during idle time
It stores in disk distributed file system HDFS, improves system burst log extensive handling capacity concurrent processing energy in real time
Power, and urgent log information is sampled using high-frequency log, non-emergent log information is sampled using low frequency log, solution
It has determined the orderly storage problem of a large amount of random burst logs.
On the basis of the above embodiments, further, described to judge whether the RDMA system running log is urgent
Log, comprising:
According to predetermined level division principle, the urgency of the RDMA system running log is determined；
If the urgency is greater than pre-programmed emergency degree threshold value, it is determined that the RDMA system running log is urgent log；
Otherwise, it determines the RDMA system running log is non-emergent log.
Specifically, in order to arrange to rambling journal file, grade classification principle can also be preset, it will
Log is divided into N number of grade, and urgency is from 1~N, and urgency numerical value is higher, and log is more important, and log sample frequency is higher.Example
Such as, the corresponding RDMA system log urgency of control message is 8, and the corresponding RDAM system log urgency of data message is 5, disorderly
The corresponding RDMA system log urgency of sequence message is 1 etc..After getting RDMA system running log, according to predetermined level
Division principle, determines the urgency of RDMA system running log, then judges whether log urgency is greater than pre-programmed emergency degree threshold
Value, for example, setting urgency threshold value is 5, then RDMA system running log of the urgency greater than 5 is urgent log, and urgency is small
In or equal to 5 RDMA system running log be non-emergent log.
RDMA system running log is divided into urgent day by urgency by log wiring method provided in an embodiment of the present invention
Will and non-emergent log will store in Hbase during idle time by urgent log storage into memory database system Hbase
Urgent log storage is into disk distributed file system HDFS, and improving system burst log, extensive handling capacity is concurrent in real time
Processing capacity, and urgent log information is sampled using high-frequency log, low frequency log is used to non-emergent log information
Sampling solves the orderly storage problem of a large amount of random burst logs.
It is further, described to judge whether the RDMA system running log is tight on the basis of the various embodiments described above
After anxious log, further includes:
The corresponding daily record data of the RDMA system running log is determined according to default semi-structured data format, wherein
When the default semi-structured data format includes: the urgency of the RDMA system running log, Log Names, log generation
Between and log numerical value.
Specifically, after getting RDMA system running log, according to predetermined level division principle, determine that RDMA system is transported
Then the corresponding urgency of row log judges whether RDMA system running log is urgent log, and semi-structured according to presetting
RDMA system running log is converted to daily record data Event by data format, and wherein Event structure is as follows:
Wherein, Level is RMDA system running log urgency level, and Name is RDMA system running log title, Value
For the specific value of RDMA system running log, Time is the time that RDMA system running log generates, and Info is RDMA system
The explanation information of running log.By the daily record data for converting RDMA system running log to semi-structured data format
Event realizes that emergency is according to priority handled.Later, daily record data Event is written in corresponding Hbase or HDFS
In.When from journal file is read in Hbase or in HDFS, so that it may the urgency of journal file is viewed, thus to urgency
High journal file priority processing.
RDMA system running log is divided into urgent day by urgency by log wiring method provided in an embodiment of the present invention
Will and non-emergent log, by the daily record data storage of the corresponding semi-structured data format of urgent log to memory database system
In Hbase, the log stored in Hbase storage is improved into system into disk distributed file system HDFS during idle time
Happen suddenly log extensive handling capacity oncurrent processing ability in real time, and is sampled to urgent log information using high-frequency log, right
Non-emergent log information is sampled using low frequency log, solves a large amount of orderly storage problems of random burst log, pass through by
Running log is converted into the daily record data of semi-structured data format, realizes emergency and according to priority handles.
On the basis of the various embodiments described above, further, if the RDMA system running log is urgent log,
It then will be in the urgent log write-in memory database system Hbase, comprising:
If the RDMA system running log is urgent log, by the corresponding daily record data of the urgent log according to tight
In anxious degree write-in memory database system Hbase.
Specifically, the RDMA system running log that will acquire is converted into the daily record data of semi-structured data format
After Event, same urgency daily record data Event is stored in chronological order in same Hbase file, it is also possible to
Urgency daily record data Event same in non-emergent log is stored in chronological order in same HDFS file, in this way, same
In a Hbase file or the same HDFS file, RDMA system running log grade is identical, can according to different Hbase files or
HDFS file performs corresponding processing, and solves the orderly storage problem of a large amount of random burst logs.
Log wiring method provided in an embodiment of the present invention, by the corresponding semi-structured data lattice of RDMA system running log
The daily record data of formula is stored according to urgency and time sequencing to memory database system Hbase or disk distributed file system
In HDFS, system burst log extensive handling capacity oncurrent processing ability in real time is improved, and use to urgent log information
High-frequency log sampling samples non-emergent log information using low frequency log, and solving a large amount of random burst logs has
Sequence storage problem realizes emergency by excellent by converting running log to the daily record data of semi-structured data format
First grade processing.
On the basis of the various embodiments described above, further, it is described by the corresponding daily record data of the urgent log according to
Urgency is written in memory database system Hbase, comprising:
The daily record data is cached according to its corresponding urgency into the first memory queue；
Daily record data in first memory queue is sequentially written in memory database system Hbase.
Specifically, since log acquisition speed in practical applications can also be by daily record data greater than log writing speed
It caches according to corresponding urgency into memory queue, is denoted as the first memory queue, the daily record data of same urgency, caching is extremely
In same first memory queue, the daily record data in the first memory queue is then sequentially written in memory database system
In Hbase, urgent journal queue is stored in Hbase, solves the problems, such as that a large amount of urgent random data are orderly written in real time.
Log wiring method provided in an embodiment of the present invention, by the corresponding semi-structured data lattice of RDMA system running log
The daily record data of formula is stored according to urgency and time sequencing to memory database system Hbase or disk distributed file system
In HDFS, system burst log extensive handling capacity oncurrent processing ability in real time is improved, and use to urgent log information
High-frequency log sampling samples non-emergent log information using low frequency log, and solving a large amount of urgent random data has
The problem of sequence is written in real time.
On the basis of the various embodiments described above, further, it is described will be urgent in the memory database system Hbase
Log is written in the disk distributed file system HDFS, comprising:
Obtain the urgent journal queue in the memory database system Hbase；
The urgent journal queue is cached into the second memory queue；
The corresponding urgent log of daily record data in second memory queue is sequentially written in disk distributed field system
In system HDFS.
Specifically, during idle time, it needs the distributed text of urgent log write-in disk in memory database system Hbase
Persistence in part system HDFS obtains the urgent journal queue in memory database system Hbase, then by urgent log first
Queue is cached into memory queue, is denoted as the second memory queue, later that the daily record data in the second memory queue is corresponding tight
Anxious log is sequentially written in disk distributed file system HDFS, and same equal level logs are stored to same HDFS text in chronological order
In part, the structure that Event data save in Hbase is the tables of data in database, can save as data text after being stored in HDFS
Part.Event message data volume in Hbase is all smaller, and generally tens arrive several hundred bytes, and HDFS file is generally tens
Mbytes arrive a few G bytes, by Hbase Event data be written HDFS when, need according to log urgency level rule will
Event data corresponding urgent day of the Event data staging into the second different memory queues, in different second memory queues
Will is aggregated to HDFS file according to chronological order, solves the orderly storage problem of a large amount of random burst logs.
Log wiring method provided in an embodiment of the present invention, the log information high to urgency are adopted using high-frequency log
Sample samples non-emergent log information using low frequency log, and the important high-frequency log that will happen suddenly is stored to Hbase internal storage data
Library system, during idle time by the important high-frequency log storage of Hbase internal storage data system to HDFS disk distributed field system
System solves the problems, such as that system burst log is handled up energy-handling capability on a large scale in real time.
It is further, described by the non-emergent distributed text of log write-in disk on the basis of the various embodiments described above
In part system HDFS, comprising:
The corresponding daily record data of the non-emergent log is cached according to its corresponding urgency into third memory queue；
The corresponding non-emergent log of daily record data in the third memory queue is sequentially written in disk distributed document
In system HDFS.
Specifically, non-emergent log can also be stored according to urgency into disk distributed file system HDFS, it is first
RDMA system running log is first obtained, then determines whether RDMA system running log is non-emergent log and its is corresponding tight
Anxious degree, converts non-emergent log to the daily record data of default semi-structured data format, and daily record data is delayed according to urgency
It deposits into memory queue, is denoted as third memory queue, later by the corresponding non-emergent day of the daily record data in third memory queue
Will is sequentially written in disk distributed file system HDFS, and same equal level logs are stored to same HDFS file in chronological order
In, solve the orderly storage problem of a large amount of random burst logs.
Log wiring method provided in an embodiment of the present invention, the log information high to urgency are adopted using high-frequency log
Sample samples non-emergent log information using low frequency log, and the important high-frequency log that will happen suddenly is stored to Hbase internal storage data
Library system, during idle time by the important high-frequency log storage of Hbase internal storage data system to HDFS disk distributed field system
System solves the problems, such as that system burst log is handled up energy-handling capability on a large scale in real time.
Fig. 2 is the structural schematic diagram of log writing station provided in an embodiment of the present invention, as shown in Fig. 2, the device includes:
It obtains module 21, judgment module 22, urgent log buffer module 23, non-emergent log memory module 24 and urgent log and stores mould
Block 25, in which:
Module 21 is obtained for obtaining remote direct data access RDMA system running log；Judgment module 22 is for judging
Whether the RDMA system running log is urgent log；If urgent log buffer module 23 runs day for the RDMA system
Will is urgent log, then will be in the urgent log write-in memory database system Hbase；Non-emergent log memory module 24 is used
If being non-emergent log in the RDMA system running log, disk distributed file system is written into the non-emergent log
In HDFS；If urgent log memory module 25 knows that RDMA system is idle for judging, by the memory database system
Urgent log in Hbase is written in the disk distributed file system HDFS.
Specifically, it obtains module 21 and RDMA system running log is obtained by Linux Shell script, running log can be with
Are as follows: register value when RDMA network interface card is run, the RDMA test such as attribute value or OFED Perftest of ethtool order output
Program operation result value.Judgment module 22 judges whether the log is urgent log, if urgent log, then urgent log buffer
Urgent log is written in memory database system Hbase module 23, realizes and is adopted to urgent log information using high-frequency log
Sample solves the orderly storage problem of a large amount of random burst logs；If non-emergent log, then non-emergent log memory module 24
Persistence in disk distributed file system HDFS directly is written into non-emergent log, judges whether system is idle later, is being
When system is idle, urgent log stored in Hbase is written in HDFS by urgent log memory module 25, empties Hbase
In urgent log, so that Hbase has enough spaces to store urgent log when subsequent acquisition RDMA system running log.This hair
The device that bright embodiment provides, for realizing the above method, function is no longer superfluous herein referring in particular to above method embodiment
It states.
RDMA system running log is divided into urgent log and non-tight by log writing station provided in an embodiment of the present invention
Anxious log stores urgent log into memory database system Hbase, the urgent log that will be stored in Hbase during idle time
It stores in disk distributed file system HDFS, improves system burst log extensive handling capacity concurrent processing energy in real time
Power, and urgent log information is sampled using high-frequency log, non-emergent log information is sampled using low frequency log, solution
It has determined the orderly storage problem of a large amount of random burst logs.
On the basis of the above embodiments, further, the judgment module is specifically used for:
According to predetermined level division principle, the urgency of the RDMA system running log is determined；
If the urgency is greater than pre-programmed emergency degree threshold value, it is determined that the RDMA system running log is urgent log；
Otherwise, it determines the RDMA system running log is non-emergent log.
Specifically, in order to arrange to rambling journal file, grade classification principle can also be preset, it will
Log is divided into N number of grade, and urgency is from 1~N, and urgency numerical value is higher, and log is more important, and log sample frequency is higher.When
After getting RDMA system running log, judgment module determines RDMA system running log according to predetermined level division principle
Urgency, then judge whether log urgency is greater than pre-programmed emergency degree threshold value, for example, setting urgency threshold value be 5, then tightly
RDMA system running log of the anxious degree greater than 5 is urgent log, and urgency is non-less than or equal to 5 RDMA system running log
Urgent log.Device provided in an embodiment of the present invention, for realizing the above method, function is implemented referring in particular to the above method
Example, details are not described herein again.
RDMA system running log is divided into urgent day by urgency by log writing station provided in an embodiment of the present invention
Will and non-emergent log will store in Hbase during idle time by urgent log storage into memory database system Hbase
Urgent log storage is into disk distributed file system HDFS, and improving system burst log, extensive handling capacity is concurrent in real time
Processing capacity, and urgent log information is sampled using high-frequency log, low frequency log is used to non-emergent log information
Sampling solves the orderly storage problem of a large amount of random burst logs.
On the basis of the various embodiments described above, further, the device further include:
Preprocessing module, for determining that the RDMA system running log is corresponding according to default semi-structured data format
Daily record data, wherein the default semi-structured data format includes: the urgency of the RDMA system running log, log
Title, log generation time and log numerical value.
Specifically, after getting RDMA system running log, preprocessing module is determined according to predetermined level division principle
Then the corresponding urgency of RDMA system running log judges whether RDMA system running log is urgent log, and according to pre-
If RDMA system running log is converted to daily record data Event by semi-structured data format, wherein Event structure is as follows:
Wherein, Level is RMDA system running log urgency level, and Name is RDMA system running log title, Value
For the specific value of RDMA system running log, Time is the time that RDMA system running log generates, and Info is RDMA system
The explanation information of running log.Device provided in an embodiment of the present invention, for realizing the above method, function is referring in particular to above-mentioned
Embodiment of the method, details are not described herein again.
RDMA system running log is divided into urgent day by urgency by log writing station provided in an embodiment of the present invention
Will and non-emergent log, by the daily record data storage of the corresponding semi-structured data format of urgent log to memory database system
In Hbase, the log stored in Hbase storage is improved into system into disk distributed file system HDFS during idle time
Happen suddenly log extensive handling capacity oncurrent processing ability in real time, and is sampled to urgent log information using high-frequency log, right
Non-emergent log information is sampled using low frequency log, solves a large amount of orderly storage problems of random burst log, pass through by
Running log is converted into the daily record data of semi-structured data format, realizes emergency and according to priority handles.
On the basis of the various embodiments described above, further, the urgent log buffer module is specifically used for:
If the RDMA system running log is urgent log, by the corresponding daily record data of the urgent log according to tight
In anxious degree write-in memory database system Hbase.
Specifically, the RDMA system running log that will acquire is converted into the daily record data of semi-structured data format
After Event, same urgency daily record data Event is stored to same Hbase text by urgent log buffer module in chronological order
In part, it is also possible to which urgency daily record data Event same in non-emergent log is stored to same HDFS text in chronological order
It, can be according in this way, RDMA system running log grade is identical in the same Hbase file or the same HDFS file in part
Different Hbase files or HDFS file perform corresponding processing, and solve the orderly storage problem of a large amount of random burst logs.This
The device that inventive embodiments provide, for realizing the above method, function is no longer superfluous herein referring in particular to above method embodiment
It states.
Log writing station provided in an embodiment of the present invention, by the corresponding semi-structured data lattice of RDMA system running log
The daily record data of formula is stored according to urgency and time sequencing to memory database system Hbase or disk distributed file system
In HDFS, system burst log extensive handling capacity oncurrent processing ability in real time is improved, and use to urgent log information
High-frequency log sampling samples non-emergent log information using low frequency log, and solving a large amount of random burst logs has
Sequence storage problem realizes emergency by excellent by converting running log to the daily record data of semi-structured data format
First grade processing.
On the basis of the various embodiments described above, further, urgent log buffer module includes:
First cache unit, for caching the daily record data according to its corresponding urgency to the first memory queue
In；
First writing unit, for the daily record data in first memory queue to be sequentially written in memory database system
In Hbase.
Specifically, since log acquisition speed is greater than log writing speed, in practical applications, the first cache unit is by day
Will data are cached according to corresponding urgency into memory queue, are denoted as the first memory queue, the daily record data of same urgency,
Caching is into same first memory queue, and then the daily record data in the first memory queue is sequentially written in by the first writing unit
In memory database system Hbase, urgent journal queue is stored in Hbase, and it is orderly to solve a large amount of urgent random data
In real time the problem of write-in.Device provided in an embodiment of the present invention, for realizing the above method, function is referring in particular to the above method
Embodiment, details are not described herein again.
Log writing station provided in an embodiment of the present invention, by the corresponding semi-structured data lattice of RDMA system running log
The daily record data of formula is stored according to urgency and time sequencing to memory database system Hbase or disk distributed file system
In HDFS, system burst log extensive handling capacity oncurrent processing ability in real time is improved, and use to urgent log information
High-frequency log sampling samples non-emergent log information using low frequency log, and solving a large amount of urgent random data has
The problem of sequence is written in real time.
On the basis of the various embodiments described above, further, the urgent log memory module includes:
Acquiring unit, for obtaining the urgent journal queue in the memory database system Hbase；
Second cache unit, for caching the urgent journal queue into the second memory queue；
Second writing unit, for the corresponding urgent log of daily record data in second memory queue to be sequentially written in
In disk distributed file system HDFS.
Specifically, acquiring unit obtains the urgent journal queue in memory database system Hbase, and then the second caching is single
Member caches urgent journal queue into memory queue, is denoted as the second memory queue, and the second writing unit is by the second memory later
The corresponding urgent log of daily record data in queue is sequentially written in disk distributed file system HDFS, by same equal level logs
It being stored in same HDFS file in chronological order, the structure that Event data save in Hbase is the tables of data in database,
Data file can be saved as after deposit HDFS.Device provided in an embodiment of the present invention, for realizing the above method, function is specific
Referring to above method embodiment, details are not described herein again.
Log writing station provided in an embodiment of the present invention, the log information high to urgency are adopted using high-frequency log
Sample samples non-emergent log information using low frequency log, and the important high-frequency log that will happen suddenly is stored to Hbase internal storage data
Library system, during idle time by the important high-frequency log storage of Hbase internal storage data system to HDFS disk distributed field system
System solves the problems, such as that system burst log is handled up energy-handling capability on a large scale in real time.
On the basis of the various embodiments described above, further, the non-emergent log memory module includes:
Third cache unit, for will the corresponding daily record data of the non-emergent log according to its corresponding urgency caching
Into third memory queue；
Third writing unit, for successively writing the corresponding non-emergent log of daily record data in the third memory queue
Enter in disk distributed file system HDFS.
Specifically, non-emergent log can also be stored according to urgency into disk distributed file system HDFS, it is first
RDMA system running log is first obtained, then determines whether RDMA system running log is non-emergent log and its is corresponding tight
Anxious degree, converts non-emergent log to the daily record data of default semi-structured data format, and third cache unit is by daily record data
It caches according to urgency into memory queue, is denoted as third memory queue, third writing unit will be in third memory queue later
The corresponding non-emergent log of daily record data be sequentially written in disk distributed file system HDFS, on time by same equal level logs
Between be sequentially stored in same HDFS file, solve a large amount of orderly storage problems of random burst log.The embodiment of the present invention mentions
The device of confession, for realizing the above method, function is referring in particular to above method embodiment, and details are not described herein again.
Log writing station provided in an embodiment of the present invention, the log information high to urgency are adopted using high-frequency log
Sample samples non-emergent log information using low frequency log, and the important high-frequency log that will happen suddenly is stored to Hbase internal storage data
Library system, during idle time by the important high-frequency log storage of Hbase internal storage data system to HDFS disk distributed field system
System solves the problems, such as that system burst log is handled up energy-handling capability on a large scale in real time.
Fig. 3 is the structural schematic diagram of log writing system provided in an embodiment of the present invention, as shown in figure 3, in practical application
In, Apache Flume Log Administration System, including the first Flume system and the 2nd Flume system can be used.Each
Flume system includes Source component, three components of channel component and Sink component.Is all deployed on every server
One Flume system is written in Hbase or in HDFS for the classification of log.A server portion is selected on multiple servers
The 2nd Flume system is affixed one's name to, for the data in Hbase to be written in HDFS.
It include Source component, emergency channel component, non-emergent event channel group in first Flume system
Part, Hbase Sink component and HDFS Sink component, the 2nd Flume system include: Hbase Source component, Mem
Channel component and HDFS sink component.Wherein: Source component is for obtaining RDMA system running log, emergency
Channel component is used to cache the corresponding daily record data Event of urgent RDMA system running log in the form of memory queue；It is non-tight
Urgent thing part channel component: for caching the corresponding daily record data of non-emergent RDMA system running log in the form of memory queue
Event, Hbase sink component: for the corresponding log of the storage RDMA system running log of memory database form in a distributed manner
Data Event；HDFS sink component: for storing RDMA system running log in the form of disk distributed file system.Fig. 4
For the structural schematic diagram of the 2nd Flume system in log writing system provided in an embodiment of the present invention, as shown in figure 4, second
Flume system includes: Hbase Source component, multiple Mem channel components and multiple HDFS sink components.More queues
2nd Flume system is used to the Event data of different urgency levels different HDFS document queues are written, and solves a large amount of urgent
The problem of random data are orderly written in real time.Wherein: Hbase source component comes from the first Flume system for receiving
The Event data of Hbase sink give Event data forwarding to downstream channel caching component, and Mem channel component is used
In Event data of the caching from the 2nd Flume system Hbase source component, each Mem channel component is for delaying
The Event data for depositing same priority, using priority query's form cache same urgency event1, event2,
Component is written in the corresponding Sink in Event data forwarding downstream by the data such as event3, and the effect of Mem channel component is main
It is a large amount of urgent Event data of caching.For example, first Mem channel component cache priority grade queue 1, second Mem
Channel component cache priority grade queue 2, third Mem channel component cache priority grade queue 3 etc..HDFS sink group
Part is for receiving the Event data from the 2nd Flume system, by the corresponding log write-in HDFS disk distribution of Event data
Formula file system.Event data in different Mem channel queues are aggregated to HDFS file according to chronological order.
The structure that Event data save in Hbase is the tables of data in database, can save as data file after being stored in HDFS.It is logical
The important high-frequency log storage that will happen suddenly is crossed to Hbase memory database system, during idle time by Hbase internal storage data system
Important high-frequency log storage solves system burst log extensive handling capacity in real time to HDFS disk distributed file system
Processing capacity problem.
Fig. 5 is the structural schematic diagram of electronic equipment provided in an embodiment of the present invention, as shown in figure 5, the equipment includes: place
Manage device (processor) 51, memory (memory) 52 and bus 53；
Wherein, processor 51 and memory 52 complete mutual communication by the bus 53；
Processor 51 is used to call the program instruction in memory 52, to execute side provided by above-mentioned each method embodiment
Method, for example, obtain remote direct data and access RDMA system running log；Whether judge the RDMA system running log
For urgent log；If the RDMA system running log is urgent log, memory database system is written into the urgent log
In system Hbase；It is if the RDMA system running log is non-emergent log, the non-emergent log write-in disk is distributed
In file system HDFS；If judgement knows that RDMA system is idle, by the urgent log in the memory database system Hbase
It is written in the disk distributed file system HDFS.
The embodiment of the present invention discloses a kind of computer program product, and the computer program product is non-transient including being stored in
Computer program on computer readable storage medium, the computer program include program instruction, when described program instructs quilt
When computer executes, computer is able to carry out method provided by above-mentioned each method embodiment, for example, obtains long-range direct
Data access RDMA system running log；Judge whether the RDMA system running log is urgent log；If the RDMA system
System running log is urgent log, then will be in the urgent log write-in memory database system Hbase；If the RDMA system
Running log is non-emergent log, then will be in the non-emergent log write-in disk distributed file system HDFS；If judgement obtains
Know that RDMA system is idle, then the disk distributed document is written into the urgent log in the memory database system Hbase
In system HDFS.
The embodiment of the present invention provides a kind of non-transient computer readable storage medium, the non-transient computer readable storage
Medium storing computer instruction, the computer instruction make the computer execute side provided by above-mentioned each method embodiment
Method, for example, obtain remote direct data and access RDMA system running log；Whether judge the RDMA system running log
For urgent log；If the RDMA system running log is urgent log, memory database system is written into the urgent log
In system Hbase；It is if the RDMA system running log is non-emergent log, the non-emergent log write-in disk is distributed
In file system HDFS；If judgement knows that RDMA system is idle, by the urgent log in the memory database system Hbase
It is written in the disk distributed file system HDFS.
Those of ordinary skill in the art will appreciate that: realize that all or part of the steps of above method embodiment can pass through
The relevant hardware of program instruction is completed, and program above-mentioned can store in computer-readable storage medium, which exists
When execution, step including the steps of the foregoing method embodiments is executed；And storage medium above-mentioned includes: ROM, RAM, magnetic or disk
Etc. the various media that can store program code.
The embodiments such as device described above are only schematical, wherein the unit as illustrated by the separation member
It may or may not be physically separated, component shown as a unit may or may not be physics list
Member, it can it is in one place, or may be distributed over multiple network units.It can be selected according to the actual needs
In some or all of the modules achieve the purpose of the solution of this embodiment.Those of ordinary skill in the art are not paying creativeness
Labour in the case where, it can understand and implement.
Through the above description of the embodiments, those skilled in the art can be understood that each embodiment can
It realizes by means of software and necessary general hardware platform, naturally it is also possible to pass through hardware.Based on this understanding, on
Stating technical solution, substantially the part that contributes to existing technology can be embodied in the form of software products in other words, should
Computer software product may be stored in a computer readable storage medium, such as ROM/RAM, magnetic disk, CD, including several fingers
It enables and using so that a computer equipment (can be personal computer, server or the network equipment etc.) executes each implementation
Method described in certain parts of example or embodiment.
Finally, it should be noted that the above various embodiments is only to illustrate the technical solution of the embodiment of the present invention, rather than it is right
It is limited；Although the embodiment of the present invention is described in detail referring to foregoing embodiments, the ordinary skill of this field
Personnel are it is understood that it is still possible to modify the technical solutions described in the foregoing embodiments, or to part
Or all technical features are equivalently replaced；And these are modified or replaceed, it does not separate the essence of the corresponding technical solution
The range of various embodiments of the present invention technical solution.
Claims (16)
1. a kind of log wiring method characterized by comprising
It obtains remote direct data and accesses RDMA system running log；
Judge whether the RDMA system running log is urgent log；
If the RDMA system running log is urgent log, memory database system Hbase is written into the urgent log
In；
If the RDMA system running log is non-emergent log, disk distributed field system is written into the non-emergent log
In system HDFS；
If judgement knows that RDMA system is idle, the magnetic is written into the urgent log in the memory database system Hbase
In disk distributed file system HDFS.
2. the method according to claim 1, wherein described judge whether the RDMA system running log is tight
Anxious log, comprising:
According to predetermined level division principle, the urgency of the RDMA system running log is determined；
If the urgency is greater than pre-programmed emergency degree threshold value, it is determined that the RDMA system running log is urgent log；
Otherwise, it determines the RDMA system running log is non-emergent log.
3. according to the method described in claim 2, it is characterized in that, described judge whether the RDMA system running log is tight
After anxious log, further includes:
The corresponding daily record data of the RDMA system running log is determined according to default semi-structured data format, wherein described
Default semi-structured data format include: the urgency of the RDMA system running log, Log Names, log generation time and
Log numerical value.
4. if according to the method described in claim 3, it is characterized in that, the RDMA system running log is urgent day
Will, then will be in the urgent log write-in memory database system Hbase, comprising:
If the RDMA system running log is urgent log, by the corresponding daily record data of the urgent log according to urgency
It is written in memory database system Hbase.
5. according to the method described in claim 4, it is characterized in that, it is described by the corresponding daily record data of the urgent log according to
Urgency is written in memory database system Hbase, comprising:
The daily record data is cached according to its corresponding urgency into the first memory queue；
Daily record data in first memory queue is sequentially written in memory database system Hbase.
6. according to the method described in claim 5, it is characterized in that, described will be tight in the memory database system Hbase
Anxious log is written in the disk distributed file system HDFS, comprising:
Obtain the urgent journal queue in the memory database system Hbase；
The urgent journal queue is cached into the second memory queue；
The corresponding urgent log of daily record data in second memory queue is sequentially written in disk distributed file system
In HDFS.
7. according to any method of claim 3-6, which is characterized in that described by the non-emergent log write-in disk point
In cloth file system HDFS, comprising:
The corresponding daily record data of the non-emergent log is cached according to its corresponding urgency into third memory queue；
The corresponding non-emergent log of daily record data in the third memory queue is sequentially written in disk distributed file system
In HDFS.
8. a kind of log writing station characterized by comprising
Module is obtained, for obtaining remote direct data access RDMA system running log；
Judgment module, for judging whether the RDMA system running log is urgent log；
Urgent log buffer module, if being urgent log for the RDMA system running log, by the urgent log write
Enter in memory database system Hbase；
Non-emergent log memory module will be described non-emergent if being non-emergent log for the RDMA system running log
Log is written in disk distributed file system HDFS；
Urgent log memory module, if for judging to know that RDMA system is idle, it will be in the memory database system Hbase
Urgent log be written in the disk distributed file system HDFS.
9. device according to claim 8, which is characterized in that the judgment module is specifically used for:
According to predetermined level division principle, the urgency of the RDMA system running log is determined；
If the urgency is greater than pre-programmed emergency degree threshold value, it is determined that the RDMA system running log is urgent log；
Otherwise, it determines the RDMA system running log is non-emergent log.
10. device according to claim 9, which is characterized in that further include:
Preprocessing module, for determining the corresponding log of the RDMA system running log according to default semi-structured data format
Data, wherein the default semi-structured data format include: the urgency of the RDMA system running log, Log Names,
Log generation time and log numerical value.
11. device according to claim 10, which is characterized in that the urgent log buffer module is specifically used for:
If the RDMA system running log is urgent log, by the corresponding daily record data of the urgent log according to urgency
It is written in memory database system Hbase.
12. device according to claim 11, which is characterized in that promptly log buffer module includes:
First cache unit, for caching the daily record data according to its corresponding urgency into the first memory queue；
First writing unit, for the daily record data in first memory queue to be sequentially written in memory database system
In Hbase.
13. device according to claim 12, which is characterized in that the urgent log memory module includes:
Acquiring unit, for obtaining the urgent journal queue in the memory database system Hbase；
Second cache unit, for caching the urgent journal queue into the second memory queue；
Second writing unit, for the corresponding urgent log of daily record data in second memory queue to be sequentially written in disk
In distributed file system HDFS.
14. any device of 0-13 according to claim 1, which is characterized in that the non-emergent log memory module includes:
Third cache unit, for caching the corresponding daily record data of the non-emergent log according to its corresponding urgency to
In three memory queues；
Third writing unit, for the corresponding non-emergent log of daily record data in the third memory queue to be sequentially written in magnetic
In disk distributed file system HDFS.
15. a kind of electronic equipment characterized by comprising
Memory and processor, the processor and the memory complete mutual communication by bus；The memory
It is stored with the program instruction that can be executed by the processor, the processor calls described program instruction to be able to carry out right such as and wants
Seek 1 to 7 any method.
16. a kind of computer readable storage medium, is stored thereon with computer program, which is characterized in that the computer program
The method as described in claim 1 to 7 is any is realized when being executed by processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810912188.5A CN109284257B (en) | 2018-08-10 | 2018-08-10 | Log writing method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810912188.5A CN109284257B (en) | 2018-08-10 | 2018-08-10 | Log writing method and device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109284257A true CN109284257A (en) | 2019-01-29 |
| CN109284257B CN109284257B (en) | 2021-01-05 |
Family
ID=65183185
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810912188.5A Active CN109284257B (en) | 2018-08-10 | 2018-08-10 | Log writing method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109284257B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112395180A (en) * | 2020-12-10 | 2021-02-23 | 南威软件股份有限公司 | Method for realizing service log full link tracking |
| CN112486913A (en) * | 2020-11-12 | 2021-03-12 | 思创数码科技股份有限公司 | Log asynchronous storage method and device based on cluster environment |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1477822A (en) * | 2003-07-31 | 2004-02-25 | 北京港湾网络有限公司 | Extension storage method of journal information |
| CN102810050A (en) * | 2011-05-31 | 2012-12-05 | 深圳市金蝶友商电子商务服务有限公司 | Log data writing method and log system |
| CN103176888A (en) * | 2011-12-22 | 2013-06-26 | 阿里巴巴集团控股有限公司 | Log recording method and log recording system |
| CN106951488A (en) * | 2017-03-14 | 2017-07-14 | 海尔优家智能科技（北京）有限公司 | A kind of log recording method and device |
-
2018
- 2018-08-10 CN CN201810912188.5A patent/CN109284257B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1477822A (en) * | 2003-07-31 | 2004-02-25 | 北京港湾网络有限公司 | Extension storage method of journal information |
| CN102810050A (en) * | 2011-05-31 | 2012-12-05 | 深圳市金蝶友商电子商务服务有限公司 | Log data writing method and log system |
| CN103176888A (en) * | 2011-12-22 | 2013-06-26 | 阿里巴巴集团控股有限公司 | Log recording method and log recording system |
| CN106951488A (en) * | 2017-03-14 | 2017-07-14 | 海尔优家智能科技（北京）有限公司 | A kind of log recording method and device |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112486913A (en) * | 2020-11-12 | 2021-03-12 | 思创数码科技股份有限公司 | Log asynchronous storage method and device based on cluster environment |
| CN112486913B (en) * | 2020-11-12 | 2023-09-22 | 思创数码科技股份有限公司 | Log asynchronous storage method and device based on cluster environment |
| CN112395180A (en) * | 2020-12-10 | 2021-02-23 | 南威软件股份有限公司 | Method for realizing service log full link tracking |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109284257B (en) | 2021-01-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107566206B (en) | Flow measuring method, equipment and system | |
| CN111835562B (en) | Log uploading method, log inquiring method and system | |
| CN112491979B (en) | Network card data packet cache management method, device, terminal and storage medium | |
| US8479219B2 (en) | Allocating space in message queue for heterogeneous messages | |
| CN111522786A (en) | Log processing system and method | |
| CN103176849A (en) | Virtual machine clustering deployment method based on resource classification | |
| CN107943802A (en) | A kind of log analysis method and system | |
| CN114257559B (en) | Data message forwarding method and device | |
| US11190620B2 (en) | Methods and electronic devices for data transmission and reception | |
| CN114039875B (en) | Data acquisition method, device and system based on eBPF technology | |
| US20240129251A1 (en) | Data processing method and apparatus, computer device, and readable storage medium | |
| CN109284257A (en) | A kind of log write-in method, apparatus, electronic equipment and storage medium | |
| CN108519987A (en) | A kind of data persistence method and apparatus | |
| CN113422808B (en) | Internet of things platform HTTP information pushing method, system, device and medium | |
| Ros-Giralt et al. | High-performance many-core networking: design and implementation | |
| CN114500403A (en) | Data processing method and device and computer readable storage medium | |
| CN109829094A (en) | Distributed reptile system | |
| CN109951532B (en) | DPDK-based automatic flow model conversion device | |
| CN107911317B (en) | Message scheduling method and device | |
| CN114020471B (en) | Sketch-based lightweight elephant flow detection method and platform | |
| CN111131083B (en) | Method, device and equipment for data transmission between nodes and computer readable storage medium | |
| CN115499513A (en) | Data request processing method and device, computer equipment and storage medium | |
| CN114185681A (en) | Automatic current-limiting processing method and device | |
| CN109120665B (en) | High-speed data packet acquisition method and device | |
| Wasi-ur-Rahman et al. | Understanding the communication characteristics in HBase: What are the fundamental bottlenecks? |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| EE01 | Entry into force of recordation of patent licensing contract | ||
| EE01 | Entry into force of recordation of patent licensing contract |
Application publication date: 20190129 Assignee: BEIJING STAR-NET RUIJIE NETWORKS Co.,Ltd. Assignor: RUIJIE NETWORKS Co.,Ltd. Contract record no.: X2022350000023 Denomination of invention: A log writing method, device, electronic device and storage medium Granted publication date: 20210105 License type: Common License Record date: 20220808 |