CN109918399A - Method for writing data and device - Google Patents
Method for writing data and device Download PDFInfo
- Publication number
- CN109918399A CN109918399A CN201810914766.9A CN201810914766A CN109918399A CN 109918399 A CN109918399 A CN 109918399A CN 201810914766 A CN201810914766 A CN 201810914766A CN 109918399 A CN109918399 A CN 109918399A
- Authority
- CN
- China
- Prior art keywords
- database
- daily record
- state
- record data
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Abstract
This disclosure relates to a kind of method for writing data and device.The method for writing data includes: to obtain to call to service corresponding calling daily record data；When waiting the duration by the calling daily record data write-in database to reach preset duration, the calling daily record data is stored in and is cached；The state for determining the database is the idle period, and the database is written in the calling daily record data in the caching in the period of the free time.The method for writing data and device of the disclosure, service access amount is called for the short time high concurrent of pulse pattern, it can be avoided calling daily record data caused by waiting the duration time-out of database connection to lose, to realize the safety for increasing substantially and calling daily record data.
Description
Technical field
This disclosure relates to big data technical field more particularly to a kind of method for writing data and device.
Background technique
Currently, many services and applications are needed using different types of data source.For example, in the neck such as government affairs and education
Domain, the data of conditional shared enterprises become a big demand, and most of business is needed through Data Mart (Data
Mar safety, managed data access) is carried out.In the related technology, pass through SOA (Service-Oriented
Architecture, Services Oriented Achitecture) data of enterprise are transformed into service, bottom difference is shielded, from different
Data source is decoupled to provide data service.
In practical business scene, for the safety for guaranteeing the shared quality of service, the health degree, shared data of service
With there is the Liability Retroact ability of safety accident, need for every time call service be monitored, record calling service name,
Service operations title calls state, calls initial time, response time, called side IP (Internet Protocol, network association
View) address, user's name, tenant's title and malloc failure malloc information (if any) etc. is called to call daily record data.
Summary of the invention
In view of this, the present disclosure proposes a kind of method for writing data and device, to solve to be directed to pulse in the related technology
The short time high concurrent of type calls service access amount, causes part to call daily record data because waiting the duration of database connection
Time-out and the problem of lose.
According to the one side of the disclosure, a kind of method for writing data is provided, the method is suitable for server, the side
Method includes:
It obtains to call and services corresponding calling daily record data；
When waiting the duration by the calling daily record data write-in database to reach preset duration, by the calling log
Data deposit caching；
The state for determining the database is the idle period, and will be in the caching in the period of the free time
The database is written in the calling daily record data.
According to another aspect of the present disclosure, a kind of data transfer apparatus is provided, described device is suitable for server, described
Device includes:
Data acquisition module services corresponding calling daily record data for obtaining to call；
Data are stored in module, for waiting the duration by the calling daily record data write-in database to reach preset duration
When, the calling daily record data is stored in and is cached；
Data write. module, for determining that the state of the database is the idle period, and when described idle
Between section the database is written into the calling daily record data in the caching.
According to another aspect of the present disclosure, a kind of data transfer apparatus is provided, comprising: processor；It is handled for storage
The memory of device executable instruction；Wherein, the processor is configured to executing the above method.
According to another aspect of the present disclosure, a kind of non-volatile computer readable storage medium storing program for executing is provided, is stored thereon with
Computer program instructions, wherein the computer program instructions realize the above method when being executed by processor.
The method for writing data and device of the disclosure obtain and call the corresponding calling daily record data of service, will adjust waiting
When reaching preset duration with the duration of daily record data write-in database, daily record data deposit caching will be called, determines database
State is the idle period, and database is written in the calling daily record data in caching in the idle period, thus directed towards
The short time high concurrent of pulse pattern calls service access amount, can be avoided tune caused by waiting the duration time-out of database connection
It is lost with daily record data, to realize the safety for increasing substantially and calling daily record data.
According to below with reference to the accompanying drawings to detailed description of illustrative embodiments, the other feature and aspect of the disclosure will become
It is clear.
Detailed description of the invention
Comprising in the description and constituting the attached drawing of part of specification and specification together illustrates the disclosure
Exemplary embodiment, feature and aspect, and for explaining the principles of this disclosure.
Fig. 1 shows the flow chart of the method for writing data according to one embodiment of the disclosure.
Fig. 2 a shows the schematic diagram of the database connection pool according to one embodiment of the disclosure.
Fig. 2 b shows the schematic diagram of the database connection pool according to one embodiment of the disclosure.
Fig. 3 shows the block diagram of the data transfer apparatus according to one embodiment of the disclosure.
Fig. 4 shows the block diagram of the data transfer apparatus according to one embodiment of the disclosure.
Specific embodiment
Various exemplary embodiments, feature and the aspect of the disclosure are described in detail below with reference to attached drawing.It is identical in attached drawing
Appended drawing reference indicate element functionally identical or similar.Although the various aspects of embodiment are shown in the attached drawings, remove
It non-specifically points out, it is not necessary to attached drawing drawn to scale.
Dedicated word " exemplary " means " being used as example, embodiment or illustrative " herein.Here as " exemplary "
Illustrated any embodiment should not necessarily be construed as preferred or advantageous over other embodiments.
In addition, giving numerous details in specific embodiment below to better illustrate the disclosure.
It will be appreciated by those skilled in the art that without certain details, the disclosure equally be can be implemented.In some instances, for
Method, means, element and circuit well known to those skilled in the art are not described in detail, in order to highlight the purport of the disclosure.
In order to make those skilled in the art more fully understand the technical solution in the disclosure, below first to being related in the disclosure
And portion of techniques term be briefly described.
Servlet (Server Applet, servlet): refer to the small routine run on the server.
Servlet full name is Java Servlet, is the server program with written in Java, for interactively browsing and modifying number
According to generation Dynamic Web (World Wide Web, global wide area network) content.
SOA: being a component model, by defining interface and agreement between each functional unit (referred to as service), from
And can be realized each service through interface, form the system with specific function.Interface defined in SOA is only
Hardware platform, operating system and the programming language for standing on the service of realization allow the service of building in systems with a kind of system
One and general mode interact.
WSO2: being a leading open source SOA solution provider, provide the relevant various infrastructure of SOA,
Technological frame and related tool etc., for integrated data storage, creation complex data view, trustship different types of data source
Data, safe, reliable, efficient data access is provided.Wherein, data source refer to provide certain required data device or
Original media, such as database etc..
Specifically, administrator can be added to required different data sources in WSO2 and be managed, for different industry
Business demand writes different SQL (Service Query Language, service-seeking language) sentence, complete paired data source
Operation, and the interface for becoming a service is issued, the publication for completing data is shared.In addition, administrator can also configure service
Shared range, the access of the service is controlled for IP address or domain.In terms of safety, User Token, signature can be used
The various ways such as encryption guarantee the safety of service institute's sharing data, are a maturation, reliable, stabilised platform.
Currently, in Web project, calling initial time, called side IP address, the calling of record service when calling service
The information such as user's name and tenant's title, when calling service to complete, record calls the end time.According to call initial time and
The end time is called, determines the response time of service.If calling service normal, above- mentioned information are encapsulated to call log number
According to deposit database.If calling serv-fail, above- mentioned information and malloc failure malloc information are encapsulated to call daily record data, is held
Longization is stored in database.
In addition, using database connection pool to improve the performance to database manipulation.Database connection pool is responsible for dividing
Match, manage and discharge database to connect, allows application program to reuse existing database and connect, rather than re-establish again
One database connection, to improve the performance to database manipulation.It is more than maximum free time length by releasing idling duration
Database connection can be avoided because omitting without database connection caused by release database connection.
Detailed process is as follows: the Servlet Request Listener monitor provided using servlet specification is being adjusted
It when with service, obtains called side IP address, call the information such as user's name and tenant's title, and recording current time is to call
Begin the time.When calling service to complete, record current time is to call the end time.It calculates to call the end time and call and originate
Difference between time, the difference are the response time serviced.It is connected to database connection pool request for data library, if success
Database connection is obtained, then will call daily record data that database is written by database connection.If database connection has not been obtained,
It then waits and retrying again.If database connection still has not been obtained after time-out, data are abandoned.
Data base method is written in calling daily record data based on Database Connection Pool, can significantly improve database reading
Write performance.But database is written for the calling daily record data of large concurrent, database connection pool can not preferably cope with this
Kind situation.
For example, being greater than 10000, and TPS (Transaction Per Second, issued transaction amount per second) for concurrency
The case where greater than 20000.During being connected to database connection pool request for data library, it will cause to wait database connection
Duration improve, and if waiting time it is overtime, this time calling daily record data database will can not be written, to cause to adjust
It is lost with daily record data.Service access amount is called for the short time high concurrent of pulse pattern as a result, it will causes to call log
Failed database is written in data, leads to that daily record data is called to lose.
For PostgreSQL database, the optimal connection number calculation formula that provides in the related technology are as follows: connection number=
((core number × 2)+effective disk number).Wherein, optimal connection number can refer to reach the ability of processor read-write database
The number of data connection into the database connection pool of optimal performance.In actual application, in order to avoid data connection failure
Causing the ability of processor read-write database reduces, and the number of data connection can be slightly larger than and be calculated in database connection pool
Optimal connection number.
For example, being directed to 4 core i7 processors, the then optimal connection number of database connection pool are as follows: connection number=((4 × 2)+1)
=9.For example, choosing connection number is 10.Thus, it is possible to database connection pool is configured that initialization connection number is 0, most
Big connection number is 10, and maximum free time connection number is 10, minimum free time connection number be 1 (because selection connection number is 10,
Optimal connection number is 9), database connection pool maximum waiting time is 60s.
It is configured based on above-mentioned database connection pool, available by statistical experiment: database is write for 20000 times,
To call daily record data that the time-consuming of database is written using database connection pool is about 21-24s；Data are write for 40000 times
Library, will call daily record data that the time-consuming of database is written using database connection pool is about 41s；Data are write for 60000 times
Library, will call daily record data that the time-consuming of database is written using database connection pool is about 64s.Data are write for 60000 times
Library has exceeded database connection pool maximum waiting time 60s, it will the duration for the database to be written such as causing is more than 60s's
Daily record data is called to lose.
The technical issues of in the presence of above-mentioned the relevant technologies, Fig. 1 is shown to be write according to the data of one embodiment of the disclosure
Enter the flow chart of method.This method is suitable for server, as shown in Figure 1, the method comprising the steps of S11 to step S13.
In step s 11, it obtains to call and services corresponding calling daily record data.
In one implementation, the Servlet Request Listener monitor provided by Servlet specification
(Servlet request monitoring device) is able to record the allocating time (call initial time and call end time) of service, and according to
The allocating time of service determines the response time of service.
Wherein, Servlet Request Listener include Request Initialized (request initialization) and
Request Destroy (request is destroyed).Preceding execution Request Initialized is serviced calling, record current time is
Call initial time.Request Destroy is executed after calling service, record current time is to call the end time.It calculates
It calls the end time and calls the difference between initial time, which is the response time serviced.In addition, according to request
The relevant information of object (i.e. the promoter of Servlet Request Listener) available called side, such as called side IP
User's name and tenant's title etc. are called in address.
In one implementation, when calling services successfully, calling daily record data includes service name, service operations name
Claim, one in calling state, calling initial time, response time, called side IP address, calling user's name and tenant's title
Or it is multinomial.
In another implementation, when calling serv-fail, calling daily record data includes service name, service operations
Title calls state, calls initial time, the response time, called side IP address, calls user's name, tenant's title and calling
It is one or more in failure information.
Wherein, service name is to refer to the information of the called service of unique identification.Service operations title is to refer to
The information of unique identification service operations (such as addition, deletion, change and inquiry etc.).Called side IP address, which refers to, calls service
The IP address of equipment.Calling user's name is to refer to the information for the user that unique identification calls service.Tenant (Tenant) name
Title is the information for referring to unique identification tenant.One tenant may include multiple users that can call service.Calling state
Refer to the state for calling service, including calls successfully or malloc failure malloc.Initial time is called to refer to the time for starting to call service.
The end time is called to refer to the time for terminating to call service.Response time refers to the response time of service, call the end time and
Calling the difference between initial time is the response time serviced.Malloc failure malloc information refers to the generation when calling serv-fail
Information, for example including call serv-fail the reason of etc. information.
In step s 12, it when waiting will call the duration of daily record data write-in database to reach preset duration, will call
Daily record data deposit caching.
In one implementation, this method further include: the service condition of database connection in database connection pool is obtained,
And the service condition connected according to database, determine the state of database；If the state of database be it is busy, waiting will adjust
Database is written with daily record data；Or, daily record data will be called to write by data connection if the state of database is the free time
Enter database.
In one implementation, it will call daily record data that database is written, comprising: to obtain and be in database connection pool
The database of unused state connects；It is connected by acquired database, will call daily record data that database is written.
Fig. 2 a and Fig. 2 b show the schematic diagram of the database connection pool according to one embodiment of the disclosure.Such as Fig. 2 a and Fig. 2 b institute
Show, database connection pool is connected with multiple databases, such as database connection 1~5.Application program is connected by connection database
The database connection in pond is connect, realization is connect with database, writes database or read from database to realize.
In one implementation, the number sum number connected according to the database for being in use state in database connection pool
According to the total number that library connects, the service condition of database connection is determined；According to the service condition that database connects, database is determined
State.The state of database can be busy or idle.
Wherein, if there is application program connection database connection, then database connection is in use state；If no
There are application program connection database connection, then database connection is in unused state.
In one implementation, the service condition connected according to database, determines the state of database, comprising: obtains
The number that database in database connection pool in use state connects；Calculate that the database in use state connects
The ratio for the total number that number is connect with database in database connection pool；If ratio is greater than or equal to threshold value, it is determined that data
The state in library is busy.
Wherein, the state of database is busy, then it represents that there is no the numbers in unused state in database connection pool
Connected according to library, even if or exist in unused state database connect, but in unused state database connection in,
Database connection, which can be only used for reading from database, calls daily record data, and may not be usable for that daily record data will be called to be written
Database.
Wherein, threshold value can according to the pre-set data of experience, such as 80% etc., the embodiment of the present disclosure does not make this
Limitation.For example, 20% database, which connects, to be used if being used to write database in database connection pool 80% database connection
In read from database, then threshold value can be set to 80%.
As an example, as shown in Figure 2 a, user 1 is connect by database connection 1 with database, and user 2 passes through number
2 connect with database according to library connection, user 3 connects 3 by database and connect with database, user 4 by database connect 4 and
Database connection.The number that database in database connection pool in use state connects is 4, number in database connection pool
Total number according to library connection is 5.Calculate data in the number 4 and database connection pool that the database in use state connects
The ratio of the total number 5 of library connection is 80%.The ratio 80% is equal to threshold value 80%, determines that the state of database is busy.
In another implementation, the service condition connected according to database, determines the state of database, comprising: obtain
The number for taking the database in database connection pool in use state to connect；Calculate what the database in use state connected
The ratio for the total number that number is connect with database in database connection pool；If ratio is less than threshold value, it is determined that database
State is the free time.
Wherein, the state of database is the free time, then it represents that there are the data in unused state in database connection pool
Library connection, and in the connection of the database in unused state, partial database connection, which can be used for reading from database, adjusts
With daily record data, partial database connection can be used for call daily record data that database is written.
As an example, as shown in Figure 2 b, user 1 is connect by database connection 1 with database, and user 2 passes through number
It is connect according to library connection 2 with database, user 3 is connect by database connection 3 with database.In use in database connection pool
The number of the database connection of state is 3, and the total number that database connects in database connection pool is 5.Calculating to be in makes
The ratio for the total number 5 being connect with the number 3 that the database of state connects with database in database connection pool is 60%.This ratio
Value 60% is less than threshold value 80%, determines the state of database for the free time.
In one implementation, in the buffer by ConcurrentHashMap (concurrent Hash Map) data structure come
Daily record data is called in storage.ConcurrentHashMap data structure is Key-Value structure.Wherein, Key value is to call day
The serial number of will data, Value value are to call daily record data Bean (object for calling daily record data to be packaged into), the attribute in Bean
To call daily record data, correspond to the corresponding field of database.
The time-consuming of daily record data deposit caching will be called smaller and smaller to the space hold of caching.Therefore, it is incited somebody to action in waiting
When the duration of daily record data write-in database being called to reach preset duration, daily record data deposit caching will be called, on the one hand, will not
Cause biggish time-consuming and caching space hold on the other hand can guarantee that the calling daily record data of waiting time time-out is deposited
Enter caching, avoids calling daily record data and be lost because of waiting time time-out, so as to increase substantially calling daily record data
Safety.
In step s 13, the state for determining database is the idle period, and in the period of the free time will cache
Calling daily record data be written database.
In one implementation, database is written into the calling daily record data in caching in the idle period, comprising:
In the idle period, obtains the database in database connection pool in unused state and connect；Pass through acquired data
Database is written in calling daily record data in caching by library connection.
In one implementation, the state for determining database is the idle period, comprising: according to every number of seconds of acquisition
According to the service condition that the increased size for calling daily record data in library is connected with database per second, training obtains prediction model；Pass through
Prediction model determines that the state of database is the idle period.
In one implementation, the execution for monitoring SQL per second obtains sample number from the monitoring script per second for executing SQL
According to, pass through sample data training and verification prediction model.It specifically, will be another using a part of sample data as training data
Part sample data is as verification data.For example, being used as training data for the 80% of sample data, the 20% of sample data is made
For verify data, the embodiment of the present disclosure to this with no restriction.Wherein, training data refers to the data for training prediction model,
Verification data refer to the data for verifying the prediction model that training obtains.
As an example, using the increased size for calling daily record data of database per second as the feature in sample data
1, the service condition that database per second is connected is as the feature 2 in sample data, using the corresponding time per second as sample number
Output in, then one day available 86400 sample data.The quantity of sample data can be accumulated with the time,
To improve the accuracy and stability of training pattern.
Assuming that M sample data is arrived in acquisition.Using m1 sample data therein as training data, then m1 trained number
According to for (x1 (i),x2 (i),y(i)) (i=1,2 ... m1).x1 (i)Indicate the value of feature 1 in i-th of training data, x2 (i)It indicates
The value of feature 2, y in i-th of training data(i)Indicate the value exported in i-th of training data.By m2 sample therein
Data are as verification data, then m2 training data is (x1 (k),x2 (k),y(k)) (k=1,2 ... m2).x1 (k)It indicates k-th
Verify the value of feature 1 in data, x2 (k)Indicate the value of feature 2 in k-th of verification data, y(k)Indicate k-th of verification data
The value of middle output.
Assuming that the state change of database meets multiple features linear regression algorithm, then the hypothesis function of prediction model
(Hypothesis Function) are as follows: hθ(x1,x2)=θ0+θ1x1+θ2x2.Wherein, hθ(x1,x2) indicate the corresponding time per second,
θ0、θ1And θ2Indicate the coefficient of training pattern, x1Indicate the increased size for calling daily record data of database per second, x2Indicate per second
The service condition of database connection.The corresponding loss function of prediction model (Loss Function) are as follows:θ is acquired according to training data0、θ1And θ2Optimal value, solved
Journey uses gradient descent algorithm to obtain the maximum value of gradient, successive iteration acquires θ that is, for obtained loss function derivation0、θ1
And θ2Optimal value.
Obtaining θ0、θ1And θ2Optimal value after, using verification data prediction model is verified.If passed through
The h that prediction model obtainsθ(x1 (k),x2 (k)) and y(k)Between error less than 5%, then it is assumed that prediction model is effective.According to obtaining
Prediction model, determine that the state of database is the idle period, and in the period of the free time by the calling day in caching
Database is written in will data.Before calling daily record data write-in database in it will cache, database can be judged again
State whether be it is busy, if it is wait, if until the calling daily record data write-in number during time-out still fails cache
According to library, then database is written into the calling daily record data in caching again in the idle period that next time recommends.Repeat this process
Until database is written in calling daily record data in will cache.
The method for writing data of the embodiment of the present disclosure obtains and calls the corresponding calling daily record data of service, will adjust waiting
When reaching preset duration with the duration of daily record data write-in database, daily record data deposit caching will be called, determines database
State is the idle period, and database is written in the calling daily record data in caching in the idle period, thus directed towards
The short time high concurrent of pulse pattern calls service access amount, can be avoided tune caused by waiting the duration time-out of database connection
It is lost with daily record data, to realize the safety for increasing substantially and calling daily record data.
Fig. 3 shows the block diagram of the data transfer apparatus according to one embodiment of the disclosure.The device is suitable for server, such as schemes
Shown in 3, described device includes:
Data acquisition module 31 services corresponding calling daily record data for obtaining to call；
Data are stored in module 32, for when waiting the duration by the calling daily record data write-in database to reach default
When long, the calling daily record data is stored in and is cached；
Data write. module 33, for determining that the state of the database is the idle period, and in the free time
The database is written in the calling daily record data in the caching by the period.
In one implementation, described device further include:
State determining module, for obtaining the service condition that database connects in database connection pool, and according to the number
According to the service condition that library connects, the state of the database is determined；
Data processing module, if the state for the database be it is busy, wait the calling daily record data
Database is written；And
The Data write. module passes through the data connection if being also used to the state of the database as the free time
The database is written into the calling daily record data.
In one implementation, the state determining module is used for:
Obtain the number that the database in the database connection pool in use state connects；
Calculate database described in the number and the database connection pool that the database in use state connects
The ratio of the total number of connection；
If the ratio is greater than or equal to threshold value, it is determined that the state of the database is busy；
Or, if the ratio is less than threshold value, it is determined that the state of the database is the free time.
In one implementation, the Data write. module is used for:
The database in database connection pool in unused state is obtained to connect；
It is connected by acquired database, the database is written into the calling daily record data.
In one implementation, described device further include:
Prediction model training module, for the increased size for calling daily record data of the database per second according to acquisition and often
The service condition of second data library connection, training obtain prediction model；
Free time section prediction module, for determined by the prediction model database state be free time when
Between section.
The data transfer apparatus of the disclosure, obtains and calls the corresponding calling daily record data of service, will call log waiting
When the duration that database is written in data reaches preset duration, daily record data deposit caching will be called, determines that the state of database is
The idle period, and database is written into the calling daily record data in caching in the idle period, thus directed towards pulse class
The short time high concurrent of type calls service access amount, can be avoided and calls log caused by waiting the duration time-out of database connection
Loss of data, to realize the safety for increasing substantially and calling daily record data.
Fig. 4 shows the block diagram of the data transfer apparatus according to one embodiment of the disclosure.Referring to Fig. 4, which may include
Processor 901, the machine readable storage medium 902 for being stored with machine-executable instruction.Processor 901 and machine readable storage are situated between
Matter 902 can be communicated via system bus 903.Also, processor 901 by read machine readable storage medium storing program for executing 902 with data
The corresponding machine-executable instruction of logic is written to execute method for writing data described above.
Machine readable storage medium 902 referred to herein can be any electronics, magnetism, optics or other physical stores
Device may include or store information, such as executable instruction, data, etc..For example, machine readable storage medium may is that
RAM (Radom Access Memory, random access memory), volatile memory, nonvolatile memory, flash memory, storage are driven
Dynamic device (such as hard disk drive), solid state hard disk, any kind of storage dish (such as CD, dvd) or similar storage medium,
Or their combination.
The presently disclosed embodiments is described above, above description is exemplary, and non-exclusive, and
It is not limited to disclosed each embodiment.Without departing from the scope and spirit of illustrated each embodiment, for this skill
Many modifications and changes are obvious for the those of ordinary skill in art field.The selection of term used herein, purport
In the principle, practical application or technological improvement to the technology in market for best explaining each embodiment, or make the art
Other those of ordinary skill can understand each embodiment disclosed herein.
Claims (10)
1. a kind of method for writing data, which is characterized in that the method is suitable for server, which comprises
It obtains to call and services corresponding calling daily record data；
When waiting the duration by the calling daily record data write-in database to reach preset duration, by the calling daily record data
Deposit caching；
The state for determining the database is the idle period, and will be described in the caching in the period of the free time
Call daily record data that the database is written.
2. the method according to claim 1, wherein the method also includes:
The service condition of database connection in database connection pool, and the service condition connected according to the database are obtained, really
The state of the fixed database；
If the state of the database be it is busy, wait by the callings daily record data write-in database；
Or, institute is written in the calling daily record data by the data connection if the state of the database is the free time
State database.
3. according to the method described in claim 2, it is characterized in that, determining institute according to the service condition that the database connects
State the state of database, comprising:
Obtain the number that the database in the database connection pool in use state connects；
The number that the database in use state connects is calculated to connect with database described in the database connection pool
Total number ratio；
If the ratio is greater than or equal to threshold value, it is determined that the state of the database is busy；
Or, if the ratio is less than threshold value, it is determined that the state of the database is the free time.
4. method according to claim 1 or 2, which is characterized in that the database is written into the calling daily record data,
Include:
The database in database connection pool in unused state is obtained to connect；
It is connected by acquired database, the database is written into the calling daily record data.
5. the method according to claim 1, wherein determine that the state of the database is the idle period,
Include:
According to the service condition that the increased size for calling daily record data of the database per second of acquisition is connected with database per second, instruction
Get prediction model；
The state that the database is determined by the prediction model is the idle period.
6. a kind of data transfer apparatus, which is characterized in that described device is suitable for server, and described device includes:
Data acquisition module services corresponding calling daily record data for obtaining to call；
Data be stored in module, for wait by it is described calling daily record data write-in database duration reach preset duration when,
The calling daily record data is stored in and is cached；
Data write. module, for determining that the state of the database is the idle period, and in the period of the free time
The database is written into the calling daily record data in the caching.
7. device according to claim 6, which is characterized in that described device further include:
State determining module, for obtaining the service condition that database connects in database connection pool, and according to the database
The service condition of connection determines the state of the database；
Data processing module, if for the database state be it is busy, wait by the callings daily record data write-in
Database；And
The Data write. module, if being also used to the state of the database as the free time, by the data connection by institute
It states and calls daily record data that the database is written.
8. device according to claim 7, which is characterized in that the state determining module is used for:
Obtain the number that the database in the database connection pool in use state connects；
The number that the database in use state connects is calculated to connect with database described in the database connection pool
Total number ratio；
If the ratio is greater than or equal to threshold value, it is determined that the state of the database is busy；
Or, if the ratio is less than threshold value, it is determined that the state of the database is the free time.
9. device according to claim 6 or 7, which is characterized in that the Data write. module is used for:
The database in database connection pool in unused state is obtained to connect；
It is connected by acquired database, the database is written into the calling daily record data.
10. device according to claim 6, which is characterized in that described device further include:
Prediction model training module, for the increased size and every number of seconds for calling daily record data of the database per second according to acquisition
According to the service condition that library connects, training obtains prediction model；
Free time section prediction module, for determining that the state of the database is the idle time by the prediction model
Section.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810914766.9A CN109918399A (en) | 2018-08-13 | 2018-08-13 | Method for writing data and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810914766.9A CN109918399A (en) | 2018-08-13 | 2018-08-13 | Method for writing data and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109918399A true CN109918399A (en) | 2019-06-21 |
Family
ID=66959503
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810914766.9A Pending CN109918399A (en) | 2018-08-13 | 2018-08-13 | Method for writing data and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109918399A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112667513A (en) * | 2020-12-31 | 2021-04-16 | 长沙市到家悠享网络科技有限公司 | Test method, test device, test equipment and storage medium |
| CN112699150A (en) * | 2021-01-04 | 2021-04-23 | 中国银联股份有限公司 | Database operation framework, method and system |
| CN116744168A (en) * | 2022-09-01 | 2023-09-12 | 荣耀终端有限公司 | Log storage method and related device |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103220167A (en) * | 2013-03-25 | 2013-07-24 | 中标软件有限公司 | Distributed monitoring system and data storing method thereof |
| CN105225020A (en) * | 2015-11-11 | 2016-01-06 | 国家电网公司 | A kind of running status Forecasting Methodology based on BP neural network algorithm and system |
| CN105302692A (en) * | 2015-10-28 | 2016-02-03 | 联动优势科技有限公司 | Internal and external parameters based database connection pool monitoring and management method and apparatus |
| US9411842B2 (en) * | 2013-07-31 | 2016-08-09 | Oracle International Corporation | Estimating a cost of performing database operations using vectorized instructions |
| CN106209426A (en) * | 2016-06-28 | 2016-12-07 | 北京北信源软件股份有限公司 | A kind of server load state assessment analysis method and system based on D S evidence theory |
| CN106326034A (en) * | 2016-08-12 | 2017-01-11 | 浪潮(北京)电子信息产业有限公司 | Method and device for storage of data backup |
| US20170116337A1 (en) * | 2015-10-23 | 2017-04-27 | International Business Machines Corporation | User interest reminder notification |
| CN106650501A (en) * | 2016-12-30 | 2017-05-10 | 迈普通信技术股份有限公司 | Database access control method and apparatus |
-
2018
- 2018-08-13 CN CN201810914766.9A patent/CN109918399A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103220167A (en) * | 2013-03-25 | 2013-07-24 | 中标软件有限公司 | Distributed monitoring system and data storing method thereof |
| US9411842B2 (en) * | 2013-07-31 | 2016-08-09 | Oracle International Corporation | Estimating a cost of performing database operations using vectorized instructions |
| US20170116337A1 (en) * | 2015-10-23 | 2017-04-27 | International Business Machines Corporation | User interest reminder notification |
| CN105302692A (en) * | 2015-10-28 | 2016-02-03 | 联动优势科技有限公司 | Internal and external parameters based database connection pool monitoring and management method and apparatus |
| CN105225020A (en) * | 2015-11-11 | 2016-01-06 | 国家电网公司 | A kind of running status Forecasting Methodology based on BP neural network algorithm and system |
| CN106209426A (en) * | 2016-06-28 | 2016-12-07 | 北京北信源软件股份有限公司 | A kind of server load state assessment analysis method and system based on D S evidence theory |
| CN106326034A (en) * | 2016-08-12 | 2017-01-11 | 浪潮(北京)电子信息产业有限公司 | Method and device for storage of data backup |
| CN106650501A (en) * | 2016-12-30 | 2017-05-10 | 迈普通信技术股份有限公司 | Database access control method and apparatus |
Non-Patent Citations (2)
| Title |
|---|
| 国冰磊等: "基于数据库负载的SQL能耗预测模型", 《计算机科学》 * |
| 杨正萍: "基于多层神经网络的服务系统可靠性预测", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112667513A (en) * | 2020-12-31 | 2021-04-16 | 长沙市到家悠享网络科技有限公司 | Test method, test device, test equipment and storage medium |
| CN112699150A (en) * | 2021-01-04 | 2021-04-23 | 中国银联股份有限公司 | Database operation framework, method and system |
| CN116744168A (en) * | 2022-09-01 | 2023-09-12 | 荣耀终端有限公司 | Log storage method and related device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20230126005A1 (en) | Consistent filtering of machine learning data | |
| JP5897747B2 (en) | Fault tolerant batch processing | |
| US11100420B2 (en) | Input processing for machine learning | |
| JP5999574B2 (en) | Database management system and computer system | |
| CN109918399A (en) | Method for writing data and device | |
| US11841861B1 (en) | Constrained query execution | |
| US11119841B2 (en) | Checking data integrity of data storage systems | |
| US10684791B2 (en) | System and method for environment aware backup and restoration | |
| US20150178062A1 (en) | Automated computer application update analysis | |
| US10261843B2 (en) | Speculative execution management in a coherent accelerator architecture | |
| US20050120117A1 (en) | Efficient connection pool validation | |
| US11797431B2 (en) | REST API parser for test automation | |
| WO2016111673A1 (en) | Multi-tenant upgrading | |
| CN114090580A (en) | Data processing method, device, equipment, storage medium and product | |
| CN114048027A (en) | Job operation parameter optimization method applied to super-computation cluster scheduling | |
| US11953979B2 (en) | Using workload data to train error classification model | |
| US20200364211A1 (en) | Predictive database index modification | |
| US20160072945A1 (en) | Call recording test suite | |
| US8352440B2 (en) | Using repeated incremental background consistency checking to detect problems with content closer in time to when a failure occurs | |
| US11580128B2 (en) | Preventing DBMS deadlock by eliminating shared locking | |
| US11487731B2 (en) | Read iterator for pre-fetching nodes of a B-tree into memory | |
| US11243857B2 (en) | Executing test scripts with respect to a server stack | |
| US20180322003A1 (en) | Fault isolation in transaction logs | |
| US20090106765A1 (en) | Predetermination and propagation of resources in a distributed software build | |
| US11567925B2 (en) | Concurrent update management |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190621 |