CN111752987B - Database access method, device, storage medium and computer equipment - Google Patents
Database access method, device, storage medium and computer equipment Download PDFInfo
- Publication number
- CN111752987B CN111752987B CN201910250179.9A CN201910250179A CN111752987B CN 111752987 B CN111752987 B CN 111752987B CN 201910250179 A CN201910250179 A CN 201910250179A CN 111752987 B CN111752987 B CN 111752987B
- Authority
- CN
- China
- Prior art keywords
- data
- data object
- time sequence
- unified
- field information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 34
- 238000003491 array Methods 0.000 claims description 14
- 238000004590 computer program Methods 0.000 claims description 10
- 238000000605 extraction Methods 0.000 claims description 3
- 238000005516 engineering process Methods 0.000 abstract description 7
- 230000007246 mechanism Effects 0.000 abstract description 4
- 230000032683 aging Effects 0.000 abstract description 3
- 230000008569 process Effects 0.000 abstract description 3
- 238000013523 data management Methods 0.000 abstract 1
- 230000005540 biological transmission Effects 0.000 description 11
- 238000006243 chemical reaction Methods 0.000 description 8
- 239000003795 chemical substances by application Substances 0.000 description 5
- 238000012545 processing Methods 0.000 description 5
- 238000012423 maintenance Methods 0.000 description 3
- 230000002238 attenuated effect Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 238000007667 floating Methods 0.000 description 2
- 239000012634 fragment Substances 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000002776 aggregation Effects 0.000 description 1
- 238000004220 aggregation Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000004806 packaging method and process Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 230000007723 transport mechanism Effects 0.000 description 1
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The application provides a database access method, a database access device, a storage medium and computer equipment. The method relates to the computer technology, and solves the problems that the existing access mode is complex in process, low in access efficiency and incapable of meeting time sequence data aging requirements. The method comprises the following steps: reading original data in a database through standard proxy service to generate time sequence data in a unified proxy layer format; and converting the time sequence data into a target format and outputting the time sequence data. The technical scheme provided by the application is suitable for real-time data management, and realizes a simplified and efficient database access mechanism.
Description
Technical Field
The present application relates to computer technologies, and in particular, to a database access method, apparatus, storage medium, and computer device.
Background
The time sequence database stores real-time data based on time points, namely time sequence data, which is abbreviated as time sequence data. The real-time data generated by each data measuring point is single and is provided with a time tag. Because the data measuring points are more and the information quantity is large, thousands of time sequence data can be generated at the same time point, and the data can reach tens of millions and billions. And requires real-time and high-efficiency concurrent reading and writing of large amounts of data to service third party systems, such as monitoring systems, requiring aggregation, classification, anomaly discovery, and the like.
The existing database access mode principle is shown in fig. 1, because the database is written in the language C, in order to provide the efficiency of accessing and writing the database, a local dynamic link library mode (dll or so) is adopted to provide the read-write operation of data. When using Java technology to read and write time series data, java local access technology (Java Native Access, JNA) is used to access the local dynamic link library, then a Java supported data access model, such as remote procedure call (Remote Procedure Call, RPC), is encapsulated to provide external data network access service, and the Restful mode encapsulates the data into a standard network transmission JSON format by packaging. It is seen that when accessing the time sequence database, the time sequence database needs to be subjected to four-layer architecture such as a database layer, a local layer, a proxy layer and a service layer, and the efficiency of reading and writing the time sequence data tends to be attenuated layer by layer after each layer of transmission of the time sequence data is subjected to. The general requirements of high throughput, high concurrency and low delay of time series data are seriously not met.
Disclosure of Invention
The present application is directed to solving the problems described above.
According to a first aspect of the present application, there is provided a database access method comprising:
reading original data in a database through standard proxy service to generate time sequence data in a unified proxy layer format;
and converting the time sequence data into a target format and outputting the time sequence data.
Preferably, the step of reading the original data in the database through the standard proxy service to generate the time series data in the unified proxy layer format includes:
reading necessary field information in at least one piece of original data;
and filling the necessary field information into the data object in the unified agent layer format to generate time sequence data.
Preferably, the step of filling the necessary field information into the data object in the unified proxy layer format, and generating the time sequence data includes:
dividing the dynamic memory into a plurality of data object arrays, wherein the sizes of the data object arrays are the same, and one data object in each data object array corresponds to one piece of original data;
splitting the read original data into a plurality of sequences with the same number, wherein the sequence length is the same as the size of the data object array, and each sequence corresponds to an idle data object array;
reading necessary field information in original data in each sequence, filling the necessary field information into the data objects in a unified proxy layer format, and storing the data objects in a corresponding data object array.
Preferably, the step of reading necessary field information in the original data and filling the necessary field information into the data object in the unified proxy layer format further includes:
after the data object array is used, clearing the content in the data object array, and releasing the data object array.
Preferably, the step of converting the time series data into the target format and outputting the time series data further comprises:
the target format of the output data is configured.
Preferably, when the time series data is converted into the target format, the time series data is directly converted through a native built-in JNA service.
According to another aspect of the present application, there is also provided a database access apparatus including:
the unified proxy module is used for reading the original data in the database through the standard proxy service and generating time sequence data in a unified proxy layer format;
and the unified service module is used for converting the time sequence data into a target format and outputting the time sequence data.
8. The database access apparatus of claim 7, wherein the unified proxy module comprises:
an information extraction unit for reading necessary field information in at least one piece of original data;
and the time sequence data generating unit is used for filling the necessary field information into the data object in the unified agent layer format to generate time sequence data.
According to another aspect of the present application, there is also provided a storage medium having stored thereon a computer program which, when executed, performs the steps of:
reading original data in a database through standard proxy service to generate time sequence data in a unified proxy layer format;
and converting the time sequence data into a target format and outputting the time sequence data.
According to another aspect of the present application, there is also provided a computer device comprising a processor, a memory and a computer program stored on the memory, the processor implementing the following steps when executing the computer program:
reading original data in a database through standard proxy service to generate time sequence data in a unified proxy layer format;
and converting the time sequence data into a target format and outputting the time sequence data.
The application provides a database access method, a database access device, a storage medium and computer equipment, which are used for reading original data in a database through standard proxy service, generating time sequence data in a unified proxy layer format, converting the time sequence data into a target format and outputting the time sequence data. The method solves the problems that the prior access mode has complicated process and low access efficiency and can not meet the time sequence data aging requirement, simplifies the structures of the proxy layer and the service layer in the database access system, and improves the database access efficiency.
Other characteristic features and advantages of the application will become apparent from the following description of exemplary embodiments, which is to be read with reference to the accompanying drawings.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description, serve to explain the principles of the application. In the drawings, like reference numerals are used to identify like elements. The drawings, which are included in the description, illustrate some, but not all embodiments of the application. Other figures can be derived from these figures by one of ordinary skill in the art without undue effort.
FIG. 1 illustrates schematically the principles of a prior art database access system;
FIG. 2 schematically illustrates a flow of a database access method according to an embodiment of the present application;
FIG. 3 schematically illustrates the flow of step 201 in FIG. 2;
FIG. 4 schematically illustrates the flow of step 302 in FIG. 3;
FIG. 5 schematically illustrates the implementation principle of the technical solution provided by the embodiment of the present application;
FIG. 6 schematically illustrates a structure of a database access apparatus according to an embodiment of the present application;
fig. 7 exemplarily illustrates a structure of the system agent module 601 of fig. 6;
fig. 8 exemplarily shows a structure of a computer apparatus provided by an embodiment of the present application.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present application more apparent, the technical solutions of the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application, and it is apparent that the described embodiments are some embodiments of the present application, but not all embodiments of the present application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application. It should be noted that, without conflict, the embodiments of the present application and features of the embodiments may be arbitrarily combined with each other.
The existing time sequence database server side is written based on the C language, but cannot flexibly and conveniently deal with a third party system and provide continuous data service. When accessing the time sequence database, the time sequence database needs to be subjected to four-layer architecture such as a database layer, a local layer, a proxy layer and a service layer, and the efficiency of reading and writing the time sequence data is gradually attenuated after the time sequence data is subjected to transmission of each layer. The general requirements of high throughput, high concurrency and low delay of time series data are seriously not met.
In order to solve the above problems, embodiments of the present application provide a database access method, apparatus, storage medium, and computer device, which reads raw data in a database through a standard proxy service, generates time series data in a unified proxy layer format, converts the time series data into a target format, and outputs the time series data. The structure of the proxy layer and the service layer in the database access system is simplified, and the database access efficiency is improved. The Java technology provides a method for efficiently and rapidly reading and writing the time sequence database, so that a third party system is conveniently and flexibly accessed, development and maintenance cost is reduced, and working efficiency is improved.
An embodiment of the present application provides a database access method, and a flow for completing database access to time-series data using the method is shown in fig. 2, including:
step 201, reading original data in a database through a standard proxy service, and generating time sequence data in a unified proxy layer format.
The flow of generating time series data in this step is specifically shown in fig. 3, and includes:
step 301, reading necessary field information in at least one piece of original data.
And 302, filling the necessary field information into the data object in the unified agent layer format to generate time sequence data.
In order to unify the proxy layer service, the built-in proxy layer JNA service is only developed once in the embodiment of the application, and the data format is unified and stable under the condition that the time sequence data specification is unchanged, and secondary development and maintenance are not needed. For example, for time series data of a measurement point, the required data fields are shown in table 1.
TABLE 1
| Fields | Description of the application | Format specification | Example |
| point | Measuring point serial number | <Device numbering>.<Measuring point number> | TEST001.WD0005 |
| time | Time stamp number of milliseconds | Long integer type | 1519286936430 |
| value | Measurement point data value | Floating point number | 12.4 |
| status | State code | 0 or a positive integer, 0 representing normal | 4 |
Since the JNA service of the unified proxy layer is required, for the data characteristics of the time series data, a Java object for storing a piece of time series data is created as a data object, and the following is an implementation code example:
the flow of generating the time series data in step 302 is shown in fig. 4, and includes:
step 401, dividing the dynamic memory into a plurality of data object arrays.
The data object arrays are the same in size, and one data object in each data object array corresponds to one piece of original data.
Step 402, splitting the read original data into a plurality of sequences of the same number.
The sequence length is the same as the size of the data object array, and each sequence corresponds to an idle data object array;
step 403, reading necessary field information in the original data in each sequence, filling the necessary field information into the data object in the unified proxy layer format, and storing the data object in the corresponding data object array.
And step 404, after the data object array is used, clearing the content in the data object array, and releasing the data object array. The freed free object array may continue to be used to read the original data.
In the embodiment of the application, in order to improve the efficiency, when the database is accessed for reading and writing, a batch processing data mode is adopted to split the original data into a plurality of data sequences with the same quantity, and the data sequences are read into a plurality of data object arrays, which are also called a sub-bucket mode, and one data object array can be called a bucket. When the method is used for reading, a certain number of measuring points are read in batches at a time, barrels in the memory are dynamically batched according to the number of the measuring points, and in order to improve the memory utilization rate of each barrel, the barrels are repeatedly utilized and recycled under a high concurrency mode, so that the time consumed by each barrel for distributing the memory is reduced by a commonly known caching barrel mechanism. The specific calculation method of the barrel number is as follows:
java implementation of a buffer bucket mechanism is mainly realized by adopting a Java concurrency chain queue API 'ConcurrentLinkedQueue', and the following is an example of realizing codes:
the use method needs to provide the total number of measuring points in a single barrel after barrel division, firstly checks whether an idle barrel (a data object array of multiple measuring points) exists, if so, the method is directly used, otherwise, a new object memory space is reassigned to a new barrel for use.
After the use of one barrel is finished, the barrel which is used completely is recycled into the cache queue through a reuse method and is used as a free barrel for other people to use.
Tests show that when a single barrel contains 1 ten thousand measuring points, the memory can be allocated once, which consumes tens to hundreds of milliseconds, and is related to the specific computer machine performance, if the buffer idle barrel mechanism can be recycled, the memory use efficiency can be greatly improved, and the memory use risk such as memory overflow can be reduced.
After the free bucket is available, the data can be read by calling the JNA service interface, and the following is an example of the code implementation of Java fragments for an interface service:
firstly, loading a corresponding library in a JNA mode, wherein only the name of a library file is needed to be provided, and a suffix name is not needed, wherein the name is "dbinterface. Dll" in a windows system, and the name is "dbinterface. So" in Linux.
Meanwhile, a connection initialization ' Init ' method and a connection Release ' method are also provided.
The method for actually reading the original data of the time sequence data is GetDataByBatch, a measuring point name sequence is required to be provided, a corresponding number of data object arrays (an idle bucket) and the number of measuring points required to be read are required to be read, and finally the integer state code of the read database is returned.
Step 202, configuring a target format of output data.
In the embodiment of the application, the data is finally provided for the external unified data format. When the data service is provided externally, a configurable mode is adopted to select a standard JSON format or a CSV format (comma separation), different data formats are selected due to different application scenes of different formats, and the conversion of the formats is directly converted through the native built-in JNA service instead of directly transmitting Java data objects, such as a PRC mode.
Step 203, converting the time sequence data into a target format and outputting the time sequence data.
In this step, specifically, when the time-series data is converted into the target format, the time-series data is directly converted by the native built-in JNA service.
If the standard JSON format is used, in compact mode, a piece of time series data appears as:
{"point":"TEST001.WD0005","time":1519286936430,"value":12.4,"status":4}
when outputting the JSON format, if the data is required to be formatted, in order to improve the efficiency of converting the time stamp and the floating point value precision, the JSON is generated and simultaneously the JSON is directly converted on the basis of the original data, so that the efficiency problem caused by secondary conversion is reduced, wherein the converted code fragments are as follows:
however, in order to improve the data transmission efficiency, reduce the amount of transmission data, facilitate batch processing, and also select CSV format, one of the time series data is expressed as:
TEST001.WD0005,1519286936430,12.4,4
it can be seen that for this piece of data, the data size is reduced by half compared to 71 characters in JSON format in compact mode, whereas CSV format requires only 35 characters. And under the same data volume, the total number of measuring points is doubled for transmission, and the transmission efficiency is obviously improved for mass data transmission.
The data format is configurable, so that the data can be conveniently and flexibly selected and removed according to service requirements. Because some systems need to perform secondary data format conversion to adapt to a third party system after obtaining time sequence data in a CSV format, the data format conversion needs to be performed piece by piece, which inevitably brings about consumption of performance, and in this case, the JSON format is more reasonable to use.
The principle of implementing database access by using the technical scheme provided by the embodiment of the application is shown in fig. 5. Compared with fig. 1, the technical scheme provided by the embodiment of the application eliminates a plurality of proxy layer services, unifies the proxy layer services and embeds the proxy layer services into a service layer, standardizes external data services on the basis, and improves the database access efficiency. High concurrency low latency response of a large amount of data in the read-write time sequence database is optimized to achieve the goal of processing millions or tens of millions of time sequence data in a second level.
An embodiment of the present application further provides a database access apparatus, where the structure of the apparatus is as shown in fig. 6, including:
the unified proxy module 601 is configured to read original data in the database through a standard proxy service, and generate timing data in a unified proxy layer format;
the unified service module 602 is configured to convert the time-series data into a target format and output the time-series data.
Preferably, the unified proxy module 601 has a structure as shown in fig. 7, and includes:
an information extraction unit 701 for reading necessary field information in at least one piece of original data;
the time sequence data generating unit 702 is configured to fill the necessary field information into the data object in the unified agent layer format, and generate time sequence data.
An embodiment of the present application also provides a storage medium having stored thereon a computer program which, when executed, performs the steps of:
reading original data in a database through standard proxy service to generate time sequence data in a unified proxy layer format;
and converting the time sequence data into a target format and outputting the time sequence data.
An embodiment of the present application further provides a computer device, whose structure is shown in fig. 8, including a processor, a memory, and a computer program stored on the memory, where the processor implements the following steps when executing the computer program:
reading original data in a database through standard proxy service to generate time sequence data in a unified proxy layer format;
and converting the time sequence data into a target format and outputting the time sequence data.
The embodiment of the application provides a database access method, a database access device, a storage medium and computer equipment, which are used for reading original data in a database through standard proxy service, generating time sequence data in a unified proxy layer format, converting the time sequence data into a target format and outputting the time sequence data. The method solves the problems that the prior access mode has complicated process and low access efficiency and can not meet the time sequence data aging requirement, simplifies the structures of the proxy layer and the service layer in the database access system, and improves the database access efficiency.
In the prior art, for each external data service provided, a set of corresponding proxy layer services needs to be developed separately to support different service layers and provide data processing capability. The technical scheme provided by the embodiment of the application eliminates the proxy layer, unifies a plurality of proxy layer services required by the service layer into one proxy service and embeds the proxy service into the service layer, thereby reducing efficiency attenuation caused by data transmission or conversion.
And meanwhile, a plurality of service layers are unified into one standard service, and the direct conversion of the original data into a standard unified data exchange format is supported, so that the format of the data is unified and the conversion efficiency is improved.
The embodiment of the application also provides a configurable output data format mode to reduce the used threshold and improve the data transmission processing efficiency, without the need of excessively focusing on the programming language itself by a developer, and the efficiency is improved by selecting a corresponding strategy in a configurable mode.
Because of the built-in and unified standards of various services, the use difficulty and the maintenance cost are greatly reduced, so that developers are more focused on data service, and do not need to pay much attention to data conversion and transmission.
Those of ordinary skill in the art will appreciate that all or some of the steps, systems, functional modules/units in the apparatus, and methods disclosed above may be implemented as software, firmware, hardware, and suitable combinations thereof. In a hardware implementation, the division between the functional modules/units mentioned in the above description does not necessarily correspond to the division of physical components; for example, one physical component may have multiple functions, or one function or step may be performed cooperatively by several physical components. Some or all of the components may be implemented as software executed by a processor, such as a digital signal processor or microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on computer readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media). The term computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data, as known to those skilled in the art. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital Versatile Disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by a computer. Furthermore, as is well known to those of ordinary skill in the art, communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media.
The above description may be implemented alone or in various combinations and these modifications are within the scope of the present application.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present application, and are not limiting. Although the application has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present application.
Claims (7)
1. A method of database access, comprising:
reading original data in a database through standard proxy service to generate time sequence data in a unified proxy layer format;
converting the time sequence data into a target format and outputting the time sequence data;
the step of reading the original data in the database by the standard proxy service to generate the time sequence data in the unified proxy layer format comprises the following steps:
reading necessary field information in at least one piece of original data;
filling the necessary field information into the data object in the unified agent layer format to generate time sequence data;
filling the necessary field information into the data object in the unified agent layer format, and generating time sequence data comprises the following steps:
dividing the dynamic memory into a plurality of data object arrays, wherein the sizes of the data object arrays are the same, and one data object in each data object array corresponds to one piece of original data;
splitting the read original data into a plurality of sequences with the same number, wherein the sequence length is the same as the size of the data object array, and each sequence corresponds to an idle data object array;
reading necessary field information in original data in each sequence, filling the necessary field information into the data objects in a unified proxy layer format, and storing the data objects in a corresponding data object array.
2. The database access method according to claim 1, wherein the step of reading necessary field information in the original data and filling the necessary field information into the data object in the unified proxy layer format further comprises:
after the data object array is used, clearing the content in the data object array, and releasing the data object array.
3. The database access method according to claim 1, wherein the step of converting and outputting the time series data into a target format further comprises, before:
the target format of the output data is configured.
4. The database access method according to claim 1, wherein the time-series data is converted directly by a native built-in JNA service when converted into a target format.
5. A database access apparatus, comprising:
the unified proxy module is used for reading the original data in the database through the standard proxy service and generating time sequence data in a unified proxy layer format;
the unified service module is used for converting the time sequence data into a target format and outputting the target format;
the unified proxy module comprises:
an information extraction unit for reading necessary field information in at least one piece of original data;
the time sequence data generating unit is used for filling the necessary field information into the data object in the unified agent layer format to generate time sequence data;
filling the necessary field information into the data object in the unified agent layer format, and generating time sequence data comprises the following steps:
dividing the dynamic memory into a plurality of data object arrays, wherein the sizes of the data object arrays are the same, and one data object in each data object array corresponds to one piece of original data;
splitting the read original data into a plurality of sequences with the same number, wherein the sequence length is the same as the size of the data object array, and each sequence corresponds to an idle data object array;
reading necessary field information in original data in each sequence, filling the necessary field information into the data objects in a unified proxy layer format, and storing the data objects in a corresponding data object array.
6. A storage medium having a computer program stored thereon, the computer program when executed performing the steps of:
reading original data in a database through standard proxy service to generate time sequence data in a unified proxy layer format;
converting the time sequence data into a target format and outputting the time sequence data;
the step of reading the original data in the database by the standard proxy service to generate the time sequence data in the unified proxy layer format comprises the following steps:
reading necessary field information in at least one piece of original data;
filling the necessary field information into the data object in the unified agent layer format to generate time sequence data;
filling the necessary field information into the data object in the unified agent layer format, and generating time sequence data comprises the following steps:
dividing the dynamic memory into a plurality of data object arrays, wherein the sizes of the data object arrays are the same, and one data object in each data object array corresponds to one piece of original data;
splitting the read original data into a plurality of sequences with the same number, wherein the sequence length is the same as the size of the data object array, and each sequence corresponds to an idle data object array;
reading necessary field information in original data in each sequence, filling the necessary field information into the data objects in a unified proxy layer format, and storing the data objects in a corresponding data object array.
7. A computer device comprising a processor, a memory and a computer program stored on the memory, the processor implementing the following steps when executing the computer program:
reading original data in a database through standard proxy service to generate time sequence data in a unified proxy layer format;
converting the time sequence data into a target format and outputting the time sequence data;
the step of reading the original data in the database by the standard proxy service to generate the time sequence data in the unified proxy layer format comprises the following steps:
reading necessary field information in at least one piece of original data;
filling the necessary field information into the data object in the unified agent layer format to generate time sequence data;
filling the necessary field information into the data object in the unified agent layer format, and generating time sequence data comprises the following steps:
dividing the dynamic memory into a plurality of data object arrays, wherein the sizes of the data object arrays are the same, and one data object in each data object array corresponds to one piece of original data;
splitting the read original data into a plurality of sequences with the same number, wherein the sequence length is the same as the size of the data object array, and each sequence corresponds to an idle data object array;
reading necessary field information in original data in each sequence, filling the necessary field information into the data objects in a unified proxy layer format, and storing the data objects in a corresponding data object array.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910250179.9A CN111752987B (en) | 2019-03-29 | 2019-03-29 | Database access method, device, storage medium and computer equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910250179.9A CN111752987B (en) | 2019-03-29 | 2019-03-29 | Database access method, device, storage medium and computer equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111752987A CN111752987A (en) | 2020-10-09 |
| CN111752987B true CN111752987B (en) | 2023-12-12 |
Family
ID=72672133
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910250179.9A Active CN111752987B (en) | 2019-03-29 | 2019-03-29 | Database access method, device, storage medium and computer equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111752987B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113176876B (en) * | 2021-04-23 | 2023-08-11 | 网易（杭州）网络有限公司 | Data reading and writing method and device and electronic equipment |
| CN114048239B (en) * | 2022-01-12 | 2022-04-12 | 树根互联股份有限公司 | Storage method, query method and device of time series data |
| CN117668090A (en) * | 2024-02-01 | 2024-03-08 | 安徽容知日新科技股份有限公司 | Data exchange method, data exchange device, electronic equipment and computer readable storage medium |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2002132644A (en) * | 2000-10-24 | 2002-05-10 | Asahi Glass Co Ltd | Time series database system |
| CN1494022A (en) * | 2002-10-30 | 2004-05-05 | 华为技术有限公司 | Method accessing data bank through protocol agency mode |
| CN102214228A (en) * | 2011-06-24 | 2011-10-12 | 国电南瑞科技股份有限公司 | Large-grid distributed real-time database system and data management method thereof |
| CN102750606A (en) * | 2012-05-16 | 2012-10-24 | 中国电力科学研究院 | Power grid scheduling cloud system |
| CN105069690A (en) * | 2015-06-11 | 2015-11-18 | 国家电网公司 | Power grid operating data processing method |
| CN105574072A (en) * | 2015-11-11 | 2016-05-11 | 国网冀北电力有限公司信息通信分公司 | IEC61970 standard-based real-time database cluster realization method |
| CN105930491A (en) * | 2016-04-28 | 2016-09-07 | 安徽四创电子股份有限公司 | Monitoring data storage method based on time sequence database InfluxDB |
| CN107015872A (en) * | 2016-12-09 | 2017-08-04 | 上海壹账通金融科技有限公司 | The processing method and processing device of monitoring data |
| CN107092628A (en) * | 2017-01-10 | 2017-08-25 | 口碑控股有限公司 | The treating method and apparatus of time series data |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10817524B2 (en) * | 2017-04-10 | 2020-10-27 | Servicenow, Inc. | Systems and methods for querying time series data |
| US11010362B2 (en) * | 2017-08-25 | 2021-05-18 | Vmware, Inc. | Method and system for caching a generated query plan for time series data |
-
2019
- 2019-03-29 CN CN201910250179.9A patent/CN111752987B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2002132644A (en) * | 2000-10-24 | 2002-05-10 | Asahi Glass Co Ltd | Time series database system |
| CN1494022A (en) * | 2002-10-30 | 2004-05-05 | 华为技术有限公司 | Method accessing data bank through protocol agency mode |
| CN102214228A (en) * | 2011-06-24 | 2011-10-12 | 国电南瑞科技股份有限公司 | Large-grid distributed real-time database system and data management method thereof |
| CN102750606A (en) * | 2012-05-16 | 2012-10-24 | 中国电力科学研究院 | Power grid scheduling cloud system |
| CN105069690A (en) * | 2015-06-11 | 2015-11-18 | 国家电网公司 | Power grid operating data processing method |
| CN105574072A (en) * | 2015-11-11 | 2016-05-11 | 国网冀北电力有限公司信息通信分公司 | IEC61970 standard-based real-time database cluster realization method |
| CN105930491A (en) * | 2016-04-28 | 2016-09-07 | 安徽四创电子股份有限公司 | Monitoring data storage method based on time sequence database InfluxDB |
| CN107015872A (en) * | 2016-12-09 | 2017-08-04 | 上海壹账通金融科技有限公司 | The processing method and processing device of monitoring data |
| CN107092628A (en) * | 2017-01-10 | 2017-08-25 | 口碑控股有限公司 | The treating method and apparatus of time series data |
Non-Patent Citations (4)
| Title |
|---|
| Time series databases and influxdb;Syeda Noor Zehra Naqvi等;《Studienarbeit, Université Libre de Bruxelles》;第1-44页 * |
| 基于时序数据库的电力运维系统关键技术研究;荣雪琴;刘勇;刘昊;卜树坡;;电测与仪表(第09期);第56-59+91页 * |
| 基于时标量测稳态数据的智能电网调控主站应用研究;李亮;《中国优秀硕士学位论文全文数据库(工程科技Ⅱ辑)》(第2018年第03期期);C042-2285 * |
| 电网调度综合数据平台设计与实现;犹锋;《电子技术与软件工程》(第17期);第169页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111752987A (en) | 2020-10-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111752987B (en) | Database access method, device, storage medium and computer equipment | |
| US9086952B2 (en) | Memory management and method for allocation using free-list | |
| KR100875836B1 (en) | Instruction instruction compression apparatus and method for parallel processing BLU computer | |
| CN110399104B (en) | Data storage method, data storage device, electronic apparatus, and storage medium | |
| US20210357258A1 (en) | Method, device and medium for allocating resource based on type of pci device | |
| CN111143446A (en) | Data structure conversion processing method and device of data object and electronic equipment | |
| CN115129621B (en) | Memory management method, device, medium and memory management module | |
| CN111241180A (en) | Information processing method, device, equipment and storage medium | |
| CN112860412B (en) | Service data processing method and device, electronic equipment and storage medium | |
| US8095768B2 (en) | VSAM smart reorganization | |
| US20210141723A1 (en) | Memory usage in managed runtime applications | |
| CN113961566A (en) | Transaction flow processing method and system | |
| CN112650512A (en) | Hardware driving method and device, terminal and storage medium | |
| CN114116790A (en) | Data processing method and device | |
| CN111459819A (en) | Software testing method and device, electronic equipment and computer readable medium | |
| CN108279973B (en) | Information statistical method and device and electronic equipment | |
| CN113110879B (en) | Instruction processing method and device | |
| US20230195527A1 (en) | Workload distribution by utilizing unused central processing unit capacity in a distributed computing system | |
| CN116126546B (en) | Performance optimization method and device, electronic equipment and medium | |
| CN110727402B (en) | High-speed FC data real-time receiving and frame loss-free storage method | |
| CN112363778B (en) | Dynamic picture loading method and device, electronic equipment and storage medium | |
| EP4194067A1 (en) | Game engine resource processing method and apparatus, and electronic device and computer-readable storage medium | |
| CN110914799B (en) | Deserialization method and device and computing equipment | |
| US9141453B2 (en) | Reduced footprint core files in storage constrained environments | |
| CN117472755A (en) | Interface testing method, device, processor and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |