CN108959587B - Data updating method and device based on column storage, server and storage medium - Google Patents

Data updating method and device based on column storage, server and storage medium Download PDF

Info

Publication number
CN108959587B
CN108959587B CN201810750473.1A CN201810750473A CN108959587B CN 108959587 B CN108959587 B CN 108959587B CN 201810750473 A CN201810750473 A CN 201810750473A CN 108959587 B CN108959587 B CN 108959587B
Authority
CN
China
Prior art keywords
data
updated
column
auxiliary table
storage
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201810750473.1A
Other languages
Chinese (zh)
Other versions
CN108959587A (en
Inventor
郭琰
王攀
周智伟
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Dameng Database Co Ltd
Original Assignee
Shanghai Dameng Database Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Dameng Database Co Ltd filed Critical Shanghai Dameng Database Co Ltd
Priority to CN201810750473.1A priority Critical patent/CN108959587B/en
Publication of CN108959587A publication Critical patent/CN108959587A/en
Application granted granted Critical
Publication of CN108959587B publication Critical patent/CN108959587B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Abstract

The invention discloses a data updating method, a device, a server and a storage medium based on column storage, relating to the field of databases, wherein the method comprises the following steps: acquiring data to be updated of a column storage table, wherein the data to be updated comprises a row number, a column number and an updated value to be updated; inserting the row number, the column number and the updated value into an updated auxiliary table corresponding to the column storage table; determining a corresponding data area of the data to be updated in the column storage table according to the row number and the column number; and recalculating the statistical information of the data area according to the data of the data area and the data to be updated, and modifying the record corresponding to the statistical information in the column storage auxiliary table corresponding to the column storage table. By adopting the technical scheme, the updating efficiency of the data based on the column storage is improved.

Description

Data updating method and device based on column storage, server and storage medium
Technical Field
The embodiment of the invention relates to the technical field of databases, in particular to a data updating method and device based on column storage, a server and a storage medium.
Background
With the continuous development of big data technology, the amount of data contained in the database is increased sharply, and the traditional query performance based on the row storage mode is challenged.
Currently, to improve the performance of database query, column storage, which is a different storage method from the conventional row storage, is considered. The column storage technique is to store a data table in units of columns, and store data of the same column in one data file or in a plurality of files according to the data size.
The column storage mode can improve the data query performance, but when data is updated, the updating of a plurality of fields needs to position and update the storage files corresponding to different columns respectively, so the data updating performance is lower than that of the row storage mode.
Disclosure of Invention
In view of this, embodiments of the present invention provide a method, an apparatus, a server, and a storage medium for updating data based on a column storage, so as to improve data updating efficiency of a column storage table.
In a first aspect, an embodiment of the present invention provides a data updating method based on column storage, where the method includes:
acquiring data to be updated of a column storage table, wherein the data to be updated comprises a row number, a column number and an updated value to be updated;
inserting the row number, the column number and the updated value into an updated auxiliary table corresponding to the column storage table;
determining a corresponding data area of the data to be updated in the column storage table according to the row number and the column number;
and recalculating the statistical information of the data area according to the data of the data area and the data to be updated, and modifying the record corresponding to the statistical information in the column storage auxiliary table corresponding to the column storage table.
In a second aspect, an embodiment of the present invention further provides a data updating apparatus based on column storage, where the apparatus includes:
the device comprises a to-be-updated data acquisition module, a to-be-updated data acquisition module and a to-be-updated data updating module, wherein the to-be-updated data acquisition module is used for acquiring to-be-updated data of a column storage table, and the to-be-updated data comprises a row number, a column number and an updated value to be updated;
the inserting module is used for inserting the row number, the column number and the updated value into an updated auxiliary table corresponding to the column storage table;
a data area determining module, configured to determine, according to the row number and the column number, a data area corresponding to the data to be updated in the column storage table;
and the record modification module is used for recalculating the statistical information of the data area according to the data of the data area and the data to be updated and modifying the record corresponding to the statistical information in the column storage auxiliary table corresponding to the column storage table.
In a third aspect, an embodiment of the present invention further provides a server, including:
one or more processors;
storage means for storing one or more programs;
when the one or more programs are executed by the one or more processors, the one or more processors implement the column storage based data update method according to any embodiment of the present invention.
In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the data updating method based on column storage according to any embodiment of the present invention.
According to the invention, the data to be updated in the column storage table is recorded by utilizing the update auxiliary table, so that the update of the column storage table is converted into the insertion of the update auxiliary table, thereby avoiding reading and writing of data files, solving the problem of low data update efficiency and improving the update efficiency of the data based on column storage.
Drawings
FIG. 1 is a flowchart of a data updating method based on column storage according to an embodiment of the present invention;
FIG. 2 is a flowchart of a data updating method based on column storage according to a second embodiment of the present invention;
FIG. 3 is a schematic structural diagram of a data updating apparatus based on column storage according to a third embodiment of the present invention;
fig. 4 is a schematic structural diagram of a server according to a fourth embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be further noted that, for the convenience of description, only some but not all of the relevant aspects of the present invention are shown in the drawings.
Example one
Fig. 1 is a flowchart of a data updating method based on column storage according to an embodiment of the present invention, where this embodiment is applicable to a case of updating data based on column storage, and the method may be executed by a data updating apparatus based on column storage, where the apparatus may be implemented by software and/or hardware, and may be generally integrated in a server, as shown in fig. 1, a technical solution provided in this embodiment is specifically as follows:
step 110, acquiring data to be updated of the column storage table, wherein the data to be updated comprises a row number to be updated, a column number and an updated value.
Column storage is to store data in units of columns, and the specific storage rule and management implementation mode directly determine the operation efficiency of a column storage table. All data is stored in a column memory table.
The column storage table stores each column of data in a partitioned manner according to a certain number of preset rows, such a region is referred to as a data region, and the preset rows are referred to as a region size. The data in the same data area are stored in the same data file, and one data file can store one to more data areas. The data area formed by the same column of data is directly stored in one data file or stored in a plurality of data files according to the data size, and is assisted by a column storage auxiliary table, an insertion auxiliary table and a deletion auxiliary table. The insertion auxiliary table is structured in a form of a row storage for buffering data inserted to the column storage table, and the number of rows of the buffered data is smaller than the area size of the data area in the column storage table. When the number of data lines inserted into the auxiliary table reaches the area size, the data inserted into the auxiliary table is written into the data file corresponding to each column in units of columns, and then the auxiliary table is emptied. The structure of the deletion auxiliary table is a line storage form, and is used for recording deleted data in each data area according to the data area, specifically, the deletion auxiliary table can record a line number where the data to be deleted is located, and when the deleted data is continuous multiple lines of data, can record a start line number of the data to be deleted and a corresponding deletion line number.
The column storage auxiliary table is used for recording control information such as offset addresses and data lengths of each area of each column in a data file and statistical information such as maximum values and minimum values of column values stored in each area, and the storage form is row storage.
Optionally, the structure of the column storage auxiliary table is shown in table 1, where the statistical information is used for the auxiliary table query.
TABLE 1 column storage auxiliary table structure
Figure BDA0001725412960000041
Figure BDA0001725412960000051
The structure of the above-described storage table is explained below:
1) column number: the column is the corresponding sequence number in the table definition when the table is created;
2) area code: different data areas have different numbers, and the number corresponding to the data area is an area number;
3) file number: the file number corresponding to the data file;
4) offset in file: for example, if three data areas are stored in the same data file, the offset in the file of the first data area is 0, the offset in the file of the second data area is the data space occupied by the first data area, and the offset in the file of the third data area is the data space occupied by the first and second data areas.
5) Zone size: the total row number of the data which can be stored in the data area is preset by a user;
6) number of valid data lines in area: removing the line number of the data in the data area after the data are deleted;
7) the size of the occupied space of the data is as follows: the number of bytes occupied by data storage;
8) number of rows of NULL values included: the number of lines occupied by the data null value in the data area;
9) number of lines where all data are different from each other: the number of lines occupied by mutually different data in the data stored in the column storage table;
10) maximum within the zone: a maximum data value in the data area;
11) minimum in zone: a minimum data value in the data region;
12) the sum of all values in the field, all data values in the data field, is summed.
Wherein, the column number, the area number, the file number, the size of the occupied space of the data and the offset in the file in the column storage auxiliary table are control information; the maximum value in the area, the minimum value in the area, the sum of all values in the area, the area size, the number of lines of effective data in the area, the number of lines of included null values and the number of lines of all data which are different from each other are statistical information.
The data to be updated in the embodiment of the present application may be any data object that needs to be updated, and may be, for example, a student data object, where the student data object generally includes fields of name, gender, school number, age, grade, and the like of a student, or a teacher data object, where the teacher data object generally includes fields of name, age, teaching age, salary, and the like. Optionally, the data to be updated may also be a field in the data object, for example, the age or grade of the student data object. Optionally, the data to be updated may also include a row number or a column number where the data to be updated is located. Illustratively, it may be the line number where a certain student data object is located.
The data to be updated may include a value of the data object to be updated, a row number where the data object is located, and a column number where the data object is located, and a plurality of records may be updated at a time each time the update is performed, each record including the row number, the column number, and the updated value.
Optionally, the obtaining of the data to be updated includes:
and acquiring an update statement of a column storage table, and determining the data to be updated according to the update statement. The data to be updated can be obtained by first obtaining an update statement of a column storage table and determining the data to be updated according to the update statement.
And step 120, inserting the row number, the column number and the updated value into an updated auxiliary table corresponding to the column storage table.
The update auxiliary table is used for recording the update data in each data area in the column storage table, wherein one update record comprises a row number, a column number and an updated value. The table structure for updating the auxiliary table in this embodiment is as follows:
table 2: updating auxiliary table structure
Column name Type (B) Description of the invention
COLID SMALLINT Updated column number
DTA_ROWID BIGINT Updated row number
VALUE VARBINARY(8188) Updated value
The update auxiliary table is used for updating the column storage table data. And when the column storage table is updated, recording the row number and the column number of the acquired data to be updated and the updated data in the update auxiliary table.
Step 130, determining a data area corresponding to the data to be updated in the column storage table according to the row number and the column number.
And comparing the row number and the column number with the row number and the column number of each data area in the column storage table, and determining the corresponding data area of the data to be updated in the column storage table.
Wherein, according to the row number and the column number, determining a data area corresponding to the data to be updated in the column storage table optionally includes:
and inquiring and determining the data area corresponding to the data to be updated in the column storage auxiliary table according to the row number and the column number.
Because the data is stored in the column storage table in an area mode, after the data to be updated in the column storage table is obtained, the data file where the data to be updated is located and which data areas in the column storage table are determined according to the row number and the column number of the data to be updated, the column number, the area number, the file number and the number of effective data lines in the areas recorded in the column storage auxiliary table.
Step 140, recalculating the statistical information of the data area according to the data of the data area and the data to be updated, and modifying the record corresponding to the statistical information in the column storage auxiliary table corresponding to the column storage table.
After the data to be updated is updated, the statistical information of the data area where the data to be updated is located in the column storage table is also changed correspondingly, for example, the statistical information of the number of rows of all the data which are different from each other, the sum of all the values in the area, and the like. Therefore, the corresponding statistical information in the column storage auxiliary table corresponding to the column storage table needs to be modified. After the data area where the data to be updated is located is determined, acquiring the data of the data area, calculating statistical information of the data area after the data to be updated is updated by the data area, and correspondingly modifying corresponding records in the column storage auxiliary table, for example, after the data to be updated is updated, if the number of lines of all data different from each other is changed, correspondingly modifying the record in the column storage auxiliary table; after updating the data to be updated, the sum of all values in the area is changed, and the record in the auxiliary table is stored corresponding to the modification column.
According to the technical scheme, when the data is updated, the updating auxiliary table is arranged, the data updating operation in the column storage table is converted into the inserting operation of the updating auxiliary table, and the data in the data file does not need to be updated frequently, so that the problem of frequent reading and writing of the data file stored based on the column is solved, and the updating efficiency of the data is improved.
On the basis of the technical scheme, the method can also optionally comprise the following steps:
and writing the data in the updated auxiliary table into the data file corresponding to the column storage table at preset time.
The preset time user can set according to needs, for example, the preset time user can set the time when the system is idle, such as 3 am every day or 3 am every weekday.
After the data of the column storage table is subjected to more updating operations, the problem of expansion of the data of the auxiliary table during updating exists. Therefore, when the system is idle or the column storage table is not operated, the data in the updated auxiliary table is reformed, a time can be preset as the preset time, and when the preset time is reached, the data in the updated auxiliary table is written into the data file corresponding to the column storage table, so that the problem of data expansion of the updated auxiliary table can be avoided, and the query efficiency of the data is improved.
Example two
Fig. 2 is a flowchart of a data updating method based on column storage according to a second embodiment of the present invention. The present embodiment provides a preferred embodiment based on the above embodiments, and reference is made to the first embodiment for details that are not described in detail in the present embodiment. As shown in fig. 2, the data updating method based on column storage according to this embodiment includes the following steps:
step 210, acquiring data to be updated of the column storage table, where the data to be updated includes a row number to be updated, a column number, and an updated value.
Step 220, scanning the record in the update auxiliary table according to the row number and the column number.
After the row number and the column number of the data to be updated are obtained, before the operation of updating the auxiliary table, whether a data record corresponding to the row number and the column number exists in the auxiliary table is scanned and updated according to the row number and the column number.
Step 230, determining whether the record of the row number and the column number exists in the update auxiliary table, if so, performing step 240, otherwise, performing step 250.
Since the original recorded content in the update supplementary table affects the current update operation on the update supplementary table, it is necessary to make a judgment before the data update operation is performed.
Step 240, modifying the corresponding updated value in the updated auxiliary table to the updated value, and then executing step 260.
And when the record of the row number and the column number already exists in the update auxiliary table by judgment, which indicates that the record corresponding to the row number and the column number is modified before the update, directly modifying the corresponding updated value in the record into the updated value.
Step 250, inserting the row number, column number and updated value into the update auxiliary table.
And when the fact that the record of the row number and the column number does not exist in the updating auxiliary table is judged, the fact that the record corresponding to the row number and the column number is not modified before the updating is shown, directly inserting the data to be updated, namely the row number, the column number and the updated value to be updated into the updating auxiliary table.
And step 260, determining a data area corresponding to the data to be updated in the column storage table according to the row number and the column number.
Step 270, recalculating the statistical information of the data area according to the data of the data area and the data to be updated, and modifying the record corresponding to the statistical information in the column storage auxiliary table corresponding to the column storage table.
According to the technical scheme, before the operation of updating the auxiliary table, whether the data record corresponding to the row number and the column number exists in the auxiliary table is scanned and updated according to the row number and the column number, and the updating rule of the data is determined according to the content stored in the auxiliary table. And when judging that the record of the row number and the column number exists in the updating auxiliary table, directly modifying the record into an updated value, otherwise, inserting the row number, the column number and the updated value into the updating auxiliary table, and keeping the updated value to be the latest value so as to ensure the query efficiency of the data. The problem of frequent reading and writing of the data files based on column storage is solved, and the data updating efficiency is improved.
EXAMPLE III
Fig. 3 is a flowchart of a data updating apparatus based on column storage according to a third embodiment of the present invention, where the apparatus is used to execute a data updating method based on column storage. As shown in fig. 3, the apparatus includes a data to be updated acquisition module 310, an insertion module 320, a data area determination module 330, and a record modification module 340.
The data to be updated acquiring module 310 is configured to acquire data to be updated of a column storage table, where the data to be updated includes a row number to be updated, a column number, and an updated value;
an inserting module 320, configured to insert the row number, the column number, and the updated value into an update auxiliary table corresponding to the column storage table;
a data area determining module 330, configured to determine, according to the row number and the column number, a data area corresponding to the data to be updated in the column storage table;
and a record modification module 340, configured to recalculate the statistical information of the data area according to the data of the data area and the data to be updated, and modify a record corresponding to the statistical information in the column storage auxiliary table corresponding to the column storage table.
Further, the insertion module includes:
the record scanning unit is used for scanning the record in the updating auxiliary table according to the line number and the column number;
a modification unit, configured to modify, if there are records of the row number and the column number in the update auxiliary table, a corresponding updated value in the update auxiliary table to the updated value;
an inserting unit, configured to insert the row number, the column number, and the updated value into the update auxiliary table if there is no record of the row number and the column number in the update auxiliary table.
Further, the to-be-updated data obtaining module 310 is specifically configured to:
and acquiring an update statement of a column storage table, and determining the data to be updated according to the update statement.
Further, the data area determining module 330 is specifically configured to:
and inquiring and determining the data area corresponding to the data to be updated in the column storage auxiliary table according to the row number and the column number.
Further, the apparatus further comprises:
and the updating table reforming module is used for writing the data in the updating auxiliary table into the data file corresponding to the column storage table at preset time.
The data updating device based on the column storage can execute the data updating method based on the column storage provided by any embodiment of the invention, and has corresponding functional modules and beneficial effects of the execution method. For details of the technology not described in detail in this embodiment, reference may be made to a data updating method based on column storage according to any embodiment of the present invention.
Example four
The fourth embodiment of the invention provides a server, and integrates the data updating device based on the column storage provided by any embodiment of the invention. Specifically, as shown in fig. 4, an embodiment of the present invention provides a server, where the server includes:
one or more processors 410, one processor 410 being exemplified in fig. 4;
a memory 420; and one or more modules.
The server may further include: an input device 430 and an output device 440. The processor 410, the memory 420, the input device 430 and the output device 440 in the server may be connected by a bus or other means, and fig. 4 illustrates the connection by a bus as an example.
The memory 420 serves as a computer-readable storage medium, and may be used for storing software programs, computer-executable programs, and modules, such as program instructions/modules corresponding to the column storage-based data updating method in the embodiment of the present invention (for example, the data acquisition module 310, the data area determination module 320, the row number recording module 330, and the record modification module 340 shown in fig. 3, the processor 410 executes various functional applications and data processing of the server by executing the software programs, instructions, and modules stored in the memory 420, that is, implementing the column storage-based data updating method in the above-described method embodiment.
The memory 420 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created according to the use of the server, and the like. Further, the memory 420 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some examples, memory 420 may further include memory located remotely from processor 410, which may be connected to a server over a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The input device 430 may be used to receive input numeric or character information and generate key signal inputs related to user settings and function control of the server. The output device 440 may include a display device such as a display screen.
The server can execute the data updating method based on the column storage provided by any embodiment of the invention, and has corresponding functional modules and beneficial effects of the execution method.
EXAMPLE five
The fifth embodiment of the present invention further provides a storage medium, where a computer program is stored, and when the computer program is executed by a processor, the method for updating data based on column storage according to the fifth embodiment of the present invention includes:
that is, the program when executed by the processor implements:
acquiring data to be updated of a column storage table, wherein the data to be updated comprises a row number, a column number and an updated value to be updated;
inserting the row number, the column number and the updated value into an updated auxiliary table corresponding to the column storage table;
determining a corresponding data area of the data to be updated in the column storage table according to the row number and the column number;
and recalculating the statistical information of the data area according to the data of the data area and the data to be updated, and modifying the record corresponding to the statistical information in the column storage auxiliary table corresponding to the column storage table.
Any combination of one or more computer-readable media may be employed. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Computer program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present invention and the technical principles employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims (8)

1. A method for updating data stored on a column basis, the method comprising:
acquiring data to be updated of a column storage table, wherein the data to be updated comprises a row number, a column number and an updated value to be updated;
scanning and updating records in the auxiliary table according to the row number and the column number;
if the record of the row number and the column number exists in the update auxiliary table, modifying the corresponding updated value in the update auxiliary table into the updated value;
if the record of the line number and the column number does not exist in the updating auxiliary table, inserting the line number, the column number and the updated value into the updating auxiliary table;
determining a corresponding data area of the data to be updated in the column storage table according to the row number and the column number;
and recalculating the statistical information of the data area according to the data of the data area and the data to be updated, and modifying the record corresponding to the statistical information in the column storage auxiliary table corresponding to the column storage table.
2. The method of claim 1, wherein obtaining data to be updated comprises:
and acquiring an update statement of a column storage table, and determining the data to be updated according to the update statement.
3. The method according to claim 1, wherein determining the corresponding data area of the data to be updated in the column storage table according to the row number and the column number comprises:
and inquiring and determining the data area corresponding to the data to be updated in the column storage auxiliary table according to the row number and the column number.
4. The method of claim 1, further comprising:
and writing the data in the updated auxiliary table into the data file corresponding to the column storage table at preset time.
5. An apparatus for updating data stored on a column basis, the apparatus comprising:
the device comprises a to-be-updated data acquisition module, a to-be-updated data acquisition module and a to-be-updated data updating module, wherein the to-be-updated data acquisition module is used for acquiring to-be-updated data of a column storage table, and the to-be-updated data comprises a row number, a column number and an updated value to be updated;
the record scanning unit is used for scanning and updating records in the auxiliary table according to the line number and the column number;
a modification unit, configured to modify, if there are records of the row number and the column number in the update auxiliary table, a corresponding updated value in the update auxiliary table to the updated value;
an inserting unit, configured to insert the row number, the column number, and the updated value into the update auxiliary table if there is no record of the row number and the column number in the update auxiliary table;
a data area determining module, configured to determine, according to the row number and the column number, a data area corresponding to the data to be updated in the column storage table;
and the record modification module is used for recalculating the statistical information of the data area according to the data of the data area and the data to be updated and modifying the record corresponding to the statistical information in the column storage auxiliary table corresponding to the column storage table.
6. The apparatus of claim 5, further comprising an updated table reformatting module, the updated table reformatting module being specifically configured to:
and writing the data in the updated auxiliary table into the data file corresponding to the column storage table at preset time.
7. A server, characterized in that the server comprises:
one or more processors;
storage means for storing one or more programs;
when executed by the one or more processors, cause the one or more processors to implement the column store based data update method of any one of claims 1-4.
8. A computer storage medium on which a computer program is stored, which program, when being executed by a processor, is adapted to carry out a method for column storage based data updating according to any one of claims 1 to 4.
CN201810750473.1A 2018-07-10 2018-07-10 Data updating method and device based on column storage, server and storage medium Active CN108959587B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201810750473.1A CN108959587B (en) 2018-07-10 2018-07-10 Data updating method and device based on column storage, server and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810750473.1A CN108959587B (en) 2018-07-10 2018-07-10 Data updating method and device based on column storage, server and storage medium

Publications (2)

Publication Number Publication Date
CN108959587A CN108959587A (en) 2018-12-07
CN108959587B true CN108959587B (en) 2021-03-02

Family

ID=64482680

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810750473.1A Active CN108959587B (en) 2018-07-10 2018-07-10 Data updating method and device based on column storage, server and storage medium

Country Status (1)

Country Link
CN (1) CN108959587B (en)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109828974A (en) * 2019-02-28 2019-05-31 上海达梦数据库有限公司 Data reforming method, device, equipment and storage medium
CN109933590B (en) * 2019-03-19 2021-04-20 上海达梦数据库有限公司 Data updating method, device, server and storage medium
CN113553335A (en) * 2021-07-26 2021-10-26 北京东方国信科技股份有限公司 Data updating method and device for columnar storage file
CN113901069B (en) * 2021-12-08 2022-03-15 威讯柏睿数据科技(北京)有限公司 Data storage method and device of distributed database

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102129458A (en) * 2011-03-09 2011-07-20 胡劲松 Method and device for storing relational database
CN103942342A (en) * 2014-05-12 2014-07-23 中国人民大学 Memory database OLTP and OLAP concurrency query optimization method
CN104715039A (en) * 2015-03-23 2015-06-17 星环信息科技(上海)有限公司 Column-based storage and research method and equipment based on hard disk and internal storage
CN106557494A (en) * 2015-09-25 2017-04-05 北京国双科技有限公司 Update the method and device of row storage table

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9785659B2 (en) * 2013-08-01 2017-10-10 International Business Machines Corporation Protecting storage data during system migration

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102129458A (en) * 2011-03-09 2011-07-20 胡劲松 Method and device for storing relational database
CN103942342A (en) * 2014-05-12 2014-07-23 中国人民大学 Memory database OLTP and OLAP concurrency query optimization method
CN104715039A (en) * 2015-03-23 2015-06-17 星环信息科技(上海)有限公司 Column-based storage and research method and equipment based on hard disk and internal storage
CN106557494A (en) * 2015-09-25 2017-04-05 北京国双科技有限公司 Update the method and device of row storage table

Also Published As

Publication number Publication date
CN108959587A (en) 2018-12-07

Similar Documents

Publication Publication Date Title
CN108959587B (en) Data updating method and device based on column storage, server and storage medium
US20190180344A1 (en) User interface and geo-parsing data structure
CN108875077B (en) Column storage method and device of database, server and storage medium
CN111258966A (en) Data deduplication method, device, equipment and storage medium
US20100274795A1 (en) Method and system for implementing a composite database
US9323798B2 (en) Storing a key value to a deleted row based on key range density
CN109471851B (en) Data processing method, device, server and storage medium
CN103186622A (en) Updating method of index information in full text retrieval system and device thereof
CN109408539B (en) Data operation method, device, server and storage medium
CN108694172B (en) Information output method and device
CN109542912B (en) Interval data storage method, device, server and storage medium
US11789940B2 (en) Natural language interface to databases
CN110222046B (en) List data processing method, device, server and storage medium
CN108984720B (en) Data query method and device based on column storage, server and storage medium
US10409799B2 (en) Supporting updatable repeated values over variable schema
CN112835905B (en) Array type column indexing method, device, equipment and storage medium
CN111190895B (en) Organization method, device and storage medium of column-type storage data
CN111522820A (en) Data storage structure, storage retrieval method, system, device and storage medium
KR20150123603A (en) Method and system for managing database
CN114547086B (en) Data processing method, device, equipment and computer readable storage medium
CN116450607A (en) Data processing method, device and storage medium
CN108984719B (en) Data deleting method and device based on column storage, server and storage medium
CN109033271B (en) Data insertion method and device based on column storage, server and storage medium
CN115374121A (en) Database index generation method, machine-readable storage medium and computer equipment
CN113722296A (en) Agricultural information processing method and device, electronic equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant