JP2015179353A - Data processing apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce the time required for updating a database.SOLUTION: A data processing apparatus 100 includes: a format conversion section 101 for forming column-format data by decomposing tabular data into components by column; and a data operation section 102 for operating the column-format data. The format conversion section 101 forms a free area between data constituting the column-format data, in converting the tabular data into the column-format data, to form the column-format data. The data operation section 102 inserts other data in the free area formed between the data, in inserting the other data between the data constituting the column-format data.

Description

  The present invention relates to a data processing apparatus, and more particularly to a data processing apparatus that processes a database.

  As a database management system (DBMS) internal structure, a relational database (RDB) is widely used in which multiple data are represented in tabular form and associated with each other. Yes. RDB is good at update processing such as transactions, but since each row (record) is used as a unit of data reference and operation, there is a disadvantage that aggregation and search in column units are slow.

  To overcome this shortcoming, FAST (Filter Array Structure) enables tabulation and high-speed aggregation and processing by processing the arrangement of the data for each column and keeping it deduplicated. A data structure called structure has been devised. FIG. 1 shows an example in which one tabular data shown in FIG. 1 (A) is converted into FAST structure data shown in FIG. 1 (B). As shown in FIG. 1B, in the FAST structure, tabular data is referred to as “order set” representing row numbers, “value list” (hereinafter referred to as “VL”) which is a unique and sorted data group. ), And is expressed by being decomposed into three components of “value number” (hereinafter referred to as “VNo”) indicating which data is included in the column. A DBMS having an internal structure designed so that column processing can be efficiently performed by considering data not in record units but in column units like this FAST structure is called a column store DB.

  In the FAST structure, when data is in memory, VNo and VL data are arranged in an array, so there is no cost for calculating the address to the data, and the data is arranged in an array. As the cache hit rate increases, it is possible to perform aggregation and search at high speed.

JP 2002-108662 A

  However, the above-described FAST structure has a problem that it takes time to perform update processing such as insertion of new data. Specifically, first, in the FAST structure, VL eliminates duplicate data and stores sorted data in an array. However, when new data that does not exist in the VL is inserted, it is necessary to rearrange the data in the VL array, and the VNo value indicating the storage destination of the VL data is also updated accordingly. Needs to be processed.

  An example of VNo and VL remodeling processing is shown in FIG. Here, consider an example in which new data “1997/07/22” is inserted into the “birth date” column in the FAST structure, as shown in FIG. “1997/07/22” is stored in the array number 3 of the VL as shown by the hatched line in FIG. 2B in order to maintain the sortability of the VL data. However, at that time, as indicated by an arrow A2 in FIG. 2A, the data that has been included in SEQ ID NOS: 3 to 8 so far is moved backward by one (remake of VL). Next, along with this data movement, it is necessary to increase the value of VNo that previously indicated the storage destination of the data of array numbers 3 to 8 of VL by 1 as shown in FIG. Remake).

  The remodeling process described above costs a certain amount of time, and generally takes longer as the VNo and VL sizes are larger. For this reason, the longer this time, the greater the TAT (Turn Around Time) when a user makes a reference to data or an operation inquiry during remodeling. As described above, the FAST structure has a problem that it takes time to perform update processing such as insertion of new data.

  Here, there is a technique disclosed in Patent Document 1 as a technique related to the present invention. In such technology, when new data is added to the database, if there is no space at the location where such data is added, the previous and next data are moved to create a space, and the new data is stored in the space. It is. However, even the technique disclosed in Patent Document 1 requires processing such as creating a vacancy in the database when storing data, and there is a problem that it takes time to perform update processing as described above.

  Therefore, an object of the present invention is to solve the above-described problem that it takes time to update the database.

A data processing apparatus according to one aspect of the present invention
A format conversion unit that converts tabular data into column format data broken down into components for each column;
A data operation unit for performing operations on the column format data,
When converting the table format data to the column format data, the format conversion unit forms a free area between the data constituting the column format data, and converts the data into the column format data.
The data operation unit inserts the other data into the empty area formed between the data when the other data is inserted between the data constituting the column format data.
The configuration is as follows.

Moreover, the program which is the other form of this invention is:
In the information processing device,
A format conversion unit that converts tabular data into column format data broken down into components for each column;
A data operation unit for performing operations on the column format data;
Is a program that realizes
When converting the table format data to the column format data, the format conversion unit forms a free area between the data constituting the column format data, and converts the data into the column format data.
The data operation unit inserts the other data into the empty area formed between the data when the other data is inserted between the data constituting the column format data.
The configuration is as follows.

In addition, a data processing method according to another aspect of the present invention includes:
Convert tabular data into column format data broken down into components for each column,
A data processing method for performing an operation on the column format data,
When converting the table format data into the column format data, a free space is formed between the data constituting the column format data, and the table format data is converted into the column format data.
When inserting other data between the data constituting the column format data, the other data is inserted into the empty area formed between the data,
The configuration is as follows.

  By configuring as described above, the present invention can suppress the time required for database update processing.

In the technique relevant to this invention, it is a figure which shows an example which converted tabular data into FAST structure data. It is a figure which shows a mode when new data are inserted in FAST structure data in the technique relevant to this invention. It is a block diagram which shows the structure of the database management system in this invention. It is a flowchart which shows operation | movement of the database management system in this invention. It is a flowchart which shows operation | movement of the database management system in this invention. It is a sequence diagram which shows operation | movement of the database management system in this invention. It is a figure which shows an example which converted the tabular data into FAST structure data in the database management system in this invention. It is the figure which compared the FAST structure data converted in the technique relevant to this invention, and the FAST structure data converted by the database management system in this invention. It is a figure which shows the mode of a process when adding new data to FAST structure data in the database management system in this invention. It is a figure which shows the mode of a process when adding new data to FAST structure data in the database management system in this invention. It is a figure which shows the mode of a process when adding new data to FAST structure data in the database management system in this invention. It is a figure which shows a mode when adjusting the empty size of FAST structure data in the database management system in this invention. It is a figure which shows an example which determines the insertion level of the new data with respect to FAST structure data in the database management system in this invention. It is a block diagram which shows the structure of the data processor in the additional remark 1 of this invention.

  A first embodiment of the present invention will be described with reference to FIGS. FIG. 1 is a block diagram showing the configuration of the database management system. 2 to 6 are diagrams for explaining the operation of the database management system. 7 to 13 are diagrams showing a state when data is operated in the database management system.

[Constitution]
The database management system 1 according to the present invention is a general information processing apparatus including an arithmetic device and a storage device. As shown in FIG. 3, the database management system 1 is operated by a user 2 and connected to a client 3 that is an information processing terminal operated by the user 2.

  As shown in FIG. 3, the database management system 1 includes an import unit 10, a database operation unit 20, and a free size adjustment unit 30 that are constructed by incorporating a program into an equipped arithmetic device. The import unit 10 includes a data conversion unit 11. The database operation unit 20 includes a data insertion processing unit 21, and the data insertion unit 21 further includes a data insertion confirmation unit 22, a VNo / VL remodeling unit 23, a data insertion unit 24, and an insertion level update unit 25. ing. In addition, the data management system 1 includes a database 40 that stores the FAST structure data D1 to be handled in the equipped storage device.

  The import unit 10 accepts tabular data D1 that is a set of data represented in tabular format such as a CSV file and imported by a user 2 who uses the system 1. Then, as shown in FIG. 1 described above, the data conversion unit 11 (format conversion unit) included in the import unit 10 converts the tabular data D1 into FAST structure data D2, which is column format data obtained by decomposing the data into components for each column. Convert and store in the database 40. Note that, as described above, the FAST structure data D2 includes tabular data in “ordered sets” representing row numbers, “value lists” (“VL”) that are unique and sorted data groups, and columns. It is expressed by decomposing into three components of “value number” (“VNo”) indicating whether data is included. However, the data converter 11 may convert the tabular data D1 into other column format data different from the FAST structure data D2.

  When the data conversion unit 11 converts the tabular data D1 shown in FIG. 7A to the FAST structure data D2, the data conversion unit 11 stores the data in each VL as shown by “NULL” in FIG. 7B described later. A free space of a predetermined capacity is inserted between the data to be processed. Furthermore, as shown in FIG. 7B, which will be described later, the data conversion unit 11 inserts an insertion level array (IL) that is a new area for storing the degree of insertion of new data for each VL data. Create FAST structure data with added and store it in the database.

  Further, the database operation unit 20 receives an inquiry issued from the client 3, operates the database 40 in response to the inquiry, and returns the inquiry result to the client 3. In particular, the data insertion processing unit 21 (data operation unit) included in the database operation unit 20 performs processing corresponding to each inquiry on the FAST structure data D2. In addition, the free size adjustment unit 30 (free space adjustment unit) also performs processing on the FAST structure data D2.

  When the new data is inserted into the FAST structure data, the data insertion processing unit 21 performs a new data entry process as follows. First, the data insertion confirmation unit 22 confirms whether there is a free area in the insertion range. If there is an empty area, the data insertion unit 24 inserts new data, and the insertion level update unit 25 updates the insertion level array “IL” in the inserted range. If there is no free area, the VNo / VL remodeling unit 23 recreates the VNo / VL and also recreates the insertion level array “IL”.

  The free size adjusting unit 30 periodically adjusts the free area of the FAST structure data. Along with this, the empty size adjustment unit 30 also recreates the insertion level array “IL”. The empty size adjusting unit 30 operates in cooperation with the data insertion processing unit 21 as necessary, and periodically adjusts the empty area of the FAST structure data even when the new data is inserted.

  Next, a specific example of the configuration of the above-described data management system 1 and its operation will be described with reference to FIGS. In the following, in particular <1. Conversion operation from tabular data to FAST structure data>, <2. <2-a. Inserting new data into FAST structure data> Insertion of new data into VL with free area> and <2-b. Operation when remodeling VNo and VL>, <3. The operation of adjusting the size of the free area> will be described separately.

<1. Conversion from tabular data to FAST structure data>
Here, the case where the “phonetic” column, which is the tabular data shown in FIG. 7A, is converted into FAST structure data will be described as an example.

  First, in the technology related to the present invention, when the “phonetic” column is converted into the FAST structure, the format shown in FIG. In this example, “VL” stores sorted data by eliminating duplication of “phonetic” column data. Also, “VNo” stores the storage destination array number of the data “VL” so that it can be understood which data is contained in the “phonetic” column of each record.

  On the other hand, when creating the “VL”, the data conversion unit 11 of the database management apparatus 1 according to the present invention stores the data by putting an empty area between the data constituting the “VL”. A specific example of this process will be described with reference to the flowchart of FIG. 4 and FIG.

  First, in the tabular data shown in FIG. 7 (A), the number of data that has been de-duplicated in the “Furigana” column is duplicated with “Takahashi” of the numbers 2 and 4, and as shown in FIG. In total, “9”. Here, when the number of free areas (size) “S” is “3”, “9 × (3 + 1) + 3 = 39”, and the size of the array of “VL” is “39” " In this way, the arrangement region of “VL” is secured (step S1 in FIG. 4).

  Next, nine “phonetic” data sorted as shown in FIG. 8A, three free areas between each data, and free data at both ends of the data as shown in FIG. 8B. The data are stored in sequence in sequence numbers 3, 7, 11, 15, 19, 23, 27, 31, and 35 so that three areas are included (step S2 in FIG. 4). At this time, “NULL” indicating a vacancy is entered in a vacant area where no element is contained (step S3 in FIG. 4). It should be noted that the empty area is not limited to “NULL”, and other values such as a special value indicating the empty may be included.

  Thereafter, as shown in FIG. 8B, the data converter 11 creates a new array “IL” (Insert Level) different from “VL” with the same size “39” as the size of “VL”. (Step S4 in FIG. 4). Then, “0” is stored in all the elements (step S5 in FIG. 4). This IL array is an array for recording the degree of insertion of new data (insertion status information) within the range of corresponding data. As will be described later, the free space between the “VL” data is adjusted according to the “insertion level” between the data determined from the “degree of insertion”.

<2. Inserting new data into FAST structure data>
Next, the operation when new data is inserted into the FAST structure data will be described with reference to the flowchart of FIG. 5, the sequence diagram of FIG. 6, and FIGS. First, an example of inserting new data “SHIMIZU” into the “Furigana” column of the FAST structure data shown in FIG. 8B created as described above, that is, inserting new data into “VL” having a free space The operation will be described.

<2-a. Inserting new data into a VL with free space>
In the “VL” of the FAST structure data shown in FIG. 8B, the range in which “stains” is present is scanned (step S11 in FIG. 5 and step S21 in FIG. 6). The array scanning method can be performed by binary search. Then, it can be seen that “Shimizu” does not yet exist in “VL” (step S12: No in FIG. 5) and is between “Sato” and “Suzuki”. Further, as shown in FIG. 8B, the element of the array element number 25 in the middle of “Sato” and “Suzuki” is “NULL” and is empty (step S13 in FIG. 5: Yes, step S22 in FIG. 6). ) As shown by hatching in FIG. 9, “stains” are inserted into the empty area (step S15 in FIG. 5 and steps S23 and S24 in FIG. 6).

  Thereafter, the “insertion degree” which is the value of the element of the “IL” array located in the range where the new data is inserted is increased by “1”. Here, since “shimuzu” is inserted between “sato” and “suzuki”, all of the “IL” array located in the range between “sato” and “suzuki” as shown by the oblique lines in FIG. The value of the element is changed from “0” to “1” (step S16 in FIG. 5 and steps S25 and S26 in FIG. 6).

  Finally, an element “25” is added at the very end of “VNo” so as to indicate “stain” of “VL” (step S17 in FIG. 5) (see the hatched line in FIG. 9). When the empty space in the range where new data is inserted is an even number, the middle place determined as the place to be inserted is the one with the smaller array number. For example, if “Suzuki” is not “27” but “26”, there are two places where “Shimizu” can enter, “24” or “25”, but if you put it in the smaller “24” To do.

  With the above processing, when the new data “SHIMIZU” is inserted into the FAST structure data shown in FIG. 8B, “VNo” and “VL” of the FAST structure data are as shown in FIG. As a result of scanning the range of the new data “SHIMIZU” in the FAST structure data (step S11 in FIG. 5), if the same data has already been entered (step S12 in FIG. 5: Yes) ), No data is inserted into “VL”.

  Next, consider a case where there is no vacancy at the location where new data is to be inserted. First, the FAST structure data is the case of FIG. 9, and it is considered to insert new data of “Shimoyanagi” and “Shouai”. In FIG. 9, “Shimoyanagi” is between “Shimizu” and “Suzuki”, and since the element of the array element number 26 in the middle is “NULL”, it is determined that insertion is possible. Insert as shown. Next, “Shouai” is located between “Shimoyanagi” and “Suzuki”, but there is no “ULL” space between them. In this case, VNo and VL are changed as follows.

<2-b. Operation when remodeling VNo and VL>
In the VNo, VL remodeling process, a space is inserted between each “VL” data as in the case where the data conversion unit 11 converts the tabular data into FAST structure data (FIG. 6). Steps S27 and S28). However, the size (number) of empty areas is changed for each data range according to the “insertion level” determined from the array “IL”. Specifically, the new empty size is (S + (number according to the insertion level)). It is estimated that the higher the insertion level of the data range, the higher the possibility that data will be inserted into the range thereafter.

  Here, a method of determining the “insertion level” based on the “IL” sequence corresponding to each data of “VL” will be described with reference to FIG. In each pattern of FIGS. 13A, 13B, and 13C, the “insertion degree” that is the value of the “IL” element set in the empty area (NULL) within the range of “VL” is all If they match, it becomes the “insertion level” corresponding to the value, and if all do not match, it becomes the “insertion level” of a low value. In these examples, “insertion levels” in the range of “VL” data “156” to “162” are (A): level 1, (B): level 0, and (C): level 0, respectively. . Also, as shown in the patterns of FIGS. 13D and 13E, when there is no space in the “VL” range, the values of the “IL” elements of the adjacent data are compared. Then, the smaller value becomes the “insertion level”. Therefore, in these examples, (D): level 1 and (E): level 0.

  Next, an example of the above-described VNo, VL remodeling process will be described with reference to FIG. Here, it is assumed that new data “Sato” is inserted into the FAST structure data of FIG. Then, “Sato” is in the range from “Kimura” to “Takahashi”, but there is no space between them, so VNo and VL are remade.

  Specifically, the VNo, VL remodeling process first inserts several new free areas between the “VL” data according to the value of the “insertion level” corresponding to the data. Do. Here, as an example, “original empty area number + (insertion level × 3)” empty areas are newly inserted between data. That is, the higher the insertion level, the more free areas are inserted in the range. However, the number of empty areas to be inserted is not limited to the number described above.

  For example, in FIG. 11A, a “NULL” empty area indicated by the diagonal line in FIG. 11B is added to the range indicated by the diagonal line whose “insertion level” based on “IL” is not “0”. . In particular, six empty areas “NULL” corresponding to “insertion level = 2” are added between “Kimura” and “Takahashi”. Then, “Sato” is inserted into the empty area added between “Kimura” and “Takahashi”. Accordingly, the range from “Kimura” to “Takahashi” in which “Sato” is inserted is “1” as the “insertion degree” that is the value of “IL”, as indicated by the hatched line in FIG. Adds to "3".

<3. Adjustment of free space size>
Here, in the range where new data is not inserted so much, that is, in the range where the “insertion level” is low, the amount of memory corresponding to the size of the initial free space is left without being inserted for a long time. Will be consumed. As a countermeasure against this, the empty size adjusting unit 30 periodically adjusts the empty area in the range where the “insertion level” is low. At this time, a process for expanding the free area is also performed in accordance with the “insertion level” performed by changing the VNo and VL described above. It is assumed that this process is automatically performed periodically in anticipation of when the database system load is low (for example, at night).

  In adjusting the size of the free space, the following processing is performed in addition to the above-described VNo and VL remodeling processing.

  First, in the range where the “insertion level” is “0”, the size of the free area, that is, the number S of free areas is set to “S / 2 (rounded down to the decimal point)”. However, new data may be inserted later, and the minimum free area size is set to “1” so that the size of the free area is not “0” at that time. For example, if there are “3” free areas in the range of “0” as “insertion level” indicated by hatching in FIG. 12A, the free areas are shown as shaded in FIG. “1” (3/2: rounded down).

  In addition to this, in the range where the “insertion level” is larger than “0”, as shown in FIG. 12B, the number of empty areas “NULL” corresponding to the value of “insertion level” is inserted as described above. To do.

  After the VNo and VL are remade as described above, the “insertion degree” of each “IL” is lowered by “1”, as indicated by the hatched lines in FIG. At this time, the portion where the “insertion degree” is “0” remains “0”.

  In actual database operations, available resources are limited. A maximum value may be set for the array size so that the required memory usage does not exceed the usable resource limit by recreating VNo and VL that take extra space size or adjusting the space size. In this case, when the array size exceeds the maximum value, it is possible to adjust the empty size a plurality of times to reduce the array size. In the above description, the case where the free area in the range where the “insertion level” is “0” is reduced is described. However, the free area may be reduced even in the range where the “insertion level” is greater than “0”. . In this case, the lower the “insertion level”, the lower the possibility that new data will be inserted later. Therefore, the lower the “insertion level”, the smaller (smaller) the size (number) of free areas is adjusted. May be.

  It should be noted that a command or API I / F for executing the adjustment asynchronously is prepared in the above-described empty size adjustment unit 30 so that it can be executed at a convenient timing from the user 2 or the client 3 used by the user. It is also possible. As a result, the database operator can make the adjustment at an explicit timing.

  As described above, according to the present invention, by reducing the number of times VNo and VL are remodeled, the time required for the remodeling can be suppressed, and as a result, the performance of the apparatus can be improved. In particular, the present invention is useful for columns in which new data is randomly and uniformly inserted into sorted column data.

  In the above description, the operation for inserting new data has been described. However, the present invention can also be applied to data update processing. For example, in the conventional system, if new data not in “VL” is overwritten, the same processing as the insertion processing of new data is performed, and it is necessary to recreate VNo and VL. Even in this case, the number of remodeling occurrences can be reduced by the method of the present invention.

<Appendix>
Part or all of the above-described embodiment can be described as in the following supplementary notes. The outline of the configuration of the data processing apparatus (see FIG. 14), program, and data processing method in the present invention will be described below. However, the present invention is not limited to the following configuration.

(Appendix 1)
A format conversion unit 101 that converts tabular data into column format data decomposed into components for each column;
A data operation unit 102 for performing operations on the column format data,
When converting the table format data into the column format data, the format conversion unit 101 forms an empty area between the data constituting the column format data, and converts the data into the column format data.
When the data operation unit 102 inserts other data between the data constituting the column format data, the data operation unit 102 inserts the other data into the empty area formed between the data.
Data processing apparatus 100.

  According to the above invention, first, when converting the tabular data to the column format data, the column format data is generated by previously forming an empty area between the data constituting the column format data. When other data is inserted between the data constituting the column format data by manipulating the column format data, the other data is inserted into the free space formed as described above. As a result, when other data is inserted into the column format data, the order is not updated, so the time required for the update can be suppressed and the performance of the data processing apparatus can be improved.

(Appendix 2)
A data processing apparatus according to attachment 1, wherein
The data operation unit stores and manages insertion status information representing an insertion status of the other data between the data for each data forming the column format data.
Data processing device.

(Appendix 3)
The data processing device according to attachment 2, wherein
According to the insertion status information, provided with a free space adjustment unit that changes the size of the free space formed between each data forming the column format data,
Data processing device.

(Appendix 4)
The data processing device according to attachment 3, wherein
The data operation unit manages the insertion degree of the other data between the data forming the column format data as the insertion status information,
The free space adjustment unit changes the size of the free space formed between corresponding data as the insertion degree is higher,
Data processing device.

(Appendix 5)
The data processing device according to appendix 3 or 4,
The data operation unit manages the insertion degree of the other data between the data forming the column format data as the insertion status information,
The free space adjustment unit changes the size of the free space formed between the corresponding data to be smaller as the insertion degree is lower,
Data processing device.

(Appendix 6)
A data processing device according to appendix 4 or 5, wherein
The data operation unit is configured to change and set the insertion degree between each data in which the size of the free space is changed by the free space adjustment unit.
Data processing device.

  As described above, by changing the size of the empty area according to the past data insertion state, it is possible to appropriately cope with subsequent data insertion. As a result, it is possible to further suppress the necessity of updating the order of the column format data by inserting other data, to save capacity, and to further improve the performance of the data processing device. it can.

(Appendix 7)
A data processing apparatus according to any one of appendices 1 to 6,
The format conversion unit forms free areas at both ends of each data located at the head and the rear end among the data constituting the column format data,
Data processing device.

(Appendix 8)
A data processing apparatus according to any one of appendices 1 to 7,
The format conversion unit converts the tabular data into column format data having a FAST (Filter Array Structure) structure that is decomposed into components of the value, order, and position, which are the data, for each column.
Data processing device.

(Appendix 9)
In the information processing device,
A format conversion unit that converts tabular data into column format data broken down into components for each column;
A data operation unit for performing operations on the column format data;
Is a program that realizes
When converting the table format data to the column format data, the format conversion unit forms a free area between the data constituting the column format data, and converts the data into the column format data.
The data operation unit inserts the other data into the empty area formed between the data when the other data is inserted between the data constituting the column format data.
program.

(Appendix 9.1)
The program according to appendix 9, wherein
The data operation unit stores and manages insertion status information representing an insertion status of the other data between the data for each data forming the column format data.
program.

(Appendix 9.2)
The program according to attachment 9.1,
In addition to the information processing apparatus,
An empty area adjustment unit that changes the size of the empty area formed between the data forming the column format data according to the insertion status information;
A program to realize

(Appendix 10)
Convert tabular data into column format data broken down into components for each column,
A data processing method for performing an operation on the column format data,
When converting the table format data into the column format data, a free space is formed between the data constituting the column format data, and the table format data is converted into the column format data.
When inserting other data between the data constituting the column format data, the other data is inserted into the empty area formed between the data,
Data processing method.

(Appendix 10.1)
A data processing method according to attachment 10, wherein
When inserting other data between the data constituting the column format data, the insertion status information indicating the insertion status of the other data between the data for each data forming the column format data Remember and manage,
Data processing method.

(Appendix 10.2)
A data processing method according to attachment 10.1,
In accordance with the insertion status information, the size of the empty area formed between the data forming the column format data is changed.
Data processing method.

  Note that the above-described program is stored in a storage device or recorded on a computer-readable recording medium. For example, the recording medium is a portable medium such as a flexible disk, an optical disk, a magneto-optical disk, and a semiconductor memory.

  Although the present invention has been described with reference to the above-described embodiment and the like, the present invention is not limited to the above-described embodiment. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

DESCRIPTION OF SYMBOLS 1 Database management system 2 User 3 Client 10 Import part 11 Data conversion part 20 Database operation part 21 Data insertion process part 22 Data insertion confirmation part 23 VNo, VL remake part 24 Data insertion part 25 Insertion level update part 30 Free size adjustment part 40 Database D1 Tabular data D2 FAST structure data 100 Data processing device 101 Format conversion unit 102 Data operation unit

Claims (10)

A format conversion unit that converts tabular data into column format data broken down into components for each column;
A data operation unit for performing operations on the column format data,
When converting the table format data to the column format data, the format conversion unit forms a free area between the data constituting the column format data, and converts the data into the column format data.
The data operation unit inserts the other data into the empty area formed between the data when the other data is inserted between the data constituting the column format data.
Data processing device. The data processing apparatus according to claim 1,
The data operation unit stores and manages insertion status information representing an insertion status of the other data between the data for each data forming the column format data.
Data processing device. The data processing apparatus according to claim 2, wherein
According to the insertion status information, provided with a free space adjustment unit that changes the size of the free space formed between each data forming the column format data,
Data processing device. The data processing apparatus according to claim 3, wherein
The data operation unit manages the insertion degree of the other data between the data forming the column format data as the insertion status information,
The free space adjustment unit changes the size of the free space formed between corresponding data as the insertion degree is higher,
Data processing device. The data processing device according to claim 3 or 4,
The data operation unit manages the insertion degree of the other data between the data forming the column format data as the insertion status information,
The free space adjustment unit changes the size of the free space formed between the corresponding data to be smaller as the insertion degree is lower,
Data processing device. The data processing device according to claim 4 or 5,
The data operation unit is configured to change and set the insertion degree between each data in which the size of the free space is changed by the free space adjustment unit.
Data processing device. A data processing apparatus according to any one of claims 1 to 6,
The format conversion unit forms free areas at both ends of each data located at the head and the rear end among the data constituting the column format data,
Data processing device. A data processing apparatus according to any one of claims 1 to 7,
The format conversion unit converts the tabular data into column format data having a FAST (Filter Array Structure) structure that is decomposed into components of the value, order, and position, which are the data, for each column.
Data processing device. In the information processing device,
A format conversion unit that converts tabular data into column format data broken down into components for each column;
A data operation unit for performing operations on the column format data;
Is a program that realizes
When converting the table format data to the column format data, the format conversion unit forms a free area between the data constituting the column format data, and converts the data into the column format data.
The data operation unit inserts the other data into the empty area formed between the data when the other data is inserted between the data constituting the column format data.
program. Convert tabular data into column format data broken down into components for each column,
A data processing method for performing an operation on the column format data,
When converting the table format data into the column format data, a free space is formed between the data constituting the column format data, and the table format data is converted into the column format data.
When inserting other data between the data constituting the column format data, the other data is inserted into the empty area formed between the data,
Data processing method.

Images