KR102619300B1 - Appratus and method for processing of storing data - Google Patents

Abstract

Relating to a data storage and processing device and method, the data storage and processing device includes a first storage unit, a second storage unit of a different type from the first storage unit, and a second storage unit that copies and writes input data to the first storage unit based on a B+ tree. Additionally, it may include a processor that migrates at least one cold data among the data added to the first storage unit to the second storage unit.

Description

Data storage processing apparatus and method {APPRATUS AND METHOD FOR PROCESSING OF STORING DATA}

It relates to a data storage processing device and method.

A database refers to a set of data built by systematically integrating multiple data so that each data can be accessed or searched. Such databases are typically built including a database management system (DBMS) to manage the database. Recently, relational databases that build relationships between keys and values in the form of tables are widely used, and these relational databases are processed using SQL (Structured Query Language), etc. Recently, new languages such as NewSQL are being introduced for processing, accuracy, scalability, and availability of large data in relational database management systems (RDMS). New SQL also optimizes the database by preserving and storing the order without hashing the keys. However, if data is managed in the same way as a typical relational database, the range of the main key appears continuously, which may cause problems in which various requests for data are concentrated on a specific node. Therefore, to distribute requests, a universal unique identifier (UUID) is used instead of a monotonically increasing or decreasing primary key. However, in this case, there was a problem in that random keys were continuously distributed and stored in New SQL's storage engine, making it difficult to classify files created over time.

A problem to be solved is to provide a data storage processing device and method that classifies generated files even when a database is managed using a universal unique identifier and allows each classified data to be appropriately and efficiently stored in a storage device. Do this.

In order to solve the above-described problems, a data storage processing device and method are provided.

A data storage processing device adds a first storage unit, a second storage unit of a different type to the first storage unit, and input data to the first storage unit based on a copy and write B+ tree, and adds the input data to the first storage unit. It may include a processor that migrates at least one cold data among the added data to the second storage unit.

The processor acquires at least one merged data by merging the at least one cold data according to a predefined setting, and stores the at least one merged data in place of the at least one cold data to the second storage unit. It can be delivered.

The processor may perform merging of the at least one cold data among the data added to the first storage unit, excluding at least one hot data.

The processor determines whether to migrate the at least one merged data to the first storage unit based on at least one of the storage capacity, load level, and input/output bandwidth usage of at least one of the first storage unit and the second storage unit. can be decided.

The cold data may include data that is accessed or updated frequently or is of low importance, and the hot data may include data that is accessed or updated frequently or is of high importance.

The first storage unit may include a storage device with relatively high performance, and the second storage unit may include a storage device with relatively low performance.

The data storage processing method includes the steps of inputting a plurality of data simultaneously or sequentially, adding the input data to the first storage unit based on a copy and write B+ tree, and selecting the data added to the first storage unit. It may include migrating at least one cold data to a second storage unit that is different from the first storage unit.

The step of migrating at least one cold data from among the data added to the first storage unit to the second storage unit includes obtaining at least one merged data by merging the at least one cold data according to predefined settings. and transmitting the at least one merged data to the second storage unit.

Obtaining at least one merged data by merging the at least one cold data according to the predefined setting includes excluding at least one hot data from the data added to the first storage unit and Depending on settings, the method may further include merging the at least one cold data.

The step of migrating at least one cold data from among the data added to the first storage unit to the second storage unit includes the storage capacity, load level, and input/output bandwidth usage of at least one of the first storage unit and the second storage unit. It is also possible to further include the step of determining whether to migrate the merged data to the first storage unit based on at least one of the following.

The cold data may include data that is accessed or updated frequently or is of low importance, and the hot data may include data that is accessed or updated frequently or is of high importance.

The first storage unit may include a storage device with relatively high performance, and the second storage unit may include a storage device with relatively low performance.

According to the above-described data storage processing device and method, even when managing a database using a universal unique identifier, the advantage of being able to classify the generated files and store the classified data appropriately and efficiently in each storage device is obtained. You can.

According to the above-described data storage processing apparatus and method, even when new data is frequently added and written to data files, data or files that are accessed and updated frequently or are not important are stored in a storage device with relatively low performance ( hard disk storage devices, etc.), and data or files that are frequently accessed and updated or are important can be stored in relatively high-performance storage devices (solid state drive devices, etc.).

According to the above-described data storage processing device and method, it is possible to adopt a plurality of heterogeneous storage devices in a storage engine such as New SQL that uses an optimization technique for preserving order, so that less important data can be stored more efficiently. It can be classified and managed.

According to the data storage processing device and method described above, it is possible to reduce manpower and costs in operation and management without deteriorating perceived performance through efficient and differential management of data based on heterogeneous storage devices.

1 is a block diagram of one embodiment of a data storage processing device.
FIG. 2 is a diagram for explaining an example of first data and second data stored in each of the first storage unit and the second storage unit.
Figure 3 is a diagram for explaining an example of a process in which data is written to the first storage unit.
Figure 4 is a first diagram to explain an example of a process for merging cold data.
Figure 5 is a second diagram for explaining an example of a process for merging cold data.
FIG. 6 is a diagram illustrating an example of a process for migrating merged data to the second storage unit.
7 is a flowchart of one embodiment of a data storage processing method.

Terms with the addition of 'unit' used in describing the following embodiments may be implemented as software or hardware, and depending on the design, one 'unit' may be implemented as one physical or logical part, or a plurality of 'units' may be implemented. ' may be implemented with one physical or logical part, or one 'part' may be implemented with a plurality of physical or logical parts. When a part is said to be connected to another part throughout the specification, this may mean a physical connection or an electrical connection depending on the part and the other part. In addition, when a part includes another part, this does not mean excluding another part other than the other part unless specifically stated to the contrary, and means that another part may be included depending on the designer's choice. do. Terms such as first or second are used to distinguish one part from another, and unless specifically stated, they do not mean sequential expressions. Additionally, singular expressions may include plural expressions, unless the context clearly makes an exception.

Hereinafter, an embodiment of a data storage processing device will be described with reference to FIGS. 1 to 6.

1 is a block diagram of one embodiment of a data storage processing device.

According to an embodiment shown in FIG. 1, the data storage and processing device 10 includes an input unit 99, a processor 100, and first to Nth storage units 110 to 130, where N is a natural number of 2 or more. ) may include. At least two of the input unit 99, the processor 100, and the first to Nth storage units 110 to 140 are connected to circuits, cables, and/or wireless communication networks (e.g., Wi-Fi, Wi-Fi Direct ( From one room to another via (Wi-Fi Direct), Bluetooth, Bluetooth Low Energy, CAN, zigbee, or Near Field Communication (NFC), or It is provided so that instructions or data can be transmitted/received simultaneously or sequentially in both directions. In addition, the data storage processing device 10 visually or audibly outputs the data stored in at least one of the first to Nth storage units 110 to 140 as a result of processing by the processor 100 to the outside, as necessary. In addition to the output unit (not shown, for example, a monitor device, data input/output terminal, or printer device, etc.) and/or the input unit 99, or separately from the input unit 99, data 80 is received from another external device (not shown). A communication unit (not shown, for example, a LAN card, short-distance communication) capable of receiving or transmitting data stored in at least one of the first to N-th storage units 110 to 140 as a result of processing by the processor 100 to another external device. It is also possible to further include modules or mobile communication modules, etc.).

The input unit 99 receives at least one data 80 according to the user's operation or predefined settings, and transfers the received data 99 to the processor 100 and the first to Nth storage units 110 to 110. 140). Data 80 coming through the input unit 99 may include letters, numbers, symbols, or a combination of at least two of these (hereinafter referred to as letters, numbers, etc.). If necessary, the data 80 may further include a key for classifying or searching the data 80. Additionally, depending on the embodiment, these data 80 may have the form of a table (for example, a sorted string table (SST)), in which case the data 80 contains a value and a key corresponding to it. It may be prepared in the form of a string containing it. Depending on the embodiment, the data 80 may include a file in a predetermined format that is a set of letters or numbers.

The input unit 99 may be provided integrally with the data storage and processing device 10, or may be provided to be physically separable. The input unit 99 includes, for example, a keyboard, mouse, tablet, touch screen, touch pad, track ball, track pad, scanner device, image capture module, ultrasonic scanner, motion detection sensor, vibration sensor, light receiving sensor, and pressure sensor. , it may include a proximity sensor and/or a microphone, etc. Additionally, depending on the embodiment, the input unit 99 may include a data input/output terminal capable of receiving data from another external device (for example, a portable memory device, etc.).

The processor 100 may control at least one of the first to Nth storage units 110 to 130 to temporarily or non-temporarily store the data 80 received from the input unit 99. According to one embodiment, the processor 100 stores at least one data 80 received from the input unit 99 in predetermined storage units 110 to 140, for example, in the first storage unit 110, according to predefined settings. ) can also be controlled to save it. In addition, the processor 100 merges all or part of the data written to at least one of the storage units 110 to 140, for example, the first storage unit 110, and generates new data (82-1 in FIG. 5, etc., hereinafter merged). data) may be generated, for example, by merging at least one cold data (81-1 to 81-3 in FIG. 3, etc.) among the data in the first storage unit 110 to create merged data 82-1. You can also create In addition, the processor 100 may migrate the generated merged data 82-1 to another storage unit, for example, the second storage unit 120. According to one embodiment, the processor 100 may include a write processor 101, a merge processor 103, and a migration processor 105 as shown in FIG. 1 to perform the above-described operations. The write processing unit 101, the merge processing unit 103, and the migration processing unit 105 may be physically and/or logically separated according to design, and if they are logically separated, at least one of them may be implemented in software. It may be possible. Detailed operations of the write processing unit 101, merge processing unit 103, and migration processing unit 105 will be described later.

The processor 100 executes a program (which may be referred to as an app, application, or software) stored in the first to Nth storage units 110 to 140 or another storage unit (not shown) provided separately, and performs the above-described writing operation. At least one of processing, merge processing, and migration processing may be performed sequentially or independently. Here, the program may be written directly by the designer and stored, modified or updated in the storage unit (eg, 110 to 140), and/or acquired or updated through an electronic software distribution network accessible through a wired or wireless communication network. It may be. According to one embodiment, the processor 100 includes, for example, a Central Processing Unit (CPU), a Micro Controller Unit (MCU), a Micro Processor (Micom), and an Application Processor (AP). , Application Processor), an electronic control unit (ECU), and/or other electronic devices capable of processing various operations and generating control signals. These devices may be implemented using, for example, one or two or more semiconductor chips.

According to one embodiment, the first to Nth storage units 110 to 140 may be physically and/or virtually separated from the other storage units 110 to 140. In this case, at least one of the first to Nth storage units 110 to 140, for example, the first storage unit 110, is a storage device with relatively high processing or transfer speed (for example, DRAM). By adopting SRAM, M.2 solid state, drive, Single Level Cell (SLC) solid state drive, Multi Level Cell (MLC: Multi Level Cell) solid state drive, etc. It may have been implemented. In addition, at least one other storage unit among the first to N-th storage units 110 to 140, for example, the second storage unit 120, is a storage device (for example, a hard disk storage device) with a relatively slow processing or transfer speed. Tripe Level Cell (TLC) Solid State Drive (SSD) or Quad Level Cell (QLC) Solid State Drive, Compact Disc (CD), DVD, and/or Laser Disc etc.) may have been implemented by adopting. In other words, some of the first to Nth storage units (110 to 140) (eg, 110) have relatively superior performance and life expectancy compared to other storage units (eg, 120), but require high costs. Some storage units (such as 120) can achieve high capacity at lower cost than other storage units (such as 110), but their performance and life expectancy may be relatively weak. Meanwhile, each of the first to Nth storage units 110 to 140 may be provided with corresponding first to Nth input/output interfaces 110A to 140A for input and output of at least one data.

FIG. 2 is a diagram for explaining an example of first data and second data stored in each of the first storage unit and the second storage unit.

The first to Nth storage units 110 to 140 are provided to temporarily or non-temporarily store the received data 80. For example, the data storage and processing device 10 may include a first storage unit 110 and a second storage unit 120 as shown in FIG. 2, and the first storage unit 110 includes at least one stores data (71-1 to 71-13, hereinafter referred to as first existing data), and the second storage unit 120 stores at least one data (72-1 to 72-16, hereinafter referred to as second existing data). You can. At least one of the first existing data 71-1 to 71-13 may be the same as or different from at least one of the second existing data 72-1 to 72-16.

As described above, when the first to Nth storage units 110 to 140 have different purposes, performance, capacity and/or price, each of the first to Nth storage units 110 to 140 has a purpose, Depending on performance, capacity, and/or price, existing data (71-1 to 71-13, 72-1 to 72-16) with predetermined classification criteria may be stored. For example, the first storage unit 110, which has a relatively fast processing or transmission speed, stores data with a high frequency of access or update or with high importance (for example, may be referred to as warm data or hot data) as first existing data ( 71-1 to 71-13), and conversely, the second storage unit 120, which has a relatively slow processing or transmission speed, stores data that is accessed or updated with a low frequency or is of low importance (can be referred to as cold data, for example). may be stored as second existing data (71-1 to 71-13). In addition to the first and second storage units 110 and 120, corresponding data or files are also written to other storage units 130 to 140 depending on the purpose, performance, capacity, and/or price of each storage unit 130. There may be.

Hereinafter, specific operations of the processor 100 will be described with reference to FIGS. 3 to 6. Hereinafter, for convenience of description of the operation of the processor 100, the data storage processing device 10 includes two storage units (i.e., a second storage unit 120 and a first storage unit 110) and an input The data 80 will be described based on an embodiment that can be classified into cold data 81-1 to 81-3 and second data 82-1. However, the details described later will be related to three or more storage units 110 to 140. ) and/or even if the input data 80 can be classified differently in addition to cold data (81-1 to 81-3) or hot data (81-4), it is used in the same or slightly modified form. It may also apply.

Figure 3 is a diagram for explaining an example of a process in which data is written to the first storage unit.

1 and 3, in one embodiment, the write processing unit 101 of the processor 100 writes at least one data 80 (81-1 to 81) input through the input unit 99. -4, hereinafter referred to as input data. Depending on the embodiment, it may include a key) is recorded in at least one storage unit 110 to 140, for example, the first storage unit 110. ) can be configured to store at least one input data (81-1 to 81-4). In this case, at least one input data (81-1 to 81-4) is written to any one of the first existing data (71-1 to 71-12) stored in the first storage unit 110, or the input data written to other newly created data (not shown) for (81-1 to 81-4), or distributed to two or more of the plurality of first existing data (71-1 to 71-12) or newly created data It may be written.

According to one embodiment, the write processing unit 101 may store data 81-1 to 81-4 in the first storage unit 110 based on a copy on write B+ tree. Copy on Write (CoW): When new data corresponding to previously stored data is input, the new data is not stored in the location (address) of the previously stored data, but rather the new data is copied to another location. Includes a process for making corrections. When at least one node in the tree needs to be updated according to external data input, the copy and write B+ tree replaces the existing node with a new node and places the new node in a new location (for example, at the end of the data file). After adding a node and creating a new corresponding parent node as needed, the data in the tree is restored by removing the existing node (i.e., all or part of the existing tree's nodes) corresponding to the additionally created node. This is a way to update. Such a copy and write B+ tree can delay writing data when used together with the Log Structured Merge Tree (LSM Tree) used in NewSQL, etc.

In detail, as shown in FIG. 3, when new data, for example, first input data 81-1, is input, the write processing unit 101 generates at least one of the first existing data 71-1 to 71-3, For example, it can be recorded in addition to the last first existing data (71-13). Depending on the embodiment, the first input data 81-1 may be added and recorded at a location corresponding to the last first existing data 71-13, for example, at the end of the first existing data 71-13. Also, similarly, if other data, for example, second input data 81-2, is further input, the second input data 81-2 may be added to the end of the first existing data 71-13. 1 It may also be written in addition to input data 81-1. Other data 81-3 and 81-4 may be added to the first existing data 71-13 in the same manner. In other words, the write processing unit 101 repeats the above-described process every time new data 80 is input, thereby storing at least one of the input data 81-1 to 81-4 in the first storage unit 110. Additions can be continued, and thus the first storage unit 110 can update and store one or more input data (81-1 to 81-4). Here, the first to fourth input data (81-1 to 81-4) may be input sequentially or simultaneously, and the write processor 101 inputs the first to fourth input data (81-1 to 81-4). ) may be added to the last first existing data 71-13 sequentially according to a predetermined order (for example, input order) or in a random order.

FIG. 4 is a first diagram for explaining an example of a process for merging cold data, and FIG. 5 is a second diagram for explaining an example of a process for merging cold data.

The merge processing unit 103 of the processor 100 merges all or part of the at least one input data 81-1 to 81-4 stored in the first storage unit 110 into at least one merge data 82-1. ) can also be created. Specifically, as shown in FIGS. 3 and 4, the first to fourth input data 81-1 to 81-4 are being written or after being written to the first storage unit 110 simultaneously and/or sequentially, and are merged. The processing unit 103 may merge two or more of the first to fourth input data 81-1 to 81-4 as shown in FIG. 5.

According to one embodiment, the merge processing unit 103 may classify at least one input data 81-1 to 81-4 written in the first storage unit 110. For example, the merge processing unit 103 analyzes at least one first to fourth input data 81-1 to 81-4 written in the first storage unit 110 according to predefined settings, and provides the analysis result. Based on , at least one data 80 can be classified as cold data or hot data. Here, cold data can contain sufficient data even if it is stored in a storage device with a relatively slow processing or transmission speed due to its relatively low frequency of access or update or low importance, while hot data, on the contrary, has a relatively low frequency of access or update. It may include data that is frequent or important and will be stored in storage devices with relatively fast processing or transfer speeds. According to one embodiment, the merge processor 103 stores the first to fourth input data 81-1 to 81-4 generated from files managed by the processor 100, for example, a database engine in the form of a key value. If it corresponds to data in the oldest layer and/or data in the lowest layer of the file hierarchy managed by a storage engine that uses a log structure merge tree, it is classified as cold data. Otherwise, it is classified as cold data. In this case, it may not be classified, or it may be classified as hot data or warm data. Accordingly, some of the first to fourth input data (81-1 to 81-4), for example, the first to third input data (81-1 to 81-3), are classified as cold data, Some other input data, for example, fourth input data 81-4, can be classified as hot data. When the input data (81-1 to 81-4) is classified in this way, the merge processing unit 103 selects cold data (81-1 to 81-3), for example, from the input data (81-1 to 81-4). At least one merged data 82-1 may be generated by merging the first to third input data 81-1 to 81-3.

According to one embodiment, the merge processing unit 103 may be arranged to perform merging in response to satisfaction of the merging condition when the merging condition is satisfied. For example, the merge processing unit 103 may determine the number of cold data 81-1 to 81-3 written to the first storage unit 110 and/or at least one of the cold data 81-1 to 81-3. It may be arranged to perform merging based on the recorded time. More specifically, the merge processing unit 103 counts the cold data 81-1 to 81-3, and when the number of cold data 81-1 to 81-3 exceeds a predefined threshold, the cold data Merging of at least two of (81-1 to 81-3) may be initiated, or the time at which one or more specific cold data (81-1 to 81-3) was written may be confirmed and predefined from the written time. When the fixed or variable time has elapsed, merging of at least two of the cold data 81-1 to 81-3 may be initiated. In addition, the merge processing unit 103 may have insufficient access to the cold data (81-1 to 81-3) and/or two or more of the cold data (81-1 to 81-3) are in one completed form. If merging into a file, etc. (eg, a sorted string table) is possible, merging of the cold data 81-1 to 81-3 may be initiated. Depending on the embodiment, the merge processing unit 103 may also perform the creation of the merged data 82-1 when the migration processing unit 105 determines that the situation is appropriate for migrating the merged data 82-1. The merged data 82-1 generated in this way can be treated as a state that can be migrated by the migration processing unit 105 of the processor 100.

FIG. 6 is a diagram illustrating an example of a process for migrating merged data to the second storage unit.

When the merging of two or more cold data (81-1 to 81-3) is completed, the migration processing unit 105 generates the merging of the cold data (81-1 to 81-3) as shown in FIG. At least one merged data 82-1 may be migrated to another storage unit, for example, the second storage unit 120. For example, the migration processing unit 105 may move the cold data 81-1 to 81-3 to a storage device with a relatively slow processing or transmission speed.

Depending on the embodiment, the migration processing unit 105 may store merged data 82 according to whether conditions predefined by a designer or a user are satisfied, according to an operation by a user, etc., and/or according to a random selection of the migration processing unit 105. It is possible to determine whether to migrate -1), and perform migration processing of the merged data (82-1) according to the decision. According to one embodiment, the migration processing unit 105 includes a storage unit in which the merge data 82-1 is stored (i.e., the first storage unit 110) and a storage unit in which the merge data 82-1 is scheduled to be stored (i.e. , the state of at least one input/output interface 110A, 120A of the second storage unit 120 may be determined, and whether to migrate the merged data 82-1 may be determined based on the determined state. For example, the migration processing unit 105 checks the bandwidth usage of each of the input/output interfaces 110A and 120A for each storage unit 110 and 120, and considers the confirmed bandwidth usage of the merged data 82-1. It is possible to determine whether migration is appropriate or possible. Specifically, it is determined that migration of merged data (82-1) is appropriate or possible only when write performance is higher than the required level based on bandwidth usage. You may. In more detail, for example, if the second storage unit 120 is a storage device with variable write performance, such as a quad-level cell solid state drive, the migration processing unit 105 may It may be decided to migrate the merged data 82-1 to the second storage unit 120 only if the write performance of the storage unit 120 is sufficiently satisfactory. According to another embodiment, the migration processing unit 105 checks the remaining storage capacity of the first storage unit 110 where the merged data 82-1 is stored and determines that the remaining storage capacity of the first storage unit 110 is the merged data ( 82-1) is determined to be insufficient to maintain storage, and/or the remaining storage capacity of the second storage unit 120 to which the merged data 82-1 is to be transferred is checked and the remaining storage capacity of the second storage unit 120 is checked. If it is determined that the remaining storage capacity is sufficiently larger than the merged data 82-1, arrangements may be made to determine migration of the merged data 82-1. According to another embodiment, the migration processing unit 105 checks the load level of each of the at least one input/output interface (110A, 120A), and if the load level of each falls within a certain range, merge data 82-1 ), it is also possible to make arrangements to decide on the migration of. If migration is determined to be appropriate or possible, the migration processing unit 105 initiates migration of the merged data 82-1. Accordingly, the merged data 82-1 can be migrated to a predetermined storage unit, for example, the second storage unit 120.

According to the above-described method, specific data, for example, cold data (81-1 to 81-3) with a low access frequency, is stored in a storage unit scheduled to store the cold data (81-1 to 81-3), for example, 2 Writing to the storage unit 120 can be delayed. That is, it is possible to delay writing the cold data 81-1 to 81-3. In the conventional method of using a universal unique identifier instead of a monotonically increasing or monotonically decreasing primary key, the input data is distributed evenly over the entire file, so that data of a specific standard (for example, cold access or update frequency or low importance) is distributed evenly over the entire file. Classifying data based on time was difficult to process. However, as described above, delaying recording of input data to be stored (i.e., cold data 81-1 to 81-3) to the original storage location (for example, the second storage unit 120) is substantially This is equivalent to sorting data in chronological order. Therefore, even when using a universal unique identifier, it is possible to classify data according to a predetermined classification standard (for example, cold data or hot data), and also through the delay process of merging and writing, data of a specific standard (for example, access or hot or warm data with frequent updates or high importance) can be filtered, and each classified data, for example, hot data (81-4) or cold data (81-1 to 81-3) can be written to the corresponding storage locations 110 and 120, respectively.

The above-described process may be employed in a database for which maintenance, management, and/or update are performed based on SQL, a system for managing the database, or a language for the same. For example, New SQL sometimes uses a universal unique identifier instead of a numeric monotonically increasing/monotonically decreasing primary key. This is because the range of the main key is continuous, so requests cannot be distributed and are concentrated on specific nodes, which may reduce the efficiency of data management and processing. In this case, classifying the created files as cold data based on creation time can be quite difficult. According to the above-described device and process, even in cases where classification is performed without using the creation time such as a universal unique identifier in New SQL, etc., data or files with relatively low access and call frequency or low importance are defined in advance. It becomes possible to properly classify and record files in a given storage device, and thus the efficiency of file management can be reconsidered.

The above-described data storage and processing device 10 may be implemented using an information processing device including a processing device and a data storage device. Here, the information processing device includes, for example, a desktop computer, a laptop computer, a smart phone, a tablet PC, a smart watch, a head mounted display (HMD) device, a navigation device, a portable game console, and a personal digital assistant (PDA). Personal Digital Assistant), digital televisions, set-top boxes, digital media player devices, artificial intelligence sound playback devices (artificial intelligence speakers), home appliances (refrigerators, washing machines, etc.), manned vehicles (passenger cars, buses, two-wheeled vehicles, etc.) , unmanned vehicles (robot vacuum cleaners, etc.), manned aircraft, unmanned aircraft (drones, etc.), household or industrial robots, industrial machines, electronic whiteboards, electronic billboards, or automated teller machines (ATMs), etc. It is not limited. Depending on the situation or conditions, designers, users, etc. may consider and apply at least one of various devices for computational processing and control of information in addition to the above-mentioned devices as the above-described data storage and processing device 10.

Hereinafter, an embodiment of a data storage and processing method will be described with reference to FIG. 7.

7 is a flowchart of one embodiment of a data storage processing method.

As shown in FIG. 7, data (i.e., input data) can be input by the user's operation of a keyboard device or the like or from another external device (for example, another information processing device or storage device) (400). In this case, the device into which data is input (i.e., data storage and processing device) may include two or more storage units, for example, first and second storage units, and the first and second storage units have processing or data transfer speeds with each other. may be different. Here, the first storage unit is implemented using a storage device (for example, a storage device for hot data storage such as a solid state drive storage device) with a relatively faster processing or transfer speed compared to the second storage unit, and the second storage unit is The unit may be implemented using a storage device (for example, a storage device for cold data storage such as a hard disk storage device) that has a relatively slow processing or transmission speed compared to the first storage unit. Figure 7 shows an embodiment of a method of performing data storage processing with two storage units, that is, first and second storage units, provided. However, the data storage processing method described later can be performed in three ways depending on the selection of the designer or user. Even when the above storage units (i.e., third to Nth storage units) are employed, they may be performed in the same or slightly modified form.

The input data may be written and stored in a predefined storage unit, for example, a first storage unit, as defined by the designer or user (410). Here, the data to be newly written may be inputted by being added to one of the existing data in the first storage unit, written by being distributed and added to a plurality of existing data, or written by being added to newly created data that is different from the existing data. According to one embodiment, writing of data may be performed based on a copy and write B+ tree. For example, data to be newly written may be added and recorded sequentially or simultaneously to at least one end of existing data. This process may be repeated, and accordingly, one or more input data will be added and written to at least one existing data or newly created data.

It may be determined whether the merge condition for at least one input data written to the first storage unit is satisfied (420). Here, the merging conditions include, for example, the number of data, the time elapsed since the data was written, whether access to the data is sufficiently insufficient, whether two or more of the written input data are merged into one complete form, and/ Alternatively, it may include whether the merged data can be migrated. If merging is possible (Yes at 420), merging of all or part of the input data written to the first storage can be performed (430). Here, the data on which merging is performed may include cold data, and hot data or warm data may be excluded from merging. For merging of cold data, input data may be classified in advance as cold data or hot data. For example, the input data corresponds to data from the oldest created hierarchy among the files managed by the database engine in the form of key values, or from the lowest hierarchy of files managed by the storage engine using a log structure merge tree. If it corresponds to data, it may be classified as cold data, and if not, it may be classified as hot data. Conversely, if the merging condition is not satisfied (No in 420), merging is not performed, and the data is transferred to the data storage processing device as needed through data input/output terminals, communication networks, etc. according to operations by the user, etc., or according to predefined settings. It can be continuously entered through.

When merged data is generated, it may be determined whether predefined migration conditions are satisfied (440). Migration conditions include the status of the input/output interface corresponding to each storage unit (e.g., bandwidth usage or load level of the input/output interface for at least one of the first storage unit and the second storage unit, etc.) and/or the storage unit to which the merged data is to be migrated. It may include status (for example, remaining capacity of the first storage unit, etc.). If the migration condition is satisfied (for example, in 440, the bandwidth usage of the input/output interface is sufficient to move the first data, etc.), the merged data is transferred to the selected storage unit (for example, the second storage unit). Can be migrated (450). Conversely, if the migration condition of the merged data is not satisfied (No in 440), the merged data is not migrated and remains written in the first storage (460). If the migration conditions are later satisfied (example of 440), the merged data may be migrated to the first storage unit independently of the other processes (410 to 430) or dependently on the other processes (410 to 430) (440, 460). ).

The above-described process may be repeated one or more times (Yes at 470) or terminated (No at 470) according to predefined settings or user manipulation.

The data storage and processing method according to the above-described embodiment may be implemented in the form of a program that can be driven by a computer device. A program may include code, libraries, data files and/or data structures, etc., singly or in combination. Additionally, the computer device may be implemented by including a processor or memory that enables program functions, and may further include a communication device as needed.

A program for implementing the above-described data storage and processing method may be recorded on a computer-readable recording medium. Recording media readable by a computer include, for example, semiconductor storage devices such as ROM, RAM, flash memory or solid state drives, magnetic disk storage devices such as hard disks or floppy disks, and optical recording media such as compact disks or DVDs. and/or may include at least one type of physical device, such as a magnetic tape, capable of storing a specific program executed in response to a call from a computer or the like.

Although an embodiment of the data storage and processing device and the data storage and processing method has been described above, the data storage and processing device and the data storage and processing method are not limited to only the above-described embodiment. Various devices or methods that can be implemented by those skilled in the art by modifying and modifying the above-described embodiments based on the above-described embodiments may also be examples of the above-described data storage and processing device or data storage and processing method. For example, the described system, structure, device, circuit, etc. may be combined or combined in some form different from that described or replaced or substituted by other components or equivalents, and/or the described method(s) may be used as described. Even if it is performed in a different order than described, it may correspond to an embodiment of the above-described data storage and processing device or data storage and processing method.

80: data 99: input unit
100: Processor 101: Write processing unit
103: Merge processing department 105: Migration processing department
110: first storage unit 120: second storage unit

Claims (12)

first storage unit;
a second storage unit that is different from the first storage unit; and
A processor that adds input data to the first storage unit based on a copy and write B+ tree, and migrates at least one cold data from the data added to the first storage unit to the second storage unit,
The processor,
Merging the at least one cold data according to predefined settings to obtain at least one merged data, and transmitting the at least one merged data to the second storage unit,
Data storage for determining whether to migrate the at least one merged data to the second storage unit based on at least one of the storage capacity, load level, and input/output bandwidth usage of at least one of the first storage unit and the second storage unit. processing unit. According to paragraph 1,
The processor is configured to transfer the at least one merged data to the second storage unit instead of the at least one cold data. According to paragraph 2,
The processor is configured to exclude at least one hot data from among the data added to the first storage unit and merge the at least one cold data. delete delete According to paragraph 1,
A data storage and processing device wherein the first storage unit includes a storage device with relatively high performance, and the second storage unit includes a storage device with relatively low performance. Entering a plurality of data simultaneously or sequentially;
Adding input data to a first storage unit based on a copy and write B+ tree;
A step of migrating at least one cold data from among the data added to the first storage unit to a second storage unit of a different type from the first storage unit,
The step of migrating at least one cold data from the data added to the first storage unit to the second storage unit,
Obtaining at least one merged data by merging the at least one cold data according to predefined settings; and
Determining whether to migrate the merged data to the second storage unit based on at least one of the storage capacity, load level, and input/output bandwidth usage of at least one of the first storage unit and the second storage unit; comprising; How data is stored and processed. In clause 7,
The step of migrating at least one cold data from the data added to the first storage unit to the second storage unit,
A data storage processing method further comprising: transmitting the at least one merged data to the second storage unit. According to clause 8,
Obtaining at least one merged data by merging the at least one cold data according to the predefined settings,
Excluding at least one hot data from the data added to the first storage unit and merging the at least one cold data according to predefined settings. delete delete In clause 7,
A data storage and processing method wherein the first storage unit includes a storage device with relatively high performance, and the second storage unit includes a storage device with relatively low performance.