KR20160139694A - Ship Design Data Management System Using Hadoop and HBase - Google Patents

Abstract

The method of managing input ship design data according to an embodiment of the present invention includes storing original data of the ship design data in HDFS and storing only metadata of the original data using HBase .

Description

Ship Design Data Management System Using Hadoop and HBase}

The present invention relates to a system using a distributed network and a database for managing ship design data, and more particularly to a system for storing, processing, and managing a large amount of ship design data generated by a design process for shipbuilding using Hadoop and HBase ≪ / RTI >

Ship design generally uses a custom design approach. Generate information on linear, basic, design, and structure to meet the requirements of the customer and receive final approval. These design processes include linear design, basic design, design design, and structural design, and a large number of drawings are generated at each step.

In order to manage such a large amount of design data, a system with a high-performance computer or a distributed network is required. However, there is a problem that the system is expensive to construct and the system expansion is limited.

The present invention is based on the technical background as described above. The present invention provides a method and apparatus for storing data safely using the Hadoop Distributed File System (HDFS) of Apache open source project Hadoop and using MapReduce And to provide a system and method for efficiently managing design data using HBase, a column-oriented database.

The objects of the present invention are not limited to the above-mentioned objects, and other objects not mentioned can be clearly understood by those skilled in the art from the following description.

According to another aspect of the present invention, there is provided a method of managing input ship design data, the method comprising: storing original data of the ship design data in an HDFS; storing only the metadata of the original data using HBase The method comprising the steps of:

According to the present invention, it is possible to achieve a high performance by constructing a distributed network using low-performance computers using the Hadoop platform, and thus it is possible to develop a system that is easy to expand and cost.

1 is a flow diagram illustrating a storage process in accordance with one embodiment of the present invention.
2 is a storage process selection pseudo code according to an embodiment of the present invention.
3 is a flowchart illustrating a search process according to an embodiment of the present invention;
4 is a view illustrating a ship design data retrieval process according to an embodiment of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS The advantages and features of the present invention and the manner of achieving them will become apparent with reference to the embodiments described in detail below with reference to the accompanying drawings. The present invention may, however, be embodied in many different forms and should not be construed as being limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. Is provided to fully convey the scope of the invention to those skilled in the art, and the invention is only defined by the scope of the claims. It is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. In the present specification, the singular form includes plural forms unless otherwise specified in the specification. As used herein, the terms " comprises, " and / or "comprising" refer to the presence or absence of one or more other components, steps, operations, and / Or additions.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

Figure 1 shows the storage process of the present invention.

Upon receiving the data, the preprocessing step S100 is performed.

In the first extraction step S102, metadata of original data is extracted. The metadata includes data such as a data map, a size, and an administrator.

The extracted data is transformed into a proper form to be stored in the HBase through the transformation step (S104) or cleaved.

In the selection step S106, the size of the data is determined, and the large-scale data is subjected to batch processing using MapReduce (S120). A small amount of data is input to the HBase by the point query method (S110) ).

When the user inputs data, the ship design data storage module can increase the input speed by extracting the size of data and performing different routines according to sizes.

FIG. 2 shows a pseudo code for selecting a routine for each data size of the storage process.

In HBase, since the image data such as the design drawing can not be directly stored in the design data, the original data of the design can be stored in the HDFS (S140) and only the metadata can be stored in the HBase.

3 shows a data retrieval process.

Apache's other project, ZooKeeper, is used to retrieve the data. In the selection step S310, a search routine is selected and a search is requested to the HMase of the HBase (S320). HMaster requests the location of the data to the main keeper and searches and returns the data required by the user based on the retrieved location information (S330).

Ship design data is managed through a column-oriented database HBase. Table 1 shows the structure of tables using HBase.

The rowkey is used to sort data by index, and the column family is used to group and search data. And column means attribute, and cell means data corresponding to attribute.

 Information on tables created by the user in HBase is .META. Manage from the table .META. Tables are maintained in the -ROOT- table. When the user retrieves the data, HBase's HMaster requests the location information of the data from the main keeper. The primary keeper is the -ROOT- table and .META. Through the table, the location of the data is retrieved from the UserTable and the user returns the desired data. Figure 4 shows the order of the tables that go through this data retrieval.

HBase does not directly modify the actual cell data in the modification of the data. HBase creates a new row with the same row key for the modified contents of the data, and applies the timestamp to manage the row data by version. By managing row data by version, you can manage the history of data modification and correct recovery of incorrectly modified data.

While the present invention has been described in detail with reference to the accompanying drawings, it is to be understood that the invention is not limited to the above-described embodiments. Those skilled in the art will appreciate that various modifications, Of course, this is possible. Accordingly, the scope of protection of the present invention should not be limited to the above-described embodiments, but should be determined by the description of the following claims.

Claims (2)

A method for managing inputted ship design data,
Storing original data of the ship design data in an HDFS,
Storing only the metadata of the original data using HBase
Wherein said method comprises the steps of: 2. The method of claim 1, wherein the storing
Extracting a data size of the ship design data,
If the data size is greater than or equal to the predetermined capacity, storing the data through MapReduce; and if the data size is less than the predetermined capacity, storing the data through the PointQuery
How to Manage Ship Design Data.

Images