WO2018218672A1 - Distributed storage system for stationery big data - Google Patents
Distributed storage system for stationery big data Download PDFInfo
- Publication number
- WO2018218672A1 WO2018218672A1 PCT/CN2017/087026 CN2017087026W WO2018218672A1 WO 2018218672 A1 WO2018218672 A1 WO 2018218672A1 CN 2017087026 W CN2017087026 W CN 2017087026W WO 2018218672 A1 WO2018218672 A1 WO 2018218672A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- stationery
- data
- storage
- module
- stationery data
- Prior art date
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
Definitions
- the invention belongs to the field of big data storage technology, and in particular relates to a stationery big data distributed storage system.
- the embodiment of the invention provides a stationery big data distributed storage system, which can realize distributed processing of stationery big data, and the data transmission is stable.
- a stationery big data distributed storage system includes:
- the stationery data distribution storage module is a module that provides a storage service as a whole by a group of storage nodes connected through a network;
- the mapping configuration module implements hierarchical partitioning of stationery data and mapping configuration with the storage node
- the stationery data distribution management module realizes distribution management of stationery data storage, and the distribution management of the stationery data includes: a stationery data storage module and a stationery data deletion module.
- the stationery data distribution storage module is configured to implement the following steps:
- Hierarchical division of stationery data hierarchically divide the correspondence data in the stationery data according to the region->sub-region->factory station or line;
- the relationship between the region -> sub-region refers to a region like "province - city - county” -> Inclusion relationship between sub-regions.
- the plant station and the line include a stationery factory, a stationery purchase station, and a line; wherein the stationery factory and the stationery purchase station are generally collectively referred to as a factory station.
- the relationship between the area and the plant station and the line refers to the inclusion relationship between the area and the stationery factory, between the area and the stationery purchase station, and between the area and the line.
- said by region -> sub-region -> The inclusion relationship between the station or the line divides the stationery data network model from the network structure into a tree structure hierarchy, so that the stationery data in the stationery data is attributed to the corresponding tree structure branch according to the association relationship, the tree shape
- the area in the structure is the root node, each area -> Sub-area ->
- the plant station or line constitutes a specific branch; thus, other stationery data can be attributed to the corresponding tree structure branch according to its association with the region, plant station or line and each other.
- the stationery data storage module implements adding and updating stationery data, and the adding and updating includes the following steps:
- the stationery data distribution management module When receiving a stationery data storage request, the stationery data distribution management module first analyzes the stationery data to determine the branch of the stationery data tree structure;
- mapping configuration module Taking the area or plant station line information corresponding to the branch of the stationery data as a parameter, querying the mapping configuration module to determine the corresponding storage node; if the data is found to be invalid, an error is returned;
- the storage node After receiving the stationery data storage request, the storage node performs data update or data creation operation on the storage medium according to whether the stationery data exists.
- the stationery data deletion module implements deletion of the stationery data, and the deleting comprises the following steps:
- the stationery data distribution management module When receiving a stationery data deletion request, the stationery data distribution management module first analyzes the stationery data to determine the branch of the stationery data tree structure;
- mapping configuration module Taking the area or plant station line information corresponding to the branch of the stationery data as a parameter, querying the mapping configuration module to determine the corresponding storage node; if the data is found to be invalid, an error is returned;
- the storage node After receiving the stationery data deletion request, the storage node performs a data deletion operation on the storage medium.
- the stationery big data distributed storage system uses the distributed characteristics associated with the stationery area and the stationery area to distribute the large-scale stationery data according to the basis of the distributed stationery.
- the data is distributed reasonably and evenly to multiple storage nodes, and provides an easy-to-use and efficient infrastructure for subsequent stationery data access, realizing distributed processing of stationery big data, and stable data transmission.
- FIG. 1 is a schematic structural diagram of a stationery big data distributed storage system according to an embodiment of the present invention.
- the stationery big data distributed storage system uses the distributed characteristics associated with the stationery area and the stationery area to distribute the large-scale stationery data according to the basis of the distributed stationery.
- the data is distributed reasonably and evenly to multiple storage nodes, and provides an easy-to-use and efficient infrastructure for subsequent stationery data access, realizing distributed processing of stationery big data, and stable data transmission.
- a stationery big data distributed storage system 100 includes a stationery data distribution storage module 110; a hierarchical division of stationery data and a mapping configuration module 120 with a storage node;
- the stationery data distribution storage module 110 is a module that provides a storage service as a whole by a group of storage nodes through a network interconnection;
- the mapping configuration module 120 implements hierarchical division of stationery data and a mapping configuration between the storage node and the storage node;
- the stationery data distribution management module 130 implements distribution management of the stationery data storage, the distribution management 130 of the stationery data includes: a stationery data storage module 131 and a stationery data deletion module 132;
- the distributed characteristics of the data and the stationery area are distributed, and based on the distributed storage of large-scale stationery data, the processed stationery data is distributed to multiple storage nodes reasonably and evenly, and is followed by Stationery data access provides an easy-to-use, efficient infrastructure that enables great data on
- the stationery data distribution storage module 130 is configured to implement the following steps:
- Hierarchical division of stationery data hierarchically divide the correspondence data in the stationery data according to the region->sub-region->factory station or line;
- the area -> The relationship between sub-regions refers to a region like "province - city - county" -> The inclusion relationship between the sub-areas; the station and the line, including the stationery factory, the stationery purchase station, and the line; wherein the stationery factory and the stationery purchase station are generally referred to as the factory station; the area and the factory station and the line The relationship between the area and the stationery factory, the area and the stationery acquisition station, and the inclusion relationship between the area and the line; -> Sub-area -> The inclusion relationship between the station or the line divides the stationery data network model from the network structure into a tree structure hierarchy, so that the stationery data in the stationery data is attributed to the corresponding tree structure branch according to the association relationship, the tree shape
- the area in the structure is the root node, each area -> Sub-area -> The plant station or line constitutes a specific branch; thus, other stationery data can be attributed to the corresponding tree structure branch according to its association with the region, plant station or line
- the stationery data storage module 131 implements adding and updating stationery data, and the adding and updating includes the following steps:
- the stationery data distribution management module When receiving a stationery data storage request, the stationery data distribution management module first analyzes the stationery data to determine the branch of the stationery data tree structure;
- mapping configuration module Taking the area or plant station line information corresponding to the branch of the stationery data as a parameter, querying the mapping configuration module to determine the corresponding storage node; if the data is found to be invalid, an error is returned;
- the storage node After receiving the stationery data storage request, the storage node performs data update or data creation operation on the storage medium according to whether the stationery data exists.
- the stationery data deletion module 132 implements deletion of the stationery data, and the deleting includes the following steps:
- the stationery data distribution management module When receiving a stationery data deletion request, the stationery data distribution management module first analyzes the stationery data to determine the branch of the stationery data tree structure;
- mapping configuration module Taking the area or plant station line information corresponding to the branch of the stationery data as a parameter, querying the mapping configuration module to determine the corresponding storage node; if the data is found to be invalid, an error is returned;
- the storage node After receiving the stationery data deletion request, the storage node performs a data deletion operation on the storage medium.
- the stationery big data distributed storage system uses the distributed characteristics associated with the stationery area and the stationery area to distribute the large-scale stationery data according to the basis of the distributed stationery.
- the data is distributed reasonably and evenly to multiple storage nodes, and provides an easy-to-use and efficient infrastructure for subsequent stationery data access, realizing distributed processing of stationery big data, and stable data transmission.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
A distributed storage system (100) for stationery big data, comprising: a stationery data distributed storage module (110); a mapping configuration module (120) for layered division of stationery data and mapping configuration between the stationery data and storage nodes; and a stationery data distribution management module (130). The stationery data distributed storage module (110) consists of a group of storage nodes connected to one another by means of the Internet and provides storage service for the outside as a whole; the mapping configuration module (120) is used for achieving layered division of the stationery data and mapping configuration between the stationery data and the storage nodes; and the stationery data distribution management module (130) is used for achieving stationery data storage distribution management. The stationery data distribution management module comprises a stationery data storage module (131) and a stationery data deletion module (132). The processed stationery data is distributed to multiple storage nodes reasonably and uniformly; easy-to-use and efficient basic facilities are provided for subsequent stationery data access; distributed processing of stationery big data is achieved; and data transmission is stable.
Description
本发明属于大数据存储技术领域，尤其涉及文具大数据分布式存储系统。The invention belongs to the field of big data storage technology, and in particular relates to a stationery big data distributed storage system.
随着信息科技的不断发展，人们的生活已经离不开数据信息，例如，在工作中需要通过数据信息来记录工作的进程等；在生活中如网络购物等，也都需要通过数据信息来进行操作、记录。这就使得数据信息的高效管理成为了一项发展的热点。目前，由于越来越多的人们、企业运用数据信息，使得数据信息的数量越来越多，普通的数据信息处理装置已经渐渐地不能够满足使用需求，然而，大数据信息处理装置的成本较高，如果全部更换大数据处理装置的话，会使得个人或企业需要花费较多的成本进行设备的更换。With the continuous development of information technology, people's lives are inseparable from data information. For example, in the work, data information is needed to record the progress of work; in life, such as online shopping, it is also necessary to use data information. Operation, recording. This makes the efficient management of data information a hot spot for development. At present, as more and more people and enterprises use data information, the number of data information is increasing, and ordinary data information processing devices have gradually failed to meet the demand for use. However, the cost of big data information processing devices is relatively high. High, if all the big data processing devices are replaced, it will cause the individual or the enterprise to spend more cost to replace the equipment.
另一方面，随着计算机产业发展，人们对电脑的性能要求越来越高，传统的机械硬盘因为其固有的机械特性，造成速度无法进一步提高，其性能已经是整个电脑的瓶颈，所以固态硬盘以优越的读写和IOPS性能，慢慢的进入了人们的生活。
On the other hand, with the development of the computer industry, people's performance requirements for computers are getting higher and higher. The traditional mechanical hard disk cannot be further improved because of its inherent mechanical characteristics. Its performance is already the bottleneck of the whole computer, so the solid state hard disk With superior read and write and IOPS performance, it has slowly entered people's lives.
本发明实施例提供一种文具大数据分布式存储系统，能够实现对文具大数据的分布式处理，并且数据传输稳定。The embodiment of the invention provides a stationery big data distributed storage system, which can realize distributed processing of stationery big data, and the data transmission is stable.
本发明实施例是这样实现的，一种文具大数据分布式存储系统，包括：The embodiment of the present invention is implemented in this manner, and a stationery big data distributed storage system includes:
文具数据分布存储模块；Stationery data distribution storage module;
文具数据的分层划分及与存储节点间的映射配置模块；Hierarchical division of stationery data and mapping configuration module with storage nodes;
文具数据分布管理模块；Stationery data distribution management module;
所述文具数据分布存储模块，由一组存储节点，通过网络互联，对外作为一个整体提供存储服务的模块；The stationery data distribution storage module is a module that provides a storage service as a whole by a group of storage nodes connected through a network;
所述映射配置模块，实现文具数据的分层划分及其与存储节点间的映射配置；The mapping configuration module implements hierarchical partitioning of stationery data and mapping configuration with the storage node;
所述文具数据分布管理模块实现文具数据存储的分布管理，所述文具数据的分布管理包括：文具数据存储模块和文具数据删除模块。The stationery data distribution management module realizes distribution management of stationery data storage, and the distribution management of the stationery data includes: a stationery data storage module and a stationery data deletion module.
优选地，所述文具数据分布存储模块用于实现如下步骤：Preferably, the stationery data distribution storage module is configured to implement the following steps:
文具数据的分层划分：将文具数据中的按照地区->子地区->厂站或线路之间的关联关系进行层次划分；Hierarchical division of stationery data: hierarchically divide the correspondence data in the stationery data according to the region->sub-region->factory station or line;
文具数据的分层划分、文具数据的分层划分与存储节点间的映射配置：将地区与地区、厂站或线路与文具数据分布存储模块中的存储节点进行映射；包括：确定每个地区、子地区对应的存储节点，并记录映射，以及，确定每个文具厂、文具收购站、线路对应的存储节点，并记录映射。Hierarchical division of stationery data, hierarchical division of stationery data and mapping configuration between storage nodes: mapping regional and regional, factory stations or lines to storage nodes in the stationery data distribution storage module; including: determining each region, The storage node corresponding to the sub-area, and record the mapping, and determine the storage node corresponding to each stationery factory, stationery purchase station, and line, and record the mapping.
优选地，所述地区 -> 子地区之间关联关系，是指类似“省 - 市 - 县”这样的地区
-> 子地区之间的包含关系。Preferably, the relationship between the region -> sub-region refers to a region like "province - city - county"
-> Inclusion relationship between sub-regions.
优选地，所述厂站及线路，包括文具厂、文具收购站、线路；其中，所述文具厂、文具收购站通常统称为厂站。Preferably, the plant station and the line include a stationery factory, a stationery purchase station, and a line; wherein the stationery factory and the stationery purchase station are generally collectively referred to as a factory station.
优选地，所述地区与厂站及线路间的关联关系，是指地区与文具厂之间、地区与文具收购站之间以及地区与线路之间的包含关系。Preferably, the relationship between the area and the plant station and the line refers to the inclusion relationship between the area and the stationery factory, between the area and the stationery purchase station, and between the area and the line.
优选地，所述按照地区 -> 子地区 ->
厂站或线路之间的包含关系，将文具数据网模型从网状结构划分树状结构层次，从而将文具数据中的文具数据按照关联关系归属到相应的树状结构分支中，所述树状结构中地区为根节点，每个地区
-> 子地区 ->
厂站或线路构成一个具体的分支；这样，其它文具数据就可以根据其与地区、厂站或线路及相互之间的关联关系，被归属到相应的树状结构分支中。Preferably, said by region -> sub-region ->
The inclusion relationship between the station or the line divides the stationery data network model from the network structure into a tree structure hierarchy, so that the stationery data in the stationery data is attributed to the corresponding tree structure branch according to the association relationship, the tree shape The area in the structure is the root node, each area
-> Sub-area ->
The plant station or line constitutes a specific branch; thus, other stationery data can be attributed to the corresponding tree structure branch according to its association with the region, plant station or line and each other.
优选地，所述文具数据存储模块，实现文具数据的新增和更新，所述新增和更新包括如下步骤：Preferably, the stationery data storage module implements adding and updating stationery data, and the adding and updating includes the following steps:
当接到一个文具数据存储请求时，文具数据分布管理模块首先分析文具数据，确定其文具数据树状结构分支；When receiving a stationery data storage request, the stationery data distribution management module first analyzes the stationery data to determine the branch of the stationery data tree structure;
以文具数据所属分支对应的地区或厂站线路信息为参数，查询映射配置模块，确定所对应的存储节点；若发现数据无效，返回错误；Taking the area or plant station line information corresponding to the branch of the stationery data as a parameter, querying the mapping configuration module to determine the corresponding storage node; if the data is found to be invalid, an error is returned;
向查询到的存储节点发起文具数据存储操作；Initiating a stationery data storage operation to the queried storage node;
存储节点接收到文具数据存储请求后，根据文具数据是否存在，分别在存储介质上执行数据更新或数据新建操作。After receiving the stationery data storage request, the storage node performs data update or data creation operation on the storage medium according to whether the stationery data exists.
优选地，所述文具数据删除模块，实现文具数据的删除，所述删除包括如下步骤：Preferably, the stationery data deletion module implements deletion of the stationery data, and the deleting comprises the following steps:
当接到一个文具数据删除请求时，文具数据分布管理模块首先分析文具数据，确定其文具数据树状结构分支；When receiving a stationery data deletion request, the stationery data distribution management module first analyzes the stationery data to determine the branch of the stationery data tree structure;
以文具数据所属分支对应的地区或厂站线路信息为参数，查询映射配置模块，确定所对应的存储节点；若发现数据无效，返回错误；Taking the area or plant station line information corresponding to the branch of the stationery data as a parameter, querying the mapping configuration module to determine the corresponding storage node; if the data is found to be invalid, an error is returned;
存储节点接收到文具数据删除请求后，在存储介质上执行数据删除操作。After receiving the stationery data deletion request, the storage node performs a data deletion operation on the storage medium.
本发明实施例提供的文具大数据分布式存储系统，通过利用文具数据与文具区域相关联的分布式特性，在对大规模文具数据进行分布式存储时，以之为依据，将所处理的文具数据合理的、均匀的分布到多个存储节点上，并为后续的文具数据访问提供易用、高效的基础设施，实现了对文具大数据的分布式处理，且数据传输稳定。
The stationery big data distributed storage system provided by the embodiment of the present invention uses the distributed characteristics associated with the stationery area and the stationery area to distribute the large-scale stationery data according to the basis of the distributed stationery. The data is distributed reasonably and evenly to multiple storage nodes, and provides an easy-to-use and efficient infrastructure for subsequent stationery data access, realizing distributed processing of stationery big data, and stable data transmission.
图1是本发明实施例提供的一种文具大数据分布式存储系统的结构示意图。FIG. 1 is a schematic structural diagram of a stationery big data distributed storage system according to an embodiment of the present invention.
为了使本发明的目的、技术方案及优点更加清楚明白，以下结合附图及实施例，对本发明进行进一步详细说明。应当理解，此处所描述的具体实施例仅仅用以解释本发明，并不用于限定本发明。The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It is understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
本发明实施例提供的文具大数据分布式存储系统，通过利用文具数据与文具区域相关联的分布式特性，在对大规模文具数据进行分布式存储时，以之为依据，将所处理的文具数据合理的、均匀的分布到多个存储节点上，并为后续的文具数据访问提供易用、高效的基础设施，实现了对文具大数据的分布式处理，且数据传输稳定。The stationery big data distributed storage system provided by the embodiment of the present invention uses the distributed characteristics associated with the stationery area and the stationery area to distribute the large-scale stationery data according to the basis of the distributed stationery. The data is distributed reasonably and evenly to multiple storage nodes, and provides an easy-to-use and efficient infrastructure for subsequent stationery data access, realizing distributed processing of stationery big data, and stable data transmission.
以下结合具体实施例对本发明的具体实现进行详细描述。The specific implementation of the present invention will be described in detail below with reference to specific embodiments.
如图1所示，在本发明实施例中，一种文具大数据分布式存储系统100，包括文具数据分布存储模块110；文具数据的分层划分及与存储节点间的映射配置模块120；文具数据分布管理模块130；所述文具数据分布存储模块110，由一组存储节点，通过网络互联，对外作为一个整体提供存储服务的模块；所述映射配置模块120，实现文具数据的分层划分及其与存储节点间的映射配置；所述文具数据分布管理模块130实现文具数据存储的分布管理，所述文具数据的分布管理130包括：文具数据存储模块131和文具数据删除模块132；通过利用文具数据与文具区域相关联的分布式特性，在对大规模文具数据进行分布式存储时，以之为依据，将所处理的文具数据合理的、均匀的分布到多个存储节点上，并为后续的文具数据访问提供易用、高效的基础设施，实现了对文具大数据的分布式处理，且数据传输稳定。As shown in FIG. 1 , in the embodiment of the present invention, a stationery big data distributed storage system 100 includes a stationery data distribution storage module 110; a hierarchical division of stationery data and a mapping configuration module 120 with a storage node; The data distribution management module 130; the stationery data distribution storage module 110 is a module that provides a storage service as a whole by a group of storage nodes through a network interconnection; the mapping configuration module 120 implements hierarchical division of stationery data and a mapping configuration between the storage node and the storage node; the stationery data distribution management module 130 implements distribution management of the stationery data storage, the distribution management 130 of the stationery data includes: a stationery data storage module 131 and a stationery data deletion module 132; The distributed characteristics of the data and the stationery area are distributed, and based on the distributed storage of large-scale stationery data, the processed stationery data is distributed to multiple storage nodes reasonably and evenly, and is followed by Stationery data access provides an easy-to-use, efficient infrastructure that enables great data on stationery Distributed processing and stable data transmission.
在本实施例中，所述文具数据分布存储模块130用于实现如下步骤：In this embodiment, the stationery data distribution storage module 130 is configured to implement the following steps:
文具数据的分层划分：将文具数据中的按照地区->子地区->厂站或线路之间的关联关系进行层次划分；Hierarchical division of stationery data: hierarchically divide the correspondence data in the stationery data according to the region->sub-region->factory station or line;
文具数据的分层划分、文具数据的分层划分与存储节点间的映射配置：将地区与地区、厂站或线路与文具数据分布存储模块中的存储节点进行映射；包括：确定每个地区、子地区对应的存储节点，并记录映射，以及，确定每个文具厂、文具收购站、线路对应的存储节点，并记录映射。其中，所述地区
-> 子地区之间关联关系，是指类似“省 - 市 - 县”这样的地区 ->
子地区之间的包含关系；所述厂站及线路，包括文具厂、文具收购站、线路；其中，所述文具厂、文具收购站通常统称为厂站；所述地区与厂站及线路间的关联关系，是指地区与文具厂之间、地区与文具收购站之间以及地区与线路之间的包含关系；所述按照地区
-> 子地区 ->
厂站或线路之间的包含关系，将文具数据网模型从网状结构划分树状结构层次，从而将文具数据中的文具数据按照关联关系归属到相应的树状结构分支中，所述树状结构中地区为根节点，每个地区
-> 子地区 ->
厂站或线路构成一个具体的分支；这样，其它文具数据就可以根据其与地区、厂站或线路及相互之间的关联关系，被归属到相应的树状结构分支中。Hierarchical division of stationery data, hierarchical division of stationery data and mapping configuration between storage nodes: mapping regional and regional, factory stations or lines to storage nodes in the stationery data distribution storage module; including: determining each region, The storage node corresponding to the sub-area, and record the mapping, and determine the storage node corresponding to each stationery factory, stationery purchase station, and line, and record the mapping. Among them, the area
-> The relationship between sub-regions refers to a region like "province - city - county" ->
The inclusion relationship between the sub-areas; the station and the line, including the stationery factory, the stationery purchase station, and the line; wherein the stationery factory and the stationery purchase station are generally referred to as the factory station; the area and the factory station and the line The relationship between the area and the stationery factory, the area and the stationery acquisition station, and the inclusion relationship between the area and the line;
-> Sub-area ->
The inclusion relationship between the station or the line divides the stationery data network model from the network structure into a tree structure hierarchy, so that the stationery data in the stationery data is attributed to the corresponding tree structure branch according to the association relationship, the tree shape The area in the structure is the root node, each area
-> Sub-area ->
The plant station or line constitutes a specific branch; thus, other stationery data can be attributed to the corresponding tree structure branch according to its association with the region, plant station or line and each other.
在本实施例中，所述文具数据存储模块131，实现文具数据的新增和更新，所述新增和更新包括如下步骤：In this embodiment, the stationery data storage module 131 implements adding and updating stationery data, and the adding and updating includes the following steps:
当接到一个文具数据存储请求时，文具数据分布管理模块首先分析文具数据，确定其文具数据树状结构分支；When receiving a stationery data storage request, the stationery data distribution management module first analyzes the stationery data to determine the branch of the stationery data tree structure;
以文具数据所属分支对应的地区或厂站线路信息为参数，查询映射配置模块，确定所对应的存储节点；若发现数据无效，返回错误；Taking the area or plant station line information corresponding to the branch of the stationery data as a parameter, querying the mapping configuration module to determine the corresponding storage node; if the data is found to be invalid, an error is returned;
向查询到的存储节点发起文具数据存储操作；Initiating a stationery data storage operation to the queried storage node;
存储节点接收到文具数据存储请求后，根据文具数据是否存在，分别在存储介质上执行数据更新或数据新建操作。After receiving the stationery data storage request, the storage node performs data update or data creation operation on the storage medium according to whether the stationery data exists.
在本实施例中，所述文具数据删除模块132，实现文具数据的删除，所述删除包括如下步骤：In this embodiment, the stationery data deletion module 132 implements deletion of the stationery data, and the deleting includes the following steps:
当接到一个文具数据删除请求时，文具数据分布管理模块首先分析文具数据，确定其文具数据树状结构分支；When receiving a stationery data deletion request, the stationery data distribution management module first analyzes the stationery data to determine the branch of the stationery data tree structure;
以文具数据所属分支对应的地区或厂站线路信息为参数，查询映射配置模块，确定所对应的存储节点；若发现数据无效，返回错误；Taking the area or plant station line information corresponding to the branch of the stationery data as a parameter, querying the mapping configuration module to determine the corresponding storage node; if the data is found to be invalid, an error is returned;
存储节点接收到文具数据删除请求后，在存储介质上执行数据删除操作。After receiving the stationery data deletion request, the storage node performs a data deletion operation on the storage medium.
本发明实施例提供的文具大数据分布式存储系统，通过利用文具数据与文具区域相关联的分布式特性，在对大规模文具数据进行分布式存储时，以之为依据，将所处理的文具数据合理的、均匀的分布到多个存储节点上，并为后续的文具数据访问提供易用、高效的基础设施，实现了对文具大数据的分布式处理，且数据传输稳定。The stationery big data distributed storage system provided by the embodiment of the present invention uses the distributed characteristics associated with the stationery area and the stationery area to distribute the large-scale stationery data according to the basis of the distributed stationery. The data is distributed reasonably and evenly to multiple storage nodes, and provides an easy-to-use and efficient infrastructure for subsequent stationery data access, realizing distributed processing of stationery big data, and stable data transmission.
以上所述仅为本发明的较佳实施例而已，并不用以限制本发明，凡在本发明的精神和原则之内所作的任何修改、等同替换和改进等，均应包含在本发明的保护范围之内。The above is only the preferred embodiment of the present invention, and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection of the present invention. Within the scope.
Claims (8)
- 一种对文具大数据分布式存储系统，其特征在于，包括： A distributed data storage system for stationery, characterized in that it comprises:文具数据分布存储模块；Stationery data distribution storage module;文具数据的分层划分及与存储节点间的映射配置模块；Hierarchical division of stationery data and mapping configuration module with storage nodes;文具数据分布管理模块；Stationery data distribution management module;所述文具数据分布存储模块，由一组存储节点，通过网络互联，对外作为一个整体提供存储服务的模块；The stationery data distribution storage module is a module that provides a storage service as a whole by a group of storage nodes connected through a network;所述映射配置模块，实现文具数据的分层划分及其与存储节点间的映射配置；The mapping configuration module implements hierarchical partitioning of stationery data and mapping configuration with the storage node;所述文具数据分布管理模块实现文具数据存储的分布管理，所述文具数据的分布管理包括：文具数据存储模块和文具数据删除模块。The stationery data distribution management module realizes distribution management of stationery data storage, and the distribution management of the stationery data includes: a stationery data storage module and a stationery data deletion module.
- 如权利要求1所述的文具大数据分布式存储系统，其特征在于，所述文具数据分布存储模块用于实现如下步骤：The stationery big data distributed storage system according to claim 1, wherein the stationery data distribution storage module is configured to implement the following steps:文具数据的分层划分：将文具数据中的按照地区->子地区->厂站或线路之间的关联关系进行层次划分；Hierarchical division of stationery data: hierarchically divide the correspondence data in the stationery data according to the region->sub-region->factory station or line;文具数据的分层划分、文具数据的分层划分与存储节点间的映射配置：将地区与地区、厂站或线路与文具数据分布存储模块中的存储节点进行映射；包括：确定每个地区、子地区对应的存储节点，并记录映射，以及，确定每个文具厂、文具收购站、线路对应的存储节点，并记录映射。Hierarchical division of stationery data, hierarchical division of stationery data and mapping configuration between storage nodes: mapping regional and regional, factory stations or lines to storage nodes in the stationery data distribution storage module; including: determining each region, The storage node corresponding to the sub-area, and record the mapping, and determine the storage node corresponding to each stationery factory, stationery purchase station, and line, and record the mapping.
- 如权利要求2所述的文具大数据分布式存储系统，其特征在于，所述地区 -> 子地区之间关联关系，是指 “省 - 市 - 县”这样的地区 -> 子地区之间的包含关系。The stationery big data distributed storage system according to claim 2, wherein the association relationship between the regions -> sub-regions means "province - city - Areas such as counties -> Inclusion relationships between sub-areas.
- 如权利要求2所述的文具大数据分布式存储系统，其特征在于，所述厂站及线路，包括文具厂、文具收购站、线路；其中，所述文具厂、文具收购站通常统称为厂站。The stationery big data distributed storage system according to claim 2, wherein the factory station and the line comprise a stationery factory, a stationery purchase station, and a line; wherein the stationery factory and the stationery purchase station are generally collectively referred to as a factory. station.
- 如权利要求2所述的文具大数据分布式存储系统，其特征在于，所述地区与厂站及线路间的关联关系，是指地区与文具厂之间、地区与文具收购站之间以及地区与线路之间的包含关系。The stationery big data distributed storage system according to claim 2, wherein the relationship between the area and the station and the line refers to the area between the area and the stationery factory, between the area and the stationery purchase station, and the area. The inclusion relationship with the line.
- 如权利要求2所述的文具大数据分布式存储系统，其特征在于，所述按照地区 -> 子地区 -> 厂站或线路之间的包含关系，将文具数据网模型从网状结构划分树状结构层次，从而将文具数据中的文具数据按照关联关系归属到相应的树状结构分支中，所述树状结构中地区为根节点，每个地区 -> 子地区 -> 厂站或线路构成一个具体的分支；这样，其它文具数据就可以根据其与地区、厂站或线路及相互之间的关联关系，被归属到相应的树状结构分支中。A stationery big data distributed storage system according to claim 2, wherein said by region -> sub-region -> The inclusion relationship between the station or the line divides the stationery data network model from the network structure into a tree structure hierarchy, so that the stationery data in the stationery data is attributed to the corresponding tree structure branch according to the association relationship, the tree shape The area in the structure is the root node, each area -> Sub-area -> The plant station or line constitutes a specific branch; thus, other stationery data can be attributed to the corresponding tree structure branch according to its association with the region, plant station or line and each other.
- 如权利要求1所述的文具大数据分布式存储系统，其特征在于，所述文具数据存储模块，实现文具数据的新增和更新，所述新增和更新包括如下步骤：The stationery big data distributed storage system according to claim 1, wherein the stationery data storage module implements adding and updating stationery data, and the adding and updating comprises the following steps:当接到一个文具数据存储请求时，文具数据分布管理模块首先分析文具数据，确定其文具数据树状结构分支；When receiving a stationery data storage request, the stationery data distribution management module first analyzes the stationery data to determine the branch of the stationery data tree structure;以文具数据所属分支对应的地区或厂站线路信息为参数，查询映射配置模块，确定所对应的存储节点；若发现数据无效，返回错误；Taking the area or plant station line information corresponding to the branch of the stationery data as a parameter, querying the mapping configuration module to determine the corresponding storage node; if the data is found to be invalid, an error is returned;向查询到的存储节点发起文具数据存储操作；Initiating a stationery data storage operation to the queried storage node;存储节点接收到文具数据存储请求后，根据文具数据是否存在，分别在存储介质上执行数据更新或数据新建操作。After receiving the stationery data storage request, the storage node performs data update or data creation operation on the storage medium according to whether the stationery data exists.
- 如权利要求1所述的文具大数据分布式存储系统，其特征在于，所述文具数据删除模块，实现文具数据的删除，所述删除包括如下步骤：The stationery big data distributed storage system according to claim 1, wherein the stationery data deletion module implements deletion of the stationery data, and the deleting comprises the following steps:当接到一个文具数据删除请求时，文具数据分布管理模块首先分析文具数据，确定其文具数据树状结构分支；When receiving a stationery data deletion request, the stationery data distribution management module first analyzes the stationery data to determine the branch of the stationery data tree structure;以文具数据所属分支对应的地区或厂站线路信息为参数，查询映射配置模块，确定所对应的存储节点；若发现数据无效，返回错误；Taking the area or plant station line information corresponding to the branch of the stationery data as a parameter, querying the mapping configuration module to determine the corresponding storage node; if the data is found to be invalid, an error is returned;存储节点接收到文具数据删除请求后，在存储介质上执行数据删除操作。 After receiving the stationery data deletion request, the storage node performs a data deletion operation on the storage medium.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| PCT/CN2017/087026 WO2018218672A1 (en) | 2017-06-02 | 2017-06-02 | Distributed storage system for stationery big data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| PCT/CN2017/087026 WO2018218672A1 (en) | 2017-06-02 | 2017-06-02 | Distributed storage system for stationery big data |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2018218672A1 true WO2018218672A1 (en) | 2018-12-06 |
Family
ID=64454248
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/CN2017/087026 WO2018218672A1 (en) | 2017-06-02 | 2017-06-02 | Distributed storage system for stationery big data |
Country Status (1)
| Country | Link |
|---|---|
| WO (1) | WO2018218672A1 (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060015529A1 (en) * | 2004-07-15 | 2006-01-19 | Hitachi, Ltd. | Method and apparatus of hierarchical storage management based on data value |
| CN104268709A (en) * | 2014-10-10 | 2015-01-07 | 浪潮集团有限公司 | Method for designing RFID system by distributed LSM tree |
| CN104636084A (en) * | 2015-01-21 | 2015-05-20 | 广东电网有限责任公司电力科学研究院 | Device and method for carrying out reasonable and efficient distributive storage on big power data |
| CN104765875A (en) * | 2015-04-24 | 2015-07-08 | 海南易建科技股份有限公司 | Distributed processing method and system for passenger behavior data |
-
2017
- 2017-06-02 WO PCT/CN2017/087026 patent/WO2018218672A1/en active Application Filing
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060015529A1 (en) * | 2004-07-15 | 2006-01-19 | Hitachi, Ltd. | Method and apparatus of hierarchical storage management based on data value |
| CN104268709A (en) * | 2014-10-10 | 2015-01-07 | 浪潮集团有限公司 | Method for designing RFID system by distributed LSM tree |
| CN104636084A (en) * | 2015-01-21 | 2015-05-20 | 广东电网有限责任公司电力科学研究院 | Device and method for carrying out reasonable and efficient distributive storage on big power data |
| CN104765875A (en) * | 2015-04-24 | 2015-07-08 | 海南易建科技股份有限公司 | Distributed processing method and system for passenger behavior data |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2012012968A1 (en) | Data partitioning method for distributed parallel database system | |
| JP2000132343A5 (en) | ||
| CN107924289A (en) | Computer system and access control method | |
| CN101903869B (en) | Hierarchical block-identified data communication for unified handling of structured data and data compression | |
| JPH0258165A (en) | Management system for distributed data base system | |
| WO2018218672A1 (en) | Distributed storage system for stationery big data | |
| JP2004054845A (en) | Data management device | |
| WO2018209644A1 (en) | Data encryption method and system for smart household | |
| WO2017120726A1 (en) | Method and system for food service delivery management | |
| WO2019006640A1 (en) | Big data management system | |
| WO2018157330A1 (en) | Big data partitioning method and system | |
| WO2018039824A1 (en) | Energy dynamic storage method and system for solar cell panel | |
| WO2018027576A1 (en) | Method and system for collecting operating time in statistics in internet of things | |
| US10846005B2 (en) | Method, apparatus and computer program product for determining ownership of a data segment | |
| Wu et al. | Analysis of data backup and recovery system | |
| Bing et al. | Design and application of traveling informatization backend database based on Web | |
| WO2018027553A1 (en) | Method and system for drawing style during refurbishment | |
| WO2018039823A1 (en) | Method and system for dynamically controlling solar cell according to load | |
| WO2018209693A1 (en) | Rdd persistence method based on ssd and hdd hybrid storage system | |
| Li et al. | Classification and Storage Method of Medical Health Monitoring Data Based on Bayesian Algorithm | |
| WO2018006256A1 (en) | Local mail data collection method and system | |
| WO2018157331A1 (en) | Storage method and system applied to big data | |
| WO2018014317A1 (en) | Method and system for sorting and saving email data | |
| WO2018014316A1 (en) | Method and system for collecting email data of local area network | |
| CN117992636A (en) | Hierarchical NFT system based on tree-like dependency |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 17911888 Country of ref document: EP Kind code of ref document: A1 |
|
| NENP | Non-entry into the national phase |
Ref country code: DE |
|
| 122 | Ep: pct application non-entry in european phase |
Ref document number: 17911888 Country of ref document: EP Kind code of ref document: A1 |