CN114185493A - Method for dividing and storing surveying and mapping data - Google Patents
Method for dividing and storing surveying and mapping data Download PDFInfo
- Publication number
- CN114185493A CN114185493A CN202210136509.3A CN202210136509A CN114185493A CN 114185493 A CN114185493 A CN 114185493A CN 202210136509 A CN202210136509 A CN 202210136509A CN 114185493 A CN114185493 A CN 114185493A
- Authority
- CN
- China
- Prior art keywords
- storage
- identifier
- capacity
- unit
- units
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0608—Saving storage space on storage systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
- G06F3/0644—Management of space entities, e.g. partitions, extents, pools
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0653—Monitoring storage devices or systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Memory System Of A Hierarchy Structure (AREA)
Abstract
The invention relates to a method for segmenting and storing mapping data, which comprises the following steps: respectively transmitting the basic data obtained by mapping according to an acquisition end, and temporarily storing the basic data in a cache library; the method comprises the steps of dividing a memory of a storage server into a plurality of storage units with the same storage capacity, setting the priority of the storage units, firstly storing the storage units with the first priority, not storing other storage units, then dynamically dividing the storage units with the first priority according to the frequency of each identifier into a plurality of storage subunits with different capacities, dynamically monitoring the frequency of each identifier to determine whether each identifier has weak direction or strong direction change, and adjusting the divided storage subunits again according to the change to enable data to be stored in a divided mode quickly and effectively.
Description
Technical Field
The invention relates to the technical field of division and storage of mapping data, in particular to a division and storage method of mapping data.
Background
The traditional mapping data are generally manually recorded, and are only single numerical values such as coordinates, longitude and latitude, length and width and the like, the existing mapping data comprise a large number of images or video data of geographic features, and typical characteristic data of the images or the video data occupy a large memory and can only be stored electronically. The existing data storage mode, as disclosed in the publication, is as follows: "CN 111694520A" discloses a method for optimizing big data storage, which includes obtaining a storage memory, dividing the storage memory into a plurality of storage nodes, and recording the virtual location and storage capacity of each storage node; setting the storage attribute of each storage node, recording the storage attribute and the storage rule, and setting and enabling the data to be stored to be distributed to the corresponding storage node by the storage rule only under the same storage attribute; the method comprises the steps of decoding data to be stored, sending the decoded data to be stored to a compiling module, writing the compiling module into storage attributes of the data to be stored according to file attributes of the data to be stored, classifying the data according to corresponding storage rules, respectively obtaining storage capacities of storage nodes with the same classification, identifying redundant data in the data to be stored by a processing module through a redundant data identification method according to the storage rules and the selection of the storage capacities, writing node codes of corresponding virtual positions in the redundant data, and storing the node codes in the corresponding storage nodes.
In the method, the storage attribute and the storage rule need to be configured, and the compiling module writes the storage attribute into the data to be stored according to the file attribute of the data to be stored, so that the method needs technical means for assistance and is difficult to implement.
Disclosure of Invention
The present invention is directed to a method for segmenting and storing mapping data, so as to solve the problems in the background art.
In order to achieve the purpose, the invention provides the following technical scheme:
the mapping data segmentation and storage method comprises the following steps:
respectively transmitting the basic data obtained by mapping according to an acquisition end, and temporarily storing the basic data in a cache library;
extracting basic data in a cache library according to a set first period, classifying according to an identifier set by an acquisition end, and recording the occurrence frequency of each identifier;
the method comprises the steps of obtaining a storage memory of a storage server, dividing the storage memory into a plurality of storage units with equal storage capacity, setting priorities among the storage units, setting the storage units as a plurality of storage subunits with different storage capacities according to the occurrence frequency of each identifier by a dynamic partitioning module according to the occurrence frequency ratio of each identifier, and storing classified basic data in the corresponding storage subunits according to corresponding identifiers.
Preferably, the dynamic partitioning module divides the storage unit into a plurality of storage sub-units with different storage capacities according to the ratio of each identifier occurrence frequency, sets a dynamically adjusted list according to the ratio of each identifier occurrence frequency, and stores the list in the dynamic partitioning module.
Preferably, the frequency of each identifier is dynamically monitored to determine whether the frequency of each identifier has a difference beyond a set threshold, and if the frequency of each identifier has a difference beyond the set threshold, one or more adjusting units are executed to adjust the dynamic partitioning module to adjust the storage capacity of the plurality of storage sub-units.
Preferably, one or more adjusting units are executed to adjust the dynamic splitting module to adjust the storage capacity of the plurality of storage sub-units and correspondingly adjust the list stored in the dynamic splitting module.
Preferably, when an adjusting unit is executed, a monitoring unit is correspondingly started, the monitoring unit monitors the storage capacity of the corresponding storage subunit, the storage capacity of the storage subunit is firstly compared with a set storage capacity lower limit to determine whether the storage subunit has allocable residual capacity, and if the storage subunit has allocable residual capacity, the storage capacity of the storage subunit is dynamically adjusted according to the ratio of the occurrence frequency of each identifier; if the allocable residual capacity does not exist, the storage capacity of the storage subunit is not adjusted.
Preferably, the plurality of monitoring units are correspondingly started when the plurality of adjusting units are executed, the plurality of monitoring units monitor the storage capacity of the corresponding storage sub-units, the storage capacity of each storage sub-unit is firstly compared with a set storage capacity lower limit to determine whether each storage sub-unit has allocable residual capacity, and if the storage sub-units have allocable residual capacity, the storage capacity of the storage sub-units is dynamically adjusted according to the ratio of the occurrence frequency of each identifier; if the storage subunit does not have the allocable residual capacity, detecting whether the corresponding identification of the storage subunit is a strengthened type or a weakened type, and if the corresponding identification of the storage subunit is the weakened type, not adjusting the storage capacity of the storage subunit; and if the storage capacity of the storage subunit is not adjusted, the storage capacity of the storage subunit is not adjusted.
Preferably, the monitoring unit is further configured to monitor the storage capacity of the storage unit to determine whether the storage capacity of the storage unit is lower than a set adjustment threshold, and if the storage capacity of the storage unit is lower than the set adjustment threshold, enter the next storage unit for split storage according to a set priority.
Preferably, the plurality of monitoring units are correspondingly started when the plurality of adjusting units are executed, the plurality of monitoring units monitor the storage capacity of the corresponding storage sub-units, the storage capacity of each storage sub-unit is firstly compared with a set storage capacity lower limit to determine whether each storage sub-unit has allocable residual capacity, and if the storage sub-units have allocable residual capacity, the storage capacity of the storage sub-units is dynamically adjusted according to the ratio of the occurrence frequency of each identifier; if the storage subunits do not have allocable residual capacity, whether the corresponding identification of the storage subunits is an enhanced type or a weakened type is detected, if the corresponding identification of the storage subunits is the weakened type and the number of the weakened type identifications is lower than a set identification threshold value, the set storage capacity lower limit reserved by all the storage subunits with the weakened type identifications is allocated at one time according to the rate of the occurrence frequency of the enhanced type identifications.
Preferably, the basic data corresponding to the weakening class identifier is temporarily stored in the cache library, the basic data corresponding to the strengthening class identifier is stored in the corresponding storage sub-unit according to the corresponding identifier, and when the storage capacity of the storage sub-unit corresponding to any strengthening class identifier is zero, the corresponding basic data is temporarily stored in the cache library until the storage capacities of the storage sub-units corresponding to all strengthening class identifiers are zero.
The method comprises the steps of firstly dividing a memory of a storage server into a plurality of storage units with the same storage capacity, then setting the priority of the storage units, firstly storing the storage units with the first priority, not storing other storage units, then dynamically dividing the storage units with the first priority according to the frequency of each identifier into a plurality of storage subunits with different capacities, dynamically monitoring the frequency of each identifier to determine whether each identifier has weak or strong change, and adjusting the divided storage subunits again according to the change so as to enable data to be stored in a quickly and effectively divided mode.
Drawings
FIG. 1 is a flow chart of the method of the present invention.
Detailed Description
The invention is described in detail below with reference to the accompanying drawings, in which reference is made to fig. 1.
The invention provides a method for segmenting and storing mapping data, which comprises the following steps:
respectively transmitting the basic data obtained by mapping according to an acquisition end, and temporarily storing the basic data in a cache library;
extracting basic data in a cache library according to a set first period, classifying according to an identifier set by an acquisition end, and recording the occurrence frequency of each identifier;
the method comprises the steps of obtaining a storage memory of a storage server, dividing the storage memory into a plurality of storage units with equal storage capacity, setting priorities among the storage units, setting the storage units as a plurality of storage subunits with different storage capacities according to the occurrence frequency of each identifier by a dynamic partitioning module according to the occurrence frequency ratio of each identifier, and storing classified basic data in the corresponding storage subunits according to corresponding identifiers.
Preferably, the dynamic partitioning module divides the storage unit into a plurality of storage sub-units with different storage capacities according to the ratio of each identifier occurrence frequency, sets a dynamically adjusted list according to the ratio of each identifier occurrence frequency, and stores the list in the dynamic partitioning module.
Preferably, the frequency of each identifier is dynamically monitored to determine whether the frequency of each identifier has a difference beyond a set threshold, and if the frequency of each identifier has a difference beyond the set threshold, one or more adjusting units are executed to adjust the dynamic partitioning module to adjust the storage capacity of the plurality of storage sub-units.
Preferably, the frequency of each identifier is dynamically monitored to determine whether the frequency of each identifier has a difference beyond a set threshold, and if the frequency of each identifier has a difference beyond the set threshold, one or more adjusting units are executed to adjust the dynamic partitioning module to adjust the storage capacity of the plurality of storage sub-units, and the list stored in the dynamic partitioning module is correspondingly adjusted.
Preferably, when an adjusting unit is executed, a monitoring unit is correspondingly started, the monitoring unit monitors the storage capacity of the corresponding storage subunit, the storage capacity of the storage subunit is firstly compared with a set storage capacity lower limit to determine whether the storage subunit has allocable residual capacity, and if the storage subunit has allocable residual capacity, the storage capacity of the storage subunit is dynamically adjusted according to the ratio of the occurrence frequency of each identifier; if the allocable residual capacity does not exist, the storage capacity of the storage subunit is not adjusted.
Preferably, the plurality of monitoring units are correspondingly started when the plurality of adjusting units are executed, the plurality of monitoring units monitor the storage capacity of the corresponding storage sub-units, the storage capacity of each storage sub-unit is firstly compared with a set storage capacity lower limit to determine whether each storage sub-unit has allocable residual capacity, and if the storage sub-units have allocable residual capacity, the storage capacity of the storage sub-units is dynamically adjusted according to the ratio of the occurrence frequency of each identifier; if the storage subunit does not have the allocable residual capacity, detecting whether the corresponding identification of the storage subunit is a strengthened type or a weakened type, and if the corresponding identification of the storage subunit is the weakened type, not adjusting the storage capacity of the storage subunit; and if the storage capacity of the storage subunit is not adjusted, the storage capacity of the storage subunit is not adjusted.
Preferably, the monitoring unit is further configured to monitor the storage capacity of the storage unit to determine whether the storage capacity of the storage unit is lower than a set adjustment threshold, and if the storage capacity of the storage unit is lower than the set adjustment threshold, enter the next storage unit for split storage according to a set priority.
Preferably, the plurality of monitoring units are correspondingly started when the plurality of adjusting units are executed, the plurality of monitoring units monitor the storage capacity of the corresponding storage sub-units, the storage capacity of each storage sub-unit is firstly compared with a set storage capacity lower limit to determine whether each storage sub-unit has allocable residual capacity, and if the storage sub-units have allocable residual capacity, the storage capacity of the storage sub-units is dynamically adjusted according to the ratio of the occurrence frequency of each identifier; if the storage subunits do not have allocable residual capacity, whether the corresponding identification of the storage subunits is an enhanced type or a weakened type is detected, if the corresponding identification of the storage subunits is the weakened type and the number of the weakened type identifications is lower than a set identification threshold value, the set storage capacity lower limit reserved by all the storage subunits with the weakened type identifications is allocated at one time according to the rate of the occurrence frequency of the enhanced type identifications.
Preferably, the basic data corresponding to the weakening class identifier is temporarily stored in the cache library, the basic data corresponding to the strengthening class identifier is stored in the corresponding storage sub-unit according to the corresponding identifier, and when the storage capacity of the storage sub-unit corresponding to any strengthening class identifier is zero, the corresponding basic data is temporarily stored in the cache library until the storage capacities of the storage sub-units corresponding to all strengthening class identifiers are zero.
Example 1
In this embodiment, when there are two acquisition ends, the data is divided and stored.
Respectively transmitting the basic data obtained by mapping according to a first acquisition end and a second acquisition end, and temporarily storing the basic data in a cache library; the first acquisition end is provided with a first identifier, and the second acquisition end is provided with a second identifier;
extracting basic data in a cache library according to a set first period, distinguishing and classifying the basic data according to a first identifier and a second identifier, and recording the occurrence frequency of the first identifier and the second identifier;
the method comprises the steps of obtaining a storage memory of a storage server, dividing the storage memory into a first storage unit and a second storage unit with equal storage capacity, setting priorities of the first storage unit and the second storage unit, wherein the priorities of the first storage unit and the second storage unit are set, for example, the priority of the first storage unit is greater than the priority of the second storage unit, the priorities can be set in a master-slave mode, or parameters can be set, the parameters of the first storage unit and the second storage unit are marked, and the setting of the priorities can be completed by setting a default storage path according to the parameters during storage. When storing, the basic data is stored in the first storage unit, and after the capacity of the first storage unit is zero, the second storage unit is stored.
According to the frequency of the first mark and the second mark, the dynamic segmentation module sets the first storage unit as two storage subunits with different storage capacities according to the ratio of the frequency of the first mark and the second mark, and stores the classified basic data in the corresponding storage subunits according to the corresponding marks.
For example, in the above, the ratio of the occurrence frequencies of the first mark and the second mark is 1: 2, the storage capacity of the storage sub-unit corresponding to the first identifier occupies one third of the total capacity of the first storage unit, and the storage capacity of the storage sub-unit corresponding to the second identifier occupies two thirds of the total capacity of the first storage unit. While forming a stored list. The storage list is stored in the dynamic partitioning module.
After the division, storing the classified basic data in the corresponding storage subunits according to the corresponding first identifier and second identifier, in the process of storage, monitoring the first identifier and second identifier by using a monitoring unit to determine whether the first identifier and second identifier have changes in the storage process, specifically, dynamically monitoring the frequencies of the first identifier and second identifier to determine whether the frequency of any one of the first identifier and second identifier has a difference beyond a set threshold (at this time, the set threshold is understood as that the frequency of the first identifier is stabilized to be one third of the sum of the frequencies of the first identifier and second identifier, the frequency of the second identifier is stabilized to be two thirds of the sum of the frequencies of the first identifier and second identifier, the set threshold is dynamically changed, if the storage capacity of the storage subunit corresponding to the first identifier is adjusted, the corresponding set threshold value is also correspondingly adjusted), if the difference is different from the set threshold value, one or two adjusting units are executed to adjust the adjustment of the dynamic dividing module to the storage capacity of the two storage sub-units. During specific adjustment, if the frequency of the first identifier and the second identifier in the monitoring process is stable at 1: 3; the storage capacity of the storage sub-unit corresponding to the first identifier is one fourth of the total capacity of the first storage unit, and the storage capacity of the storage sub-unit corresponding to the second identifier is three fourths of the total capacity of the first storage unit. While forming a stored list. The storage list is stored in the dynamic segmentation module, and the set threshold value is changed into: the appearance frequency of the first mark is stabilized to be one fourth of the sum of the appearance frequencies of the first mark and the second mark, and the appearance frequency of the second mark is stabilized to be three fourths of the sum of the appearance frequencies of the first mark and the second mark.
In the above, when an adjustment unit is executed, a monitoring unit is correspondingly started, and assuming that the first monitoring unit monitors the storage capacity of the first storage subunit, the storage capacity of the first storage subunit is first compared with a set storage capacity lower limit to determine whether the first storage subunit has allocable residual capacity, and if so, the storage capacity of the storage subunit is dynamically adjusted according to the ratio of the occurrence frequency of the first identifier to the occurrence frequency of the second identifier; that is, if the frequency of the first mark and the second mark after stabilization is stable in multiple cycles in the monitoring process is 1: 3; the storage capacity of the storage sub-unit corresponding to the first identifier is one fourth of the total capacity of the first storage unit, and the storage capacity of the storage sub-unit corresponding to the second identifier is three fourths of the total capacity of the first storage unit. While forming a stored list. The storage list is stored in the dynamic partitioning module. If the allocable residual capacity does not exist, the storage capacity of the storage subunit is not adjusted.
In the above, the rules for storing the list are: the storage subunit corresponding to the first identifier depends on the storage subunit corresponding to the second identifier, and the dynamic segmentation module can directly call the storage list to know the capacity information of the storage subunit corresponding to the first identifier and the storage subunit corresponding to the second identifier.
In the above, when the two adjustment units are executed, the two monitoring units are correspondingly started, the two monitoring units monitor the storage capacity of the corresponding storage sub-units, the storage capacity of each storage sub-unit is firstly compared with a set lower limit of the storage capacity to determine whether each storage sub-unit has allocable residual capacity, and if yes, the storage capacity of the storage sub-units is dynamically adjusted according to the ratio of the occurrence frequency after the first identifier and the second identifier are stable; and if the storage subunit does not have allocable residual capacity, detecting whether the corresponding identifier of the storage subunit is an enhanced class or a weakened class, wherein the frequency of the first identifier and the second identifier after being stable is stable in 1: and 3, it is indicated that the first identifier tends to be weakened, the second identifier tends to be strengthened, and it is proved that the flow of the basic data corresponding to the first identifier is gradually reduced compared with that after the first segmentation, and the capacity of the basic data required to be stored correspondingly is also reduced. Of course, if there are more than three identifiers, for example, the third identifier is also a strengthened identifier, and the frequency of the third identifier is greater than the frequency of the second identifier, then at this time, the storage list corresponding to the third identifier includes the first identifier and the second identifier, and the storage list corresponding to the second identifier can only be the first identifier, and cannot include the third identifier.
In the above description, it is a matter of course that there are many cases, if there are 5 identifiers, it may be set that, when the number of the weakened class identifiers is less than 2, a set lower storage capacity limit reserved for all the storage subunits with the weakened class identifiers (for example, each storage subunit has a lower storage limit of 5% or 10%, and when the lower limit is reached, the storage space capacity is substantially full, and the storage efficiency of its own is also reduced), then all the remaining capacities corresponding to the weakened class identifiers may include an index that the set lower storage capacity limit is completely allocated to the strengthened class. The method aims to achieve the purposes that when the storage subunit reaches the lower storage limit, the capacity of the storage space is basically full, the storage efficiency of the storage subunit is reduced, the flow of basic data corresponding to the weakening type index is low, the occupied memory of the cache library is small, the time is short, the storage of the basic data is suspended, the storage of the weakening type index is accelerated only, the residual storage capacity corresponding to the weakening type index is completely distributed to the strengthening type index, and certainly, the storage subunit which reaches the set lower storage capacity limit can be distributed again in the strengthening type index. To speed up the storage efficiency late in the first storage unit. And after the storage capacity of the storage subunit corresponding to any enhanced class identifier is zero, the corresponding basic data is temporarily stored in the cache library until the storage capacity of the storage subunit corresponding to all enhanced class identifiers is zero.
In the foregoing, the monitoring unit is further configured to monitor the storage capacity of the first storage unit to determine whether the storage capacity of the first storage unit is lower than a set adjustment threshold (for example, when 0.5% remains, the efficiency of storing again decreases linearly), and if the storage capacity of the first storage unit is lower than the set adjustment threshold, enter the split storage of the second storage unit according to a set priority.
The principles and embodiments of the present invention are explained herein using specific examples, which are presented only to assist in understanding the method and its core concepts of the present invention. The foregoing is only a preferred embodiment of the present invention, and it should be noted that there are objectively infinite specific structures due to the limited character expressions, and it will be apparent to those skilled in the art that a plurality of modifications, decorations or changes may be made without departing from the principle of the present invention, and the technical features described above may be combined in a suitable manner; such modifications, variations, combinations, or adaptations of the invention using its spirit and scope, as defined by the claims, may be directed to other uses and embodiments.
Claims (9)
1. The mapping data segmentation and storage method is characterized by comprising the following steps:
respectively transmitting the basic data obtained by mapping according to an acquisition end, and temporarily storing the basic data in a cache library;
extracting basic data in a cache library according to a set first period, classifying according to an identifier set by an acquisition end, and recording the occurrence frequency of each identifier;
the method comprises the steps of obtaining a storage memory of a storage server, dividing the storage memory into a plurality of storage units with equal storage capacity, setting priorities among the storage units, setting the storage units as a plurality of storage subunits with different storage capacities according to the occurrence frequency of each identifier by a dynamic partitioning module according to the occurrence frequency ratio of each identifier, and storing classified basic data in the corresponding storage subunits according to corresponding identifiers.
2. The method for dividing and storing mapping data according to claim 1, wherein the dynamic division module divides the storage unit into a plurality of storage sub-units with different storage capacities according to the ratio of each identifier occurrence frequency, and sets a dynamically adjusted list according to the ratio of each identifier occurrence frequency, and stores the list in the dynamic division module.
3. The method of claim 1, wherein the frequency of each marker is dynamically monitored to determine whether the frequency of each marker has a difference beyond a set threshold, and if so, one or more adjustment units are executed to adjust the dynamic partitioning module to adjust the storage capacity of the plurality of storage sub-units.
4. The method of claim 3, wherein one or more adjustment units are implemented to adjust the dynamic segmentation module to adjust the storage capacity of the plurality of storage sub-units and correspondingly adjust the list stored in the dynamic segmentation module.
5. The method for segmenting and storing mapping data according to claim 4, wherein a monitoring unit is correspondingly activated when an adjustment unit is executed, the monitoring unit monitors the storage capacity of the corresponding storage subunit, the storage capacity of the storage subunit is compared with a set storage capacity lower limit to determine whether the storage subunit has allocable residual capacity, and if the storage subunit has allocable residual capacity, the storage capacity of the storage subunit is dynamically adjusted according to the ratio of the occurrence frequency of each identifier; if the allocable residual capacity does not exist, the storage capacity of the storage subunit is not adjusted.
6. The method for segmenting and storing mapping data according to claim 4, wherein the plurality of monitoring units are correspondingly activated when the plurality of adjustment units are executed, the plurality of monitoring units monitor the storage capacity of the corresponding storage sub-unit, the storage capacity of each storage sub-unit is compared with a set storage capacity lower limit to determine whether each storage sub-unit has allocable residual capacity, and if so, the storage capacity of the storage sub-unit is dynamically adjusted according to the ratio of the occurrence frequency of each identifier; if the storage subunit does not have the allocable residual capacity, detecting whether the corresponding identification of the storage subunit is a strengthened type or a weakened type, and if the corresponding identification of the storage subunit is the weakened type, not adjusting the storage capacity of the storage subunit; and if the storage capacity of the storage subunit is not adjusted, the storage capacity of the storage subunit is not adjusted.
7. The method for segmenting and storing mapping data according to claim 5 or 6, wherein the monitoring unit is further configured to monitor the storage capacity of the storage unit to determine whether the storage capacity of the storage unit is lower than a set adjustment threshold, and if the storage capacity of the storage unit is lower than the set adjustment threshold, the segmentation storage of the next storage unit is performed according to a set priority.
8. The method for segmenting and storing mapping data according to claim 6, wherein the plurality of monitoring units are correspondingly activated when the plurality of adjustment units are executed, the plurality of monitoring units monitor the storage capacity of the corresponding storage sub-unit, the storage capacity of each storage sub-unit is compared with a set storage capacity lower limit to determine whether each storage sub-unit has allocable residual capacity, and if so, the storage capacity of the storage sub-unit is dynamically adjusted according to the ratio of the occurrence frequency of each identifier; if the storage subunits do not have allocable residual capacity, whether the corresponding identification of the storage subunits is an enhanced type or a weakened type is detected, if the corresponding identification of the storage subunits is the weakened type and the number of the weakened type identifications is lower than a set identification threshold value, the set storage capacity lower limit reserved by all the storage subunits with the weakened type identifications is allocated at one time according to the rate of the occurrence frequency of the enhanced type identifications.
9. The method for segmenting and storing mapping data according to claim 6, wherein the basic data corresponding to the weakened class identifier is temporarily stored in a buffer library, the basic data corresponding to the strengthened class identifier is stored in the corresponding storage sub-unit according to the corresponding identifier, and when the storage capacity of the storage sub-unit corresponding to any strengthened class identifier is zero, the corresponding basic data is temporarily stored in the buffer library until the storage capacity of the storage sub-units corresponding to all strengthened class identifiers is zero.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210136509.3A CN114185493B (en) | 2022-02-15 | 2022-02-15 | Method for dividing and storing surveying and mapping data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210136509.3A CN114185493B (en) | 2022-02-15 | 2022-02-15 | Method for dividing and storing surveying and mapping data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114185493A true CN114185493A (en) | 2022-03-15 |
| CN114185493B CN114185493B (en) | 2022-05-06 |
Family
ID=80607043
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210136509.3A Active CN114185493B (en) | 2022-02-15 | 2022-02-15 | Method for dividing and storing surveying and mapping data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114185493B (en) |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101566926A (en) * | 2008-04-24 | 2009-10-28 | 联咏科技股份有限公司 | Memory access device and display using same |
| US20110145200A1 (en) * | 2009-12-16 | 2011-06-16 | Teradata Us, Inc. | Precedence based storage |
| CN104978361A (en) * | 2014-04-11 | 2015-10-14 | 深圳中兴力维技术有限公司 | Storage method and device of dynamic environment real-time monitoring data |
| CN108763465A (en) * | 2018-05-25 | 2018-11-06 | 广州钱柜软件科技有限公司 | A kind of video storage distribution method based on big data |
| CN111399756A (en) * | 2019-09-29 | 2020-07-10 | 杭州海康威视系统技术有限公司 | Data storage method, data downloading method and device |
| CN111694520A (en) * | 2020-06-11 | 2020-09-22 | 邦尼集团有限公司 | Method and device for optimizing big data storage |
| CN112817543A (en) * | 2021-02-24 | 2021-05-18 | 浙江大华技术股份有限公司 | Data storage method, system, device, equipment and medium |
| CN113473039A (en) * | 2021-06-18 | 2021-10-01 | 苏州浪潮智能科技有限公司 | Multi-channel video storage method and system |
| CN113486026A (en) * | 2021-08-02 | 2021-10-08 | 北京字节跳动网络技术有限公司 | Data processing method, device, equipment and medium |
| CN113791733A (en) * | 2021-09-06 | 2021-12-14 | 青岛中科曙光科技服务有限公司 | Information storage method, device, equipment and storage medium |
-
2022
- 2022-02-15 CN CN202210136509.3A patent/CN114185493B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101566926A (en) * | 2008-04-24 | 2009-10-28 | 联咏科技股份有限公司 | Memory access device and display using same |
| US20110145200A1 (en) * | 2009-12-16 | 2011-06-16 | Teradata Us, Inc. | Precedence based storage |
| CN104978361A (en) * | 2014-04-11 | 2015-10-14 | 深圳中兴力维技术有限公司 | Storage method and device of dynamic environment real-time monitoring data |
| CN108763465A (en) * | 2018-05-25 | 2018-11-06 | 广州钱柜软件科技有限公司 | A kind of video storage distribution method based on big data |
| CN111399756A (en) * | 2019-09-29 | 2020-07-10 | 杭州海康威视系统技术有限公司 | Data storage method, data downloading method and device |
| CN111694520A (en) * | 2020-06-11 | 2020-09-22 | 邦尼集团有限公司 | Method and device for optimizing big data storage |
| CN112817543A (en) * | 2021-02-24 | 2021-05-18 | 浙江大华技术股份有限公司 | Data storage method, system, device, equipment and medium |
| CN113473039A (en) * | 2021-06-18 | 2021-10-01 | 苏州浪潮智能科技有限公司 | Multi-channel video storage method and system |
| CN113486026A (en) * | 2021-08-02 | 2021-10-08 | 北京字节跳动网络技术有限公司 | Data processing method, device, equipment and medium |
| CN113791733A (en) * | 2021-09-06 | 2021-12-14 | 青岛中科曙光科技服务有限公司 | Information storage method, device, equipment and storage medium |
Non-Patent Citations (3)
| Title |
|---|
| 宋亚奇: "云平台下电力设备监测大数据存储优化与并行处理技术研究", 《中国博士学位论文全文数据库 工程科技Ⅱ辑》 * |
| 栾绍鹏: "云平台中国土资源信息存储检测仿真研究", 《计算机仿真》 * |
| 王珺: "低占空比无线传感器网络节点调度算法研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114185493B (en) | 2022-05-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106383880B (en) | GIF file playing method and system | |
| EP1736885A2 (en) | Method for managing a memory in a mobile terminal | |
| US20030097496A1 (en) | Method and system for obtaining the status of a device | |
| CN107436725A (en) | A kind of data are write, read method, apparatus and distributed objects storage cluster | |
| CN105797381A (en) | Game cold data storage and reading methods and devices | |
| CN114465956B (en) | Method and device for limiting flow rate of virtual machine, electronic equipment and storage medium | |
| CN113473039A (en) | Multi-channel video storage method and system | |
| CN101650718A (en) | Method and device for matching character strings | |
| CN114185493B (en) | Method for dividing and storing surveying and mapping data | |
| US20090190607A1 (en) | Method and apparatus for transmitting data and method and apparatus for performing data task | |
| CN114339378A (en) | Audio and video code stream processing method and device and electronic equipment | |
| US11250001B2 (en) | Accurate partition sizing for memory efficient reduction operations | |
| CN107423188B (en) | Log processing method and device | |
| CN111208953A (en) | Distributed storage method and device | |
| CN113703678B (en) | Method, device, equipment and medium for re-splitting storage bucket index | |
| CN108287793A (en) | The way to play for time and server of response message | |
| CN109669623A (en) | A kind of file management method, document management apparatus, electronic equipment and storage medium | |
| CN111343404B (en) | Imaging data processing method and device | |
| CN108874753B (en) | Method and device for searching response of subject post and computer equipment | |
| CN110502480B (en) | Distributed storage data retrieval method, system, device and readable storage medium | |
| CN110365342B (en) | Waveform decoding method and device | |
| CN111865794B (en) | Logical port association method, system, equipment and data transmission system | |
| CN113645585A (en) | Internet of things data acquisition method | |
| US20100169322A1 (en) | Efficient access of bitmap array with huge usage variance along linear fashion, using pointers | |
| CN112333189A (en) | Data processing method based on Internet of things communication and live broadcast platform and cloud computing center |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |