CN106383668B - Information storage method, storage management equipment and client - Google Patents
Information storage method, storage management equipment and client Download PDFInfo
- Publication number
- CN106383668B CN106383668B CN201610830142.XA CN201610830142A CN106383668B CN 106383668 B CN106383668 B CN 106383668B CN 201610830142 A CN201610830142 A CN 201610830142A CN 106383668 B CN106383668 B CN 106383668B
- Authority
- CN
- China
- Prior art keywords
- storage
- client
- information
- transmission connection
- storage device
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0604—Improving or facilitating administration, e.g. storage management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
- G06F3/0643—Management of files
Abstract
The invention provides an information storage method, a storage management device and a client. The method comprises the following steps: the client sends an information storage request to the storage management equipment; the storage management equipment responds to the information storage request, selects a storage device as a target storage device of the client according to a preset rule, and sends the information of the target storage device to the client; the method comprises the steps that a client establishes transmission connection with a target storage device, the duration of the transmission connection is monitored, when the duration of the transmission connection reaches a first preset duration, the transmission connection is released, and a new information storage request is sent to a storage management device again. The information storage method can be applied to the storage management equipment and the client. Through the design, the information to be stored in the client can be uniformly distributed in each storage device according to time, so that the load balance of each storage device is realized.
Description
Technical Field
The invention relates to the technical field of distributed storage, in particular to an information storage method, storage management equipment and a client.
Background
In a distributed file system, load balancing is one of key technologies for improving system performance, and can ensure the stability and service quality of the system, so that it is necessary to ensure that information to be stored of each client can be uniformly distributed on all storage devices. Most of the existing distributed file systems are implemented in a centralized stream receiving and storing manner, that is, each client directly sends information to be stored to a data distribution device, and the data distribution device uniformly distributes the information to different storage devices.
However, the storage method has strong dependency on the data distribution device, and when the data distribution device has a problem, the information of all the clients cannot be stored. And with the increase of the amount of information to be stored, one data distribution device may be connected with thousands of clients, and in this case, the performance requirement on the data distribution device is high, and the performance of the data distribution device will directly affect the performance of the whole distributed file system.
Disclosure of Invention
In view of this, the present invention provides an information storage method, a storage management device, and a client, so as to solve the problem that the existing distributed file system has a strong dependency on a data distribution device.
The preferred embodiment of the present invention provides an information storage method, which is applied to a storage management device, wherein the storage management device can communicate with a client and a plurality of storage devices, and the method comprises the following steps:
receiving an information storage request sent by the client at intervals of a first preset duration;
selecting a storage device as a target storage device of the client according to a preset rule aiming at each information storage request;
and sending the information of the target storage equipment to the client so as to enable the client to establish transmission connection with the target storage equipment.
Another preferred embodiment of the present invention provides an information storage method, which is applied to a client, where the client is capable of communicating with a storage management device and a plurality of storage devices, and the method includes:
sending an information storage request to the storage management device;
receiving information of a target storage device sent by the storage management device, and establishing transmission connection with the target storage device;
and monitoring the duration of the transmission connection, releasing the transmission connection when the duration of the transmission connection reaches a first preset duration, and sending a new information storage request to the storage management equipment again.
Another preferred embodiment of the present invention provides a storage management device, which is capable of communicating with a client and a plurality of storage devices, and comprises a storage request receiving module, a storage target selecting module, and a target information sending module;
the storage request receiving module is used for receiving information storage requests sent by the client every first preset time interval;
the storage target selection module is used for selecting a storage device as a target storage device of the client according to a preset rule aiming at each information storage request; and
and the target information sending module is used for sending the information of the target storage equipment to the client so as to enable the client to establish transmission connection with the target storage equipment.
Another preferred embodiment of the present invention provides a client, where the client is capable of communicating with a storage management device and a plurality of storage devices, and the client includes a storage request sending module, a transmission connection establishing module, and a transmission connection monitoring module;
the storage request sending module is used for sending an information storage request to the storage management equipment;
the transmission connection establishing module is used for receiving the information of the target storage device sent by the storage management device and establishing transmission connection with the target storage device; and
and the transmission connection monitoring module is used for monitoring the duration of the transmission connection, releasing the transmission connection when the duration of the transmission connection reaches a first preset duration, and sending a new information storage request to the storage management equipment again.
Compared with the prior art, according to the information storage method, the storage management device and the client provided by the embodiment of the invention, the client applies for the target storage device from the storage management device every first preset time interval, and the storage management device allocates the new target storage device to the client according to the preset rule, so that the client directly sends the information to be stored to the target storage device for storage. The management and distribution of the storage management equipment can ensure that the information to be stored in the client is uniformly distributed in each storage equipment according to time, and a series of problems caused by a storage mode that the data distribution equipment is adopted to receive the information of all the clients and then uniformly distribute the information in the prior art are avoided.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present invention and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained according to the drawings without inventive efforts.
Fig. 1 is a schematic block diagram of a distributed file system according to an embodiment of the present invention.
Fig. 2 is a schematic block diagram of a storage management device according to an embodiment of the present invention.
Fig. 3 is a schematic flowchart of an information storage method according to an embodiment of the present invention.
Fig. 4 is a schematic view of the substeps of step S102 in fig. 3.
Fig. 5 is a schematic flowchart of another information storage method according to an embodiment of the present invention.
Fig. 6 is a schematic flowchart of another information storage method according to an embodiment of the present invention.
Fig. 7 is a schematic view of the substeps of step S301 in fig. 6.
Fig. 8 is a functional block diagram of a storage management device according to an embodiment of the present invention.
Fig. 9 is a connection block diagram of sub-modules of a storage target selecting module according to an embodiment of the present invention.
Fig. 10 is a functional block diagram of a client according to an embodiment of the present invention.
Fig. 11 is a connection block diagram of sub-modules of a storage request sending module according to an embodiment of the present invention.
Icon: 100-a distributed file system; 110-a storage management device; 111-a memory; 112-a memory controller; 113-a processor; 114-peripheral interfaces; 115-input-output unit; 116-a communication unit; 1101-a storage request receiving module; 1102-a storage target selection module; 201-information record query submodule; 202-a first selection submodule; 203-a second selection submodule; 1103-target information sending module; 1104-task number receiving module; 1105, reporting to a monitoring module; 120-a client; 121-storage request sending module; 1211-identity obtaining submodule; 1212-finding a sub-module; 1213-request sending submodule; 122-transmission connection establishment module; 123-transmission connection monitoring module; 130-a storage device; 200-network.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. The components of embodiments of the present invention generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations.
Thus, the following detailed description of the embodiments of the present invention, presented in the figures, is not intended to limit the scope of the invention, as claimed, but is merely representative of selected embodiments of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.
Fig. 1 is a block diagram of a distributed file system 100 according to an embodiment of the present invention. The distributed file system 100 includes a storage management device 110, a client 120, and a plurality of storage devices 130. The client 120 and the storage devices 130 are respectively connected with the storage management device 110 through a network 200 in a communication manner, and the client 120 and the storage devices 130 can establish a transmission connection through the network 200 to perform data communication or interaction.
According to actual requirements, the client 120 may be an information acquisition device, and may also include an exchange server and a plurality of information acquisition devices communicatively connected to the exchange server.
The storage management device 110 and the storage device 130 may be servers, computers, terminals, and the like having storage and information processing capabilities.
Fig. 2 is a block diagram of a storage management device 110 according to an embodiment of the present invention. The storage management device 110 includes a memory 111, a storage controller 112, a processor 113, a peripheral interface 114, an input-output unit 115, and a communication unit 116.
The memory 111, the memory controller 112, the processor 113, the peripheral interface 114, the input/output unit 115, and the communication unit 116 are electrically connected to each other directly or indirectly to implement data transmission or interaction. For example, the components may be electrically connected to each other via one or more communication buses or signal lines.
The Memory 111 may be, but is not limited to, a Random Access Memory (RAM), a Read Only Memory (ROM), a Programmable Read-Only Memory (PROM), an Erasable Read-Only Memory (EPROM), an electrically Erasable Read-Only Memory (EEPROM), and the like.
The memory 111 may be used to store software programs and modules, and the processor 113 executes the software programs and modules stored in the memory 111 after receiving the execution instruction, so as to execute corresponding functional applications and information processing.
The processor 113 may be an integrated circuit chip having information processing capabilities. The Processor 113 may be a general-purpose Processor, and includes a Central Processing Unit (CPU), a Network Processor (NP), and the like. But may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components. The various methods, steps and logic blocks disclosed in embodiments of the present invention may be implemented or performed. A general purpose processor may be a microprocessor or the processor 113 may be any conventional processor or the like.
The peripheral interface 114 is used to couple various input/output devices to the processor 113 and memory 111. In some embodiments, the peripheral interface 114, the processor 113, and the memory controller 112 may be implemented in a single chip, and in other embodiments, they may be implemented separately by separate chips.
The communication unit 116 is configured to establish a communication connection between the storage management device 110 and an external communication terminal through the network 200, where the external communication terminal includes the client 120 and the storage device 130 in the embodiment of the present invention.
It will be appreciated that the configuration shown in FIG. 2 is merely illustrative, and that storage management device 110 may also include more or fewer components than shown in FIG. 2, or have a different configuration than shown in FIG. 2. The components shown in fig. 2 may be implemented in hardware, software, or a combination thereof.
It should be noted that specific structures and connection manners of the components in the storage device 130, the client 120, and other servers and terminals disclosed in the embodiments of the present invention are the same as those of the corresponding components in the storage management device 110. For example, the client 120 stores corresponding software programs and modules, and the corresponding functional applications can be implemented by executing the software programs and modules. The storage device 130 stores corresponding software programs and modules, and the corresponding functional applications can be realized by executing the software programs and modules.
Referring to fig. 3, a flowchart of an information storage method according to an embodiment of the present invention is shown, where the information storage method is applicable to the storage management device 110 shown in fig. 1. The specific process and steps shown in fig. 3 will be described in detail below.
Step S101: receiving an information storage request sent by the client 120 at intervals of a first preset duration.
The client 120 sends an information storage request to the storage management device 110 at intervals of a first preset duration, and optionally, the information storage request carries identity information of the client 120. The first preset duration can be flexibly set according to actual conditions, and this embodiment does not limit this. For example, the first preset time period may be 5 minutes.
Step S102: for each information storage request, a storage device 130 is selected as a target storage device of the client 120 according to a preset rule.
Each time the storage management device 110 receives an information storage request sent by the client 120, it responds to the information storage request and allocates a new target storage device to the client 120 according to a preset rule.
Optionally, the preset rule may be multiple, for example: the storage management device 110 queries the current task number of each storage device 130, and selects the storage device 130 with the minimum current task number as the target storage device of the client 120. The number of tasks of each storage device 130 is the number of clients 120 in transmission connection with each storage device 130. In implementation, the storage management device 110 queries the number of clients 120 currently in transmission connection with each storage device 130 after receiving the information storage request. Another example is: the storage management device 110 queries the remaining storage space of each storage device 130, and selects the storage device 130 with the most remaining storage space as the target storage device of the client 120.
When the preset rule is that the storage management device 110 queries the current task number of each storage device 130, and selects the storage device 130 with the smallest current task number as the target storage device of the client 120, optionally, the storage management device 110 may query the current task number of each storage device 130 by sequentially accessing each storage device 130, or may query the current task number of each storage device 130 stored by itself by receiving and storing the current task number periodically reported by each storage device 130, and querying the current task number of each storage device 130 stored by itself when needed. It should be noted that the period of reporting the current task number by each storage device 130 is less than the first preset time.
Step S103: and sending the information of the target storage device to the client 120, so that the client 120 establishes a transmission connection with the target storage device.
The storage management device 110 sends the information of the target storage device to the client 120, and the client 120 establishes a transmission connection with the target storage device after receiving the information of the target storage device.
Optionally, in order to better achieve load balancing and prevent data loss, the storage device 130 that has established a transmission connection with the client 120 needs to be removed each time the target storage device is newly allocated to the client 120. Each time a target storage device is allocated to the client 120, the storage management device 110 may associate information of the target storage device with the client 120 and store the information, that is, the storage management device 110 stores information records of each storage device 130 that establishes a transmission connection with the client 120 within the first preset time period.
As shown in fig. 4, in step S102, the step of selecting one storage device 130 as the target storage device of the client 120 according to the preset rule may include three substeps, i.e., step S201, step S202, and step S203.
Step S201: and inquiring whether a storage device 130 which does not establish transmission connection with the client 120 exists according to the information record. If not, the storage management device 110 executes step S202, and if so, the storage management device 110 executes step S203.
Step S202: deleting the information record of each storage device 130 establishing transmission connection with the client 120 within the first preset time length in the storage management device 110, and taking the storage device 130 with the minimum current task number as a target storage device of the client 120.
Step S203: and using the storage device 130 with the least current task number in the storage devices 130 which have not established transmission connection with the client 120 as the target storage device of the client 120.
Each time a target storage device is selected for a client 120, a storage device 130 with the least number of tasks currently needs to be preferentially selected from storage devices 130 that have not established a transmission connection with the client 120 as the target storage device of the client 120. After all the storage devices 130 have established transmission connection with the client 120, the information records of the target storage device of the client 120 in the storage management device 110 are deleted, the current task numbers of all the storage devices 130 are inquired again, and the information records of the target storage device of the client 120 are recorded again.
Optionally, in the implementation process of the information storage method, each storage device 130 may report the current task number to the storage management device 110 at regular time, so that the storage management device 110 can query its own storage content when needed, without polling each storage device 130.
As shown in fig. 5, optionally, the information storage method may further include three steps of step S104, step S105, and step S106, which are described in detail below.
Step S104: and receiving the current task number sent by each storage device 130 every second preset time interval.
By reporting the current task number to the storage management device 110 every second preset time interval by each storage device 130, the time for the storage management device 110 to query the current task number of each storage device 130 can be saved, and even if the storage management device 110 is abnormally restarted, the synchronization of the current task number of each storage device 130 in the storage management device 110 can be ensured. It should be noted that the second preset time period is shorter than the first preset time period, for example, the first preset time period is 5 minutes, and the second preset time period may be 30 seconds.
Step S105: recording the current task number sent by each storage device 130, and judging whether a storage device 130 which does not send the current task number for a third preset time length exists.
Step S106: when the storage device 130 which does not send the current task number for the third preset time length exists, determining that the storage device 130 is down, and deleting the information of the storage device 130 recorded in the storage management device 110.
The storage management device 110 records the current task number sent by each storage device 130, so that the storage management device 110 can query when needed. The storage management device 110 determines whether there is a downtime of the storage device 130 by determining whether there is a storage device 130 that has not sent the current task number for a third preset time. When the storage management device 110 does not receive the current task number reported by one of the storage devices 130 for the third preset time period, it is determined that the storage device 130 is down, and the information of the storage device 130 recorded in the storage management device 110 is deleted, so that the information stored in the storage management device 110 is synchronized with the actual situation of each storage device 130. After the information of the storage device 130 is deleted, when the storage management device 110 reallocates the storage device 130 for the client 120, the storage device 130 is not queried.
The third preset time length is longer than the second preset time length, and optionally, the third preset time length may be an integral multiple of the second preset time length. For example, when the second preset time period is 15 seconds, the third preset time period may be 30 seconds.
In the information storage method provided in the embodiment of the present invention, it is only necessary to ensure that each storage device 130 is enough to store the information generated by each client 120 in the retention period. Thus, to conserve storage space, the capacity of each storage device 130 may be allocated as follows: each client 120 has a maximum information acquisition amount in the preset retention period, selects a maximum value of all the maximum information acquisition amounts, and takes the product of the maximum value and the number of the clients 120 as the sum of the storage capacities of all the storage devices 130.
The preset retention period is a time for which the user wants to retain information in each storage device 130, and can be adjusted according to actual situations. For example, the preset shelf life may be 30 days.
Because the information storage method is uniformly distributed according to the number of tasks, not according to the storage amount, in order to avoid data loss caused by the fact that some storage devices 130 with smaller storage capacity are full when the retention period is not reached, and in order to better realize load balancing, the capacities of the storage devices 130 in the embodiment of the invention can be equal.
According to the above capacity allocation manner, when any one of the storage devices 130 is full, it may be considered that the information storage time in the storage device 130 reaches the preset retention period.
Optionally, under the configuration, in order to prevent data loss caused by full storage of each storage device 130, a monitoring server may be provided, and the monitoring server deletes the storage content before a preset retention period before the current time in each storage device 130 every fourth preset time interval.
In practice, each storage device 130 may record the storage time when storing the information sent by each client 120, and form a file directory, which includes the storage time of all the information. The monitoring server scans the file directory once every fourth preset time interval, and deletes the storage content before the scanning time is reversed by a preset retention period, so as to avoid the storage devices 130 from being full of storage. It should be noted that the fourth preset time period is less than the retention period. For example, the preset retention period may be 30 days, and the fourth preset duration may be one day.
The capacity of each storage device 130 is allocated according to the preset retention period, and when the storage capacities of the storage devices 130 are equal, it is equivalent to equally allocate the sum of the information amounts collected by each client 120 in one preset retention period to each storage device 130 according to time.
Assuming that there are N storage devices 130 in total, when each storage device 130 is operating normally, the amount of information allocated to each storage device 130 is the sum of the amounts of information collected by the clients 120 in the preset retention period of 1/N.
However, in practical applications, a situation may occur in which a single storage device 130 cannot be used. When a single storage device 130 is down, the amount of information allocated to the storage device 130 cannot be stored, that is, the time for which the information stored in each storage device 130 can be retained is reduced by 1/N. Thus, some storage devices 130 may become full when the preset retention period has not yet been reached, and this may not be detected.
Since the information storage method provided in this embodiment allocates the target storage device to each client 120 according to the current task number of each storage device 130, and the current task number of each storage device 130 refers to the number of clients 120 in transmission connection with each storage device 130, some clients 120 may continue to select a storage device 130 that is already fully stored for information storage, thereby causing data loss.
In order to solve the above problem, optionally, the information storage method may further include: each storage device 130 sends an alarm signal to the monitoring server when full.
When the monitoring server receives the alarm signal, the monitoring server can remind the user to enable the user to check whether the storage devices 130 which are down exist, and when the storage devices 130 which are down exist, partial contents in the storage devices 130 can be manually deleted according to the number of the storage devices 130 which are down, so that data loss is avoided.
As shown in fig. 6, an information storage method according to another embodiment of the present invention is applied to a client 120, and the client 120 can communicate with a storage management device 110 and a plurality of storage devices 130. The specific process and steps shown in fig. 6 will be described in detail below.
Step S301: an information storage request is sent to the storage management device 110.
According to actual requirements, the client 120 may be an information collecting device for collecting image information, parameter information, and the like. In this embodiment, for example, the image information is collected, and the information collecting device may be a camera, a video camera, or the like, and is configured to collect image information of a vehicle passing through a traffic gate.
In implementation, a task control block may be pre-stored in the client 120, and a timer is set in the task control block. When the timer expires a first preset time length, the timer sends an information storage request to the storage management device 110, so that the storage management device 110 allocates the storage device 130 to the client 120 according to the information storage request.
Step S302: receiving the information of the target storage device sent by the storage management device 110, and establishing a transmission connection with the target storage device.
Optionally, the client 120 needs to establish a transmission connection with one storage device 130 within each first preset time period, regardless of whether the client 120 has collected information within the preset time period. Through the design, the information collected by the client 120 can be uniformly distributed in each storage device 130 according to time.
Step S303: monitoring the duration of the transmission connection, releasing the transmission connection when the duration of the transmission connection reaches a first preset duration, and sending a new information storage request to the storage management device 110 again.
In practice, the client 120 may monitor the duration of the transmission connection through the timer. The timer may start timing from a time when the client 120 sends a request to the storage management device 110, or may start timing from a time when the client 120 establishes a transmission connection with the target storage device, and since a time difference between two times is in the order of milliseconds, specifically, starting timing from which time has no influence on implementation of the present solution, this embodiment does not limit this.
When the timer expires the first preset duration, the client 120 releases the transmission connection and resends a new information storage request to the storage management device 110, and the timer restarts counting.
That is, the client 120 needs to send an information storage request to the storage management device 110 once every a first preset time interval, so that the storage management device 110 reallocates the target storage device to the client 120 according to the current task number of each storage device 130, and thus the information to be stored of the client 120 can be uniformly distributed in each storage device 130 according to time.
According to actual needs, optionally, the client 120 may include an exchange server and a plurality of information collecting devices communicatively connected to the exchange server.
In the above case, as shown in fig. 7, the step S301 may include three substeps, i.e., a step S401, a step S402, and a step S403.
Step S401: the exchange server acquires the identity information of the information acquisition equipment.
Step S402: and the exchange server searches out the task control block corresponding to the identity information according to the identity information.
When the method is implemented, the plurality of information acquisition devices send acquired information to the exchange server for storage, wherein the information carries the identity information of each information acquisition device. The exchange server stores a plurality of task control blocks and the corresponding relation between each task control block and the identity information of each information acquisition device, and the corresponding relation can be stored in a form of a hash table.
Each task control block corresponds to one information acquisition device, and each time the exchange server receives information sent by the information acquisition device, the exchange server needs to query the task control block corresponding to the identity information of the information acquisition device. And if the task control block corresponding to the identity information of the information acquisition equipment cannot be found, establishing a new task control block corresponding to the identity information. Optionally, the identity information may be encoded information of the information collecting apparatus.
According to actual needs, when implemented, the number of the clients 120 may be multiple, and multiple clients 120 are communicatively connected to the storage management device 110 through the network 200.
When the client 120 is an information collecting device, the distributed file system 100 includes a storage management device 110, a plurality of storage devices 130, and a plurality of information collecting devices. When the client 120 includes a switching server and a plurality of information collecting devices communicatively connected to the switching server, the distributed file system 100 includes a storage management device 110, a plurality of storage devices 130, a plurality of switching servers, and a plurality of information collecting devices, the plurality of information collecting devices being divided into a plurality of groups, each group of information collecting devices being communicatively connected to one of the switching servers.
Step S403: the searched task control block sends an information storage request to the storage management device 110.
Through the design, the information collected by the client 120 can be uniformly distributed to each storage device 130 for storage according to the number of tasks, that is, according to the time, so that load balancing is better realized.
Referring to fig. 8, a functional block diagram of a storage management device 110 according to an embodiment of the present invention is shown, where the storage management device 110 is capable of communicating with a client 120 and a plurality of storage devices 130.
The storage management device 110 includes a storage request receiving module 1101, a storage target selecting module 1102, and a target information sending module 1103.
The storage request receiving module 1101 is configured to receive an information storage request sent by the client 120 every interval of a first preset duration. The detailed description about the storage request receiving module 1101 may specifically refer to the description of step S101 shown in fig. 3, that is, step S101 may be performed by the storage request receiving module 1101.
The storage target selecting module 1102 is configured to select, according to a preset rule, one storage device 130 as a target storage device of the client 120 for each information storage request. The detailed description of the storage target selection module 1102 may specifically refer to the description of step S102 shown in fig. 3, that is, step S102 may be performed by the storage target selection module 1102.
The target information sending module 1103 is configured to send the information of the target storage device to the client 120, so that the client 120 establishes a transmission connection with the target storage device. The detailed description about the object information sending module 1103 may specifically refer to the description of step S103 shown in fig. 3, that is, step S103 may be performed by the object information sending module 1103.
Optionally, the storage management device 110 may further include a task number receiving module 1104 and a report monitoring module 1105.
The task number receiving module 1104 is configured to receive the current task number sent by each storage device 130 every second preset time interval. The detailed description about the task number receiving module 1104 may specifically refer to the description of step S104 shown in fig. 5, that is, step S104 may be performed by the task number receiving module 1104.
The report monitoring module 1105 is configured to record the current task number sent by each storage device 130, determine whether there is a storage device 130 that does not send the current task number for a third preset time, determine that the storage device 130 is down when there is a storage device 130 that does not send the current task number for the third preset time, and delete the information of the storage device 130 recorded in the storage management device 110. And the third preset time length is greater than the second preset time length.
For the detailed description of the reporting monitor module 1105, reference may be made to the description of the step S105 and the step S106 shown in fig. 5, that is, the step S105 and the step S106 may be executed by the reporting monitor module 1105.
Optionally, the storage management device 110 stores information records of the storage device 130 that establishes transmission connection with the client 120 within each first preset time period. As shown in fig. 9, the storage target selecting module 1102 includes an information record querying sub-module 201, a first selecting sub-module 202, and a second selecting sub-module 203.
The information record query submodule 201 is configured to query whether there is a storage device 130 that has not established a transmission connection with the client 120 according to the information record. The detailed description about the information record query submodule 201 may specifically refer to the description of step S201 shown in fig. 4, that is, step S201 may be performed by the information record query submodule 201.
The first selecting sub-module 202 is configured to, when there is no storage device 130 that has not established a transmission connection with the client 120, delete the information record of each storage device 130 that has established a transmission connection with the client 120 within the first preset time duration in the storage management device 110, and use the storage device 130 with the smallest current task number as the target storage device of the client 120. The detailed description about the first selection sub-module 202 may specifically refer to the description about the step S202 shown in fig. 4, that is, the step S202 may be executed by the first selection sub-module 202.
The second selecting submodule 203 is configured to, when there is a storage device 130 that has not established a transmission connection with the client 120, use the storage device 130 that has the smallest current task number in the storage devices 130 that have not established a transmission connection with the client 120 as a target storage device of the client 120. The detailed description about the second selection submodule 203 may specifically refer to the description about the step S203 shown in fig. 4, that is, the step S203 may be performed by the second selection submodule 203.
Referring to fig. 10, a functional module block diagram of a client 120 according to another embodiment of the present invention is provided, where the client 120 is capable of communicating with a storage management device 110 and a plurality of storage devices 130, and the client 120 includes a storage request sending module 121, a transmission connection establishing module 122, and a transmission connection monitoring module 123.
The storage request sending module 121 is configured to send an information storage request to the storage management device 110. The detailed description about the storage request sending module 121 may specifically refer to the description of step S301 shown in fig. 6, that is, step S301 may be executed by the storage request sending module 121.
The transmission connection establishing module 122 is configured to receive information of the target storage device sent by the storage management device 110, and establish a transmission connection with the target storage device. The detailed description of the transmission connection establishing module 122 may specifically refer to the description of step S302 shown in fig. 6, that is, step S302 may be performed by the transmission connection establishing module 122.
The transmission connection monitoring module 123 is configured to monitor a duration of the transmission connection, release the transmission connection when the duration of the transmission connection reaches a first preset duration, and send a new information storage request to the storage management device 110 again. The detailed description about the transmission connection monitoring module 123 may specifically refer to the description of step S303 shown in fig. 6, that is, step S303 may be performed by the transmission connection monitoring module 123.
Alternatively, the client 120 may include a switching server including a plurality of task control blocks and a plurality of information gathering devices in communication with the switching server. At this time, as shown in fig. 11, the storage request module may include an identity acquisition sub-module 1211, a lookup sub-module 1212, and a request sending sub-module 1213, which are stored in the switching server.
The identity obtaining submodule 1211 is configured to obtain identity information of the information acquisition device. The detailed description about the identity acquisition submodule 1211 can refer to the description of step S401 shown in fig. 7, that is, step S401 can be performed by the identity acquisition submodule 1211.
The searching sub-module 1212 is configured to search, according to the identity information, a task control block corresponding to the identity information, where a correspondence between the plurality of task control blocks and the identity information of the plurality of information acquisition devices is pre-stored in the exchange server. The detailed description about the lookup sub-module 1212 specifically refers to the description about the step S402 shown in fig. 7, that is, the step S402 may be executed by the lookup sub-module 1212.
The request sending sub-module 1213 is configured to control the searched task control block to send an information storage request to the storage management device 110. The detailed description about the request transmission submodule 1213 may specifically refer to the description of step S403 shown in fig. 7, that is, step S403 may be performed by the request transmission submodule 1213.
In summary, the information storage method, the storage management device 110, and the client 120 provided in the embodiments of the present invention allocate the target storage device to the client 120 according to the preset rule, so that the information to be stored in each client 120 can be uniformly distributed in each storage device 130 according to time, and the problem of the prior art that the dependency on the data distribution device is large is avoided.
In the embodiments provided in the present invention, it should be understood that the disclosed system and method can be implemented in other ways. The system and method embodiments described above are merely illustrative, for example, the flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
In addition, the functional modules in the embodiments of the present invention may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.
The functions, if implemented in the form of software functional modules and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, an electronic device, or a network device) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes. It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
The above description is only for the specific embodiments of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present invention, and all the changes or substitutions should be covered within the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (8)
1. An information storage method applied to a storage management device, wherein the storage management device can communicate with a client and a plurality of storage devices, the method comprising:
receiving an information storage request sent by the client at intervals of a first preset duration; the storage management device stores information records of storage devices which establish transmission connection with the client within each first preset time length;
for each information storage request, inquiring whether storage equipment which is not in transmission connection with the client exists or not according to the information record;
when no storage device which is not in transmission connection with the client exists, deleting the information record of the storage device which is in transmission connection with the client within each first preset time length in the storage management device, and taking the storage device with the minimum current task number as the target storage device of the client; the current task number refers to the number of clients which are in current transmission connection with the storage device;
when a storage device which does not establish transmission connection with the client exists, taking the storage device which does not establish transmission connection with the client as a target storage device of the client;
and sending the information of the target storage equipment to the client so as to enable the client to establish transmission connection with the target storage equipment.
2. The information storage method of claim 1, further comprising:
receiving the current task number sent by each storage device every second preset time interval;
recording the current task number sent by each storage device, judging whether a storage device which does not send the current task number for a third preset time length exists, judging that the storage device is down when the storage device which does not send the current task number for the third preset time length exists, and deleting the information of the storage device recorded in the storage management device, wherein the third preset time length is longer than the second preset time length.
3. An information storage method is applied to a client and is characterized in that the client can communicate with a storage management device and a plurality of storage devices, wherein the storage management device stores information records of the storage devices which establish transmission connection with the client within each first preset time length; the method comprises the following steps:
sending an information storage request to the storage management device;
receiving information of a target storage device sent by the storage management device, and establishing transmission connection with the target storage device;
monitoring the duration of the transmission connection holding, releasing the transmission connection when the duration of the transmission connection holding reaches a first preset duration, and resends a new information storage request to the storage management device, so that when the storage management device receives the information storage request, inquiring whether a storage device which does not establish transmission connection with the client exists according to the information record, when no storage device which is not in transmission connection with the client exists, deleting the information record of the storage device which is in transmission connection with the client within each first preset time length in the storage management device, and taking the storage device with the minimum current task number as the target storage device of the client, and when the storage device which does not establish transmission connection with the client exists, taking the storage device which does not establish transmission connection with the client as a target storage device of the client.
4. The information storage method according to claim 3, wherein the client includes a switching server and a plurality of information collecting devices communicating with the switching server, the switching server including a plurality of task control blocks;
the step of sending an information storage request to the storage management device includes:
the exchange server acquires identity information of the information acquisition equipment;
the exchange server searches out task control blocks corresponding to the identity information according to the identity information, wherein the exchange server is pre-stored with corresponding relations between the plurality of task control blocks and the identity information of the plurality of information acquisition devices; and
and the searched task control block sends an information storage request to the storage management equipment.
5. The storage management equipment is characterized by being capable of communicating with a client and a plurality of storage equipment and comprising a storage request receiving module, a storage target selecting module and a target information sending module;
the storage request receiving module is used for receiving information storage requests sent by the client every first preset time interval; the storage management device stores information records of storage devices which establish transmission connection with the client within each first preset time length;
the storage target selection module is configured to:
for each information storage request, inquiring whether storage equipment which is not in transmission connection with the client exists or not according to the information record; when no storage device which is not in transmission connection with the client exists, deleting the information record of the storage device which is in transmission connection with the client within each first preset time length in the storage management device, and taking the storage device with the minimum current task number as the target storage device of the client; the current task number refers to the number of clients which are in current transmission connection with the storage device; when a storage device which does not establish transmission connection with the client exists, taking the storage device which does not establish transmission connection with the client as a target storage device of the client; and
and the target information sending module is used for sending the information of the target storage equipment to the client so as to enable the client to establish transmission connection with the target storage equipment.
6. The storage management device according to claim 5, wherein the storage management device further comprises a task number receiving module and a report monitoring module;
the task number receiving module is used for receiving the current task number sent by each storage device at intervals of second preset time; and
the report monitoring module is configured to record the current task number sent by each storage device, determine whether there is a storage device for which the third preset time length does not send the current task number, determine that the storage device is down when there is a storage device for which the third preset time length does not send the current task number, and delete the information of the storage device recorded in the storage management device, where the third preset time length is greater than the second preset time length.
7. The client is characterized in that the client can communicate with a storage management device and a plurality of storage devices, wherein information records of the storage devices which establish transmission connection with the client within each first preset time length are stored in the storage management device; the client comprises a storage request sending module, a transmission connection establishing module and a transmission connection monitoring module;
the storage request sending module is used for sending an information storage request to the storage management equipment;
the transmission connection establishing module is used for receiving the information of the target storage device sent by the storage management device and establishing transmission connection with the target storage device; and
the transmission connection monitoring module is configured to monitor a duration of the transmission connection, release the transmission connection when the duration of the transmission connection reaches a first preset duration, and send a new information storage request to the storage management device again, so that the storage management device, when receiving the information storage request, queries whether there is a storage device that has not established a transmission connection with the client according to the information record, when there is no storage device that has not established a transmission connection with the client, deletes the information record of the storage device that has established a transmission connection with the client within each first preset duration in the storage management device, and uses the storage device with the smallest number of current tasks as a target storage device of the client, when there is a storage device that has not established a transmission connection with the client, and taking the storage equipment which has not established transmission connection with the client as the client to obtain target storage equipment.
8. The client of claim 7, wherein the client comprises a switching server and a plurality of information gathering devices in communication with the switching server, wherein the switching server comprises a plurality of task control blocks;
the storage request sending module comprises an identity acquisition sub-module, a search sub-module and a request sending sub-module;
the identity acquisition submodule is used for acquiring identity information of the information acquisition equipment;
the search submodule is used for searching the task control blocks corresponding to the identity information according to the identity information, wherein the corresponding relations between the plurality of task control blocks and the identity information of the plurality of information acquisition devices are prestored in the exchange server;
and the request sending submodule is used for controlling the searched task control block to send an information storage request to the storage management equipment.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610830142.XA CN106383668B (en) | 2016-09-18 | 2016-09-18 | Information storage method, storage management equipment and client |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610830142.XA CN106383668B (en) | 2016-09-18 | 2016-09-18 | Information storage method, storage management equipment and client |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106383668A CN106383668A (en) | 2017-02-08 |
| CN106383668B true CN106383668B (en) | 2020-06-19 |
Family
ID=57936559
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610830142.XA Active CN106383668B (en) | 2016-09-18 | 2016-09-18 | Information storage method, storage management equipment and client |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106383668B (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110519310B (en) * | 2018-05-21 | 2021-12-28 | 浙江宇视科技有限公司 | Storage task allocation method and device |
| CN112241239B (en) * | 2019-07-18 | 2023-03-14 | 浙江宇视科技有限公司 | Storage task management and control method, storage task management and control device and storage equipment |
| CN111193804B (en) * | 2020-01-02 | 2022-09-09 | 深圳市网心科技有限公司 | Distributed storage method and device, network node and storage medium |
| CN111600395A (en) * | 2020-07-23 | 2020-08-28 | 南京万形电气有限公司 | Low-voltage distribution network topology identification method and system |
| CN112235809B (en) * | 2020-08-14 | 2023-07-28 | 赛特斯信息科技股份有限公司 | 5G inter-base station communication method and communication device |
| CN115938013B (en) * | 2023-03-14 | 2023-06-13 | 禾多科技（北京）有限公司 | Method, apparatus, device and computer readable medium for monitoring data |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102207891A (en) * | 2011-06-10 | 2011-10-05 | 浙江大学 | Method for achieving dynamic partitioning and load balancing of data-partitioning distributed environment |
| CN102262680A (en) * | 2011-08-18 | 2011-11-30 | 北京新媒传信科技有限公司 | Distributed database proxy system based on massive data access requirement |
| CN105450679A (en) * | 2014-08-06 | 2016-03-30 | 杭州海康威视系统技术有限公司 | Method and system for performing data cloud storage |
-
2016
- 2016-09-18 CN CN201610830142.XA patent/CN106383668B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102207891A (en) * | 2011-06-10 | 2011-10-05 | 浙江大学 | Method for achieving dynamic partitioning and load balancing of data-partitioning distributed environment |
| CN102262680A (en) * | 2011-08-18 | 2011-11-30 | 北京新媒传信科技有限公司 | Distributed database proxy system based on massive data access requirement |
| CN105450679A (en) * | 2014-08-06 | 2016-03-30 | 杭州海康威视系统技术有限公司 | Method and system for performing data cloud storage |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106383668A (en) | 2017-02-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106383668B (en) | Information storage method, storage management equipment and client | |
| CN108683720B (en) | Container cluster service configuration method and device | |
| EP3221795B1 (en) | Service addressing in distributed environment | |
| US10678757B2 (en) | Reliable generation of a unique key in a distributed system | |
| EP3419247A1 (en) | Method and device for storage resource allocation for video cloud storage | |
| CN106708697B (en) | Method and device for detecting application program used by user | |
| CN107656807B (en) | Automatic elastic expansion method and device for virtual resources | |
| CN108121511B (en) | Data processing method, device and equipment in distributed edge storage system | |
| JP2004206495A (en) | Management system, management computer, management method, and program | |
| CN110888889A (en) | Data information updating method, device and equipment | |
| JP2015191523A (en) | Configuration management apparatus, configuration management system, and configuration management program | |
| US20160344582A1 (en) | Call home cluster | |
| CN112243044B (en) | Container address allocation method and device | |
| CN105357584A (en) | User behavior log classification uploading method and system based on intelligent television | |
| KR20130093202A (en) | Method of distributed indexing and searching for effective analysis of time series data in search system | |
| CN112506870A (en) | Data warehouse increment updating method and device and computer equipment | |
| CN108664321B (en) | System resource allocation adjusting method and device | |
| CN111694505B (en) | Data storage management method, device and computer readable storage medium | |
| CN114564450A (en) | Processing method, device, system, medium and equipment of distributed file system | |
| CN114745358A (en) | IP address management method, system and controller in load balancing service | |
| EP2776941B1 (en) | Managing capacity in a data center by suspending tenants | |
| CN110543410A (en) | Method for processing cluster index, method and device for inquiring cluster index | |
| CN111352746B (en) | Message flow limiting method and storage medium | |
| US20170149893A1 (en) | Metadata server, network device and automatic resource management method | |
| CN113965538B (en) | Equipment state message processing method, device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20221111 Address after: 15 / F, Zhongchuang Plaza, 385 Hangtian Middle Road, national civil aerospace industrial base, Xi'an City, Shaanxi Province 710100 Patentee after: Xi'an Yu vision Mdt InfoTech Ltd. Address before: 310000 1-11 / F, South Block, building 10, No. 88, Jiangling Road, Xixing street, Binjiang District, Hangzhou City, Zhejiang Province Patentee before: ZHEJIANG UNIVIEW TECHNOLOGIES Co.,Ltd. |
|
| TR01 | Transfer of patent right |