CN117909311A - Data request method, device and storage medium - Google Patents

Data request method, device and storage medium Download PDF

Info

Publication number
CN117909311A
CN117909311A CN202410033851.XA CN202410033851A CN117909311A CN 117909311 A CN117909311 A CN 117909311A CN 202410033851 A CN202410033851 A CN 202410033851A CN 117909311 A CN117909311 A CN 117909311A
Authority
CN
China
Prior art keywords
storage device
client
storage
storage devices
data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202410033851.XA
Other languages
Chinese (zh)
Inventor
唐蜜
过晓春
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
China United Network Communications Group Co Ltd
Unicom Digital Technology Co Ltd
Unicom Cloud Data Co Ltd
Original Assignee
China United Network Communications Group Co Ltd
Unicom Digital Technology Co Ltd
Unicom Cloud Data Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by China United Network Communications Group Co Ltd, Unicom Digital Technology Co Ltd, Unicom Cloud Data Co Ltd filed Critical China United Network Communications Group Co Ltd
Priority to CN202410033851.XA priority Critical patent/CN117909311A/en
Publication of CN117909311A publication Critical patent/CN117909311A/en
Pending legal-status Critical Current

Links

Abstract

The application provides a data request method, a data request device and a storage medium, relates to the technical field of information processing, and is used for reducing response time of data reading and improving data reading efficiency. The method comprises the following steps: the device type of each first storage device is obtained. Under the condition that the device types of the plurality of first storage devices are different, acquiring the reading performance of each first storage device, and determining the first storage device with the reading performance meeting the preset condition as a target storage device; in the case where the reading performance is the reading speed, the preset condition is that the reading speed is greater than the preset speed threshold. And sending a read request message to the target storage device, wherein the read request message is used for requesting target data.

Description

Data request method, device and storage medium
Technical Field
The present application relates to the field of information processing technologies, and in particular, to a data request method, apparatus, and storage medium.
Background
The hadoop distributed file system (hadoop distributed FILE SYSTEM, HDFS) may provide file storage capability for large data services. The name node (naminode) in HDFS is responsible for assigning and querying the data node (datanode) to which the data corresponds. When the client writes data, N data nodes can be allocated to store the data; when a client reads data, the data can be read from a certain data node according to a certain strategy.
At present, when a client reads data, according to the number of node devices between each data node storing the data and the client in a network topology diagram, the client can select to send a read request to the data node with the minimum number of node devices between the client and the data node. However, the HDFS cluster may use storage devices with different read-write capabilities as data nodes, such as HDDs, SSDs, magnetic tapes, etc. The client selects the data node according to the network distance only, which may result in longer response time for reading data and lower efficiency for reading data.
Disclosure of Invention
The application provides a data request method, a data request device and a storage medium, which are used for reducing response time of data reading and improving data reading efficiency.
In order to achieve the above purpose, the application adopts the following technical scheme:
In a first aspect, the present application provides a data request method. The method is applied to a client, the client is connected with a plurality of first storage devices, the plurality of first storage devices store target data, and the number of node devices between each first storage device and the client is the same. The method comprises the following steps: the client obtains the device type of each first storage device. And under the condition that the device types of the plurality of first storage devices are different, the client acquires the reading performance of each first storage device, and determines the first storage device with the reading performance meeting the preset condition as the target storage device. In the case where the reading performance is the reading speed, the preset condition is that the reading speed is greater than the preset speed threshold. The client sends a read request message to the target storage device, the read request message being used to request target data.
The technical scheme provided by the application has at least the following beneficial effects: the client obtains the device type of each first storage device. Under the condition that the device types of the plurality of first storage devices are different, the client can acquire the reading performance of each first storage device, and determine the first storage device with the reading performance meeting the preset condition as the target storage device. In the case where the reading performance is the reading speed, the preset condition is that the reading speed is greater than the preset speed threshold. The client may send a read request message to the target storage device, the read request message requesting the target data. Therefore, when the client reads data, the storage device with better reading performance can be selected, the response time for reading the data is reduced, and the data reading efficiency is improved.
Optionally, the client is connected to a plurality of second storage devices, the plurality of second storage devices including a plurality of first storage devices. The data request method further comprises the following steps: the client acquires the number of node devices between each second storage device and the client. And the client determines a plurality of first storage devices from the plurality of second storage devices according to the number of node devices between each second storage device and the client, wherein the first storage device is the storage device with the minimum number of node devices between the plurality of second storage devices and the client.
Optionally, the "acquiring the device type of each first storage device" includes: the client receives a heartbeat message from each first storage device, the heartbeat message including a device type of the first storage device.
Optionally, the device type includes at least one of: random access memory RAM, solid state disk SSD, hard disk drive HDD, archive tape device.
In a second aspect, the present application provides a data requesting apparatus, where the apparatus is applied to a client, and the client is connected to a plurality of first storage devices, where the plurality of first storage devices each store target data, and the number of node devices between each first storage device and the client is the same. The device comprises: the device comprises an acquisition module, a processing module and a sending module.
And the acquisition module is used for acquiring the device type of each first storage device. The acquisition module is further used for acquiring the reading performance of each first storage device under the condition that the device types of the plurality of first storage devices are different. The processing module is used for determining the first storage device with the reading performance meeting the preset condition as the target storage device; in the case where the reading performance is the reading speed, the preset condition is that the reading speed is greater than the preset speed threshold. And the sending module is used for sending a reading request message to the target storage device, wherein the reading request message is used for requesting target data.
Optionally, the client is connected to a plurality of second storage devices, the plurality of second storage devices including a plurality of first storage devices. The acquisition module is further used for acquiring the number of node devices between each second storage device and the client. The processing module is further configured to determine, from the plurality of second storage devices, a plurality of first storage devices according to the number of node devices between each second storage device and the client, where the first storage device is a storage device with a minimum number of node devices between the plurality of second storage devices and the client.
Optionally, the acquiring module is specifically configured to receive a heartbeat message from each first storage device, where the heartbeat message includes a device type of the first storage device.
Optionally, the device type includes at least one of: random access memory RAM, solid state disk SSD, hard disk drive HDD, archive tape device.
In a third aspect, the present application provides a data requesting apparatus, comprising: a processor and a memory. The processor and the memory are coupled. The memory is for storing one or more programs, the one or more programs comprising computer-executable instructions that, when executed by the data requesting device, the processor executes the computer-executable instructions stored by the memory to implement the data requesting method as described in any one of the possible implementations of the first aspect.
In a fourth aspect, the present application provides a computer readable storage medium having instructions stored therein which, when run on a computer, cause the computer to perform the data request method described in any one of the possible implementations of the first aspect.
In a fifth aspect, the application provides a computer program product comprising a computer program which, when executed by a processor, causes the computer to implement a data request method as described in any one of the possible implementations of the first aspect.
In the foregoing solutions, the technical problems and the achieved technical effects that can be solved by the data requesting apparatus, the computer device, the computer storage medium, or the computer program product may be referred to the technical problems and the technical effects that can be solved by the foregoing first aspect, and are not described herein again.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and together with the description, serve to explain the principles of the application and do not constitute a undue limitation on the application.
FIG. 1 is a schematic diagram of an architecture of a data request system, shown in accordance with an exemplary embodiment;
FIG. 2 is a flowchart illustrating a method of data request, according to an example embodiment;
FIG. 3 is a schematic diagram illustrating a number of node devices between a storage device and a client, according to an example embodiment;
FIG. 4 is a flowchart illustrating another data request method according to an example embodiment;
FIG. 5 is a schematic diagram illustrating the number of node devices between another storage device and a client, according to an example embodiment;
FIG. 6 is a block diagram illustrating a data requesting device according to an exemplary embodiment;
FIG. 7 is a schematic diagram of a data requesting device, according to an example embodiment;
fig. 8 is a conceptual partial view of a computer program product according to an exemplary embodiment.
Detailed Description
The following description of the embodiments of the present application will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
The character "/" herein generally indicates that the associated object is an "or" relationship. For example, A/B may be understood as A or B.
The terms "first" and "second" in the description and in the claims of the application are used for distinguishing between different objects and not for describing a particular sequential order of objects.
Furthermore, references to the terms "comprising" and "having" and any variations thereof in the description of the present application are intended to cover a non-exclusive inclusion. For example, a process, method, system, article, or apparatus that comprises a list of steps or modules is not limited to only those steps or modules but may include other steps or modules not listed or inherent to such process, method, article, or apparatus.
In addition, in the embodiments of the present application, words such as "exemplary" or "such as" are used to mean serving as examples, illustrations, or descriptions. Any embodiment or design described herein as "exemplary" or "e.g." should not be taken as preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "such as" is intended to present concepts in a concrete fashion.
For ease of understanding, the terms involved in the embodiments of the present application will be described first.
The hadoop distributed file system (hadoop distributed FILE SYSTEM, HDFS) is a distributed file system in the apache hadoop ecosystem for storing and managing large-scale datasets.
A name node (name node) is a metadata node of the HDFS file system, and is responsible for managing the namespace and the data block mapping information of the file system, maintaining the metadata and the data block replication state of the file system, and the like.
The data node (datanode) is a data node of the HDFS file system and is responsible for storing and managing data blocks and executing read-write operations of the data blocks.
Before describing the data request method of the embodiment of the present application in detail, the implementation environment and application field Jing Jinhang of the embodiment of the present application will be described.
The hadoop distributed file system (hadoop distributed FILE SYSTEM, HDFS) may provide file storage capability for large data services. The name node (naminode) in HDFS is responsible for assigning and querying the data node (datanode) to which the data corresponds. When the client writes data, N data nodes can be allocated to store the data; when a client reads data, the data can be read from a certain data node according to a certain strategy.
At present, when a client reads data, according to the number of node devices between each data node storing the data and the client in a network topology diagram, the client can select to send a read request to the data node with the minimum number of node devices between the client and the data node. However, the HDFS cluster may use storage devices with different read-write capabilities as data nodes, such as HDDs, SSDs, magnetic tapes, etc. The client selects the data node according to the network distance only, which may result in longer response time for reading data and lower efficiency for reading data.
In order to solve the above problems, an embodiment of the present application provides a data request method, including: the client obtains the device type of each first storage device. Under the condition that the device types of the plurality of first storage devices are different, the client can acquire the reading performance of each first storage device, and determine the first storage device with the reading performance meeting the preset condition as the target storage device. In the case where the reading performance is the reading speed, the preset condition is that the reading speed is greater than the preset speed threshold. The client may send a read request message to the target storage device, the read request message requesting the target data. Therefore, when the client reads data, the storage device with better reading performance can be selected, the response time for reading the data is reduced, and the data reading efficiency is improved.
The following describes an implementation environment of an embodiment of the present application.
FIG. 1 is a schematic architecture diagram of a data request system, according to an example embodiment. The architecture comprises: a client 101, a target server 102, a plurality of second storage devices (e.g., second storage device 103, second storage device 104, second storage device 105). The client 101 and the target server 102 may be in wired/wireless communication, the client 101 and each second storage device may be in wired/wireless communication, and the target server 102 and each second storage device may be in wired/wireless communication.
The client 101 may send a read request message to the second storage device, where the read request message is used to request the target data. The client 101 may receive a device type from a storage device of the target server 102 and may also receive a network topology map from the target server 102.
It should be noted that, in the embodiment of the present application, the client may perform data transmission with the target server. Wherein the client may be a program that provides local services to the user. For example, the client may be a locally running application. For another example, the client may be a web browser used by the world wide web. For another example, the client may be instant messaging software. The client may be deployed on an electronic device or on other hardware devices.
The server (e.g., target server 102) may be a single physical server or may be a server cluster of multiple servers. Or the server cluster may also be a distributed cluster. Or the server may be a cloud server. The embodiment of the application does not limit the specific implementation mode of the server.
The target server 102 may act as a name node (naminode) for assigning and querying data nodes (datanode) to which the data corresponds. The target server 102 stores a device type of each of the plurality of second storage devices, and a network topology map. The target server 102 may send the device type of the plurality of second storage devices and the network topology map to the client.
The plurality of second storage devices includes a plurality of first storage devices. A second storage device acts as a data node (datanode) for storing the target data. The second storage device may also send the device type of the second storage device to the target server.
Embodiments of the present application will be described in detail below with reference to the drawings attached to the specification. As shown in fig. 2, a data request method provided in an embodiment of the present application includes:
S201, the client acquires the equipment type of each first storage equipment.
In the embodiment of the application, the client is connected with a plurality of first storage devices, and the plurality of first storage devices all store target data. The number of node devices between each first storage device and the client is the same.
Optionally, the network distance between each first storage device and the client is the same, and the network distance is the number of links between the first storage device and the client in the network topology map.
Illustratively, as shown in FIG. 3, a schematic diagram of the number of node devices between a storage device and a client is shown. If the plurality of first storage devices includes: first storage device a, first storage device B, first storage device C. The client is connected with the first storage device A through the router I, the router II and the router III, the client is connected with the first storage device B through the router I, the router II and the router IV, and the client is connected with the first storage device C through the router I, the router II and the router IV. Therefore, the number of node devices between the first storage device a, the first storage device B, the first storage device C and the client is the same, i.e., the number of node devices is 3. The number of links between the client and the first storage device a, the first storage device B and the first storage device C is the same, i.e. the network distance is 4.
In one possible implementation, for each first storage device, the first storage device may send a heartbeat message to the client according to a preset period, where the heartbeat message includes a device type of the first storage device. The client may receive heartbeat messages from each first storage device to obtain the device type of each first storage device.
Specifically, the first storage device may serve as a data node (datanode), and the target server serves as a name node (naminode). A target server is connected to each first storage device and the target server is connected to the client. The first storage device may send a heartbeat message to the target server at a preset period. The target server may receive heartbeat messages from the first storage devices to obtain the device types of each of the first storage devices. The target server may send the device type of each first storage device to the client. The client may receive the device type of each first storage device from the target server to obtain the device type of each first storage device.
Note that, in the embodiment of the present application, datanode carries the device types of the respective nodes when sending the heartbeat message to the naminode periodically, and the naminode may store the device types sent by datanode in a structure representing datanode information in an internal memory (i.e. datanodeInfo).
In another possible implementation, for each first storage device, the client may send a type request message to the first storage device, the type request message requesting a device type of the first storage device. The first storage device may receive a type request message from the client. In response to the type request message, the first storage device may send a device type of the first storage device to the client. The client may receive a device type of each first storage device to obtain the device type of each first storage device.
In one possible design, the device type includes at least one of the following: random access memory (random access memory, RAM), solid State Disk (SSD) hard disk STATE DRIVES, hard disk drive (hard-DISK DRIVE, HDD), archive tape device.
In some embodiments, the client may perform S202-S203 in the case that the device types of the plurality of first storage devices are different.
S202, the client acquires the reading performance of each first storage device, and determines the first storage device with the reading performance meeting the preset condition as the target storage device.
In one possible implementation, the target server stores the read performance of a plurality of preset storage devices, including a plurality of first storage devices. The target server may read the performance of each first storage device to the client. The client may receive the read performance of each first storage device from the target server to obtain the read performance of each first storage device.
Under the condition that the reading performance is the reading speed, the preset condition is that the reading speed is larger than a preset speed threshold.
In the embodiment of the application, under the condition that the reading performance is the reading speed, for the reading speed of each first storage device, the client can compare the reading speed of the first storage device with the preset speed threshold to determine whether the reading speed of the first storage device is greater than the preset speed threshold.
In one possible design, the client may determine the first storage device as the target storage device if the read speed of the first storage device is greater than a preset speed threshold.
Alternatively, the preset condition may be that the reading speed is maximum. If the reading speed of the first storage device is the largest, the client may determine the first storage device as the target storage device.
Illustratively, provided that the plurality of first storage devices includes: first storage device a, first storage device B, first storage device C. Wherein the device type of the first storage device A is HDD, and the reading speed is 120 megabits per second; the device type of the first storage device B is SSD, and the reading speed is 480 megabits per second; the device type of the first storage device C is an archive tape device and the read speed is 85 mega per second. If the preset speed threshold is 400 mega per second, the first storage device B is the target storage device.
S203, the client sends a read request message to the target storage device.
Wherein the read request message is used to request the target data.
In the embodiment of the application, after the client sends the read request message to the target storage device, the target storage device may receive the read request message from the client. In response to the read request message, the target storage device may send target data to the client.
It is understood that the client obtains the device type of each first storage device. Under the condition that the device types of the plurality of first storage devices are different, the client can acquire the reading performance of each first storage device, and determine the first storage device with the reading performance meeting the preset condition as the target storage device. In the case where the reading performance is the reading speed, the preset condition is that the reading speed is greater than the preset speed threshold. The client may send a read request message to the target storage device, the read request message requesting the target data. Therefore, when the client reads data, the storage device with better reading performance can be selected, the response time for reading the data is reduced, and the data reading efficiency is improved.
In some embodiments, as shown in fig. 4, before the client obtains the device type of each first storage device, the data request method may further include: S401-S402.
S401, the client acquires the number of node devices between each second storage device and the client.
In one possible implementation, the target server stores the number of node devices between each second storage device and the client. The target server may send the number of node devices between each second storage device and the client to the client. The client may receive the number of node devices between each second storage device from the target server and the client.
In another possible implementation, the target server stores a network topology map, the network topology map including: a client and a plurality of second storage devices. The target server may send the network topology map to the client. The client may receive the network topology map from the target server. The client may determine the number of node devices between each second storage device and the client based on the network topology.
S402, the client determines a plurality of first storage devices from the plurality of second storage devices according to the number of node devices between each second storage device and the client.
The first storage device is the storage device with the minimum number of node devices between the plurality of second storage devices and the client.
Illustratively, provided that the plurality of second storage devices includes: a second storage device D, a second storage device E, and a second storage device F. The client is connected with the second storage device D through the first router, the client is connected with the second storage device E through the first router, and the client is connected with the second storage device F through the first router, the second router and the third router. Therefore, the number of node devices between the second storage device D, the second storage device E, and the client is 1, and the number of node devices between the second storage device F and the client is 3. The number of node devices between the second storage device D, the second storage device E, and the client is the smallest, and the plurality of first storage devices includes: a second storage device D, a second storage device E.
In the embodiment of the present application, if the number of first storage devices determined from the plurality of second storage devices is 1, the client may use the first storage device as the target storage device.
Exemplary, as shown in fig. 5, a schematic diagram of the number of node devices between another storage device and a client is shown. If the plurality of second storage devices includes: a second storage device D, a second storage device E, and a second storage device F. The client is connected with the second storage device D through the first router, the second router and the third router, the second storage device E through the first router, the second router and the third router, and the second storage device F through the first router, the second router and the third router. Therefore, the number of node devices between the second storage device D and the client is 1, and the number of node devices between the second storage device E, the second storage device F, and the client is 3. And if the number of node devices between the second storage device D and the client is minimum, the second storage device D serves as the first storage device. Since the first storage device only has the second storage device D, the client may then use the second storage device D as the target storage device.
It will be appreciated that the client may obtain the number of node devices between each second storage device and the client. The client may determine a plurality of first storage devices from the plurality of second storage devices based on a number of node devices between each second storage device and the client. Therefore, the network overhead is generally higher than the disk overhead, so that the storage equipment with fewer node equipment between the client is screened out, the network overhead for reading data is reduced, the time for reading the data can be reduced, and the data reading efficiency is improved.
In some embodiments, after the client obtains the device type of each first storage device, the method for determining the target storage device may further include: the client may formulate a prioritization rule for the device type. And then, the client can sort the plurality of first storage devices according to the priority sorting rule of the device type to obtain a target sequence. The client may take the first storage device in the target sequence as the target storage device.
It should be noted that, in the embodiment of the present application, the prioritization rule of the device type may be formulated according to the reading performance of the device. The higher the read performance of the device type, the higher the priority of the device type.
Illustratively, as shown in Table 1, a prioritization rule for one device type is shown. The preset device types may include: RAM, SSD, HDD, archiving tape devices. When the read performance is the read speed, the device type with the highest read speed is the RAM, next the SSD, next the HDD, and finally the archive tape device. In the present example, the higher the read performance of the device type, the higher the priority of the storage device, and the smaller the value corresponding to the priority.
Table 1 prioritization rules for device types
Device type Priority level
RAM 1
SSD 2
HDD 3
Archiving tape apparatus 4
That is, in the case where the device type is RAM, the priority of the storage device is 1. In the case where the device type is SSD, the priority of the storage device is 2. In the case where the device type is HDD, the priority of the storage device is 3. In the case where the device type is an archive tape device, the priority of the storage device is 4.
The method for determining the target storage device in the present application is described below with reference to specific embodiments.
In one possible design, if the client determines a first storage device according to the number of node devices between each second storage device and the client, the client may determine the first storage device as the target storage device.
For example, if the target data is stored in three second storage devices, the initial sequence of the three second storage devices is: a second storage device D (HDD), a second storage device F (archive tape device), and a second storage device E (SSD). The number of node devices between the second storage device D and the client is 1, and the number of node devices between the second storage device E, the second storage device F and the client is 3. The client may sort the three second storage devices according to the number of node devices between the second storage devices and the client, to obtain a first sequence, where the first sequence is: a second storage device D (HDD), a second storage device F (archive tape device), and a second storage device E (SSD). Then, the client may obtain a second sequence according to the priority ordering rule of the device type (in combination with table 1), where the second sequence is: a second storage device D (HDD), a second storage device E (SSD), a second storage device F (archive tape device). The client may then take the second storage device D as the target storage device.
In another possible design, if the client determines the plurality of first storage devices according to the number of node devices between each second storage device and the client, the client may determine the target storage device from the plurality of first storage devices according to the device type of each first storage device.
For example, if the target data is stored in three second storage devices, the initial sequence of the three second storage devices is: a second storage device D (HDD), a second storage device F (archive tape device), and a second storage device E (SSD). The number of node devices among the second storage device D, the second storage device E, the second storage device F and the client is 3. The client may sort the three second storage devices according to the number of node devices between the second storage devices and the client, to obtain a first sequence, where the first sequence is: second storage device D (HDD), second storage device F (archive tape device), second storage device E (SSD) (still ordered in the initial sequence due to the same number of node devices between the three second storage devices and the client). Then, the client may obtain a second sequence according to the priority ordering rule of the device type (in combination with table 1), where the second sequence is: a second storage device E (SSD), a second storage device D (HDD), a second storage device F (archive tape device). The client may then take the second storage device E as the target storage device.
The foregoing description of the solution provided by the embodiments of the present application has been presented primarily in terms of a computer device. It will be appreciated that the computer device, in order to carry out the functions described above, comprises corresponding hardware structures and/or software modules that perform the respective functions. Those of skill in the art will readily appreciate that the various illustrative data request method steps described in connection with the disclosed embodiments of the application may be implemented as hardware or a combination of hardware and computer software. Whether a function is implemented as hardware or computer software driven hardware depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
The embodiment of the application also provides a data request device. The data request device may be a server, a computer device, a CPU in the computer device, a module for requesting data in the computer device, or a client for requesting data in the computer device.
The embodiment of the application can divide the functional modules or functional units for the data request according to the method example, for example, each functional module or functional unit can be divided corresponding to each function, or two or more functions can be integrated in one processing module. The integrated modules may be implemented in hardware, or in software functional modules or functional units. The division of the modules or units in the embodiment of the present application is schematic, which is merely a logic function division, and other division manners may be implemented in practice.
Fig. 6 is a block diagram of a data request device according to an embodiment of the present application. The data requesting device is used for executing the data requesting method shown in fig. 2 and 4. The device is applied to a client, the client is connected with a plurality of first storage devices, the plurality of first storage devices are all used for storing target data, and the number of node devices between each first storage device and the client is the same. The data requesting means may comprise an acquisition module 601, a processing module 602 and a sending module 603.
An obtaining module 601 is configured to obtain a device type of each first storage device. The obtaining module 601 is further configured to obtain a read performance of each first storage device when the device types of the plurality of first storage devices are different. A processing module 602, configured to determine, as a target storage device, a first storage device whose reading performance meets a preset condition; in the case where the reading performance is the reading speed, the preset condition is that the reading speed is greater than the preset speed threshold. The sending module 603 is configured to send a read request message to the target storage device, where the read request message is used to request target data.
Optionally, the client is connected to a plurality of second storage devices, the plurality of second storage devices including a plurality of first storage devices. The obtaining module 601 is further configured to obtain the number of node devices between each second storage device and the client. The processing module 602 is further configured to determine, from the plurality of second storage devices, a plurality of first storage devices according to the number of node devices between each second storage device and the client, where the first storage device is a storage device with a minimum number of node devices between the plurality of second storage devices and the client.
Optionally, the acquiring module 601 is specifically configured to receive a heartbeat message from each first storage device, where the heartbeat message includes a device type of the first storage device.
Optionally, the device type includes at least one of: random access memory RAM, solid state disk SSD, hard disk drive HDD, archive tape device.
Fig. 7 is a schematic diagram illustrating a structure of a data requesting device according to an exemplary embodiment. The data requesting apparatus may include a processor 701, the processor 701 being configured to execute application code to implement the data requesting method of the present application.
The processor 701 may be a central processing unit (central processing unit, CPU), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the programs of the present application.
As shown in fig. 7, the data requesting device may also include a memory 702. The memory 702 is used for storing application program codes for executing the scheme of the present application, and the processor 701 controls the execution.
The memory 702 may be, but is not limited to, read-only memory (ROM) or other type of static storage device that can store static information and instructions, random access memory (random access memory, RAM) or other type of dynamic storage device that can store information and instructions, as well as electrically erasable programmable read-only memory (ELECTRICALLY ERASABLE PROGRAMMABLE READ-only memory, EEPROM), compact disc read-only memory (compact disc read-only memory) or other optical disc storage, optical disc storage (including compact disc, laser disc, optical disc, digital versatile disc, blu-ray disc, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer. The memory 702 may be implemented separately and coupled to the processor 701 via bus 704. Memory 702 may also be integrated with processor 701.
As shown in fig. 7, the data requesting device may further comprise a communication interface 703, wherein the processor 701, the memory 702, the communication interface 703 may be coupled to each other, e.g. via a bus 704. The communication interface 703 is used for information interaction with other devices, for example, supporting information interaction of the data requesting apparatus with other apparatuses.
It should be noted that the device structure shown in fig. 7 does not constitute a limitation of the data requesting device, and the data requesting device may include more or less components than those shown in fig. 7, or may combine some components, or may be arranged with different components.
In actual implementation, the functions implemented by the processing module 602 may be implemented by the processor 701 invoking program code in the memory 702 as shown in fig. 7.
The present application also provides a computer-readable storage medium having instructions stored thereon that, when executed by a processor of a computer device, enable the computer to perform the data request method provided by the above-described illustrated embodiments. For example, the computer readable storage medium may be a memory 702 including instructions executable by the processor 701 of the computer device to perform the methods described above. Alternatively, the computer readable storage medium may be a non-transitory computer readable storage medium, for example, ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, and the like.
Fig. 8 schematically illustrates a conceptual partial view of a computer program product provided by an embodiment of the application, the computer program product comprising a computer program for executing a computer process on a computing device.
In one embodiment, a computer program product is provided using signal bearing medium 800. Signal bearing medium 800 may include one or more program instructions that when executed by one or more processors may provide the functionality or portions of the functionality described above with respect to fig. 2 and 4. Thus, for example, referring to the embodiment shown in FIG. 2, one or more features of S201-S203 may be carried by one or more instructions associated with signal bearing medium 800. Further, the program instructions in fig. 8 also describe example instructions.
In some examples, signal bearing medium 800 may comprise a computer readable medium 801 such as, but not limited to, a hard disk drive, compact Disk (CD), digital Video Disk (DVD), digital tape, memory, read-only memory (ROM), or random access memory (random access memory, RAM), among others.
In some implementations, the signal bearing medium 800 may comprise a computer recordable medium 802 such as, but not limited to, memory, read/write (R/W) CD, R/W DVD, and the like.
In some implementations, the signal bearing medium 800 may include a communication medium 803 such as, but not limited to, a digital and/or analog communication medium (e.g., fiber optic cable, waveguide, wired communications link, wireless communications link, etc.).
The signal bearing medium 800 may be conveyed by a communication medium 803 in wireless form. The one or more program instructions may be, for example, computer-executable instructions or logic-implemented instructions.
In some examples, a data requesting device, such as described with respect to fig. 6, may be configured to provide various operations, functions, or actions in response to program instructions through one or more of computer readable medium 801, computer recordable medium 802, and/or communication medium 803.
It will be apparent to those skilled in the art from this description that, for convenience and brevity of description, only the above-described division of the functional modules is illustrated, and in practical application, the above-described functional allocation may be performed by different functional modules according to needs, i.e. the internal structure of the apparatus is divided into different functional modules, so as to perform all the above-described classification or part of the functions.
In the several embodiments provided by the present application, it should be understood that the disclosed apparatus and method may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of modules or units is merely a logical function division, and there may be additional divisions when actually implemented, e.g., multiple units or components may be combined or integrated into another apparatus, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.
The units described as separate units may or may not be physically separate, and the units shown as units may be one physical unit or a plurality of physical units, may be located in one place, or may be distributed in a plurality of different places. The purpose of the embodiment scheme can be achieved by selecting part or all of the classification part units according to actual needs.
In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.
The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a readable storage medium. Based on such understanding, the technical solution of the embodiments of the present application, or the portion contributing to the prior art or the whole classification portion or portion of the technical solution, may be embodied in the form of a software product stored in a storage medium, where the software product includes several instructions to cause a device (may be a single-chip microcomputer, a chip or the like) or a processor (processor) to execute the whole classification portion or part of the steps of the method of the embodiments of the present application. The storage medium includes a usb disk, a removable hard disk, a ROM, a RAM, a magnetic disk, or an optical disk, etc. which can store the program codes.
The present application is not limited to the above embodiments, and any changes or substitutions within the technical scope of the present application should be covered by the scope of the present application. Therefore, the protection scope of the application is subject to the protection scope of the claims.

Claims (10)

1. The data request method is characterized by being applied to a client, wherein the client is connected with a plurality of first storage devices, the plurality of first storage devices all store target data, and the number of node devices between each first storage device and the client is the same; the method comprises the following steps:
Acquiring the device type of each first storage device;
Under the condition that the device types of the plurality of first storage devices are different, acquiring the reading performance of each first storage device, and determining the first storage device with the reading performance meeting the preset condition as a target storage device; under the condition that the reading performance is the reading speed, the preset condition is that the reading speed is larger than a preset speed threshold;
And sending a read request message to the target storage device, wherein the read request message is used for requesting the target data.
2. The method of claim 1, wherein the client is connected to a plurality of second storage devices, the plurality of second storage devices comprising the plurality of first storage devices; before the obtaining the device type of each first storage device, the method further includes:
acquiring the number of node devices between each second storage device and the client;
And determining the first storage devices from the second storage devices according to the number of node devices between each second storage device and the client, wherein the first storage devices are the storage devices with the minimum number of node devices between the second storage devices and the client.
3. The method according to claim 1 or 2, wherein the obtaining the device type of each of the first storage devices comprises:
a heartbeat message is received from each of the first storage devices, the heartbeat message including a device type of the first storage device.
4. The method according to claim 1 or 2, wherein the device type comprises at least one of: random access memory RAM, solid state disk SSD, hard disk drive HDD, archive tape device.
5. The data request device is characterized by being applied to a client, wherein the client is connected with a plurality of first storage devices, the plurality of first storage devices are all used for storing target data, and the number of node devices between each first storage device and the client is the same; the device comprises:
the acquisition module is used for acquiring the equipment type of each first storage equipment;
the acquisition module is further configured to acquire a reading performance of each of the first storage devices when the device types of the plurality of first storage devices are different;
The processing module is used for determining the first storage device with the reading performance meeting the preset condition as a target storage device; under the condition that the reading performance is the reading speed, the preset condition is that the reading speed is larger than a preset speed threshold;
and the sending module is used for sending a read request message to the target storage device, wherein the read request message is used for requesting the target data.
6. The apparatus of claim 5, wherein the client is connected to a plurality of second storage devices, the plurality of second storage devices comprising the plurality of first storage devices;
The acquisition module is further used for acquiring the number of node devices between each second storage device and the client;
The processing module is further configured to determine, from the plurality of second storage devices, the plurality of first storage devices according to the number of node devices between each of the second storage devices and the client, where the first storage device is a storage device with a minimum number of node devices between the plurality of second storage devices and the client.
7. The apparatus of claim 5 or 6, wherein the device comprises a plurality of sensors,
The acquisition module is specifically configured to receive a heartbeat message from each first storage device, where the heartbeat message includes a device type of the first storage device.
8. The apparatus of claim 5 or 6, wherein the device type comprises at least one of: random access memory RAM, solid state disk SSD, hard disk drive HDD, archive tape device.
9. A data requesting apparatus, comprising: a processor and a memory; the processor and the memory are coupled; the memory is configured to store one or more programs, the one or more programs comprising computer-executable instructions that, when executed by the data requesting device, cause the data requesting device to perform the data requesting method of any of claims 1-4.
10. A computer readable storage medium having instructions stored therein, which when executed by a computer, performs the data requesting method of any one of claims 1-4.
CN202410033851.XA 2024-01-09 2024-01-09 Data request method, device and storage medium Pending CN117909311A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202410033851.XA CN117909311A (en) 2024-01-09 2024-01-09 Data request method, device and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202410033851.XA CN117909311A (en) 2024-01-09 2024-01-09 Data request method, device and storage medium

Publications (1)

Publication Number Publication Date
CN117909311A true CN117909311A (en) 2024-04-19

Family

ID=90693266

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202410033851.XA Pending CN117909311A (en) 2024-01-09 2024-01-09 Data request method, device and storage medium

Country Status (1)

Country Link
CN (1) CN117909311A (en)

Similar Documents

Publication Publication Date Title
CN107943421B (en) Partition division method and device based on distributed storage system
CN110537169B (en) Cluster resource management in a distributed computing system
CN104603739A (en) Block-level access to parallel storage
US10263876B2 (en) Adaptive service timeouts
CN103324533A (en) distributed data processing method, device and system
CN112947860B (en) Hierarchical storage and scheduling method for distributed data copies
WO2017207049A1 (en) A node of a network and a method of operating the same for resource distribution
CN111208934A (en) Data storage method and device
CN110381136B (en) Data reading method, terminal, server and storage medium
CN109840051B (en) Data storage method and device of storage system
CN117909311A (en) Data request method, device and storage medium
CN115604294A (en) Method and device for managing storage resources
CN111209304B (en) Data processing method, device and system
CN110543362B (en) Graphics processor management method and device and server
CN110895520B (en) File migration method, related device and equipment
CN113746920A (en) Data forwarding method and device, electronic equipment and computer readable storage medium
CN115129709A (en) Data processing method, server and system
CN111464600B (en) Point-to-point storage network establishing method, device, medium and electronic equipment
CN110213314B (en) Method, device and server for determining storage node
CN113918644A (en) Method and related device for managing data of application program
CN115904211A (en) Storage system, data processing method and related equipment
CN116501255A (en) Data migration method, device, computing equipment and storage medium
CN115185887A (en) Metadata processing method and device
CN113867928A (en) Load balancing method and device and server
CN114327273A (en) Data storage method and device, storage medium and electronic equipment

Legal Events

Date Code Title Description
PB01 Publication