CN112732169A - Data writing or reading method, device, storage medium and electronic device - Google Patents
Data writing or reading method, device, storage medium and electronic device Download PDFInfo
- Publication number
- CN112732169A CN112732169A CN201911033412.4A CN201911033412A CN112732169A CN 112732169 A CN112732169 A CN 112732169A CN 201911033412 A CN201911033412 A CN 201911033412A CN 112732169 A CN112732169 A CN 112732169A
- Authority
- CN
- China
- Prior art keywords
- target
- disk
- data
- path information
- determining
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0625—Power saving in storage systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
- G06F1/26—Power supply means, e.g. regulation thereof
- G06F1/32—Means for saving power
- G06F1/3203—Power management, i.e. event-based initiation of a power-saving mode
- G06F1/3234—Power saving characterised by the action undertaken
- G06F1/325—Power saving in peripheral device
- G06F1/3275—Power saving in memory, e.g. RAM, cache
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0629—Configuration or reconfiguration of storage systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0683—Plurality of storage devices
- G06F3/0689—Disk arrays, e.g. RAID, JBOD
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a method and a device for writing or reading data, a storage medium and an electronic device, wherein the method comprises the following steps: acquiring a target request from a target client, wherein the target request carries target characteristics of target data; determining a target disk corresponding to the target characteristics; and acquiring target path information corresponding to the target disk, and returning the target path information to the target client to instruct the target client to write the target data into the target disk according to the target path information or read the target data from the target disk according to the target path information.
Description
Technical Field
The present invention relates to the field of communications, and in particular, to a method and an apparatus for writing or reading data, a storage medium, and an electronic apparatus.
Background
With the development of big data technology, storage of data and processing of data are also more challenging. In a large-scale storage system, high performance, high reliability and low cost are three important indicators. Particularly in the field of large data storage, the centralization of data is becoming an important trend. For example, the rise of the concept of data lake puts higher requirements on the centralized storage of data and the discrete distribution of data blocks after the centralized storage.
As a data center for data storage and processing, the increase in physical cost and energy consumption due to the scale-up is also an important point affecting the profit of an enterprise. The equipment power consumption of a data center is roughly divided into three main categories: the power consumption of the server, the power consumption of the air conditioner and the power consumption of other equipment are large, wherein the power consumption of the server and the power consumption of the air conditioner are large in power consumption of the whole data center, and the power consumption of the air conditioner can be indirectly influenced by the use of the server. For example, as the number of servers operating in a certain space increases, the amount of heat emitted increases, and the operating load of the air conditioner also increases. How to achieve the purpose of energy saving by reducing the energy consumption of the data center server becomes an important research direction for energy saving of the data center.
Current industry solutions for data center server power savings are generally based on server granularity. For example, the stored data is divided into cold data and hot data according to the access times, the data with different cold degrees and hot degrees are distributed on different servers, and the servers where the cold data are located are in standby, so that energy conservation is realized. However, energy savings are achieved through server-granular control, but there are still some problems: a certain amount of data movement is carried out in the storage file system, so that the consumption of server nodes and the consumption of internal network bandwidth of the distributed file system are increased; for the control of the server node, the control granularity is too large, only a few data blocks of the disk in the server are probably operated, and all the disks are kept online; if the cold data is to be operated, the wake-up process of the standby server is very long, so that the whole data operation flow is prolonged; because the file system continuously has data input and output, the calculation of data distribution is continuously carried out, and a large amount of system resources are occupied;
in view of the above problems in the related art, an effective solution for saving energy of a data center server based on server granularity does not exist at present.
Disclosure of Invention
Embodiments of the present invention provide a data writing or reading method and apparatus, a storage medium, and an electronic apparatus, so as to at least solve the above problems in a scheme for saving energy of a data center server based on server granularity in the related art.
According to an embodiment of the present invention, there is provided a data writing or reading method including: acquiring a target request from a target client, wherein the target request carries target characteristics of target data; determining a target disk corresponding to the target characteristics; and acquiring target path information corresponding to the target disk, and returning the target path information to the target client to instruct the target client to write the target data into the target disk according to the target path information or read the target data from the target disk according to the target path information.
Optionally, the obtaining of the target path information corresponding to the target disk includes: determining a target data node corresponding to the target disk according to a preset corresponding relation between the disk and the data node, wherein the target disk is located in the target data node; determining path information of the target data node; determining the target path information based on the path information of the target data node.
Optionally, before determining the target disk corresponding to the target feature, the method further includes: receiving configuration information of disks in at least two data nodes, wherein the configuration information comprises at least one of the following: the identification information of the data node where the disk is located, the identification information of the disk, the path information of the disk in the data node, and the dimension label of the disk, wherein the dimension label is used for indicating the characteristics of the data corresponding to the disk; determining a target disk corresponding to the target feature comprises: and determining the target disk corresponding to the target feature based on the configuration information.
Optionally, after determining the target disk corresponding to the target feature, the method further includes: and sending a wake-up signal to the target disk to wake up the target disk when the target disk is determined to be in the dormant state.
Optionally, the method further comprises: and sending a sleep instruction to the target disk to indicate that the target disk enters a sleep state within a preset time period.
According to another embodiment of the present invention, there is provided a data writing or reading apparatus including: the system comprises an acquisition module, a processing module and a processing module, wherein the acquisition module is used for acquiring a target request from a target client, and the target request carries target characteristics of target data; the determining module is used for determining a target disk corresponding to the target characteristic; and the indicating module is used for acquiring target path information corresponding to the target disk and returning the target path information to the target client so as to indicate the target client to write the target data into the target disk according to the target path information or read the target data from the target disk according to the target path information.
Optionally, the indication module further includes: the first determining unit is used for determining a target data node corresponding to the target disk according to a preset corresponding relation between the disk and the data node, wherein the target disk is positioned in the target data node; a second determining unit, configured to determine path information of the target data node; a third determining unit, configured to determine the target path information based on the path information of the target data node.
Optionally, the apparatus further comprises: a receiving module, configured to receive configuration information of disks in the data nodes sent by at least two data nodes before determining a target disk corresponding to the target feature, where the configuration information includes at least one of: the identification information of the data node where the disk is located, the identification information of the disk, the path information of the disk in the data node, and the dimension label of the disk, wherein the dimension label is used for indicating the characteristics of the data corresponding to the disk; the determining module is further configured to determine the target disk corresponding to the target feature based on the configuration information.
According to a further embodiment of the present invention, there is also provided a storage medium having a computer program stored therein, wherein the computer program is arranged to perform the steps of any of the above method embodiments when executed.
According to yet another embodiment of the present invention, there is also provided an electronic device, including a memory in which a computer program is stored and a processor configured to execute the computer program to perform the steps in any of the above method embodiments.
According to the invention, the target request from the target client is obtained, wherein the target request carries the target characteristics of the target data; determining a target disk corresponding to the target characteristics; and acquiring target path information corresponding to the target disk, and returning the target path information to the target client to instruct the target client to write target data into the target disk according to the target path information or read the target data from the target disk according to the target path information. Thus, the above-described problems with a solution for saving energy on data center servers based on server granularity may be solved. By the scheme, the energy-saving control granularity of the storage server can be refined. The method and the device achieve the purposes of reducing frequent data movement, enabling the power-off control range to be more refined, and ensuring high performance, high reliability and the like of data access and storage of the distributed file system.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the invention without limiting the invention. In the drawings:
fig. 1 is a block diagram of a hardware structure of a mobile terminal according to a data writing or reading method of an embodiment of the present invention;
FIG. 2 is a flow chart of writing or reading of data according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a server partitioned into multiple disks in accordance with an alternative embodiment of the present invention;
FIG. 4 is a schematic diagram of a disk configuration according to an alternative embodiment of the present invention;
FIG. 5 is a schematic illustration of an input-output hierarchy of a distributed file system in accordance with an alternative embodiment of the present invention;
FIG. 6 is a schematic diagram of a classified data distribution of a disk according to an alternative embodiment of the invention;
FIG. 7 is a flow diagram illustrating the distribution of disk configuration information according to an alternative embodiment of the present invention;
FIG. 8 is a schematic flow chart illustrating writing of the target data to the target disk according to an alternative embodiment of the present invention;
FIG. 9 is a schematic diagram of the operation of a disk management module according to an alternative embodiment of the invention;
fig. 10 is a block diagram of a data writing or reading apparatus according to an embodiment of the present invention.
Detailed Description
The invention will be described in detail hereinafter with reference to the accompanying drawings in conjunction with embodiments. It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict.
It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order.
Example 1
The method provided by the first embodiment of the present application may be executed in a mobile terminal, a computer terminal, or a similar computing device. Taking the example of the present invention running on a mobile terminal, fig. 1 is a block diagram of a hardware structure of the mobile terminal according to an embodiment of the present invention. As shown in fig. 1, the mobile terminal 10 may include one or more (only one shown in fig. 1) processors 102 (the processor 102 may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, and optionally may also include a transmission device 106 for communication functions and an input-output device 108. It will be understood by those skilled in the art that the structure shown in fig. 1 is only an illustration, and does not limit the structure of the mobile terminal. For example, the mobile terminal 10 may also include more or fewer components than shown in FIG. 1, or have a different configuration than shown in FIG. 1.
The memory 104 may be used to store computer programs, for example, software programs and modules of application software, such as computer programs corresponding to the data writing or reading methods in the embodiments of the present invention, and the processor 102 executes various functional applications and data processing by running the computer programs stored in the memory 104, so as to implement the methods described above. The memory 104 may include high speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory located remotely from the processor 102, which may be connected to the mobile terminal 10 via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The transmission device 106 is used for receiving or transmitting data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider of the mobile terminal 10. In one example, the transmission device 106 includes a Network adapter (NIC), which can be connected to other Network devices through a base station so as to communicate with the internet. In one example, the transmission device 106 may be a Radio Frequency (RF) module, which is used for communicating with the internet in a wireless manner.
In this embodiment, a method for writing or reading data running on the mobile terminal is provided, and fig. 2 is a flowchart of writing or reading data according to an embodiment of the present invention, as shown in fig. 2, the flowchart includes the following steps:
step S202, a target request from a target client is obtained, wherein the target request carries target characteristics of target data;
the target feature may represent a dimension of target data, for example, a time dimension of data, such as data in a telecommunication field, may have an obvious time period, and data read and written in a cluster set in a certain specific period domain is fixed in a certain period of time. For example, financial data is also the case when high-density input and output is not available until the end of the month or the quarter. Thus, data may be stored based on the time dimension.
The target characteristics may also represent the traffic type characteristics to which the data corresponds. For example, each A client served by B corporation will have certain data. The target characteristics may represent a first party customer to which the target data corresponds. Thus, storage may be based on the object represented by the data.
Specifically, the target feature may represent various dimensional features of the target data, and may be arbitrarily adjusted according to actual conditions.
Step S204, determining a target disk corresponding to the target characteristics;
fig. 3 is a schematic diagram of dividing a server into a plurality of disks according to an alternative embodiment of the present invention, where data nodes DataNode1, DataNode2, and DataNode3 respectively represent different storage servers, A, B, C, E, F is a disk divided in the server, and each disk corresponds to data of one feature. Taking the time dimension data dividing characteristic as an example, the time dimensions corresponding to different disks in different data nodes are different. For example, in the data node1, the corresponding time dimension of the a disk is 1 to 4 months of 2019, the corresponding time dimension of the B disk is 5 to 8 months of 2019, and the corresponding time dimension of the E disk is 9 to 12 months of 2019. That is, for data appearing in 2019, there is a disk corresponding to the data in each data node. The disk corresponding to the target data can be determined in the data node according to the time characteristic of the target data, for example, for the target data of 7 months in 2019, the disk corresponding to the target data in the data node1 is B.
Correspondingly, if the target feature indicates that the service type corresponding to the data is an example, the three disks B, A, F divided by the data node DataNode2 in fig. 3 correspond to the first customer, the second customer and the third customer, respectively. Then the disk corresponding to the first customer's data is disk B of data node DataNode 2.
Step S206, obtaining target path information corresponding to the target disk, and returning the target path information to the target client, so as to instruct the target client to write the target data into the target disk according to the target path information or read the target data from the target disk according to the target path information.
Each disk in the data node corresponds to a path, each disk corresponds to data of one characteristic, and the reading and writing of the data are directed at the corresponding disk.
Through the steps, the target request from the target client is obtained, wherein the target request carries the target characteristics of the target data; determining a target disk corresponding to the target characteristics; and acquiring target path information corresponding to the target disk, and returning the target path information to the target client to instruct the target client to write target data into the target disk according to the target path information or read the target data from the target disk according to the target path information. Thus, the above-described problems with a solution for saving energy on data center servers based on server granularity may be solved. By the scheme, the energy-saving control granularity of the storage server can be refined. The method and the device achieve the purposes of reducing frequent data movement, enabling the power-off control range to be more refined, and ensuring high performance, high reliability and the like of data access and storage of the distributed file system.
Alternatively, the execution subject of the above steps may be a terminal or the like, but is not limited thereto.
In an alternative embodiment, obtaining the target path information corresponding to the target disk includes: determining a target data node corresponding to the target disk according to a preset corresponding relation between the disk and the data node, wherein the target disk is located in the target data node; determining path information of the target data node; determining the target path information based on the path information of the target data node. In this embodiment, each disk corresponds to one data node, and the disk names included in different data nodes may be the same or different, for example, in fig. 3, the DataNode1 and the DataNode2 both include a and B. Each data node corresponds to a path, and the paths of the disks are different in different data nodes even if the disk names are the same. That is, each disk corresponds to a path, and the paths of different disks are different. The disk can be uniquely determined from the path. For example, in FIG. 3, the path of disk A in DataNode1 may be: the path of disk A in DataNode1/A, DataNode2 may be: DataNode 2/A. The path of the disk may be determined based on the path of the data node where the disk is located.
In an alternative embodiment, before determining the target disk corresponding to the target feature, the method further includes: receiving configuration information of disks in at least two data nodes, wherein the configuration information comprises at least one of the following: the identification information of the data node where the disk is located, the identification information of the disk, the path information of the disk in the data node, and the dimension label of the disk, wherein the dimension label is used for indicating the characteristics of the data corresponding to the disk; determining a target disk corresponding to the target feature comprises: and determining the target disk corresponding to the target feature based on the configuration information. In this embodiment, the disk configuration information may be configured by a user in a client, and as shown in fig. 4, is a disk configuration diagram according to an alternative embodiment of the present invention, which is used for configuring the dimensions of a disk in the client through a dimension configuration module. The dimension can be defined by a user, and can also be obtained by performing characteristic analysis on data and preorder calculation or artificial intelligence analysis on the data. For example, data in the field of telecommunications is integrated into storage clusters on an hourly basis. The access heat of part of the media data in the movie and television media data has strong correlation with the working time specified by the state. The user can log in a configuration interface in the client to configure the dimensionality of the disk, the client sends configuration information to the corresponding data node, and the data node configures the disk in the data node according to the configuration information. Taking the data node diagram shown in fig. 3 as an example, a user logs in a configuration interface of a client, configures the dimension of a disk a in the DataNode1 to be 1 to 4 months in 2019, configures the dimension of a disk B to be 5 to 8 months in 2019, and configures the dimension of a disk E to be 9 to 12 months in 2019. Then the configuration information may be DN 1: [ dimension: 2019/1-2019/4 disk A), DN 1: [ dimension: 2019/5-2019/8 disk B), DN 1: [ dimension: 2019/9-2019/8 disk E ], wherein DN1 represents the identity of the data node, disk A, disk B, disk E represent the identity of the disk, and the time dimensions 2019/1-2019/4 represent the dimension labels of the disk. After the client sends the disk configuration information to the data node DataNode1, the DataNode1 configures the disk in the DataNode1 according to the configuration information, the data characteristic dimension corresponding to the disk a in the DataNode1 is data in 2019, 1 to 4 months, the data characteristic dimension corresponding to the disk B in the DataNode1 is data in 2019, 5 to 8 months, and the data characteristic dimension corresponding to the disk E in the DataNode1 is data in 2019, 9 to 12 months.
In an alternative embodiment, after determining the target disk corresponding to the target feature, the method further includes: and sending a wake-up signal to the target disk to wake up the target disk when the target disk is determined to be in the dormant state. In this embodiment, the disk without data input/output is in a sleep state, so as to achieve the effect of saving energy. And after determining a target disk corresponding to the written or read target data, sending a wake-up signal to the target disk to wake up the disk in the dormant state.
In an alternative embodiment, the method further comprises: and sending a sleep instruction to the target disk to indicate that the target disk enters a sleep state within a preset time period. In this embodiment, after the target data is written or read, a hibernation instruction may be sent to the disk, so that the non-operating disk is in a hibernation state, and an effect of saving energy with a more refined disk strength is achieved.
The present application is illustrated by the following specific examples.
According to the method and the system, when the data are stored in the data center, the data characteristics are taken as dimensions and fall into different storage domains (corresponding to the disks), data movement and subsequent frequent system operation are avoided, and selective disk dormancy and awakening can be performed on the disks in the areas without data access through control over the granularity of the disks in the server.
Fig. 5 is a schematic diagram illustrating an input/output hierarchy of a distributed file system according to an alternative embodiment of the present invention, in which a mapping of a corresponding domain and a storage path is added to a file directory system. That is, the configuration of the disk in the data node is added, and the dimension of the disk and the corresponding disk path are configured. The sleep awakening module of the disk is added and reduced in the equipment management module so as to send sleep and awakening signals to the disk, control the sleep and awakening states of the disk and achieve the effect of energy conservation.
Fig. 6 is a schematic diagram illustrating data distribution after disk classification according to an alternative embodiment of the present invention. The disk of each data node is divided into different domains in advance, and each domain corresponds to data of one characteristic dimension. Specifically, the present embodiment includes the following modules:
a module A: and a dimension configuration module. The module is used for providing configuration items of disk dimension division for users. Each dimension domain may map 1-N specific disk paths. This module also provides a dimension dormancy policy configuration. The dimension can be defined by a user, and can also be obtained by performing characteristic analysis on data (for example, data in the telecommunication field is integrated into a storage cluster according to an hour rule; the access heat of part of media data in the film and television media data has strong correlation with the working time specified by the country) and preorder calculation or artificial intelligence analysis of the data. If the dimension is not configured when the storage system is powered on, the default is that the dimension and the disk universe are scattered, namely the default IO mode of the file system. The dimension sleep policy configuration is mainly configured based on the offset of the dimension, and for example, the configuration may be that all dimensions except two hours after the current time are sleep.
And a module B: and a dimension management module. The module is used for ensuring that input and output can be carried out according to the division dimension in the specific file system operation process. In a distributed file system, file paths need to be found for data input and output. The module is responsible for reporting dimension information in the data node DataNode and disk path information corresponding to the dimension to the name node NameNode at regular time, so that the name node can select a correct data storage path when inputting and outputting data. Specifically, dimension management and disk path mapping may be configured in a key value manner, e.g., [00:00-01:00 ]: data1/hdfs/ods1/, [09:00-22:00 ]: data9/hdfs/ods9/,/data10/hdfs/ods 10/. Specifically, as shown in fig. 7, a flowchart of issuing disk configuration information according to an embodiment of the present invention is shown.
And a module C: and a disk management module. The module is used for controlling the dormancy and the awakening of a specific disk. The disk management module needs to acquire the current time, compare the current time with the dimension and disk path information configured by the dimension configuration module, and perform specific operation according to the sleep strategy of the dimension configuration module through the comparison result. The disk management module provides a specific disk sleeping and waking interface for system regulation under special conditions besides its own operation logic. The disk needs to have self-regulation capability in the processes of dormancy, awakening and re-dormancy, so that frequent operation is avoided.
In this embodiment, the method for dimension configuration management includes the following steps:
step 1-1: a user logs in a dimension configuration management interface;
step 1-2: the user carries out time range configuration, each time range corresponds to one disk path configuration item, and the disk path configuration items can be used and carry out multiple path separation;
step 1-3: the user carries out overall dimension strategy configuration, and the system provides established strategy options and allows combination of multiple strategies.
Step 1-4: and the configured dimension and the mapping path information configure and issue the specified DataNode.
The data and DataNode dimension management module in the embodiment relates to the following steps:
step 2-1: the data node DataNode writes the configuration information sent by the dimension configuration management module into the configuration file of the data node DataNode
Step 2-2: the data node DataNode reports the dimension written in the configuration file and the specific disk path information (such as [ dimension 1:/disk 1]) to the name node NameNode in the heartbeat message, and the NameNode stores the relevant information in the memory.
In a specific data writing or output IO process, a client needs to carry a specific dimension to which a written file belongs, a subsequent client applies specific block information to a NameNode in a formal data stream writing process, and the NameNode judges which disk paths specifically needing to apply for the data blocks to which DataNode are written or output according to configuration information of the dimension. After receiving the specific data IO, the DataNode writes the data IO to the designated disk. Fig. 8 is a schematic flowchart of writing the target data to the target disk according to an alternative embodiment of the present invention.
Fig. 9 is a schematic diagram illustrating operations of a disk management module according to an alternative embodiment of the present invention, where the disk management module involves the following steps:
step 3-1: the disk management module continuously operates in the background to acquire the current time;
step 3-2: the disk management module acquires configuration information and strategy information of the configuration management module;
step 3-3: a disk management reference strategy and a domain disk corresponding to time designation are subjected to dormancy and awakening operations;
step 3-4: the disk management module also provides an external API, allowing a user to manually wake up the corresponding disk domain.
Through the above description of the embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but the former is a better implementation mode in many cases. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (e.g., ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal device (e.g., a mobile phone, a computer, a server, or a network device) to execute the method according to the embodiments of the present invention.
Example 2
In this embodiment, a data writing or reading apparatus is further provided, and the apparatus is used to implement the foregoing embodiments and preferred embodiments, and the description of the apparatus is omitted for brevity. As used below, the term "module" may be a combination of software and/or hardware that implements a predetermined function. Although the means described in the embodiments below are preferably implemented in software, an implementation in hardware, or a combination of software and hardware is also possible and contemplated.
Fig. 10 is a block diagram of a structure of an apparatus for writing or reading data according to an embodiment of the present invention, as shown in fig. 10, the apparatus including: an obtaining module 1002, configured to obtain a target request from a target client, where the target request carries a target feature of target data; a determining module 1004, configured to determine a target disk corresponding to the target feature; an indicating module 1006, configured to obtain target path information corresponding to the target disk, and return the target path information to the target client, so as to instruct the target client to write the target data to the target disk according to the target path information or read the target data from the target disk according to the target path information.
In an alternative embodiment, the indication module further comprises: the first determining unit is used for determining a target data node corresponding to the target disk according to a preset corresponding relation between the disk and the data node, wherein the target disk is positioned in the target data node; a second determining unit, configured to determine path information of the target data node; a third determining unit, configured to determine the target path information based on the path information of the target data node.
In an alternative embodiment, the apparatus further comprises: a receiving module, configured to receive configuration information of disks in the data nodes sent by at least two data nodes before determining a target disk corresponding to the target feature, where the configuration information includes at least one of: the identification information of the data node where the disk is located, the identification information of the disk, the path information of the disk in the data node, and the dimension label of the disk, wherein the dimension label is used for indicating the characteristics of the data corresponding to the disk; the determining module is further configured to determine the target disk corresponding to the target feature based on the configuration information.
In an optional embodiment, the apparatus is further configured to, after determining the target disk corresponding to the target feature, send a wake-up signal to the target disk to wake up the target disk when determining that the target disk is in the hibernation state.
In an optional embodiment, the apparatus is further configured to send a hibernation instruction to the target disk to instruct the target disk to enter a hibernation state within a predetermined time period.
It should be noted that, the above modules may be implemented by software or hardware, and for the latter, the following may be implemented, but not limited to: the modules are all positioned in the same processor; alternatively, the modules are respectively located in different processors in any combination.
Example 3
Embodiments of the present invention also provide a storage medium having a computer program stored therein, wherein the computer program is arranged to perform the steps of any of the above method embodiments when executed.
Alternatively, in the present embodiment, the storage medium may be configured to store a computer program for executing the steps of:
s1, acquiring a target request from a target client, wherein the target request carries target characteristics of target data;
s2, determining a target disk corresponding to the target characteristics;
s3, obtaining target path information corresponding to the target disk, and returning the target path information to the target client, so as to instruct the target client to write the target data to the target disk according to the target path information or read the target data from the target disk according to the target path information.
Optionally, in this embodiment, the storage medium may include, but is not limited to: various media capable of storing computer programs, such as a usb disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic disk, or an optical disk.
Embodiments of the present invention also provide an electronic device comprising a memory having a computer program stored therein and a processor arranged to run the computer program to perform the steps of any of the above method embodiments.
Optionally, the electronic apparatus may further include a transmission device and an input/output device, wherein the transmission device is connected to the processor, and the input/output device is connected to the processor.
Optionally, in this embodiment, the processor may be configured to execute the following steps by a computer program:
s1, acquiring a target request from a target client, wherein the target request carries target characteristics of target data;
s2, determining a target disk corresponding to the target characteristics;
s3, obtaining target path information corresponding to the target disk, and returning the target path information to the target client, so as to instruct the target client to write the target data to the target disk according to the target path information or read the target data from the target disk according to the target path information.
Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments and optional implementation manners, and this embodiment is not described herein again.
It will be apparent to those skilled in the art that the modules or steps of the present invention described above may be implemented by a general purpose computing device, they may be centralized on a single computing device or distributed across a network of multiple computing devices, and alternatively, they may be implemented by program code executable by a computing device, such that they may be stored in a storage device and executed by a computing device, and in some cases, the steps shown or described may be performed in an order different than that described herein, or they may be separately fabricated into individual integrated circuit modules, or multiple ones of them may be fabricated into a single integrated circuit module. Thus, the present invention is not limited to any specific combination of hardware and software.
The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. A method for writing or reading data, comprising:
acquiring a target request from a target client, wherein the target request carries target characteristics of target data;
determining a target disk corresponding to the target characteristics;
and acquiring target path information corresponding to the target disk, and returning the target path information to the target client to instruct the target client to write the target data into the target disk according to the target path information or read the target data from the target disk according to the target path information.
2. The method of claim 1, wherein obtaining target path information corresponding to the target disk comprises:
determining a target data node corresponding to the target disk according to a preset corresponding relation between the disk and the data node, wherein the target disk is located in the target data node;
determining path information of the target data node;
determining the target path information based on the path information of the target data node.
3. The method of claim 2,
before determining the target disk corresponding to the target feature, the method further includes: receiving configuration information of disks in at least two data nodes, wherein the configuration information comprises at least one of the following: the identification information of the data node where the disk is located, the identification information of the disk, the path information of the disk in the data node, and the dimension label of the disk, wherein the dimension label is used for indicating the characteristics of the data corresponding to the disk;
determining a target disk corresponding to the target feature comprises: and determining the target disk corresponding to the target feature based on the configuration information.
4. The method of claim 1, wherein after determining the target disk corresponding to the target feature, the method further comprises:
and sending a wake-up signal to the target disk to wake up the target disk when the target disk is determined to be in the dormant state.
5. The method of claim 1, further comprising:
and sending a sleep instruction to the target disk to indicate that the target disk enters a sleep state within a preset time period.
6. An apparatus for writing or reading data, comprising:
the system comprises an acquisition module, a processing module and a processing module, wherein the acquisition module is used for acquiring a target request from a target client, and the target request carries target characteristics of target data;
the determining module is used for determining a target disk corresponding to the target characteristic;
and the indicating module is used for acquiring target path information corresponding to the target disk and returning the target path information to the target client so as to indicate the target client to write the target data into the target disk according to the target path information or read the target data from the target disk according to the target path information.
7. The apparatus of claim 6, wherein the indication module further comprises:
the first determining unit is used for determining a target data node corresponding to the target disk according to a preset corresponding relation between the disk and the data node, wherein the target disk is positioned in the target data node;
a second determining unit, configured to determine path information of the target data node;
a third determining unit, configured to determine the target path information based on the path information of the target data node.
8. The apparatus of claim 7, further comprising:
a receiving module, configured to receive configuration information of disks in the data nodes sent by at least two data nodes before determining a target disk corresponding to the target feature, where the configuration information includes at least one of: the identification information of the data node where the disk is located, the identification information of the disk, the path information of the disk in the data node, and the dimension label of the disk, wherein the dimension label is used for indicating the characteristics of the data corresponding to the disk;
the determining module is further configured to determine the target disk corresponding to the target feature based on the configuration information.
9. A storage medium, in which a computer program is stored, wherein the computer program is arranged to perform the method of any of claims 1 to 5 when executed.
10. An electronic device comprising a memory and a processor, wherein the memory has stored therein a computer program, and wherein the processor is arranged to execute the computer program to perform the method of any of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911033412.4A CN112732169A (en) | 2019-10-28 | 2019-10-28 | Data writing or reading method, device, storage medium and electronic device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911033412.4A CN112732169A (en) | 2019-10-28 | 2019-10-28 | Data writing or reading method, device, storage medium and electronic device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112732169A true CN112732169A (en) | 2021-04-30 |
Family
ID=75589347
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911033412.4A Pending CN112732169A (en) | 2019-10-28 | 2019-10-28 | Data writing or reading method, device, storage medium and electronic device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112732169A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113568570A (en) * | 2021-06-22 | 2021-10-29 | 阿里巴巴新加坡控股有限公司 | Data processing method and device |
-
2019
- 2019-10-28 CN CN201911033412.4A patent/CN112732169A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113568570A (en) * | 2021-06-22 | 2021-10-29 | 阿里巴巴新加坡控股有限公司 | Data processing method and device |
| CN113568570B (en) * | 2021-06-22 | 2024-04-12 | 阿里巴巴创新公司 | Data processing method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101981531B (en) | Aggregating recurrent schedules to optimize resource consumption | |
| WO2016041468A1 (en) | Wakeup method, device and terminal | |
| Kumar | Efficient resource allocation and QoS enhancements of IoT with fog network | |
| WO2013090124A1 (en) | Discovery and mining of performance information of a device for anticipatorily sending updates to the device | |
| CN110505679B (en) | Power consumption control method and device of communication terminal and storage medium | |
| CN113037771B (en) | Method and device for controlling device shadow, storage medium and electronic device | |
| WO2020228289A1 (en) | Log acquisition method and apparatus, and terminal and storage medium | |
| CN111767144B (en) | Transaction route determination method, device, equipment and system for transaction data | |
| CN110765165B (en) | Method, device and system for synchronously processing cross-system data | |
| CN109857239B (en) | Storage control method and device | |
| CN107306443B (en) | Information processing method and terminal | |
| CN108924203B (en) | Data copy self-adaptive distribution method, distributed computing system and related equipment | |
| CN111623486B (en) | Method and device for determining starting time, storage medium and electronic device | |
| CN107329801B (en) | Node management method and device and multi-subsatellite server | |
| CN112181678A (en) | Service data processing method, device and system, storage medium and electronic device | |
| CN107086937A (en) | A kind of monitoring method and equipment | |
| CN112732169A (en) | Data writing or reading method, device, storage medium and electronic device | |
| CN114710571A (en) | Data packet processing system | |
| WO2021042368A1 (en) | Power consumption control and scheme generation method, device, system, and storage medium | |
| CN115525603A (en) | Storage statistics method and device, computer readable storage medium and AI device | |
| CN109116815A (en) | One kind is based on MQTT injection molding machine management system, method and storage medium | |
| CN113900692A (en) | Data management method and device, electronic equipment and storage medium | |
| CN113467723A (en) | Data storage method, device, equipment and medium | |
| Forshaw | Operating policies for energy efficient large scale computing | |
| CN111506646A (en) | Data synchronization method, device, system, storage medium and processor |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |