CN116301645A - Transmission channel allocation method and device, electronic equipment and storage medium - Google Patents
Transmission channel allocation method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN116301645A CN116301645A CN202310301797.8A CN202310301797A CN116301645A CN 116301645 A CN116301645 A CN 116301645A CN 202310301797 A CN202310301797 A CN 202310301797A CN 116301645 A CN116301645 A CN 116301645A
- Authority
- CN
- China
- Prior art keywords
- data
- transmission
- transmission channel
- target
- major
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/061—Improving I/O performance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
- G06F13/38—Information transfer, e.g. on bus
- G06F13/42—Bus transfer protocol, e.g. handshake; Synchronisation
- G06F13/4282—Bus transfer protocol, e.g. handshake; Synchronisation on a serial bus, e.g. I2C bus, SPI bus
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0629—Configuration or reconfiguration of storage systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2213/00—Indexing scheme relating to interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
- G06F2213/0026—PCI express
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Communication Control (AREA)
Abstract
The application discloses a transmission channel allocation method, a transmission channel allocation device, an electronic device and a computer readable storage medium, wherein the transmission channel allocation method comprises the following steps: determining target data to be transmitted in the current time period; dividing the target data into a plurality of major classes according to the first data characteristics, and dividing the data of the major classes into a plurality of minor classes according to the second data characteristics; and allocating a corresponding stack for the data of each major class, and allocating one or more transmission channels in the corresponding stack for the data of each minor class in each major class. The data transmission efficiency is improved.
Description
Technical Field
The present invention relates to the field of storage technologies, and in particular, to a transmission channel allocation method and apparatus, an electronic device, and a computer readable storage medium.
Background
In the big data age, higher requirements are put forward on the reliability of the storage array, and particularly, the high efficiency of the storage system for storing data is achieved; the data transmission of the storage device is stressed by exponentially increasing a large amount of service data, the current data transmission link is upgraded to PCIE (high-speed serial computer expansion bus, peripheral component interconnect express) 5.0, and the work exchange efficiency of PCIE SWITCH (high-speed serial computer expansion bus exchange device) in the transmission process affects the data transmission efficiency; in the traditional channel allocation process, channels are allocated only on a physical layer, the channels are secondarily allocated in the data transmission process without utilizing the PCIE SWITCH chip channel dynamic allocation function, and the stored data information is not fully considered in the channel allocation process, so that the efficiency and the accuracy of data transmission are seriously affected.
Therefore, how to improve the data transmission efficiency is a technical problem that needs to be solved by those skilled in the art.
Disclosure of Invention
The invention aims to provide a transmission channel allocation method and device, electronic equipment and a computer readable storage medium, and data transmission efficiency is improved.
In order to achieve the above object, the present application provides a transmission channel allocation method, including:
determining target data to be transmitted in the current time period;
dividing the target data into a plurality of major classes according to the first data characteristics, and dividing the data of the major classes into a plurality of minor classes according to the second data characteristics;
and allocating a corresponding stack for the data of each major class, and allocating one or more transmission channels in the corresponding stack for the data of each minor class in each major class.
Wherein the classifying the target data into a plurality of major classes according to the first data characteristic includes:
dividing the target data into a plurality of major classes according to a data transmission stage; wherein the data transfer stage includes any one or a combination of any of processing by the processor, server transfer to the processor, and processor allocation to the storage device.
Wherein said classifying the data of the plurality of said major classes into a plurality of minor classes according to the second data characteristics comprises:
data of a plurality of the major classes is divided into a plurality of minor classes according to data type and/or data use.
Wherein said allocating one or more transmission channels in a corresponding stack for data of each subclass of each said major class comprises:
distributing one or more transmission channels in a corresponding stack for the data of each subclass according to the data volume of the data of each subclass in each subclass; wherein the number of allocated transmission channels is positively correlated with the amount of data.
Wherein after allocating one or more transmission channels in the corresponding stack for each subclass in each major class, the method further comprises:
detecting the data transmission conditions of all the transmission channels;
if the abnormal transmission channel with the blockage is detected, the data transmitted through the abnormal transmission channel is expanded to the adjacent transmission channel of the abnormal transmission channel for transmission.
The method further comprises the steps of after expanding the data transmitted through the abnormal transmission channel to the adjacent transmission channel of the abnormal transmission channel for transmission, and the method comprises the following steps:
detecting whether a target transmission channel with the data transmission rate smaller than a preset value exists or not;
if yes, expanding the data transmitted through the abnormal transmission channel to the target transmission channel for transmission;
if not, the data quantity of the target data to be transmitted in the current period is reduced.
The method further comprises the steps of after expanding the data transmitted through the abnormal transmission channel to the adjacent transmission channel of the abnormal transmission channel for transmission, and the method comprises the following steps:
and determining a target stack with the minimum current transmission pressure, and performing redundant transmission on data transmitted from the abnormal transmission channel to other transmission channels by using the transmission channel in the target stack.
To achieve the above object, the present application provides a transmission channel allocation apparatus, including:
the determining module is used for determining target data to be transmitted in the current period;
the dividing module is used for dividing the target data into a plurality of major classes according to the first data characteristics and dividing the data of the major classes into a plurality of minor classes according to the second data characteristics;
and the allocation module is used for allocating a corresponding stack for the data of each major class and allocating one or more transmission channels in the corresponding stack for the data of each minor class in each major class.
To achieve the above object, the present application provides an electronic device, including:
a memory for storing a computer program;
and a processor for implementing the steps of the transmission channel allocation method as described above when executing the computer program.
To achieve the above object, the present application provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the transmission channel allocation method as described above.
As can be seen from the above scheme, the transmission channel allocation method provided in the present application includes: determining target data to be transmitted in the current time period; dividing the target data into a plurality of major classes according to the first data characteristics, and dividing the data of the major classes into a plurality of minor classes according to the second data characteristics; and allocating a corresponding stack for the data of each major class, and allocating one or more transmission channels in the corresponding stack for the data of each minor class in each major class.
According to the transmission channel allocation method, before data transmission, the transmission channels in each stack are divided according to the data characteristics, namely, the target data to be transmitted in the current period is divided into a plurality of major categories according to the first data characteristics, the data of the major categories are divided into a plurality of minor categories according to the second data characteristics, then the data of each major category are mapped into each stack, and the data of each minor category are mapped into each transmission channel. The application also discloses a transmission channel distribution device, electronic equipment and a computer readable storage medium, and the technical effects can be achieved.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art. The accompanying drawings are included to provide a further understanding of the disclosure, and are incorporated in and constitute a part of this specification, illustrate the disclosure and together with the description serve to explain, but do not limit the disclosure. In the drawings:
fig. 1 is a flow chart illustrating a transmission channel allocation method according to an exemplary embodiment;
FIG. 2 is a block diagram of a STACK, according to an exemplary embodiment;
fig. 3 is a flow chart illustrating another transmission channel allocation method according to an exemplary embodiment;
fig. 4 is a flowchart of a transmission channel allocation method provided in an application embodiment of the present application;
FIG. 5 is a flow chart of a real-time data distribution scheme provided in an application embodiment of the present application;
FIG. 6 is a flow chart of a method for creating a replica data processing scheme provided in an embodiment of the application of the present application;
fig. 7 is a block diagram illustrating a transmission channel allocation apparatus according to an exemplary embodiment;
fig. 8 is a block diagram of an electronic device, according to an example embodiment.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. It will be apparent that the described embodiments are only some, but not all, of the embodiments of the present application. All other embodiments, which can be made by one of ordinary skill in the art based on the embodiments herein without making any inventive effort, are intended to be within the scope of the present application. In addition, in the embodiments of the present application, "first," "second," and the like are used to distinguish similar objects, and are not necessarily used to describe a particular order or sequence.
The embodiment of the application discloses a transmission channel allocation method, which improves the data transmission efficiency.
Referring to fig. 1, a flowchart of a transmission channel allocation method according to an exemplary embodiment is shown, as shown in fig. 1, including:
s101: determining target data to be transmitted in the current time period;
s102: dividing the target data into a plurality of major classes according to the first data characteristics, and dividing the data of the major classes into a plurality of minor classes according to the second data characteristics;
in this embodiment, in order to ensure that the data processing capacity of each STACK is balanced and efficient during the working process, each STACK is appropriately partitioned according to functions before data transmission. In specific implementation, firstly, target data to be transmitted in a current period is determined, data characteristics of the target data are extracted, the target data are divided into a plurality of major classes according to the first data characteristics, and data of each major class are divided into a plurality of minor classes according to the second data characteristics, so that a transmission data model tree is established.
As a possible implementation manner, the classifying the target data into a plurality of major classes according to the first data feature includes: dividing the target data into a plurality of major classes according to a data transmission stage; wherein the data transfer stage includes any one or a combination of any of processing by the processor, server transfer to the processor, and processor allocation to the storage device.
In a specific implementation, the data transmission stage may be determined according to an instruction requirement of the front end of the server, and may include that the processor is processing, the server transmits to the processor, the processor distributes to the storage device, and the like, and the target data is divided into a plurality of major classes according to the data transmission stage.
As a possible implementation manner, the classifying the data of the plurality of major classes into a plurality of minor classes according to the second data feature includes: data of a plurality of the major classes is divided into a plurality of minor classes according to data type and/or data use.
In particular implementations, the data of each major class may be further subdivided by data type and/or data usage, dividing the data of each major class into a plurality of minor classes. The data types herein may include audio types, image types, etc., and the data uses may include finance, medical, internet, etc., without being particularly limited thereto.
S103: and allocating a corresponding stack for the data of each major class, and allocating one or more transmission channels in the corresponding stack for the data of each minor class in each major class.
In a specific implementation, each major class corresponds to one STACK to ensure data processing capability balance, and the subdivision result of the minor class in data transmission corresponds to one or transmission channels in each STACK. The structure of a STACK is shown in FIG. 2, and a subclass of data may correspond to multiple consecutive lanes in the STACK, such as lane 4 or lane 8 at the upper layer. The data model STACK allocation establishes a mapping allocation mode, so that data transmission balance is guaranteed, data transmission accuracy is guaranteed, and delay conflict caused by channel multiplexing is avoided.
As a possible implementation manner, the allocating one or more transmission channels in the corresponding stack for the data of each subclass in each major class includes: distributing one or more transmission channels in a corresponding stack for the data of each subclass according to the data volume of the data of each subclass in each subclass; wherein the number of allocated transmission channels is positively correlated with the amount of data.
In a specific implementation, if the data size of a certain subclass is large, several transmission channels are allocated to the subclass, for example, an upper channel x 4 or a lower channel x 8 is allocated to the subclass, and if the data size of the certain subclass is small, a lower channel x 1 is allocated to the subclass.
According to the transmission channel allocation method provided by the embodiment of the application, before data transmission, the transmission channels in each stack are divided according to the data characteristics, namely, the target data to be transmitted in the current period is divided into a plurality of major categories according to the first data characteristics, the data of the major categories are divided into a plurality of minor categories according to the second data characteristics, then the data of each major category is mapped into each stack, and the data of each minor category is mapped into each transmission channel.
The embodiment of the application discloses a transmission channel allocation method, and compared with the previous embodiment, the technical scheme of the embodiment is further described and optimized. Specific:
referring to fig. 2, a flowchart of another transmission channel allocation method according to an exemplary embodiment is shown, and as shown in fig. 2, includes:
s201: determining target data to be transmitted in the current time period;
s202: dividing the target data into a plurality of major classes according to the first data characteristics, and dividing the data of the major classes into a plurality of minor classes according to the second data characteristics;
s203: and allocating a corresponding stack for the data of each major class, and allocating one or more transmission channels in the corresponding stack for the data of each minor class in each major class.
S204: detecting the data transmission conditions of all the transmission channels;
s205: if the abnormal transmission channel with the blockage is detected, the data transmitted through the abnormal transmission channel is expanded to the adjacent transmission channel of the abnormal transmission channel for transmission.
It can be understood that in the PCIE channel data transmission process, the situations of delay, data transmission error code, etc. caused by the unexpected situation may occur, at this time, the data transmission situation between channels is not balanced any more, and interference may be caused for transmission of a large amount of data, so as to avoid such situations, dynamic channel allocation adjustment is performed in the transmission process for the transmission channel in each STACK of PCIE SWITCH. In specific implementation, firstly, tracking of the current data transmission condition is performed in real time through a data monitoring and checking unit in the PCIE SWITCH chip, when delay blocking exists in data transmission among channels, an abnormal transmission channel is rapidly positioned, an instruction is sent to a CPU (Central Processing unit ) to read specific changes of the flow condition of the channel processing data, and after the CPU reads out specific growing conditions of the flow data in the channel, the data transmitted by the abnormal transmission channel is expanded to an adjacent transmission channel for transmission.
For example, the current PCIE SWITCH channel can be adjusted according to the growth rate, when the data of the x 1 channel is increased suddenly, the x 1 channel is expanded to the x 2 channel in the current stack to be transmitted respectively so as to ensure the data transmission rate to be improved, and when the data amount transmitted in the x 2 channel is increased suddenly, the data amount in the x 2 channel is expanded to the x 4 channel to be transmitted respectively, and so on.
As a preferred embodiment, after expanding the data transmitted through the abnormal transmission channel to the adjacent transmission channel of the abnormal transmission channel for transmission, further comprising: detecting whether a target transmission channel with the data transmission rate smaller than a preset value exists or not; if yes, expanding the data transmitted through the abnormal transmission channel to the target transmission channel for transmission; if not, the data quantity of the target data to be transmitted in the current period is reduced.
It can be understood that in the process of increasing the transmission rate of the data in the manner of expanding and adjusting the transmission channel upwards, the data in the rapid increase is distributed to the adjacent channels for transmission, so that the pressure of the rapid increase of the current data can be relieved, but the data pressure of the whole channel area after adjustment can be increased from a long time scale, so that in the process of expanding and adjusting the channel upwards again, the CPU monitors whether the data transmission rate is lower than the expected target transmission channel, if yes, the data processing flow of the previous channel area is partially transferred to the target transmission channel, if not, the task amount of the data processing of the STACK area in the current period is reduced, and the redundant task is arranged to the next period.
S206: and determining a target stack with the minimum current transmission pressure, and performing redundant transmission on data transmitted from the abnormal transmission channel to other transmission channels by using the transmission channel in the target stack.
In a specific implementation, in order to ensure that the data transmission rate is improved during channel adjustment, no error code condition occurs. When pressure exists in data transmission, after the PCIE SWITCH channel is dynamically distributed upwards, PCIE SWITCH counts total data transmission pressure on each internal STACK, and selects one STACK with the minimum current data transmission pressure, namely a target STACK, which is used for specially backing up transmission data after expansion, and the target STACK additionally backs up transmission data with the increased data quantity after the expansion of the last process channel through the dynamic channel distribution function of PCIE SWITCH, and transmits the backed-up data to a single interval of the back-end storage device, namely a duplicate data interval. After the establishment of the duplicate data interval is completed, storing the current backup data to the next time period, if the upward channel allocation process occurs in the next time period, fully establishing the duplicate process again, storing the added data to the duplicate unit until the end of each time period, and storing the newly-added storage data in each time period by the duplicate unit. After the data storage process is finished, the system automatically performs data verification, the continuous field where the error code is located is preferentially sent to the copy unit after the error code is found in the data in the verification process, when the data segment where the error code is located and the error code are located in the copy unit, the data segment in the copy unit is replaced by the data segment, and the error code is the error code brought by the improvement of the transmission rate after the channel expansion, and the repair is finished; when the error code in the data segment is not positioned in the copy unit, the error code is caused by the non-channel expansion, and at the moment, a data checking instruction is sent to the system, and the system performs positioning and error code modification.
Therefore, the distribution of the transmission channel is realized, the rapid and accurate transmission and storage of the data flow through PCIE SWITCH are facilitated, and the method comprises a unit distribution scheme based on data modeling, a real-time data distribution scheme and a copy data processing scheme. The unit allocation before data transmission is to initially partition the STACK in PCIE SWITCH, the partition mode is based on the characteristic of the stored data, the thought of modeling the stored data is provided for carrying out characteristic analysis on the data, the stored data is pre-divided into all STACK in PCIE SWITCH according to the data characteristic, and all channels are allocated, and the method for carrying out channel allocation according to the data characteristic can improve the data transmission rate. The current data flow is recorded in real time in the data transmission process, and the real-time secondary distribution of channels in the STACK is carried out according to the current data condition, wherein the distribution purpose is to store the data in the storage device at the rear end in the most efficient mode according to the dynamic change condition of the current data transmission. The processing of the duplicate data in the data transmission process is to take out the duplicate data for real-time monitoring processing in the processing process, so that the error code can be repaired in time when the data is efficiently stored, the data transmission rate is improved, and the accuracy and the reliability of the data transmission are ensured.
In the following, an application embodiment provided by the application is introduced, a PCIE SWITCH unit and a channel dynamic allocation method on a controller are designed in a unified storage array, so that data flow is convenient to quickly and accurately transmit and store, and as shown in fig. 4, the application comprises a unit allocation scheme based on data modeling before data transmission, a real-time data allocation scheme in data processing and a copy establishment data processing scheme in the processing process, wherein the three schemes are coordinated and matched, and the schemes two and three are based on the scheme one.
The unit allocation before data transmission is to perform preliminary partition on the STACK in PCIE SWITCH so as to facilitate data transmission management, and the data partition mode is to perform characteristic analysis on the data based on the characteristic of the stored data and the thought of modeling the stored data, pre-divide the stored data into all STACK of PCIE SWITCH according to the data characteristic and allocate all channels, and reserve a certain STACK space for the channel requirement of copy data transmission in PCIE SWITCH according to the actual requirement. The current data flow is recorded in real time in the data transmission process, and the real-time secondary distribution of channels in the STACK is carried out according to the current data condition, wherein the distribution purpose is to store the data in a hard disk in the most efficient mode according to the dynamic change condition of the current data transmission. The processing of the duplicate data in the data transmission process is to take out the duplicate data for real-time monitoring processing in the processing process, so that the error code can be repaired in time when the data is efficiently stored, the data transmission rate is improved, and the accuracy and the reliability of the data transmission are ensured.
The specific flow of the scheme I is as follows: the first step is to classify and extract the data characteristics, classify and evaluate the preprocessed data according to the data magnitude and transmission stage characteristics, determine the stage of data storage according to the instruction requirement of the front end of the server, classify the data in each stage process, subdivide the data in the current several major classes according to the characteristics of data use, data type and the like after classification, and establish a complete data model tree. The second step is to map the data transmission model tree to PCIE SWITCH internal STACK structure, the major class of data transmission corresponds to each STACK in the interior to ensure the data processing capability balance, the subdivision result of the minor class in the data transmission corresponds to the subdivision result of the channel transmission in each STACK, one minor class can correspond to a plurality of continuous channels in the STACK, such as the channel x 4 or the channel x 8 at the upper layer, a plurality of channels can be allocated if the data amount of a certain class in the data type is large, and the channel x 1 at the lower layer can be allocated if the data amount is small.
The real-time data distribution scheme in the data processing is the improvement of the previous stack partition and channel distribution scheme in the data transmission process. In the PCIE channel data transmission process, the conditions of time delay, data transmission error code and the like caused by unexpected conditions can occur, the data transmission conditions among the channels are not balanced any more, the transmission of large data volume can cause interference, and in order to avoid the occurrence of the conditions, a dynamic data transmission balancing scheme is provided on the basis of the previous partition allocation. According to the allocation algorithm provided by the user, the transmission channel in each STACK of PCIE SWITCH can be dynamically allocated and adjusted in the transmission process, the specific flow of the algorithm is shown as shown in fig. 5, firstly, the current data transmission condition is tracked in real time through a data monitoring and checking unit in PCIE SWITCH chips, when delay blocking exists in data transmission among channels, abnormal transmission channel block areas are rapidly located, instructions are sent to a CPU processor to read specific changes of the flow condition of data processed by the channels, after the CPU reads specific growth conditions of the data in the channels, the current PCIE SWITCH channels are adjusted according to the growth rate, when the data of the X1 channels are increased suddenly, the X1 channels are expanded to the X2 channels in the current STACK to be respectively transmitted to ensure the data transmission rate to be increased, and when the data quantity transmitted in the X2 channels is increased suddenly, the data quantity in the X2 channels is expanded upwards to the X4 channels to be respectively transmitted, and so on. In the process of increasing the transmission rate of the proliferated data in a mode of expanding and adjusting the transmission channel upwards, the proliferated data are distributed to adjacent channels for transmission, the pressure of the proliferated current data is relieved, but the data pressure of the whole channel area after adjustment can be increased from a long time scale, so that in the process of expanding and adjusting the channel upwards again, a CPU monitors whether the data transmission rate is lower than that of an expected channel or not, if yes, the data processing flow of the previous channel area is partially transferred to the area, if not, the task amount of the data processing of the STACK area in the current stage is reduced, and the redundant task is arranged to the next transmission stage.
The copy data processing scheme is established in the processing process to ensure that the error code condition can not occur while the data transmission rate is improved in the channel adjustment process. As shown in FIG. 6, a flow chart of a duplicate data processing scheme is established, after the data transmission has pressure and the PCIE SWITCH channel is dynamically distributed upwards, PCIE SWITCH counts the total data transmission pressure on each internal STACK, and selects one STACK with the minimum current data transmission pressure; the selected STACK is used for backing up the transmission data after expansion, the data quantity increased after the expansion of the last process channel is completed is additionally backed up and transmitted through the dynamic channel distribution function of PCIE SWITCH, and the backed up data is transmitted to a single interval of the back-end SSD, namely a duplicate data interval. After the establishment of the duplicate data interval is completed, storing the current backup data to the next time period, if the upward channel allocation process occurs in the next time period, fully establishing the duplicate process again, storing the added data to the duplicate unit until the end of each time period, and storing the newly-added storage data in each time period by the duplicate unit. After the data storage process is finished, the system automatically performs data verification, the continuous field where the error code is located is preferentially sent to the copy unit after the error code is found in the data in the verification process, when the data segment where the error code is located and the error code are located in the copy unit, the data segment in the copy unit is replaced by the data segment, and the error code is the error code brought by the improvement of the transmission rate after the channel expansion, and the repair is finished; when the error code in the data segment is not positioned in the copy unit, the error code is caused by the non-channel expansion, and at the moment, a data checking instruction is sent to the system, and the system performs positioning and error code modification.
A transmission channel allocation device provided in the embodiments of the present application is described below, and a transmission channel allocation device described below and a transmission channel allocation method described above may be referred to with each other.
Referring to fig. 7, a structure diagram of a transmission channel distribution device according to an exemplary embodiment is shown, as shown in fig. 7, including:
a determining module 701, configured to determine target data that needs to be transmitted in a current period;
a dividing module 702, configured to divide the target data into a plurality of major classes according to a first data feature, and divide the data of the major classes into a plurality of minor classes according to a second data feature;
an allocating module 703, configured to allocate a corresponding stack for each of the major classes of data, and allocate one or more transmission channels in the corresponding stack for each of the minor classes of data in each of the major classes.
According to the transmission channel distribution device provided by the embodiment of the application, before data transmission, the transmission channels in each stack are divided according to the data characteristics, namely, the target data to be transmitted in the current period is divided into a plurality of major categories according to the first data characteristics, the data of the major categories are divided into a plurality of minor categories according to the second data characteristics, then the data of each major category is mapped into each stack, and the data of each minor category is mapped into each transmission channel.
Based on the above embodiment, as a preferred implementation manner, the dividing module 702 includes:
the first dividing unit is used for dividing the target data into a plurality of major classes according to the data transmission stage; wherein the data transmission stage comprises any one or a combination of any of processing by the processor, transmitting the server to the processor, and distributing the processor to the storage device;
and the second dividing unit is used for dividing the data of a plurality of major categories into a plurality of minor categories according to the data type and/or the data use.
Based on the above examples, as a preferred implementation, the distribution module 70 is specifically configured to: distributing corresponding stacks for the data of each major class, and distributing one or more transmission channels in the corresponding stacks for the data of each minor class according to the data amount of the data of each minor class in each major class; wherein the number of allocated transmission channels is positively correlated with the amount of data.
On the basis of the above embodiment, as a preferred implementation manner, the method further includes:
the first detection module is used for detecting the data transmission conditions of all the transmission channels;
and the first expansion module is used for expanding the data transmitted through the abnormal transmission channel to the adjacent transmission channel of the abnormal transmission channel for transmission when the abnormal transmission channel with the blockage is detected.
On the basis of the above embodiment, as a preferred implementation manner, the method further includes:
the second detection module is used for detecting whether a target transmission channel with the data transmission rate smaller than a preset value exists or not;
the second expansion module is used for expanding the data transmitted through the abnormal transmission channel to the target transmission channel for transmission when the target transmission channel with the data transmission rate smaller than the preset value is detected;
and the reducing module is used for reducing the data quantity of the target data to be transmitted in the current period when the target transmission channel with the data transmission rate smaller than the preset value is not detected.
On the basis of the above embodiment, as a preferred implementation manner, the method further includes:
and the transmission module is used for determining a target stack with the minimum current transmission pressure, and performing redundant transmission on the data transmitted by expanding the abnormal transmission channel to other transmission channels by utilizing the transmission channel in the target stack.
The specific manner in which the various modules perform the operations in the apparatus of the above embodiments have been described in detail in connection with the embodiments of the method, and will not be described in detail herein.
Based on the hardware implementation of the program modules, and in order to implement the method of the embodiments of the present application, the embodiments of the present application further provide an electronic device, fig. 8 is a block diagram of an electronic device according to an exemplary embodiment, and as shown in fig. 8, the electronic device includes:
a communication interface 1 capable of information interaction with other devices such as network devices and the like;
and the processor 2 is connected with the communication interface 1 to realize information interaction with other devices and is used for executing the transmission channel allocation method provided by one or more technical schemes when running the computer program. And the computer program is stored on the memory 3.
Of course, in practice, the various components in the electronic device are coupled together by a bus system 4. It will be appreciated that the bus system 4 is used to enable connected communications between these components. The bus system 4 comprises, in addition to a data bus, a power bus, a control bus and a status signal bus. But for clarity of illustration the various buses are labeled as bus system 4 in fig. 8.
The memory 3 in the embodiment of the present application is used to store various types of data to support the operation of the electronic device. Examples of such data include: any computer program for operating on an electronic device.
It will be appreciated that the memory 3 may be either volatile memory or nonvolatile memory, and may include both volatile and nonvolatile memory. Wherein the nonvolatile Memory may be Read Only Memory (ROM), programmable Read Only Memory (PROM, programmable Read-Only Memory), erasable programmable Read Only Memory (EPROM, erasable Programmable Read-Only Memory), electrically erasable programmable Read Only Memory (EEPROM, electrically Erasable Programmable Read-Only Memory), magnetic random access Memory (FRAM, ferromagnetic random access Memory), flash Memory (Flash Memory), magnetic surface Memory, optical disk, or compact disk Read Only Memory (CD-ROM, compact Disc Read-Only Memory); the magnetic surface memory may be a disk memory or a tape memory. The volatile memory may be random access memory (RAM, random Access Memory), which acts as external cache memory. By way of example, and not limitation, many forms of RAM are available, such as static random access memory (SRAM, static Random Access Memory), synchronous static random access memory (SSRAM, synchronous Static Random Access Memory), dynamic random access memory (DRAM, dynamic Random Access Memory), synchronous dynamic random access memory (SDRAM, synchronous Dynamic Random Access Memory), double data rate synchronous dynamic random access memory (ddr SDRAM, double Data Rate Synchronous Dynamic Random Access Memory), enhanced synchronous dynamic random access memory (ESDRAM, enhanced Synchronous Dynamic Random Access Memory), synchronous link dynamic random access memory (SLDRAM, syncLink Dynamic Random Access Memory), direct memory bus random access memory (DRRAM, direct Rambus Random Access Memory). The memory 3 described in the embodiments of the present application is intended to comprise, without being limited to, these and any other suitable types of memory.
The method disclosed in the embodiments of the present application may be applied to the processor 2 or implemented by the processor 2. The processor 2 may be an integrated circuit chip with signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware in the processor 2 or by instructions in the form of software. The processor 2 described above may be a general purpose processor, DSP, or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, or the like. The processor 2 may implement or perform the methods, steps and logic blocks disclosed in the embodiments of the present application. The general purpose processor may be a microprocessor or any conventional processor or the like. The steps of the method disclosed in the embodiments of the present application may be directly embodied in a hardware decoding processor or implemented by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium in the memory 3 and the processor 2 reads the program in the memory 3 to perform the steps of the method described above in connection with its hardware.
The processor 2 implements corresponding flows in the methods of the embodiments of the present application when executing the program, and for brevity, will not be described in detail herein.
In an exemplary embodiment, the present application also provides a storage medium, i.e. a computer storage medium, in particular a computer readable storage medium, for example comprising a memory 3 storing a computer program executable by the processor 2 for performing the steps of the method described above. The computer readable storage medium may be FRAM, ROM, PROM, EPROM, EEPROM, flash Memory, magnetic surface Memory, optical disk, CD-ROM, etc.
Those of ordinary skill in the art will appreciate that: all or part of the steps for implementing the above method embodiments may be implemented by hardware associated with program instructions, where the foregoing program may be stored in a computer readable storage medium, and when executed, the program performs steps including the above method embodiments; and the aforementioned storage medium includes: a removable storage device, ROM, RAM, magnetic or optical disk, or other medium capable of storing program code.
Alternatively, the integrated units described above may be stored in a computer readable storage medium if implemented in the form of software functional modules and sold or used as a stand-alone product. Based on such understanding, the technical solutions of the embodiments of the present application may be embodied in essence or a part contributing to the prior art in the form of a software product stored in a storage medium, including several instructions for causing an electronic device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the methods described in the embodiments of the present application. And the aforementioned storage medium includes: a removable storage device, ROM, RAM, magnetic or optical disk, or other medium capable of storing program code.
The foregoing is merely specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily think about changes or substitutions within the technical scope of the present application, and the changes and substitutions are intended to be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims (10)
1. A transmission channel allocation method, comprising:
determining target data to be transmitted in the current time period;
dividing the target data into a plurality of major classes according to the first data characteristics, and dividing the data of the major classes into a plurality of minor classes according to the second data characteristics;
and allocating a corresponding stack for the data of each major class, and allocating one or more transmission channels in the corresponding stack for the data of each minor class in each major class.
2. The transmission channel allocation method according to claim 1, wherein said dividing said target data into a plurality of major classes according to the first data characteristic comprises:
dividing the target data into a plurality of major classes according to a data transmission stage; wherein the data transfer stage includes any one or a combination of any of processing by the processor, server transfer to the processor, and processor allocation to the storage device.
3. The transmission channel allocation method according to claim 1, wherein said dividing the data of the plurality of said major categories into a plurality of minor categories according to the second data characteristics comprises:
data of a plurality of the major classes is divided into a plurality of minor classes according to data type and/or data use.
4. The transmission channel allocation method according to claim 1, wherein said allocating one or more transmission channels in a corresponding stack for data of each of said major classes comprises:
distributing one or more transmission channels in a corresponding stack for the data of each subclass according to the data volume of the data of each subclass in each subclass; wherein the number of allocated transmission channels is positively correlated with the amount of data.
5. The transmission channel allocation method according to claim 1, wherein after allocating one or more transmission channels in a corresponding stack for each subclass in each said subclass, further comprising:
detecting the data transmission conditions of all the transmission channels;
if the abnormal transmission channel with the blockage is detected, the data transmitted through the abnormal transmission channel is expanded to the adjacent transmission channel of the abnormal transmission channel for transmission.
6. The transmission channel allocation method according to claim 5, wherein after expanding the data transmitted through the abnormal transmission channel to a transmission channel adjacent to the abnormal transmission channel for transmission, further comprising:
detecting whether a target transmission channel with the data transmission rate smaller than a preset value exists or not;
if yes, expanding the data transmitted through the abnormal transmission channel to the target transmission channel for transmission;
if not, the data quantity of the target data to be transmitted in the current period is reduced.
7. The transmission channel allocation method according to claim 5, wherein after expanding the data transmitted through the abnormal transmission channel to a transmission channel adjacent to the abnormal transmission channel for transmission, further comprising:
and determining a target stack with the minimum current transmission pressure, and performing redundant transmission on data transmitted from the abnormal transmission channel to other transmission channels by using the transmission channel in the target stack.
8. A transmission channel allocation apparatus, comprising:
the determining module is used for determining target data to be transmitted in the current period;
the dividing module is used for dividing the target data into a plurality of major classes according to the first data characteristics and dividing the data of the major classes into a plurality of minor classes according to the second data characteristics;
and the allocation module is used for allocating a corresponding stack for the data of each major class and allocating one or more transmission channels in the corresponding stack for the data of each minor class in each major class.
9. An electronic device, comprising:
a memory for storing a computer program;
processor for implementing the steps of the transmission channel allocation method according to any one of claims 1 to 7 when executing said computer program.
10. A computer readable storage medium, characterized in that the computer readable storage medium has stored thereon a computer program which, when executed by a processor, implements the steps of the transmission channel allocation method according to any of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310301797.8A CN116301645A (en) | 2023-03-24 | 2023-03-24 | Transmission channel allocation method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310301797.8A CN116301645A (en) | 2023-03-24 | 2023-03-24 | Transmission channel allocation method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116301645A true CN116301645A (en) | 2023-06-23 |
Family
ID=86816603
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310301797.8A Pending CN116301645A (en) | 2023-03-24 | 2023-03-24 | Transmission channel allocation method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116301645A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116701287A (en) * | 2023-08-09 | 2023-09-05 | 西安甘鑫科技股份有限公司 | PCIE-based multi-device compatible device expansion method |
| CN117520432A (en) * | 2023-11-30 | 2024-02-06 | 广州方舟信息科技有限公司 | Template-based data transmission method, device, equipment and storage medium |
-
2023
- 2023-03-24 CN CN202310301797.8A patent/CN116301645A/en active Pending
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116701287A (en) * | 2023-08-09 | 2023-09-05 | 西安甘鑫科技股份有限公司 | PCIE-based multi-device compatible device expansion method |
| CN116701287B (en) * | 2023-08-09 | 2023-12-08 | 西安甘鑫科技股份有限公司 | PCIE-based multi-device compatible device expansion method |
| CN117520432A (en) * | 2023-11-30 | 2024-02-06 | 广州方舟信息科技有限公司 | Template-based data transmission method, device, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN116301645A (en) | Transmission channel allocation method and device, electronic equipment and storage medium | |
| US11061580B2 (en) | Storage device and controllers included in storage device | |
| CN103688247A (en) | Block management schemes in hybrid slc/mlc memory | |
| US8850104B2 (en) | Independent management of data and parity logical block addresses | |
| KR102448787B1 (en) | System for providing retrieval service based on blockchain and method of the same | |
| US10346039B2 (en) | Memory system | |
| US10514848B2 (en) | Data storage method for selectively storing data in a buffer preset in a memory of an electronic device or an inherent buffer in an SSD | |
| CN106126124A (en) | A kind of data processing method and electronic equipment | |
| CN111984204B (en) | Data reading and writing method and device, electronic equipment and storage medium | |
| CN111562884B (en) | Data storage method and device and electronic equipment | |
| CN110389712B (en) | Data writing method and device, solid state disk and computer readable storage medium | |
| CN110928496B (en) | Data processing method and device on multi-control storage system | |
| CN112148226A (en) | Data storage method and related device | |
| CN111708812A (en) | Distributed data processing method | |
| CN115712594B (en) | Channel allocation method and device, storage system and equipment and medium | |
| WO2019072250A1 (en) | Document management method, document management system, electronic device and storage medium | |
| CN114546267B (en) | Solid state disk based on big data calculation and solid state disk system | |
| CN106708445B (en) | Link selecting method and device | |
| CN116483740B (en) | Memory data migration method and device, storage medium and electronic device | |
| CN104503924A (en) | Area distribution method and device of tiered storage system | |
| CN111506254A (en) | Distributed storage system and management method and device thereof | |
| CN109951506B (en) | Method and equipment for evaluating performance of storage cluster | |
| CN114089924B (en) | Block chain account book data storage system and method | |
| CN112650441B (en) | Stripe cache allocation method and device, electronic equipment and storage medium | |
| CN111767165A (en) | Data processing method and device and control equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |