CN112148222A - Configuration method and device of database hard disk, storage medium and electronic device - Google Patents
Configuration method and device of database hard disk, storage medium and electronic device Download PDFInfo
- Publication number
- CN112148222A CN112148222A CN202010998010.4A CN202010998010A CN112148222A CN 112148222 A CN112148222 A CN 112148222A CN 202010998010 A CN202010998010 A CN 202010998010A CN 112148222 A CN112148222 A CN 112148222A
- Authority
- CN
- China
- Prior art keywords
- disk
- database
- hard
- hard disk
- group
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0614—Improving the reliability of storage systems
- G06F3/0619—Improving the reliability of storage systems in relation to data integrity, e.g. data losses, bit errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/061—Improving I/O performance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0629—Configuration or reconfiguration of storage systems
- G06F3/0634—Configuration or reconfiguration of storage systems by changing the state or mode of one or more devices
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
- G06F3/0644—Management of space entities, e.g. partitions, extents, pools
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0683—Plurality of storage devices
- G06F3/0688—Non-volatile semiconductor memory arrays
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The embodiment of the invention provides a configuration method, a configuration device, a storage medium and an electronic device of a database hard disk, wherein the method comprises the following steps: determining a target disk group comprising a plurality of hard disks; determining a disk group working disk included in the target disk group; configuring other hard disks included in the target disk group except the disk group working disk as a master database hard disk. By the method and the device, the problem of low data processing efficiency in the related technology is solved.
Description
Technical Field
The embodiment of the invention relates to the field of communication, in particular to a configuration method and device of a database hard disk, a storage medium and an electronic device.
Background
In the related technology, the embedded device receives videos, pictures, pre-intelligent service data and local post-intelligent analysis data collected and reported by a front end, and stores the videos, the pictures, the pre-intelligent service data and the local post-intelligent analysis data in different partitions of a local hard disk in an organization mode of files, databases and the like.
The embedded device may support a multi-disk hard disk (including a multi-disk physical hard disk or a virtual hard disk), as shown in fig. 1, the multi-disk hard disk may be divided into different disk groups, and the video storage of different channels may be concurrently stored into different disk groups. Each disk group may be composed of one or more hard disks, the hard disk currently used for recording media data such as video and the like of the corresponding channel is a working disk, and only one working disk is used, when the working disk in each disk group is full, the hard disk is automatically switched to the next free disk or the hard disk with the oldest data, that is, the switching of the working disk is generated, as shown in fig. 2.
The media data such as videos and pictures are stored in the normal partition, and the database is located in the hard disk reserved partition. When the media data of the normal partition of the current working disk is fully written and the working disk needs to be switched, the database in the reserved partition needs to be migrated. During database migration, intelligent database services need to be suspended, which may cause loss of real-time data. In addition, the event reporting of the intelligent service is very frequent, and since the database and the media data (e.g., videos and pictures) are located in the same hard disk, the frequent operation of the database may cause delayed writing or loss of other data on the hard disk, or the real-time storage and playback of the videos may also indirectly affect the access efficiency of the database data.
Aiming at the problem of low data processing efficiency in the related art, no effective solution is provided at present.
Disclosure of Invention
The embodiment of the invention provides a configuration method and device of a database hard disk, a storage medium and an electronic device, which are used for at least solving the problem of low data processing efficiency in the related technology.
According to an embodiment of the present invention, a method for configuring a database hard disk is provided, including: determining a target disk group comprising a plurality of hard disks; determining a disk group working disk included in the target disk group; configuring other hard disks included in the target disk group except the disk group working disk as a master database hard disk.
According to another embodiment of the present invention, there is provided a configuration apparatus for a database hard disk, including: the first determining module is used for determining a target disk group comprising a plurality of hard disks; a second determining module for determining a disk group working disk included in the target disk group; a configuration module, configured to configure other hard disks included in the target disk group except the disk group working disk as a master database hard disk.
According to a further embodiment of the present invention, there is also provided a computer-readable storage medium having a computer program stored thereon, wherein the computer program is arranged to perform the steps of any of the above method embodiments when executed.
According to yet another embodiment of the present invention, there is also provided an electronic device, including a memory in which a computer program is stored and a processor configured to execute the computer program to perform the steps in any of the above method embodiments.
According to the invention, the disk group working disk and the main database are configured on different hard disks, so that the storage operation of the media data and the read-write operation in the database can be performed concurrently and independently, the operation of the database and the data operation on the working disk are not influenced mutually, and the data processing efficiency is effectively improved.
Drawings
FIG. 1 is a schematic diagram of a disk pack and hard disk in the related art;
fig. 2 is a schematic diagram of a working disc switching in the related art;
fig. 3 is a block diagram of a hardware structure of a mobile terminal according to a configuration method of a database hard disk according to an embodiment of the present invention;
FIG. 4 is a flow chart of a method for configuring a database hard disk according to an embodiment of the present invention;
FIG. 5 is a schematic overall flow diagram according to an embodiment of the invention;
fig. 6 is a schematic diagram of a relative relationship between an optional working disk and a master-slave database according to an embodiment of the present invention;
FIG. 7 is a schematic diagram of a write database operation according to an embodiment of the invention;
FIG. 8 is a schematic diagram of a working disk switch according to an embodiment of the present invention;
FIG. 9 is a flow chart of switching between master and slave databases according to an embodiment of the present invention;
fig. 10 is a block diagram of a configuration apparatus of a database hard disk according to an embodiment of the present invention.
Detailed Description
Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings in conjunction with the embodiments.
It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order.
As can be seen from the foregoing description, the embedded device receives data such as video, pictures, pre-intelligent service data, local post-intelligent analysis data, and the like collected and reported by the front end, and can store the data in different partitions of the local hard disk in an organization manner such as a file, a database, and the like. The storage structure condition in the aspect of the embedded device includes the following:
hard disks supporting a single disk (low-end products, few application scenes);
hard disks supporting multiple disk positions (middle-end products, more series models and more common application scenes);
the hard disk supporting the eSATA port is generally used as a backup disk and is unrelated to a database;
supporting multiple stages of expansion cabinets (higher end products, such as X86) of SAS port, each supporting multiple disk positions;
and the multi-raid disk application has better safety or storage efficiency.
In the embodiment of the present invention, a solution is provided for a problem of low data processing efficiency in an embedded device supporting a hard disk with multiple disk positions (for example, the concurrent storage efficiency of an intelligent service database in an embedded application scenario is low, and there is a possibility of intelligent service data loss during working disk switching), and the following describes the present invention with reference to the embodiment:
the method embodiments provided in the embodiments of the present application may be executed in a mobile terminal, a computer terminal, or a similar computing device. Taking the example of being operated on a mobile terminal, fig. 3 is a hardware structure block diagram of the mobile terminal of a configuration method of a database hard disk according to an embodiment of the present invention. As shown in fig. 3, the mobile terminal may comprise one or more (only one shown in fig. 3) processors 302 (the processor 302 may comprise, but is not limited to, a processing means such as a microprocessor MCU or a programmable logic device FPGA) and a memory 304 for storing data, wherein the mobile terminal may further comprise a transmission device 306 for communication functions and an input-output device 308. It will be understood by those skilled in the art that the structure shown in fig. 3 is only an illustration, and does not limit the structure of the mobile terminal. For example, the mobile terminal may also include more or fewer components than shown in FIG. 3, or have a different configuration than shown in FIG. 3.
The memory 304 may be used to store computer programs, for example, software programs and modules of application software, such as computer programs corresponding to the configuration method of the database hard disk in the embodiment of the present invention, and the processor 302 executes various functional applications and data processing by running the computer programs stored in the memory 304, so as to implement the above-mentioned method. The memory 304 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 304 may further include memory located remotely from the processor 302, which may be connected to the mobile terminal through a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The transmission device 306 is used for receiving or sending data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 306 includes a Network adapter (NIC), which can be connected to other Network devices through a base station so as to communicate with the internet. In one example, the transmission device 306 can be a Radio Frequency (RF) module, which is used to communicate with the internet in a wireless manner.
In this embodiment, a configuration method of a database hard disk is provided, and fig. 4 is a flowchart of the configuration method of the database hard disk according to the embodiment of the present invention, as shown in fig. 4, the flowchart includes the following steps:
step S402, determining a target disk group comprising a plurality of hard disks;
step S404, determining a disk group working disk included in the target disk group;
step S406, configuring other hard disks included in the target disk group except for the disk group working disk as a master database hard disk.
The above process may be used to configure a database hard disk of an embedded device, the above steps may be executed by the embedded device, for example, may be executed by a processor in the embedded device, or may be executed by other processing terminals, where the processing terminals may be terminals relatively independent from the embedded device, or terminals having a specific connection relationship with the embedded device, and the above steps may also be executed by a processor independent from the embedded device or a processor located in another device.
In the above embodiment, there may be a plurality of disk groups, and each disk group may include one or more hard disks, and before configuring the database hard disk, a target disk group may be determined from the plurality of disk groups, and the target disk group is used as the disk group currently used by the device, so as to configure the database hard disk from the target disk group.
Through the steps, the disk group working disk and the main database are configured on different hard disks, so that the storage operation of the media data and the read-write operation in the database can be performed concurrently and independently, the operation of the database and the data operation on the working disk are not affected mutually, and the data processing efficiency is effectively improved.
In an alternative embodiment, configuring the other hard disks included in the target disk group other than the disk group working disk as a master database hard disk includes: sequencing the plurality of hard disks included in the target disk group according to the sequence of hard disk parameter values from high to low; determining the hard disks ranked in the first two digits as candidate hard disks; under the condition that the candidate hard disks comprise the disk group working disks, configuring non-disk group working disks in the candidate hard disks as the main database hard disks; and under the condition that the candidate hard disks do not comprise the disk group working disks, configuring the hard disk ranked first in the candidate hard disks as the main database hard disk. In this embodiment, each hard disk in the target disk group corresponds to a hard disk parameter value, the hard disk parameter value is determined based on a specific parameter value of each hard disk, the larger the hard disk parameter value is, the better the performance of the hard disk is, and in actual operation, the hard disk with the best performance is preferably used as the hard disk where the database is located. It should be noted that, in practical applications, in addition to selecting the hard disk according to the parameter value of the hard disk, the hard disk may also be selected according to other selection conditions, for example, the hard disk may be selected according to a preset selection order (for example, the hard disk numbers are selected in an order from large to small, or the hard disk numbers are selected in an order from small to large), or the hard disk may be selected according to the size of the remaining available storage space of the hard disk, or the hard disk may be selected in a random selection manner, and the like.
In an optional embodiment, after configuring other hard disks included in the target disk group other than the disk group working disk as a master database hard disk, the method further comprises: and configuring the candidate hard disks which are not configured as the main database hard disk as standby database hard disks. In this embodiment, the spare database hard disk may be configured from the candidate hard disks selected from the foregoing embodiments, so as to ensure that the spare database hard disks are all hard disks with good performance, and in addition, in practical application, the backup database may be configured from other hard disks included in the disk group except the candidate hard disk, that is, the working disk, the master database hard disk and the backup database hard disk may be different hard disks from each other, or the working disk and the backup database hard disk may be the same hard disk, it should be noted that, under the condition that the working disk and the standby database hard disk are the same hard disk, the working disk is not changed, when the master database needs to be switched, the master database is preferably switched to the reserved partition of other non-working disks included in the target disk group, so that the switched master database and the switched working disks are prevented from being located in the same hard disk.
In an optional embodiment, before sorting the plurality of hard disks included in the target disk group in order of hard disk parameter values from high to low, the method further includes: determining hard disk parameter values of the plurality of hard disks included in the target disk group based on at least one of the following hard disk parameters: hard disk rotation speed, hard disk controller cache and hard disk health degree for indicating the bad track state of the hard disk. In this embodiment, for the hard disk rotation speed included in the hard disk parameters, the higher the hard disk rotation speed, the better the performance of the hard disk is represented, mainly because the higher rotation speed can reduce the uniform seek time and the practical read-write time of the hard disk, thereby improving the read-write speed on the hard disk; for the cache of the hard disk controller included in the hard disk parameters, the larger the cache is, the better the read-write performance of the hard disk is; for the hard disk health degree included in the hard disk parameters, the hard disk health degree is used for indicating the health state of the hard disk, for example, the bad track condition of the hard disk is indicated, and the greater the bad track degree is, the lower the hard disk health degree is. It should be noted that the hard disk parameters may include other hard disk parameters besides the above listed hard disk parameters, for example, the remaining available storage space of the hard disk, the version of the hard disk, and so on.
In an alternative embodiment, determining a target disk group including a plurality of hard disks comprises: in the case where it is determined that a plurality of disk groups exist, sorting the plurality of disk groups in order of disk group parameter values from high to low; the disc group that is ranked first is determined as the target disc group. In this embodiment, each disk group in the plurality of disk groups corresponds to a disk group parameter value, the disk group parameter value is determined based on a specific parameter value of each disk group, the larger the disk group parameter value is, the better the performance of the disk group is, and in actual operation, the disk group with the best performance is preferably used as the disk group to be actually used. It should be noted that, in practical applications, in addition to selecting the disk groups according to the parameter values of the disk groups, the disk groups may also be selected according to other selection conditions, for example, the disk groups may be selected according to a preset selection order (for example, the disk group numbers are selected in an order from large to small, or the disk group numbers are selected in an order from small to large), or the hard disks included in the hard disks are selected according to the number of the hard disks, or the hard disks are selected in a random selection manner, and the like.
In an optional embodiment, in the case that it is determined that there are a plurality of disk groups, before sorting the plurality of disk groups in order of disk group parameter values from high to low, the method further comprises: determining a disk group parameter value for each disk group of the plurality of disk groups based on at least one of: the average value of hard disk parameters of hard disks in the disk group and the code stream value of a channel corresponding to the disk group. In this embodiment, the hard disk parameter average value of the hard disks included in the disk group is a value determined by averaging hard disk parameter values of the hard disks included in the disk group, where the hard disk parameter value at least includes at least one of a hard disk rotation speed, a hard disk controller cache, and a hard disk health degree for indicating a bad track state of the hard disk; the code stream value of the channel corresponding to the disk group is substantially scored, wherein the larger the code stream value is, the more data (e.g., video data) that needs to be stored by the hard disk is, the larger the pressure of the hard disk is, the lower the score is correspondingly. It should be noted that the above disk group parameters may include, in addition to the above listed disk group parameters, other disk group parameters, such as the number of free hard disks of the hard disks included in the disk group, identification information of the disk group, and the like.
In an optional embodiment, after configuring other hard disks included in the target disk group other than the disk group working disk as a master database hard disk, the method further comprises: under the condition that the disk group working disks need to be switched, judging whether a target disk group working disk to be switched is the main database hard disk or not; directly switching the disk group working disk to the target disk group working disk under the condition that the judgment result is that the target disk group working disk is not the main database hard disk; and switching a master database in the master database hard disks to other hard disks except the target disk group working disk under the condition that the target disk group working disk is the master database hard disk according to the judgment result. In this embodiment, after the working disks in the disk group are fully written, or after the working disks are in an abnormal state, the working disks need to be switched, for example, the working disks may be automatically switched to a next free disk or a hard disk with the oldest data, or to a free disk with the largest remaining storage space, or the like. In addition, in practical application, when the target disk group working disk is determined to be the hard disk where the master database is located, it may be considered to reselect the target disk group working disk, that is, to select another hard disk not configured with the master database as the target disk group working disk, so that the master database does not need to be switched.
In an optional embodiment, switching the master database in the master database hard disks to other hard disks except the target disk group working disk comprises: and when determining that the target disk group is configured with a spare database hard disk, switching a main database in the main database hard disk and a spare database in the spare database hard disk in the following way to switch the main database to the spare database hard disk: suspending a batch flash thread of a buffer queue, wherein the buffer queue is used for taking out data processing requests from the buffer queue in batch and sequentially executing the taken out data processing requests in the main database, and the buffer queue is used for sequentially buffering the received data processing requests; suspending a main and standby synchronous monitoring thread, wherein the main and standby synchronous monitoring thread is used for monitoring the hard disk state of a main and standby database, and dynamically synchronizing the incrementally updated main database data into a standby database based on a monitoring result; performing switching between the master database and the standby database if it is determined that the data in the master database and the standby database are consistent, synchronizing the master database and the standby database if it is determined that the data in the master database and the standby database are inconsistent, and performing switching between the master database and the standby database if it is determined that the data in the master database and the standby database are consistent; and opening the batch flash thread of the cache queue and the main and standby synchronous monitoring thread. In this embodiment, in order to improve the write efficiency of the master database more greatly, a buffer queue mechanism may be added in advance, where the buffer queue mechanism may receive write requests of different upper-layer service modules to the master database, where the received write requests may be sequentially stored in the buffer queue. In the embodiment of the present invention, a timing write-through thread (i.e., the above-mentioned batch write-through thread for the cache queue) may be additionally configured to take out write requests from the cache queue in batch and write the write requests into the primary database, and in addition, a timing monitor thread (or called primary and secondary synchronous monitor thread) may be additionally configured, which is mainly used for monitoring the hard disk activity of the primary and secondary databases and dynamically synchronizing the incrementally updated primary database data into the secondary database. Under the condition that the threads are established in advance, the timing flashing thread needs to be paused (the cache queue can still normally receive the write request and indicates that the write request is temporarily not executed, and after the switching of the database is completed, the write request in the cache queue is executed, so that data loss is avoided, and data synchronization in the main and standby databases is ensured), that is, the operation of flashing the data corresponding to the write request in the cache queue in the main database is paused, the timing monitoring thread is paused, and the timing flashing thread, namely the timing monitoring thread, is restarted after the switching of the database is completed, so that the problem of data asynchronization in the main and standby databases is avoided.
The invention is described below with the disk group in the embedded device as an example:
in this embodiment, details of design of the technology will be developed in detail from the four scenarios of program initialization, data library writing, data library reading, and working disk switching, fig. 5 is a schematic overall flow diagram according to an embodiment of the present invention, and the following describes the four scenarios in detail with reference to fig. 5:
1) program initialization
The embedded application layer manages hard disks based on the disk group, the databases are divided into a main database and a standby database, the main database and the standby database are respectively stored in different hard disks of the same disk group, the main database is used for database reading and writing operation of normal services, and the standby database is used for disaster recovery and auxiliary working disks to quickly switch the databases. Therefore, the disk group in which the database is located requires that at least one disk group with a number of hard disks greater than 1 exists or a free disk group with a single hard disk exists.
If a plurality of database disk groups which meet the conditions exist, determining the comprehensive score of each disk group according to the hard disk scores under the specific disk groups, wherein the highest comprehensive score is the database disk group.
And then sequentially selecting two hard disks with highest hard disk scores and highest hard disk scores from the database disk group as storage media of the main database and the standby database, wherein the hard disk of the current non-disk group working disk is selected as the main database hard disk, and the hard disk in the current working disk is selected as the standby database. If neither of the two hard disks is a disk group working disk, the hard disk with the highest score is a main database hard disk, and fig. 6 is a schematic diagram of a relative relationship between an optional working disk and a main database according to an embodiment of the present invention.
The scoring rules of the hard disk and the disk group can be specifically seen in table 1:
TABLE 1
2) Write data library
The main database and the current working disk of the disk group belong to different hard disks, that is, the storage operation of media data such as video and the like and the database reading operation can be performed concurrently and independently, and in order to improve the writing efficiency of the main database more greatly, a buffer queue mechanism is added, which is specifically shown in fig. 7.
The method comprises the steps of receiving write requests of different service modules of an upper layer under a cache queue mechanism, sequentially storing the write requests into a queue, and providing a write request completion notification mechanism. One timing refreshing thread is responsible for taking out writing requests from the cache queue in batch, refreshing the writing requests into the main database, monitoring the hard disk activity condition of the main database and the standby database through the other timing monitoring thread, and dynamically synchronizing the incrementally updated main database data into the standby database.
3) Read database
The main database and the current working disk of the disk group belong to different hard disks, namely, the storage operation of media data such as video and the like and the database reading operation can be carried out concurrently and independently, and the data in the main database can be directly read.
4) Working disc switching
When the normal partition of the working disk is full, it needs to switch to the next free disk in the disk group, and there are two cases:
if the next spare disk is not the hard disk where the main database is located, the working disk can be directly switched without database migration or main and standby database switching;
the next spare disk is a hard disk where the master database is located, and database migration or master-slave database switching needs to be performed, a schematic diagram of working disk switching may be shown in fig. 8, where after the working disk is switched, the process shown in fig. 9 needs to be executed, which specifically includes:
s902, stopping the batch flash thread of the buffer queue;
s904, stopping the main and standby synchronous monitoring threads;
s906, checking whether the main database and the standby database are consistent, if so, executing the step S910, otherwise, executing the step S908;
s908, performing incremental synchronization on the main database and the standby database;
s910, switching between the main database and the standby database;
s912, starting a cache queue batch flash thread;
s914, starting the main and standby synchronous monitoring thread.
Under the condition, the cache queue can still normally receive the write requests, and after the switching of the main database and the standby database is completed, the write requests in the cache queue are written into the switched main database in batches, so that the loss of service data during the switching of the working disk is prevented.
Through the above description of the embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but the former is a better implementation mode in many cases. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (e.g., ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal device (e.g., a mobile phone, a computer, a server, or a network device) to execute the method according to the embodiments of the present invention.
In this embodiment, a configuration device for a database hard disk is further provided, where the configuration device is used to implement the foregoing embodiments and preferred embodiments, and details are not repeated for what has been described. As used below, the term "module" may be a combination of software and/or hardware that implements a predetermined function. Although the means described in the embodiments below are preferably implemented in software, an implementation in hardware, or a combination of software and hardware is also possible and contemplated.
Fig. 10 is a block diagram of a configuration apparatus of a database hard disk according to an embodiment of the present invention, and as shown in fig. 10, the apparatus includes:
a first determining module 1002, configured to determine a target disk group including a plurality of hard disks;
a second determining module 1004 for determining a disk group working disk included in the target disk group;
a configuration module 1006, configured to configure other hard disks included in the target disk group except for the disk group working disk as a master database hard disk.
In an alternative embodiment, the configuration module 1006 is configured to configure other hard disks included in the target disk group except for the disk group working disk as a master database hard disk by: sequencing the plurality of hard disks included in the target disk group according to the sequence of hard disk parameter values from high to low; determining the hard disks ranked in the first two digits as candidate hard disks; under the condition that the candidate hard disks comprise the disk group working disks, configuring non-disk group working disks in the candidate hard disks as the main database hard disks; and under the condition that the candidate hard disks do not comprise the disk group working disks, configuring the hard disk ranked first in the candidate hard disks as the main database hard disk.
In an optional embodiment, the apparatus is further configured to configure, after configuring other hard disks included in the target disk group except for the disk group working disk as a master database hard disk, a candidate hard disk not configured as the master database hard disk among the candidate hard disks as a standby database hard disk.
In an optional embodiment, the apparatus is further configured to, before the plurality of hard disks included in the target disk group are sorted in order of hard disk parameter values from high to low, determine hard disk parameter values of the plurality of hard disks included in the target disk group based on at least one of the following hard disk parameters: hard disk rotation speed, hard disk controller cache and hard disk health degree for indicating the bad track state of the hard disk.
In an alternative embodiment, the first determining module 1002 may determine the target disk group including a plurality of hard disks by: in the case where it is determined that a plurality of disk groups exist, sorting the plurality of disk groups in order of disk group parameter values from high to low; the disc group that is ranked first is determined as the target disc group.
In an optional embodiment, the apparatus is further configured to, in a case where it is determined that there are a plurality of disk groups, determine a disk group parameter value of each of the plurality of disk groups based on at least one of the following disk group parameters before sorting the plurality of disk groups in order of disk group parameter value from high to low: the average value of hard disk parameters of hard disks in the disk group and the code stream value of a channel corresponding to the disk group.
In an optional embodiment, the apparatus is further configured to, after configuring other hard disks included in the target disk group except for the disk group working disk as a master database hard disk, in a case where it is determined that the disk group working disk needs to be switched, determine whether the target disk group working disk to be switched is the master database hard disk; directly switching the disk group working disk to the target disk group working disk under the condition that the judgment result is that the target disk group working disk is not the main database hard disk; and switching a master database in the master database hard disks to other hard disks except the target disk group working disk under the condition that the target disk group working disk is the master database hard disk according to the judgment result.
In an optional embodiment, the apparatus is configured to switch the master database in the master database hard disk to another hard disk except for the target disk group working disk by: and when determining that the target disk group is configured with a spare database hard disk, switching a main database in the main database hard disk and a spare database in the spare database hard disk in the following way to switch the main database to the spare database hard disk: suspending a batch flash thread of a buffer queue, wherein the buffer queue is used for taking out data processing requests from the buffer queue in batch and sequentially executing the taken out data processing requests in the main database, and the buffer queue is used for sequentially buffering the received data processing requests; suspending a main and standby synchronous monitoring thread, wherein the main and standby synchronous monitoring thread is used for monitoring the hard disk state of a main and standby database, and dynamically synchronizing the incrementally updated main database data into a standby database based on a monitoring result; performing switching between the master database and the standby database if it is determined that the data in the master database and the standby database are consistent, synchronizing the master database and the standby database if it is determined that the data in the master database and the standby database are inconsistent, and performing switching between the master database and the standby database if it is determined that the data in the master database and the standby database are consistent; and opening the batch flash thread of the cache queue and the main and standby synchronous monitoring thread.
It should be noted that, the above modules may be implemented by software or hardware, and for the latter, the following may be implemented, but not limited to: the modules are all positioned in the same processor; alternatively, the modules are respectively located in different processors in any combination.
Embodiments of the present invention also provide a computer-readable storage medium having a computer program stored thereon, wherein the computer program is arranged to perform the steps of any of the above-mentioned method embodiments when executed.
In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to: various media capable of storing computer programs, such as a usb disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic disk, or an optical disk.
Embodiments of the present invention also provide an electronic device comprising a memory having a computer program stored therein and a processor arranged to run the computer program to perform the steps of any of the above method embodiments.
In an exemplary embodiment, the electronic apparatus may further include a transmission device and an input/output device, wherein the transmission device is connected to the processor, and the input/output device is connected to the processor.
For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary embodiments, and details of this embodiment are not repeated herein.
It will be apparent to those skilled in the art that the various modules or steps of the invention described above may be implemented using a general purpose computing device, they may be centralized on a single computing device or distributed across a network of computing devices, and they may be implemented using program code executable by the computing devices, such that they may be stored in a memory device and executed by the computing device, and in some cases, the steps shown or described may be performed in an order different than that described herein, or they may be separately fabricated into various integrated circuit modules, or multiple ones of them may be fabricated into a single integrated circuit module. Thus, the present invention is not limited to any specific combination of hardware and software.
The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the principle of the present invention should be included in the protection scope of the present invention.
Claims (11)
1. A configuration method of a database hard disk is characterized by comprising the following steps:
determining a target disk group comprising a plurality of hard disks;
determining a disk group working disk included in the target disk group;
configuring other hard disks included in the target disk group except the disk group working disk as a master database hard disk.
2. The method of claim 1, wherein configuring the other hard disks included in the target disk group other than the disk group working disk as a master database hard disk comprises:
sequencing the plurality of hard disks included in the target disk group according to the sequence of hard disk parameter values from high to low;
determining the hard disks ranked in the first two digits as candidate hard disks;
under the condition that the candidate hard disks comprise the disk group working disks, configuring non-disk group working disks in the candidate hard disks as the main database hard disks;
and under the condition that the candidate hard disks do not comprise the disk group working disks, configuring the hard disk ranked first in the candidate hard disks as the main database hard disk.
3. The method of claim 2, wherein after configuring other hard disks included in the target disk group other than the disk group working disk as a master database hard disk, the method further comprises:
and configuring the candidate hard disks which are not configured as the main database hard disk as standby database hard disks.
4. The method of claim 2, wherein before sorting the plurality of hard disks included in the target disk group in order of hard disk parameter values from high to low, the method further comprises:
determining hard disk parameter values of the plurality of hard disks included in the target disk group based on at least one of the following hard disk parameters: hard disk rotation speed, hard disk controller cache and hard disk health degree for indicating the bad track state of the hard disk.
5. The method of claim 1, wherein determining a target disk group comprising a plurality of hard disks comprises:
in the case where it is determined that a plurality of disk groups exist, sorting the plurality of disk groups in order of disk group parameter values from high to low;
the disc group that is ranked first is determined as the target disc group.
6. The method of claim 5, wherein in the case that it is determined that there are a plurality of disk groups, before sorting the plurality of disk groups in order of disk group parameter values from high to low, the method further comprises:
determining a disk group parameter value for each disk group of the plurality of disk groups based on at least one of: the average value of hard disk parameters of hard disks in the disk group and the code stream value of a channel corresponding to the disk group.
7. The method of claim 1, wherein after configuring other hard disks included in the target disk group other than the disk group working disk as a master database hard disk, the method further comprises:
under the condition that the disk group working disks need to be switched, judging whether a target disk group working disk to be switched is the main database hard disk or not;
directly switching the disk group working disk to the target disk group working disk under the condition that the judgment result is that the target disk group working disk is not the main database hard disk;
and switching a master database in the master database hard disks to other hard disks except the target disk group working disk under the condition that the target disk group working disk is the master database hard disk according to the judgment result.
8. The method of claim 7, wherein switching the master database in the master database hard disk to another hard disk except the target disk group working disk comprises:
and when determining that the target disk group is configured with a spare database hard disk, switching a main database in the main database hard disk and a spare database in the spare database hard disk in the following way to switch the main database to the spare database hard disk:
suspending a batch flash thread of a buffer queue, wherein the buffer queue is used for taking out data processing requests from the buffer queue in batch and sequentially executing the taken out data processing requests in the main database, and the buffer queue is used for sequentially buffering the received data processing requests;
suspending a main and standby synchronous monitoring thread, wherein the main and standby synchronous monitoring thread is used for monitoring the hard disk state of a main and standby database, and dynamically synchronizing the incrementally updated main database data into a standby database based on a monitoring result;
performing switching between the master database and the standby database if it is determined that the data in the master database and the standby database are consistent, synchronizing the master database and the standby database if it is determined that the data in the master database and the standby database are inconsistent, and performing switching between the master database and the standby database if it is determined that the data in the master database and the standby database are consistent;
and opening the batch flash thread of the cache queue and the main and standby synchronous monitoring thread.
9. An apparatus for configuring a hard disk of a database, comprising:
the first determining module is used for determining a target disk group comprising a plurality of hard disks;
a second determining module for determining a disk group working disk included in the target disk group;
a configuration module, configured to configure other hard disks included in the target disk group except the disk group working disk as a master database hard disk.
10. A computer-readable storage medium, in which a computer program is stored, wherein the computer program is arranged to perform the method of any of claims 1 to 8 when executed.
11. An electronic device comprising a memory and a processor, wherein the memory has stored therein a computer program, and wherein the processor is arranged to execute the computer program to perform the method of any of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010998010.4A CN112148222B (en) | 2020-09-21 | 2020-09-21 | Configuration method and device of database hard disk, storage medium and electronic device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010998010.4A CN112148222B (en) | 2020-09-21 | 2020-09-21 | Configuration method and device of database hard disk, storage medium and electronic device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112148222A true CN112148222A (en) | 2020-12-29 |
| CN112148222B CN112148222B (en) | 2023-08-25 |
Family
ID=73893607
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010998010.4A Active CN112148222B (en) | 2020-09-21 | 2020-09-21 | Configuration method and device of database hard disk, storage medium and electronic device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112148222B (en) |
Citations (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030229650A1 (en) * | 2002-06-06 | 2003-12-11 | International Business Machines Corporation | Method and apparatus for selective caching of transactions in a computer system |
| US20050086263A1 (en) * | 2003-09-04 | 2005-04-21 | Oracle International Corporation | Self-managing performance statistics repository for databases |
| US20100268876A1 (en) * | 2009-04-20 | 2010-10-21 | Microsoft Corporation | Sliding-window multi-class striping |
| CN103116661A (en) * | 2013-03-20 | 2013-05-22 | 广东宜通世纪科技股份有限公司 | Data processing method of database |
| CN103780417A (en) * | 2012-10-23 | 2014-05-07 | 上海盛大网络发展有限公司 | Database failure transfer method based on cloud hard disk and device thereof |
| US20160203054A1 (en) * | 2015-01-12 | 2016-07-14 | Actifio, Inc. | Disk group based backup |
| CN106227470A (en) * | 2016-08-05 | 2016-12-14 | 浪潮(北京)电子信息产业有限公司 | A kind of SRM method and device |
| US20170316017A1 (en) * | 2016-04-28 | 2017-11-02 | inXtron. Inc. | Multi hard-disk file management system and method thereof |
| CN109522176A (en) * | 2017-09-20 | 2019-03-26 | 阿里巴巴集团控股有限公司 | A kind of monitoring method of storage system, device, electronic equipment and storage medium |
| CN109684298A (en) * | 2018-09-07 | 2019-04-26 | 平安科技（深圳）有限公司 | Database migration method, apparatus, equipment and storage medium |
| CN110515535A (en) * | 2018-05-22 | 2019-11-29 | 杭州海康威视数字技术股份有限公司 | Disk read-write control method, device, electronic equipment and storage medium |
| CN110658995A (en) * | 2019-09-05 | 2020-01-07 | 苏州浪潮智能科技有限公司 | Solid state disk, configuration data management method and device thereof, and storage medium |
| CN110750213A (en) * | 2019-09-09 | 2020-02-04 | 华为技术有限公司 | Hard disk management method and device |
| CN110851443A (en) * | 2019-10-28 | 2020-02-28 | 网联清算有限公司 | Database storage management method and device, storage medium and electronic equipment |
| CN110858122A (en) * | 2018-08-23 | 2020-03-03 | 杭州海康威视系统技术有限公司 | Method and device for storing data |
| US20200097355A1 (en) * | 2018-09-26 | 2020-03-26 | Beijing Baidu Netcom Science And Technology Co., Ltd. | Method and apparatus for outputting information |
-
2020
- 2020-09-21 CN CN202010998010.4A patent/CN112148222B/en active Active
Patent Citations (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030229650A1 (en) * | 2002-06-06 | 2003-12-11 | International Business Machines Corporation | Method and apparatus for selective caching of transactions in a computer system |
| US20050086263A1 (en) * | 2003-09-04 | 2005-04-21 | Oracle International Corporation | Self-managing performance statistics repository for databases |
| US20100268876A1 (en) * | 2009-04-20 | 2010-10-21 | Microsoft Corporation | Sliding-window multi-class striping |
| CN103780417A (en) * | 2012-10-23 | 2014-05-07 | 上海盛大网络发展有限公司 | Database failure transfer method based on cloud hard disk and device thereof |
| CN103116661A (en) * | 2013-03-20 | 2013-05-22 | 广东宜通世纪科技股份有限公司 | Data processing method of database |
| US20160203054A1 (en) * | 2015-01-12 | 2016-07-14 | Actifio, Inc. | Disk group based backup |
| US20170316017A1 (en) * | 2016-04-28 | 2017-11-02 | inXtron. Inc. | Multi hard-disk file management system and method thereof |
| CN106227470A (en) * | 2016-08-05 | 2016-12-14 | 浪潮(北京)电子信息产业有限公司 | A kind of SRM method and device |
| CN109522176A (en) * | 2017-09-20 | 2019-03-26 | 阿里巴巴集团控股有限公司 | A kind of monitoring method of storage system, device, electronic equipment and storage medium |
| CN110515535A (en) * | 2018-05-22 | 2019-11-29 | 杭州海康威视数字技术股份有限公司 | Disk read-write control method, device, electronic equipment and storage medium |
| CN110858122A (en) * | 2018-08-23 | 2020-03-03 | 杭州海康威视系统技术有限公司 | Method and device for storing data |
| CN109684298A (en) * | 2018-09-07 | 2019-04-26 | 平安科技（深圳）有限公司 | Database migration method, apparatus, equipment and storage medium |
| US20200097355A1 (en) * | 2018-09-26 | 2020-03-26 | Beijing Baidu Netcom Science And Technology Co., Ltd. | Method and apparatus for outputting information |
| CN110658995A (en) * | 2019-09-05 | 2020-01-07 | 苏州浪潮智能科技有限公司 | Solid state disk, configuration data management method and device thereof, and storage medium |
| CN110750213A (en) * | 2019-09-09 | 2020-02-04 | 华为技术有限公司 | Hard disk management method and device |
| CN110851443A (en) * | 2019-10-28 | 2020-02-28 | 网联清算有限公司 | Database storage management method and device, storage medium and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112148222B (en) | 2023-08-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106777126B (en) | Data online migration method supporting heterogeneous time sequence database | |
| KR101231563B1 (en) | Real time data replication | |
| CN110413685B (en) | Database service switching method, device, readable storage medium and computer equipment | |
| CN111880993B (en) | Cluster operation and maintenance state diagnosis method, operation and maintenance monitoring system, terminal and storage medium | |
| CN108628717A (en) | A kind of Database Systems and monitoring method | |
| CN107919977B (en) | Online capacity expansion and online capacity reduction method and device based on Paxos protocol | |
| JP2008108145A (en) | Computer system, and management method of data using the same | |
| US20090063807A1 (en) | Data redistribution in shared nothing architecture | |
| US11281509B2 (en) | Shared memory management | |
| CN111049928B (en) | Data synchronization method, system, electronic device and computer readable storage medium | |
| US11513939B2 (en) | Multi-core I/O trace analysis | |
| CN111475104A (en) | Storage device data online migration method, system, terminal and storage medium | |
| CN112199240A (en) | Method for switching nodes during node failure and related equipment | |
| US20070204071A1 (en) | Apparatus, system, and method for automated device configuration and testing | |
| CN112148222A (en) | Configuration method and device of database hard disk, storage medium and electronic device | |
| CN108733808A (en) | Big data software systems switching method, system, terminal device and storage medium | |
| CN110554992B (en) | Distributed metadata path management method, system, terminal and storage medium | |
| CN115970295A (en) | Request processing method and device and electronic equipment | |
| US20220391116A1 (en) | Control Device Switching Method, Control Device, and Storage System | |
| US10884881B2 (en) | Scale-out storage system and configuration information control method for implementing high-availability, high-speed failover | |
| CN113609245A (en) | Index fragment expansion method and system | |
| EP3869314A1 (en) | Control device switching method, control device and storage system | |
| CN108599982A (en) | Data reconstruction method and relevant device | |
| US9641613B2 (en) | Volume hierarchy download in a storage area network | |
| CN115858101B (en) | Method and device for isolating container resource view and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |