WO2021237431A1 - Procédé et appareil de traitement de données, dispositif de traitement et système de stockage de données - Google Patents

Procédé et appareil de traitement de données, dispositif de traitement et système de stockage de données Download PDF

Info

Publication number
WO2021237431A1
WO2021237431A1 PCT/CN2020/092196 CN2020092196W WO2021237431A1 WO 2021237431 A1 WO2021237431 A1 WO 2021237431A1 CN 2020092196 W CN2020092196 W CN 2020092196W WO 2021237431 A1 WO2021237431 A1 WO 2021237431A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
database
stored
database instance
instance
Prior art date
Application number
PCT/CN2020/092196
Other languages
English (en)
Chinese (zh)
Inventor
郭子亮
Original Assignee
深圳市欢太科技有限公司
Oppo广东移动通信有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市欢太科技有限公司, Oppo广东移动通信有限公司 filed Critical 深圳市欢太科技有限公司
Priority to PCT/CN2020/092196 priority Critical patent/WO2021237431A1/fr
Priority to CN202080094838.5A priority patent/CN115004160A/zh
Publication of WO2021237431A1 publication Critical patent/WO2021237431A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance

Definitions

  • This application relates to the field of databases, and more specifically, to a data processing method, device, processing equipment, and data storage system.
  • the data to be stored can be transmitted to the back-end database instance by the intermediate device, and then the data storage can be performed by the database instance.
  • the intermediate device still has the risk of data loss.
  • this application proposes a data processing method, device, processing equipment and data storage system to improve the above problems.
  • this application provides a data processing method that is applied to a database agent included in a data storage system.
  • the data storage system also includes a database instance for storing and reading data.
  • the method includes: the database agent terminal obtains the data to be stored; obtains the operating state of the database instance; if the operating state indicates that the database instance is in a fault state, writing the data to be stored to the database agent The corresponding disk at the end.
  • this application provides a data processing method applied to a data storage system.
  • the data storage system includes a gateway, a database proxy, and a database instance.
  • the database instance is used to store and read data.
  • the method includes: the gateway receives the to-be-stored data sent by the client and sends the to-be-stored data to a database agent; the database agent obtains the to-be-stored data and obtains the running status of the database instance If the running state indicates that the database instance is in a fault state, the data to be stored is written to the disk corresponding to the database agent.
  • this application provides a data processing device that runs on a database agent included in a data storage system.
  • the data storage system also includes a database instance, which is used to store and read data.
  • the device includes: a data acquisition unit for acquiring data to be stored; a state detection unit for acquiring the operating state of the database instance; and a data processing unit for if the operating state indicates that the database instance is in a faulty state, The data to be stored is written into the disk corresponding to the database agent.
  • this application provides a data storage system.
  • the data storage system includes a gateway, a database proxy, and a database instance.
  • the database instance is used to store and read data; and the gateway is used to receive data.
  • the data to be stored sent by the client, and the data to be stored is sent to the database agent; the database agent is used to obtain the data to be stored and obtain the running status of the database instance, if the running The state indicates that the database instance is in a fault state, and the data to be stored is written to the disk corresponding to the database agent.
  • the present application provides a processing device including one or more processors and a memory; one or more programs are stored in the memory and configured to be executed by the one or more processors, The one or more programs are configured to perform the methods described above.
  • the present application provides a computer-readable storage medium having program code executable by a processor, the program code causing the processor to execute the above-mentioned method.
  • the present application provides a data processing method, device, processing equipment, and data storage system.
  • the data storage system includes a database agent and a database instance
  • the database agent will first obtain the data to be stored after obtaining the data to be stored.
  • the operating state of the database instance if the operating state indicates that the database instance is in a fault state, the data to be stored is written to the disk corresponding to the database agent. Therefore, when the running state indicates that the database instance is in a fault state, the data to be stored will be written to the disk corresponding to the database agent, instead of being written into the memory corresponding to the database agent. Avoid memory overflow caused by too much data stored in the memory, and then avoid database agent crash and data loss in memory caused by memory overflow, so as to enhance data security.
  • Figure 1 shows a schematic diagram of an application scenario proposed by an embodiment of the present application
  • Figure 2 shows a schematic diagram of another application scenario proposed by an embodiment of the present application
  • FIG. 3 shows a flowchart of a data processing method proposed in an embodiment of the present application
  • FIG. 4 shows a flowchart of another data processing method proposed by an embodiment of the present application.
  • FIG. 5 shows a flowchart of yet another data processing method proposed in an embodiment of the present application
  • FIG. 6 shows a flowchart of yet another data processing method proposed by an embodiment of the present application.
  • Fig. 7 shows a structural block diagram of a data processing device proposed in an embodiment of the present application.
  • Fig. 8 shows a structural block diagram of a processing device proposed in an embodiment of the present application.
  • Fig. 9 is a storage unit for storing or carrying program code for implementing the data processing method according to the embodiment of the present application according to an embodiment of the present application.
  • data storage is inevitable.
  • special intermediate devices will be configured to obtain the data to be stored, and then the data to be stored will be sent to the back-end database instances, and these database instances will then store the data.
  • the intermediate device will store the data to be stored that has not been sent to the database instance in the local memory.
  • the inventor found in the research of this related high-availability method If there is too much data currently needed to be stored in the memory, it will cause a memory overflow, which in turn will cause the program that relies on the memory in the intermediate device to crash, and at the same time cause the data in the memory to be lost.
  • the inventor proposes the data processing method, device, processing equipment, and data storage system in this application.
  • the data storage system includes a database agent and a database instance
  • the database agent is After the data to be stored is reached, the operating state of the database instance is first obtained. If the operating state indicates that the database instance is in a fault state, the data to be stored is written to the disk corresponding to the database agent.
  • the running state indicates that the database instance is in a fault state
  • the data to be stored will be written to the disk corresponding to the database agent, instead of being written into the memory corresponding to the database agent. Avoid memory overflow caused by too much data stored in the memory, and then avoid database agent crash and data loss in memory caused by memory overflow, so as to enhance data security.
  • the data storage system 100 shown in FIG. 1 includes a database proxy terminal 110, a database instance 120, a gateway 130 and a query terminal 140.
  • the database proxy terminal 110 is used to forward the data to be stored and the query request.
  • the data to be stored may be data obtained by the gateway 130 from the client, or other data corresponding to the client obtained by the gateway 130.
  • the database agent 110 may forward the obtained data to be stored to the database instance 120, and then the database instance 120 may store the received data to be stored.
  • the database agent 110 may select one of the multiple database instances 120 as the receiving target for sending the data to be stored.
  • the client can send a query request to the database agent 110 through the query terminal 140, so that the database agent 110 sends the received query request to the corresponding database instance 120, so that the database instance 120 that has received the query request executes the query request. Inquiry request.
  • the number of database agents 110 may also be multiple.
  • the gateway 130 may select one or more database agents from the plurality of database agents 110 as the receiving target for storing the data to be stored.
  • the query terminal and the gateway can communicate with the client through the UDP (User Datagram Protocol) protocol or the TCP (Transmission Control Protocol) protocol, and the query terminal and the gateway can communicate with the database agent through the Http protocol.
  • the multiple database agents may all run in the same hardware device or may run in different hardware devices.
  • the multiple database instances may all be in the same hardware device, or they may run on different hardware devices.
  • the database instance 120 may be a database instance of a time series database.
  • the time series database may be an Influxdb database.
  • a data processing method provided by this application is applied to a database agent included in a data storage system, and the data storage system further includes a database instance, which is used to store and read data,
  • the method includes:
  • the data to be stored is the data that will be subsequently stored through the database instance.
  • the database agent can obtain the data to be stored in a variety of ways.
  • the database agent can use the data uploaded by the client as the data to be stored.
  • the user can upload the file to be stored through the client, and the client will send the file uploaded by the user to the database agent, and the database agent will use the file as the data to be stored after receiving the file.
  • the client can also automatically upload some data for storage on a regular basis.
  • the client can be configured with a monitoring task of a specified component, and the client can run the corresponding component collected during the execution of the monitoring task.
  • the data is automatically uploaded to the database agent through the client. After the database agent receives the operating data, it can use the operating data as the data to be stored.
  • the components can be processors, memory, or disks.
  • the client can communicate with the corresponding back-end server to operate on the data in the back-end server, and the process data and result data generated during the operation of the data in the back-end server can be represented by the database End as the data to be stored.
  • the process data can be understood as the data that records the operation behavior. For example, if the client opens the file A in the server, the server will record the behavior of the client opening A as a process data. Also, if the client modifies the data in file A, the modification behavior itself can also be used as process data, and the modified data obtained after the modification can be used as result data.
  • a database instance can be understood as a process program responsible for storing, reading, deleting, or changing data.
  • the running state characterizes the current state of the database instance.
  • the database instance corresponds to a faulty state or a normal state. Among them, when the database instance is in a faulty state, the database instance cannot store, read, delete, or change data. When the database instance is in a normal state, you can perform data storage, read, delete, or change operations. .
  • the database agent has multiple ways to obtain the running status of the database instance.
  • the data agent can send a status acquisition request to the database instance before sending the data to be stored. If the information returned by the database instance in response to the status acquisition request can be successfully obtained, the operating status of the database instance can be determined To signify that it is in a normal state. Correspondingly, if the information returned by the database instance in response to the status acquisition request cannot be successfully obtained, it represents the database instance, and the operating state of the database instance is determined to indicate that the database instance is in a fault state.
  • the database agent can divide the data to be stored into the first part and the second part, and add the first part of the data to the status acquisition request, and then use the status acquisition request to convert the data to be stored
  • the first part of the data is sent to the database instance.
  • the database agent can add an identification field to the status acquisition request to enable the database instance to determine whether the received status acquisition request carries part of the data to be stored.
  • the value of the identification field can be used to determine whether the status acquisition request carries part of the data in the data to be stored. If the value of the identification field is 1, it means that the status acquisition request carries part of the data in the data to be stored. If the value of the identification field is 0, part of the data in the data to be stored is not carried in the representation state acquisition request.
  • the database agent can determine whether the data to be stored needs to be divided into the first part and the second part according to the parameters of the data to be stored. For example, if the data volume of the data to be stored is relatively small, even if the data to be stored is divided into the first part and the second part for transmission, the time-consuming reduction may not be obvious, and it is also necessary in the database instance The splicing of the first part and the second part will increase the consumption of processing resources of the database instance. Then as a certain way, the database agent can first obtain the data amount of the data to be stored. If the data amount is less than the threshold data amount, the database agent will successfully obtain the information returned by the database instance in response to the status obtaining request.
  • the data to be stored will be divided into the first part and the second part, and the data to be stored will be processed in the manner of the foregoing embodiment. send.
  • the amount of data corresponding to the first part is smaller than the amount of data corresponding to the second part.
  • the database agent may periodically send a status acquisition request to the database instance, and store the operating status determined according to the sent status acquisition request to the database agent locally.
  • the database agent directly queries the running status of the database instance locally before sending the data to be stored, thereby avoiding the time-consuming sending of status acquisition requests in real time and improving the efficiency of data storage.
  • the disk corresponding to the database agent can be understood as the disk of the hardware device that the database agent runs.
  • the database agent in the case that the data storage system includes a database agent and a database instance, the database agent will first obtain the running status of the database instance after obtaining the data to be stored.
  • the running state indicates that the database instance is in a fault state, and the data to be stored is written to the disk corresponding to the database agent. Therefore, when the running state indicates that the database instance is in a fault state, the data to be stored will be written to the disk corresponding to the database agent, instead of being written into the memory corresponding to the database agent. Avoid memory overflow caused by too much data stored in the memory, and then avoid database agent crash and data loss in memory caused by memory overflow, so as to enhance data security.
  • a data processing method provided in this application is applied to a database agent included in a data storage system, and the data storage system further includes a database instance, which is used to store and read data,
  • the method includes:
  • the sending the to-be-stored data to the database instance includes: obtaining a target database instance, where the target database instance is one of the plurality of database instances corresponding to the operating state characterizing being normal A working database instance; sending the data to be stored to the target database instance.
  • the number of the target database instance can also be multiple, so that the data to be stored can be sent to multiple target database instances for storage at the same time, so that it can be realized that there is a database instance (for example, the aforementioned target database instance) in a fault state.
  • the previously stored data to be stored can also be read from other database instances to improve the high availability of the data storage system.
  • the determined target database instance includes database instance A, database instance B, and database instance C
  • the database agent will send the data D to be stored to database instance A, database instance B, and database instance C, respectively.
  • the database agent can still read the data D to be stored from the database C.
  • the sending the data to be stored to the target database instance includes: obtaining the data type of the data to be stored; obtaining the type of each data stored in the target database instance; The data type of the stored data is matched with the type of the data stored in each of the target database instances to obtain a matching target database instance; the data to be stored is sent to the matching target database instance.
  • the sending the data to be stored to the target database instance includes: obtaining the respective failure rates of the target database instances; and using the target database instances whose failure rates are lower than a threshold as A new target database instance; sending the data to be stored to the new target database instance.
  • different database instances may be in different hardware devices or different network environments, so some database instances may have a higher failure rate than other database instances.
  • the target database instance whose failure rate is lower than the threshold can be used as the new target database instance, which can effectively reduce the failure rate of the database instance.
  • the burden of data processing so that it can alleviate the failure rate caused by the higher burden of data processing (data storage and data reading). After the data processed by a database instance with a higher failure rate is reduced, more data can be allocated for storage as the failure rate decreases, so that the best balance between data processing and failure rate can be achieved.
  • the corresponding disk and memory can be used for data storage.
  • the data to be stored is divided into the first part and the second part in the aforementioned way, the first part of the data to be stored can be stored in the case where the working status of the corresponding database instance is in a fault state.
  • the second part is stored in the disk, so that if you need to send a state acquisition request again to detect whether the database instance is back to working state, you can directly read the first part of the data to be stored from the memory to generate the first part of the data to be stored. Part of the status acquisition request to increase the generation rate of the status acquisition request.
  • the second part can be read from the disk to the memory in advance, and then the first part of the data to be stored can be directly read from the memory after the recovery of the working state of the database instance is detected through the status acquisition request.
  • the second part is sent to the database instance, and if it is detected through the status acquisition request that the database instance is still in a faulty state, the second part of the data to be stored read into the memory will be deleted. In this way, the risk of memory overflow caused by memory usage can be effectively reduced, and the database agent can send the data to be stored to the database instance more quickly after the database instance is restored to its working state. .
  • the database agent in the case that the data storage system includes a database agent and a database instance, the database agent will first obtain the running status of the database instance after obtaining the data to be stored.
  • the running state indicates that the database instance is in a fault state, and the data to be stored is written to the disk corresponding to the database agent.
  • the data to be stored will be written to the disk corresponding to the database agent, instead of being written into the memory corresponding to the database agent. Avoid memory overflow caused by too much data stored in the memory, and then avoid database agent crash and data loss in memory caused by memory overflow, so as to enhance data security.
  • the data to be stored can be simultaneously sent to the target database instances in the multiple database instances whose corresponding running states are all working states, so that the data to be stored can be It is stored by multiple database instances at the same time, which further improves the high availability of the data storage system.
  • a data processing method provided by the present application is applied to a database agent included in a data storage system.
  • the data storage system also includes a database instance, a gateway, and a query terminal.
  • the database instance is used to perform data processing. For storing and reading, the method includes:
  • S310 The database agent terminal receives the data to be stored sent by the gateway.
  • the method provided in this embodiment further includes: the database agent terminal receives the data query request sent by the query terminal; and sends the query request to the corresponding database instance for the received data
  • the database instance of the query request performs data query.
  • the method further includes: obtaining a database corresponding to the data query request The running status of the instance. If the operating state of the database instance corresponding to the data query request indicates that the database instance is in a normal state, execute the sending of the query request to the corresponding database instance for the database that received the query request Instance for data query.
  • the method further includes: if the operating state of the database instance corresponding to the data query request indicates that the database instance is in a fault state, executing the query request on a disk corresponding to the database agent.
  • the database agent will first store the data to be stored on the disk, or it will be Part of the stored data is stored in the disk, and the other part is stored in the memory. In this case, the data to be stored has not been sent to the database instance for storage in the data table due to the failure of the database instance itself.
  • the query request can be executed in the corresponding disk and memory of the database agent when the database instance is in a fault state.
  • the execution of the query request can be understood as finding the target data corresponding to the query request.
  • the method further includes: if the query request fails to be executed on the disk corresponding to the database agent, caching the query request; if the database instance is detected after the query request is cached
  • the running state of is in a normal state, and the query request is sent to the corresponding database instance for data query by the database instance that has received the query request.
  • the database instance can be distinguished by the granularity of the data table (for example, the measurement in the Influxdb database).
  • the distinction based on the granularity of the data table can be understood as different data are stored in different data tables.
  • system load monitoring data can be stored in one data table
  • network card load monitoring data can be stored in another data table.
  • the corresponding data table can be searched based on the exact matching method first, and if the exact search fails, the corresponding data table can be searched for by fuzzy matching with the data table prefix.
  • the data table about the processor monitoring data includes three data tables named CPU_1, CPU_2, and CPU_3. If the data table carried in the query request as the query target is CPU_4, then in the exact matching mode, it is found that CPU_1, CPU_2, and CPU_3 and CPU_4 will not be exactly the same, and then fuzzy matching will be started.
  • the database agent in the case that the data storage system includes a database agent and a database instance, the database agent will first obtain the running status of the database instance after obtaining the data to be stored.
  • the running state indicates that the database instance is in a fault state, and the data to be stored is written to the disk corresponding to the database agent.
  • the database agent terminal is correspondingly configured with a query terminal, and then the query terminal can be used to send a query request to the database agent terminal, so that the database agent terminal has a data query function.
  • the data storage system includes a gateway, a database proxy terminal, and a database instance.
  • the database instance is used to store and read data.
  • the method includes:
  • the gateway receives the data to be stored sent by the client, and sends the data to be stored to the database agent.
  • the database agent terminal obtains the data to be stored and obtains the operating state of the database instance, and if the operating state indicates that the database instance is in a fault state, write the data to be stored into the database In the disk corresponding to the agent.
  • writing the data to be stored into the disk corresponding to the database agent terminal further includes:
  • the database agent After the database agent writes the data to be stored to the disk, if it detects that the running state of the database instance is in a normal state, it reads the data to be stored from the disk for use Send the data to be stored to the database instance.
  • the gateway receives the data to be stored sent by the client, and sends the data to be stored to the database agent, including: the gateway receives the data sent by the client Data to be stored; the gateway obtains the data source identifier corresponding to the data to be stored; the gateway obtains the database agent corresponding to the data source identifier; the gateway sends the data to be stored to the data source Identifies the corresponding database agent.
  • the gateway obtains the data to be stored, it can send the data to be stored according to the data source identifier Go to the database agent in the corresponding computer room.
  • it includes a first computer room, a second computer room, and a third computer room, wherein the first computer room is configured with a database agent terminal E, the second computer room is configured with a database agent terminal F, and the third computer room is configured with a database agent End G.
  • the method before sending the data to be stored to the database agent, the method further includes: the gateway verifies the data to be stored; if the verification is successful, compressing the data to be stored, Get the compressed data to be stored.
  • the data storage system further includes a query terminal
  • the method further includes: the query terminal obtains a data query request sent by the client, and sends the data query request to the database agent terminal;
  • the database agent terminal sends the data query request to the database instance for the database instance to execute the data query request.
  • the gateway, database agent, and database instance provided in this embodiment may not only execute the corresponding steps in the foregoing embodiment, but also have a data statistics function.
  • the database agent can also count the system throughput (Transaction Per Second) corresponding to the data storage system.
  • the system throughput can be understood as the number of transactions processed by the system per second, where , A transaction refers to the process in which the database agent responds to the received request.
  • the database agent can also make statistics on the query rate per second (Queries-per-second) of the data storage system.
  • the query rate per second characterizes the amount of processing traffic corresponding to a specific time within a specified period of time.
  • the database agent in the case that the data storage system includes a database agent and a database instance, the database agent will first obtain the running status of the database instance after obtaining the data to be stored.
  • the running state indicates that the database instance is in a fault state, and the data to be stored is written to the disk corresponding to the database agent. Therefore, when the running state indicates that the database instance is in a fault state, the data to be stored will be written to the disk corresponding to the database agent, instead of being written into the memory corresponding to the database agent. Avoid memory overflow caused by too much data stored in the memory, and then avoid database agent crash and data loss in memory caused by memory overflow, so as to enhance data security.
  • a data processing device 500 provided by the present application runs on a database agent included in a data storage system.
  • the data storage system also includes a database instance for storing and reading data.
  • the device 500 includes:
  • the data acquisition unit 510 is configured to acquire data to be stored
  • the status detection unit 520 is configured to obtain the running status of the database instance
  • the data processing unit 530 is configured to write the data to be stored into the disk corresponding to the database agent if the operating state indicates that the database instance is in a fault state; if the operating state indicates the database instance In a normal state, the data to be stored is sent to the database instance.
  • the data processing unit 530 is further configured to, after writing the data to be stored to the disk, read from the disk if it is detected that the running state of the database instance is in a normal state.
  • the data to be stored is used to send the data to be stored to the database instance.
  • the number of the database instance is multiple.
  • the data processing unit 530 is specifically configured to obtain a target database instance, where the target database instance is a database instance in a normal working state corresponding to the running state of the multiple database instances;
  • the data processing unit 530 is specifically configured to obtain the data type of the data to be stored; obtain the type of each data stored in the target database instance; compare the data type of the data to be stored with the target The types of data stored in the respective database instances are matched to obtain a matching target database instance; the data to be stored is sent to the matching target database instance.
  • the data processing unit 530 is specifically configured to obtain the respective failure rate of the target database instance; use the target database instance whose failure rate is lower than the threshold as the new target database instance; The data to be stored is sent to the new target database instance.
  • the data storage system further includes a gateway.
  • the data acquisition unit 510 is specifically configured to receive the data to be stored sent by the gateway.
  • the data storage system further includes a query terminal.
  • the data obtaining unit 510 is further configured to receive a data query request sent by the query terminal.
  • the data processing unit 530 is further configured to send the query request to a corresponding database instance for data query by the database instance that has received the query request.
  • the data obtaining unit 510 is also configured to obtain the operating status of the database instance corresponding to the data query request; if the operating status of the database instance corresponding to the data query request indicates that the database instance is in a normal state, The sending of the query request to the corresponding database instance is performed, so as to use the database instance that has received the query request to perform data query.
  • the data acquisition unit 510 is further configured to execute the query request on the disk corresponding to the database agent if the running state of the database instance corresponding to the data query request indicates that the database instance is in a fault state.
  • the data obtaining unit 510 is further configured to: if the query request fails to be executed on the disk corresponding to the database agent, cache the query request; if the query request is detected after the query request is cached
  • the running state of the database instance indicates that it is in a normal state, and the query request is sent to the corresponding database instance for data query by the database instance that has received the query request.
  • the database agent when the data storage system includes a database agent and a database instance, the database agent will first obtain the running status of the database instance after obtaining the data to be stored.
  • the running state indicates that the database instance is in a fault state, and the data to be stored is written to the disk corresponding to the database agent. Therefore, when the running state indicates that the database instance is in a fault state, the data to be stored will be written to the disk corresponding to the database agent, instead of being written into the memory corresponding to the database agent. Avoid memory overflow caused by too much data stored in the memory, and then avoid database agent crash and data loss in memory caused by memory overflow, so as to enhance data security.
  • the content in the device embodiment corresponds to the content in the foregoing method embodiment.
  • the content in the foregoing method embodiment please refer to the content in the foregoing method embodiment, which will not be repeated here. .
  • another embodiment of the present application also provides another processing device 200 that can execute the foregoing data processing method.
  • the processing device 200 further includes a memory 104 and a network module 106.
  • the memory 104 stores a program that can execute the content in the foregoing embodiment, and the processor 102 can execute the program stored in the memory 104.
  • the processor 102 uses various interfaces and lines to connect various parts of the entire processing device 200, by running or executing instructions, programs, code sets, or instruction sets stored in the memory 104, and calling data stored in the memory 104 , Perform various functions of the processing device 200 and process data.
  • the processor 102 may adopt at least one of digital signal processing (Digital Signal Processing, DSP), Field-Programmable Gate Array (Field-Programmable Gate Array, FPGA), and Programmable Logic Array (Programmable Logic Array, PLA).
  • DSP Digital Signal Processing
  • FPGA Field-Programmable Gate Array
  • PLA Programmable Logic Array
  • the processor 102 may integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), a modem, and the like.
  • the CPU mainly processes the operating system, user interface and application programs, etc.; the GPU is used for rendering and drawing of display content; the modem is used for processing wireless communication. It can be understood that the above-mentioned modem may not be integrated into the processor 102, but may be implemented by a communication chip alone.
  • the memory 104 may include random access memory (RAM) or read-only memory (Read-Only Memory).
  • the memory 104 may be used to store instructions, programs, codes, code sets or instruction sets.
  • the memory 104 may include a storage program area and a storage data area, where the storage program area may store instructions for implementing the operating system and instructions for implementing at least one function (such as touch function, sound playback function, image playback function, etc.) , Instructions used to implement the following various method embodiments, etc.
  • the data storage area can also store data created during use of the terminal 100 (such as phone book, audio and video data, chat record data) and the like.
  • the network module 106 is used to receive and send electromagnetic waves, and realize the mutual conversion between electromagnetic waves and electrical signals, so as to communicate with a communication network or other equipment, such as communicating with an audio playback device.
  • the network module 106 may include various existing circuit elements for performing these functions, for example, an antenna, a radio frequency transceiver, a digital signal processor, an encryption/decryption chip, a subscriber identity module (SIM) card, a memory, etc. .
  • SIM subscriber identity module
  • the network module 106 can communicate with various networks, such as the Internet, an intranet, and a wireless network, or communicate with other devices through a wireless network.
  • the aforementioned wireless network may include a cellular telephone network, a wireless local area network, or a metropolitan area network.
  • the network module 106 can exchange information with the base station.
  • FIG. 9 shows a structural block diagram of a computer-readable storage medium provided by an embodiment of the present application.
  • the computer readable medium 1100 stores program code, and the program code can be invoked by a processor to execute the method described in the foregoing method embodiment.
  • the computer-readable storage medium 1100 may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read Only Memory), EPROM, hard disk, or ROM.
  • the computer-readable storage medium 1100 includes a non-transitory computer-readable storage medium.
  • the computer-readable storage medium 1100 has a storage space for executing the program code 810 of any method step in the above-mentioned method. These program codes can be read from or written into one or more computer program products.
  • the program code 1110 may be compressed in an appropriate form, for example.
  • the data processing method, device, processing equipment, and data storage system in the case that the data storage system includes a database agent and a database instance, the database agent obtains the data to be stored After that, the running status of the database instance is acquired first, and if the running status indicates that the database instance is in a fault state, the data to be stored is written to the disk corresponding to the database agent. Therefore, when the running state indicates that the database instance is in a fault state, the data to be stored will be written to the disk corresponding to the database agent, instead of being written into the memory corresponding to the database agent. Avoid memory overflow caused by too much data stored in the memory, and then avoid database agent crash and data loss in memory caused by memory overflow, so as to enhance data security.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Quality & Reliability (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

L'invention concerne un procédé et un appareil de traitement de données, un dispositif de traitement et support de stockage de données (100). Le système de stockage de données (100) comprend une extrémité d'agent de base de données (110), et comprend en outre une instance de base de données (120), l'instance de base de données (120) servant à stocker et lire des données. Le procédé consiste : à obtenir, par l'extrémité d'agent de base de données (110), des données à stocker (S110) ; à obtenir un état de fonctionnement de l'instance de base de données (120) (S120) ; et si l'état de fonctionnement indique que l'instance de base de données (120) est dans un état de défaillance, à écrire lesdites données dans un disque correspondant à l'extrémité d'agent de base de données (110) (S130). Par conséquent, si l'état de fonctionnement indique que l'instance de base de données (120) est dans l'état de défaillance, lesdites données sont écrites dans le disque correspondant à l'extrémité d'agent de base de données (110) au lieu d'être écrites dans une mémoire correspondant à l'extrémité d'agent de base de données (110), de sorte qu'un dépassement de mémoire provoqué par une trop grande quantité de données stockées dans la mémoire soit évité, et qu'un plantage de l'extrémité d'agent de base de données (110) provoqué par le dépassement de mémoire ainsi que la perte de données dans la mémoire soient évités, ce qui permet d'améliorer la sécurité des données.
PCT/CN2020/092196 2020-05-26 2020-05-26 Procédé et appareil de traitement de données, dispositif de traitement et système de stockage de données WO2021237431A1 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/CN2020/092196 WO2021237431A1 (fr) 2020-05-26 2020-05-26 Procédé et appareil de traitement de données, dispositif de traitement et système de stockage de données
CN202080094838.5A CN115004160A (zh) 2020-05-26 2020-05-26 数据处理方法、装置、处理设备以及数据存储系统

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2020/092196 WO2021237431A1 (fr) 2020-05-26 2020-05-26 Procédé et appareil de traitement de données, dispositif de traitement et système de stockage de données

Publications (1)

Publication Number Publication Date
WO2021237431A1 true WO2021237431A1 (fr) 2021-12-02

Family

ID=78745166

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/092196 WO2021237431A1 (fr) 2020-05-26 2020-05-26 Procédé et appareil de traitement de données, dispositif de traitement et système de stockage de données

Country Status (2)

Country Link
CN (1) CN115004160A (fr)
WO (1) WO2021237431A1 (fr)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1614566A (zh) * 2004-11-29 2005-05-11 中兴通讯股份有限公司 一种用户数据库的备份恢复方法
CN101707399A (zh) * 2009-11-20 2010-05-12 深圳市科陆电子科技股份有限公司 电能信息采集方法及系统
CN105183614A (zh) * 2015-11-03 2015-12-23 华夏银行股份有限公司 一种数据库故障预测方法及装置
US20190004888A1 (en) * 2017-06-30 2019-01-03 Intel Corporation Drive-based storage scrubbing
CN109194515A (zh) * 2018-09-12 2019-01-11 郑州云海信息技术有限公司 服务器集群系统及实现服务器集群系统高可用的方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1614566A (zh) * 2004-11-29 2005-05-11 中兴通讯股份有限公司 一种用户数据库的备份恢复方法
CN101707399A (zh) * 2009-11-20 2010-05-12 深圳市科陆电子科技股份有限公司 电能信息采集方法及系统
CN105183614A (zh) * 2015-11-03 2015-12-23 华夏银行股份有限公司 一种数据库故障预测方法及装置
US20190004888A1 (en) * 2017-06-30 2019-01-03 Intel Corporation Drive-based storage scrubbing
CN109194515A (zh) * 2018-09-12 2019-01-11 郑州云海信息技术有限公司 服务器集群系统及实现服务器集群系统高可用的方法

Also Published As

Publication number Publication date
CN115004160A (zh) 2022-09-02

Similar Documents

Publication Publication Date Title
CN106980625B (zh) 一种数据同步方法、装置及系统
WO2021164487A1 (fr) Procédé et appareil d'interrogation de données
CN108924274B (zh) 域名系统dns处理方法、装置、存储介质及电子设备
CN109462631B (zh) 数据处理方法、装置、存储介质及电子装置
CN113568884A (zh) 一种文件管理方法、装置、电子设备及存储介质
CN113010818A (zh) 访问限流方法、装置、电子设备及存储介质
CN110753095B (zh) 一种网卡的数据处理方法、设备以及存储介质
US20170289243A1 (en) Domain name resolution method and electronic device
CN110198251B (zh) 一种获得客户端地址的方法及装置
CN110764688B (zh) 对数据进行处理的方法和装置
WO2023020187A1 (fr) Procédés et appareils d'obtention de données, dispositif électronique et support d'enregistrement
CN113014510B (zh) 惯导系统分布式测试中数据缓存方法及装置
WO2021237431A1 (fr) Procédé et appareil de traitement de données, dispositif de traitement et système de stockage de données
CN110879760B (zh) 一种统一存储系统及方法、电子设备
CN109766347B (zh) 一种数据更新方法、装置、系统、计算机设备及存储介质
CN107168648B (zh) 文件存储方法、装置及终端
CN107483637B (zh) 一种基于nfs的客户端链接管理方法及装置
CN112711466B (zh) 悬挂事务巡检方法和装置、电子设备和存储介质
CN110990313B (zh) 一种i3c总线处理时钟拉伸的方法、设备以及存储介质
CN112416698B (zh) 监控系统的扩展方法及装置、存储介质及电子设备
CN111274176A (zh) 一种信息处理方法、电子设备、系统及存储介质
CN114466032B (zh) 一种cdn系统合并回源方法、装置及存储介质
CN113468195B (zh) 服务器数据缓存更新方法、系统和主数据库服务器
CN114676166B (zh) 数据处理方法及装置
CN117201518B (zh) 数据的传输方法、系统、装置、存储介质和电子设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20937744

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 110423)

122 Ep: pct application non-entry in european phase

Ref document number: 20937744

Country of ref document: EP

Kind code of ref document: A1