CN114722043A - Data storage method, device, equipment and medium - Google Patents

Data storage method, device, equipment and medium Download PDF

Info

Publication number
CN114722043A
CN114722043A CN202210347555.8A CN202210347555A CN114722043A CN 114722043 A CN114722043 A CN 114722043A CN 202210347555 A CN202210347555 A CN 202210347555A CN 114722043 A CN114722043 A CN 114722043A
Authority
CN
China
Prior art keywords
data
target
storage
target data
storage mode
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210347555.8A
Other languages
Chinese (zh)
Inventor
桑坤
王加站
冯亚洲
杜永良
韩珂
安茂众
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shantui Chutian Construction Machinery Co Ltd
Original Assignee
Shantui Chutian Construction Machinery Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shantui Chutian Construction Machinery Co Ltd filed Critical Shantui Chutian Construction Machinery Co Ltd
Priority to CN202210347555.8A priority Critical patent/CN114722043A/en
Publication of CN114722043A publication Critical patent/CN114722043A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22—Indexing; Data structures therefor; Storage structures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02—Addressing or allocation; Relocation
    • G06F12/08—Addressing or allocation; Relocation in hierarchically structured memory systems, e.g. virtual memory systems
    • G06F12/0802—Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches
    • G06F12/0877—Cache access modes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/23—Updating
    • G06F16/2379—Updates performed during online database operations; commit processing
    • G06F16/2386—Bulk updating operations
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/245—Query processing
    • G06F16/2455—Query execution
    • G06F16/24552—Database cache management
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5061—Partitioning or combining of resources
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2209/00—Indexing scheme relating to G06F9/00
    • G06F2209/50—Indexing scheme relating to G06F9/50
    • G06F2209/5011—Pool
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2209/00—Indexing scheme relating to G06F9/00
    • G06F2209/50—Indexing scheme relating to G06F9/50
    • G06F2209/5018—Thread allocation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2212/00—Indexing scheme relating to accessing, addressing or allocation within memory systems or architectures
    • G06F2212/16—General purpose computing application
    • G06F2212/163—Server or database system

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Databases & Information Systems (AREA)
  • Software Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Computational Linguistics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention discloses a data storage method, a data storage device, data storage equipment and a data storage medium, and belongs to the technical field of data processing. The method comprises the following steps: under the condition that a storage mode subscription service is started, if a data consumer is monitored to consume target data, selecting a target storage mode associated with the target data from candidate storage modes; and storing the target data by adopting the target storage mode. By the technical scheme, the mass data can be stored in a differentiated mode.

Description

Data storage method, device, equipment and medium
Technical Field
The present invention relates to the field of data processing technologies, and in particular, to a data storage method, apparatus, device, and medium.
Background
In recent years, the internet of things technology and the big data technology in China are widely applied to the engineering machinery industry, and along with the requirements of the state and the local government on vehicle positioning, vehicle monitoring, emission supervision and the like, the engineering machinery industry is continuously developed towards the direction of intellectualization and informatization, and a large amount of data processing and storing requirements follow.
At present, in the field of engineering machinery, a large data platform mostly adopts a high-throughput distributed publish-subscribe message system of Kafka, but when facing fast-changing service requirements and diversified query requirements, a single data storage mode cannot meet fast iterative update of the service requirements.
Disclosure of Invention
The invention provides a data storage method, a data storage device, data storage equipment and a data storage medium, which are used for realizing differentiated storage of mass data.
According to an aspect of the present invention, there is provided a data storage method, the method including:
under the condition that a storage mode subscription service is started, if a data consumer is monitored to consume target data, selecting a target storage mode associated with the target data from candidate storage modes;
and storing the target data by adopting the target storage mode.
According to another aspect of the present invention, there is provided a data storage device comprising:
the target storage mode determining module is used for selecting a target storage mode associated with the target data from the candidate storage modes if it is monitored that a data consumer consumes the target data under the condition that the storage mode subscription service is started;
and the target data storage module is used for storing the target data by adopting the target storage mode.
According to another aspect of the present invention, there is provided an electronic apparatus including:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores a computer program executable by the at least one processor, the computer program being executable by the at least one processor to enable the at least one processor to perform the data storage method of any of the embodiments of the invention.
According to another aspect of the present invention, there is provided a computer-readable storage medium storing computer instructions for causing a processor to implement a data storage method according to any one of the embodiments of the present invention when the computer instructions are executed.
According to the technical scheme of the embodiment of the invention, under the condition that the storage mode subscription service is started, if the data consumer is monitored to consume the target data, the target storage mode related to the target data is selected from the candidate storage modes, and then the target storage mode is adopted to store the target data. According to the technical scheme, differentiated storage of mass data is achieved, and decoupling of data storage and services is achieved.
It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present invention, nor do they necessarily limit the scope of the invention. Other features of the present invention will become apparent from the following description.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1A is a flowchart of a data storage method according to an embodiment of the present invention;
FIG. 1B is a schematic diagram illustrating a configuration of a storage mode according to an embodiment of the present invention;
FIG. 1C is a schematic diagram of a target data determination according to an embodiment of the present invention;
FIG. 2 is a flowchart of a data storage method according to a second embodiment of the present invention;
FIG. 3 is a schematic structural diagram of a data storage device according to a third embodiment of the present invention;
fig. 4 is a schematic structural diagram of an electronic device implementing the data storage method according to the embodiment of the present invention.
Detailed Description
In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that the terms "object", "candidate", and the like in the description and claims of the present invention and the above-described drawings are used for distinguishing similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
Example one
Fig. 1A is a flowchart of a data storage method according to an embodiment of the present invention. The embodiment is applicable to how to store data, and as applicable to how to store data in the field of engineering machinery, the method may be executed by a data storage device, which may be implemented in the form of hardware and/or software, and may be integrated in an electronic device carrying a data storage function, such as a large data platform, and in particular, may be a platform that employs a high-throughput distributed publish-subscribe message system, such as Kafka. As shown in fig. 1A, the data storage method of this embodiment may include:
s110, under the condition that the storage mode subscription service is started, if the data consumer is monitored to consume the target data, selecting a target storage mode associated with the target data from the candidate storage modes.
In this embodiment, the storage schema subscription service is a service for subscribing to a storage schema, and may be implemented by Redis, which may support a data consumer to subscribe to the storage schema subscription service.
The data consumer is a user or a receiver of the data.
The candidate storage mode is a storage policy for storing data, and may include: the data source is a data source, the storage source is a destination where data is stored, and the table name is a file name and a consumption theme when the data is stored; further, a storage template for storing data, such as a data module, a name, an attribute identifier, a field type, a length, a decimal point, a mapping, and the like, may be included. Alternatively, the candidate storage mode may be set by the developer according to actual requirements through a configuration interface provided by the platform, as shown in fig. 1B for example.
It should be noted that, when the data generator uploads the target data to the platform, the platform acquires a subscription message of the data consumer to the storage mode subscription service, and if the subscription message is received to be turned on, the storage mode subscription service is turned on in response to the turning on operation of the storage mode subscription service, and the candidate storage mode is loaded. Wherein, the data producer is the sender of the data. In addition, after the candidate storage mode is loaded, the cache policy in the candidate storage mode is cached and initialized.
The target data is data in a JSON format; optionally, the terminal acquires data of engine information and instrument information such as engine speed, fuel level, accumulated engine running time, transmission oil temperature, coolant temperature, engine oil pressure, fuel consumption and the like in a CAN bus of the bulldozer, converts the acquired data into a binary code stream according to preset enterprise-standard vehicle cloud protocol requirements, reports the binary code stream to the platform, analyzes the data into a JSON format, and pushes the JSON format to a kafka message queue, namely pushes the data to the kafka message queue.
It should be noted that, in response to the storage suspension and/or storage opening operation on the at least two original data, the target data is selected from the at least two original data. Specifically, a developer can perform pause storage and/or start storage operation on at least two pieces of original data through an interface provided in the platform; correspondingly, the platform responds to the pause storage and/or the start storage operation of the developer on at least two original data, and starts to store the corresponding original data as target data; illustratively, as shown in FIG. 1C.
In this embodiment, the platform, when the storage mode subscription service is started, determines whether the target data is consumed through the kafka listener, that is, whether the data consumer consumes the target data; and if monitoring that the consumer consumes the target data, selecting a target storage mode associated with the target data from the candidate storage modes.
Alternatively, a target storage mode associated with the target data may be selected from the candidate storage modes according to the target consumption topic. Specifically, the target data includes a target consumption topic, and a target storage mode associated with the target data is selected from the candidate storage modes by using the target consumption topic as an index.
Alternatively, a target storage mode associated with the target data may be selected from the candidate storage modes according to the data source. Specifically, a target storage mode associated with the target data is selected from the candidate storage modes by taking a data source of the target data as an index.
In yet another alternative, a target storage schema associated with the target data may be selected from the candidate storage schemas based on the target consumption topic and the data source.
And S120, storing the target data by adopting a target storage mode.
Specifically, the target data is stored by adopting a cache strategy in a target storage mode. For example, the target data is stored to a storage source specified in the caching policy.
According to the technical scheme of the embodiment of the invention, under the condition that the storage mode subscription service is started, if the data consumer is monitored to consume the target data, the target storage mode related to the target data is selected from the candidate storage modes, and then the target storage mode is adopted to store the target data. According to the technical scheme, differentiated storage of mass data is achieved, and decoupling of data storage and services is achieved.
Example two
Fig. 2 is a flowchart of a data storage method according to a second embodiment of the present invention. On the basis of the above embodiments, the present embodiment further optimizes "storing target data by using a target storage mode", and provides an optional implementation scheme. As shown in fig. 2, the data storage method of the present embodiment may include:
s210, under the condition that the storage mode subscription service is started, if the data consumer is monitored to consume the target data, selecting a target storage mode associated with the target data from the candidate storage modes.
S220, caching the target data into a memory queue.
In this embodiment, the target data is cached in the memory queue.
And S230, processing the target data in batch according to a preset memory queue consumption strategy.
In this embodiment, the memory queue consumption policy may include a consumption thread number, a memory queue size, a thread pool size, and the like. For example, in order to adapt to different hardware performances, the memory queue consumption configuration may be set to a dynamic configuration based on the Nacos registration function, so that the backend server may reasonably configure the memory queue consumption policy according to the performances.
According to an optional mode, batch data can be carried out on target data according to the size of a memory queue and the size of a thread pool in a preset memory queue consumption strategy.
In yet another alternative, messages in the target data may be counted; and processing the target data in batches according to the counting result. Specifically, a counter is added into the memory queue to count the messages in the target data, and the target data is stored in batch by combining with a preset memory queue consumption strategy. For example, if the number of consuming threads in the preset memory queue consumption policy is 10, and the number of messages in the target data is 90 after counting, the number of consuming threads is modulo (modulo 10 by 90) by the number of messages, that is, the data processed by each consuming thread in batch is 9 messages.
It can be understood that through counting, multithreading consumption balance is guaranteed, and the problem that efficiency is reduced due to the fact that the size deviation of a memory queue is too large because multithreading cannot balance consumption data when the consumption queue.
And S240, storing the processed target data by adopting a target storage mode.
Specifically, a target storage template corresponding to the target storage mode may be obtained; and correspondingly storing the processed target data to the mapping field in the target storage template. Specifically, a target storage template corresponding to the target storage mode is obtained from the cache, the target data after batch processing are sequentially filled into mapping fields in the target storage template according to the fields, and the mapping fields are stored in a storage source in the target storage mode.
According to the technical scheme, under the condition that the storage mode subscription service is started, if it is monitored that a data consumer consumes target data, a target storage mode related to the target data is selected from candidate storage modes, then the target data are cached to the memory queue, then batch processing is performed on the target data according to a preset memory queue consumption strategy, and the processed target data are stored by adopting the target storage mode. According to the technical scheme, the memory queue is introduced, so that the problem of low data storage efficiency caused by overlarge consumption data is solved, and the problem of data block breakdown caused by overlarge consumption data is avoided.
EXAMPLE III
Fig. 3 is a schematic structural diagram of a data storage device according to a third embodiment of the present invention. The embodiment can be applied to the case of how to store data, and as the embodiment is applied to the case of how to store data in the field of engineering machinery, the device can be implemented in the form of hardware and/or software, and can be integrated into an electronic device carrying a data storage function, such as a server. As shown in fig. 3, the data storage device of the present embodiment may include:
a target storage mode determining module 310, configured to select a target storage mode associated with target data from candidate storage modes if it is monitored that a data consumer consumes the target data when the storage mode subscription service is turned on;
and a target data storage module 320, configured to store the target data in a target storage mode.
According to the technical scheme of the embodiment of the invention, under the condition that the storage mode subscription service is started, if the data consumer is monitored to consume the target data, the target storage mode related to the target data is selected from the candidate storage modes, and then the target storage mode is adopted to store the target data. According to the technical scheme, differentiated storage of mass data is achieved, and decoupling of data storage and services is achieved.
Optionally, the target storage mode determining module 310 is specifically configured to:
and selecting a target storage mode associated with the target data from the candidate storage modes according to the target consumption subject and/or the data source.
Optionally, the target data storage module 320 includes:
the cache unit is used for caching the target data to a memory queue;
the batch processing unit is used for carrying out batch processing on the target data according to a preset memory queue consumption strategy;
and the target data storage unit is used for storing the processed target data by adopting a target storage mode.
Optionally, the target data storage unit is specifically configured to:
acquiring a target storage template corresponding to a target storage mode;
and correspondingly storing the processed target data to the mapping field in the target storage template.
Optionally, the batch processing unit is specifically configured to:
counting messages in the target data;
and processing the target data in batches according to the counting result.
Optionally, the apparatus further comprises:
and the candidate storage mode loading module is used for responding to the starting operation of the storage mode subscription service, starting the storage mode subscription service and loading the candidate storage mode.
Optionally, the apparatus further comprises:
and the target data determining module is used for responding to the pause storage and/or the start storage operation of the at least two original data and selecting the target data from the at least two original data.
The data storage device provided by the embodiment of the invention can execute the data storage method provided by any embodiment of the invention, and has corresponding functional modules and beneficial effects of the execution method.
Example four
Fig. 4 is a schematic structural diagram of an electronic device implementing the XXX method of embodiments of the present invention. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the inventions described and/or claimed herein.
As shown in fig. 4, the electronic device 10 includes at least one processor 11, and a memory communicatively connected to the at least one processor 11, such as a Read Only Memory (ROM)12, a Random Access Memory (RAM)13, and the like, wherein the memory stores a computer program executable by the at least one processor, and the processor 11 may perform various suitable actions and processes according to the computer program stored in the Read Only Memory (ROM)12 or the computer program loaded from the storage unit 18 into the Random Access Memory (RAM) 13. In the RAM 13, various programs and data necessary for the operation of the electronic apparatus 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other via a bus 14. An input/output (I/O) interface 15 is also connected to bus 14.
A number of components in the electronic device 10 are connected to the I/O interface 15, including: an input unit 16 such as a keyboard, a mouse, or the like; an output unit 17 such as various types of displays, speakers, and the like; a storage unit 18 such as a magnetic disk, an optical disk, or the like; and a communication unit 19 such as a network card, modem, wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information/data with other devices via a computer network such as the internet and/or various telecommunication networks.
The processor 11 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various processors running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, or the like. The processor 11 performs the various methods and processes described above, such as a data storage method.
In some embodiments, the data storage method may be implemented as a computer program tangibly embodied in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and/or installed onto the electronic device 10 via the ROM 12 and/or the communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the data storage method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the data storage method by any other suitable means (e.g., by means of firmware).
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), system on a chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.
A computer program for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer programs, when executed by the processor, cause the functions/acts specified in the flowchart and/or block diagram block or blocks to be performed. A computer program can execute entirely on a machine, partly on a machine, as a stand-alone software package partly on a machine and partly on a remote machine or entirely on a remote machine or server.
In the context of the present invention, a computer-readable storage medium may be a tangible medium that can contain, or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. A computer readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, the computer readable storage medium may be a machine readable signal medium. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the electronic device. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), blockchain networks, and the internet.
The computing system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical host and VPS service are overcome.
It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present invention may be executed in parallel, sequentially, or in different orders, and are not limited herein as long as the desired results of the technical solution of the present invention can be achieved.
The above-described embodiments should not be construed as limiting the scope of the invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (10)

1. A method of storing data, comprising:
under the condition that a storage mode subscription service is started, if a data consumer is monitored to consume target data, selecting a target storage mode associated with the target data from candidate storage modes;
and storing the target data by adopting the target storage mode.
2. The method of claim 1, wherein selecting the target storage mode associated with the target data from the candidate storage modes comprises:
and selecting a target storage mode associated with the target data from the candidate storage modes according to the target consumption subject and/or the data source.
3. The method of claim 1, wherein the storing the target data in the target storage mode comprises:
caching the target data to a memory queue;
processing the target data in batches according to a preset memory queue consumption strategy;
and storing the processed target data by adopting the target storage mode.
4. The method of claim 3, wherein storing the processed target data in the target storage mode comprises:
acquiring a target storage template corresponding to the target storage mode;
and correspondingly storing the processed target data to a mapping field in the target storage template.
5. The method of claim 3, wherein the batch processing of the target data comprises:
counting messages in the target data;
and processing the target data in batches according to the counting result.
6. The method of claim 1, further comprising:
and responding to the starting operation of the storage mode subscription service, starting the storage mode subscription service, and loading the candidate storage mode.
7. The method of claim 1, further comprising:
in response to pausing storage and/or opening storage operations on at least two original data, target data is selected from the at least two original data.
8. A data storage device, comprising:
the target storage mode determining module is used for selecting a target storage mode associated with the target data from the candidate storage modes if it is monitored that a data consumer consumes the target data under the condition that the storage mode subscription service is started;
and the target data storage module is used for storing the target data by adopting the target storage mode.
9. An electronic device, characterized in that the electronic device comprises:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores a computer program executable by the at least one processor, the computer program being executable by the at least one processor to enable the at least one processor to perform the data storage method of any one of claims 1-7.
10. A computer-readable storage medium storing computer instructions for causing a processor to perform the data storage method of any one of claims 1-7 when executed.
CN202210347555.8A 2022-04-01 2022-04-01 Data storage method, device, equipment and medium Pending CN114722043A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210347555.8A CN114722043A (en) 2022-04-01 2022-04-01 Data storage method, device, equipment and medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210347555.8A CN114722043A (en) 2022-04-01 2022-04-01 Data storage method, device, equipment and medium

Publications (1)

Publication Number Publication Date
CN114722043A true CN114722043A (en) 2022-07-08

Family

ID=82241293

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210347555.8A Pending CN114722043A (en) 2022-04-01 2022-04-01 Data storage method, device, equipment and medium

Country Status (1)

Country Link
CN (1) CN114722043A (en)

Similar Documents

Publication Publication Date Title
CN112099937A (en) Resource management method and device
CN113760982A (en) Data processing method and device
CN116594563A (en) Distributed storage capacity expansion method and device, electronic equipment and storage medium
CN114722043A (en) Data storage method, device, equipment and medium
CN115905322A (en) Service processing method and device, electronic equipment and storage medium
CN115543416A (en) Configuration updating method and device, electronic equipment and storage medium
CN115599571A (en) Data processing method and device, electronic equipment and storage medium
CN113282455A (en) Monitoring processing method and device
CN112749204A (en) Method and device for reading data
CN116431523B (en) Test data management method, device, equipment and storage medium
CN113297002B (en) Database working mode switching method and device
CN116431928A (en) Time sequence data pushing method, device, equipment and storage medium
CN113032402B (en) Method, device, equipment and storage medium for storing data and acquiring data
CN114449031B (en) Information acquisition method, device, equipment and storage medium
CN115857987A (en) Configuration file updating method and device, electronic equipment and storage medium
CN117156398A (en) Message processing method, device, electronic equipment and storage medium
CN115203139A (en) Log query method and device, electronic equipment and storage medium
CN115801718A (en) Message processing method and device, electronic equipment and message processing system
CN115098520A (en) Device data updating method and device, electronic device and storage medium
CN115730108A (en) Data processing method and device, electronic equipment and storage medium
CN117411761A (en) Message processing method, device, computer equipment and computer readable storage medium
CN116828036A (en) Network acceleration product recommendation method and device, electronic equipment and medium
CN114356505A (en) Method, apparatus, device, medium and product for scheduling groups of containers
CN117081939A (en) Traffic data processing method, device, equipment and storage medium
CN114466079A (en) Request processing method, device, proxy server and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination