CN110780817B - Data recording method and apparatus, storage medium, and electronic apparatus - Google Patents
Data recording method and apparatus, storage medium, and electronic apparatus Download PDFInfo
- Publication number
- CN110780817B CN110780817B CN201911003589.XA CN201911003589A CN110780817B CN 110780817 B CN110780817 B CN 110780817B CN 201911003589 A CN201911003589 A CN 201911003589A CN 110780817 B CN110780817 B CN 110780817B
- Authority
- CN
- China
- Prior art keywords
- target
- data
- container
- data structure
- operating system
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0614—Improving the reliability of storage systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0629—Configuration or reconfiguration of storage systems
- G06F3/0631—Configuration or reconfiguration of storage systems by allocating resources to storage systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
- G06F3/0643—Management of files
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
Abstract
The invention relates to the field related to cloud technologies, and discloses a data recording method and device, a storage medium and an electronic device. Wherein, the method comprises the following steps: determining a data structure in an idle state from N data structures of an operating system as a target data structure, wherein N is an integer greater than 1; binding a target data structure with a target container, wherein an operating system corresponds to M containers, each container comprises a preset number of Central Processing Units (CPUs), M is an integer greater than 1, and the target container is a container in a running state in the M containers; data information of the target container is recorded in the target data structure. The invention solves the technical problem of data conflict caused by sharing the data structure in a high concurrent processing scene.
Description
Technical Field
The present invention relates to the field of data processing technologies, and in particular, to a data recording method and apparatus, a storage medium, and an electronic apparatus.
Background
With the rapid development of hardware platforms, the number of integrated CPUs of high-end servers is increased, and the concurrency capability of hardware is also enhanced. The bottleneck of high concurrency in conventional software is that all connections share a data structure, thus inevitably requiring queuing when modifying the data structure. In this case, the larger the concurrent connection size, the more obvious the collision. The container technology adopts a light isolation technology, each group shares one operating system, and partial key data structures are isolated inside the operating system. However, due to the dispersion and the trivial data in the operating system, the existing isolation does not achieve the effect of isolation in many cases.
Aiming at the problem of data conflict caused by shared data structure in high concurrent processing scenes in the prior art, no effective solution is provided at present.
Disclosure of Invention
Embodiments of the present invention provide a data recording method and apparatus, a storage medium, and an electronic apparatus, so as to at least solve a technical problem of data collision caused by a shared data structure in a high-concurrency processing scenario.
According to an aspect of an embodiment of the present invention, there is provided a data recording method including:
determining a data structure in an idle state from N data structures of an operating system as a target data structure, wherein N is an integer greater than 1;
binding the target data structure with a target container, wherein the operating system corresponds to M containers, each container comprises a preset number of Central Processing Units (CPUs), M is an integer greater than 1, and the target container is a container in a running state in the M containers;
and recording the data information of the target container in the target data structure.
According to another aspect of the embodiments of the present invention, there is also provided a data recording apparatus including:
a first determining module, configured to determine, as a target data structure, one data structure in an idle state from N data structures of an operating system, where N is an integer greater than 1;
a binding module, configured to bind the target data structure with a target container, where the operating system corresponds to M containers, each container includes a predetermined number of Central Processing Units (CPUs), M is an integer greater than 1, and the target container is a container in an operating state among the M containers;
and the recording module is used for recording the data information of the target container in the target data structure.
Optionally, the recording module includes:
an obtaining unit, configured to obtain connection status data of a target CPU in the target container, where the connection status data at least includes one of: a network protocol corresponding to the target CPU, port data corresponding to the target CPU, and a connection state parameter of the target CPU;
a recording unit for recording the connection state data of the target CPU in the target data structure.
Optionally, the first determining module includes:
and the determining unit is used for selecting one data structure which is not bound with any container from the N data structures and determining the data structure as the target data structure.
Optionally, the apparatus further comprises:
and the releasing module is used for releasing the binding between the target container and the target data structure after the target container is cancelled so as to enable the target data structure to recover the idle state.
Optionally, the apparatus further comprises:
an allocation module, configured to allocate N data structures in a database corresponding to the operating system in a process of starting the operating system, where the data structures include: and (6) a hash data table.
Optionally, the apparatus further comprises:
a second determining module, configured to determine a host corresponding to the operating system when the target container is started and there is no data structure in an idle state in the N data structures;
and a third determining module, configured to determine a data structure corresponding to the host as the target data structure.
According to still another aspect of the embodiments of the present invention, there is also provided a computer-readable storage medium in which a computer program is stored, wherein the computer program is configured to execute the above-mentioned data recording method when running.
According to another aspect of the embodiments of the present invention, there is also provided an electronic apparatus, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the data recording method through the computer program.
In the embodiment of the invention, a plurality of data structures are distributed in the operating system, an independent data structure is distributed for each container in the running state, and the data information of each container can be recorded in the data structure bound with the container, so that the aim of independently recording the data information in each container in different data structures is fulfilled, the technical effect of effectively avoiding data collision in a high-concurrency scene is realized, and the technical problem of data collision caused by sharing the data structures in the high-concurrency processing scene is solved.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the invention without limiting the invention. In the drawings:
FIG. 1 is a block diagram of an alternative container and operating system configuration according to an embodiment of the present invention;
FIG. 2 is a diagram of a hardware environment for an alternative data recording method according to an embodiment of the present invention;
FIG. 3 is a flow chart of an alternative data recording method according to an embodiment of the present invention;
FIG. 4 is a flow chart of an alternative data recording method according to an embodiment of the present invention;
FIG. 5 is an interactive flow diagram of an alternative method for binding containers to data structures according to an embodiment of the present invention;
FIG. 6 is an interactive flow diagram of an alternative data recording method according to an embodiment of the present invention;
FIG. 7 is a block diagram of an alternative configuration of a data recording apparatus according to an embodiment of the present invention;
fig. 8 is a schematic structural diagram of an alternative electronic device according to an embodiment of the invention.
Detailed Description
In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
The embodiment of the invention relates to the related field of cloud technology. The cloud technology is to unify hardware, software, network and other resources in a wide area network or a local area network to realize data calculation,(Storage)Treating and mixingSharingASupport tubeProvided is a technique.
Cloud technology (Cloud technology) is based on a general term of network technology, information technology, integration technology, management platform technology, application technology and the like applied in a Cloud computing business model, can form a resource pool, is used as required, and is flexible and convenient. Cloud computing technology will become an important support. Background services of the technical network system require a large amount of computing and storage resources, such as video websites, picture-like websites and more web portals. With the high development and application of the internet industry, each article may have its own identification mark and needs to be transmitted to a background system for logic processing, data in different levels are processed separately, and various industrial data need strong system background support and can only be realized through cloud computing.
Cloud storage is a new concept extended and developed from a cloud computing concept, and a distributed cloud storage system (hereinafter referred to as a storage system) refers to a storage system which integrates a large number of storage devices (storage devices are also referred to as storage nodes) of different types in a network through application software or application interfaces to cooperatively work through functions of cluster application, a grid technology, a distributed storage file system and the like, and provides data storage and service access functions to the outside.
At present, a storage method of a storage system is as follows: logical volumes are created, and when created, each logical volume is allocated physical storage space, which may be the disk composition of a certain storage device or of several storage devices. The client stores data on a certain logical volume, that is, the data is stored on a file system, the file system divides the data into a plurality of parts, each part is an object, the object not only contains the data but also contains additional information such as data Identification (ID), the file system writes each object into a physical storage space of the logical volume, and the file system records storage location information of each object, so that when the client requests to access the data, the file system can allow the client to access the data according to the storage location information of each object.
The process of allocating physical storage space for the logical volume by the storage system specifically includes: physical storage space is divided in advance into stripes according to a group of capacity measures of objects stored in a logical volume (the measures often have a large margin with respect to the capacity of the actual objects to be stored) and Redundant Array of Independent Disks (RAID), and one logical volume can be understood as one stripe, thereby allocating physical storage space to the logical volume.
In short, the database can be regarded as an electronic file cabinet, namely a place for storing electronic files, and a user can add, query, update, delete and the like to data in the files.
A "database" is a collection of data that is stored together in a manner that can be shared by multiple users, has as little redundancy as possible, and is independent of the application.
A Database Management System (DBMS) is a computer software System designed for managing a Database, and generally has basic functions of storage, interception, security assurance, backup, and the like. The database management system may be categorized according to the database model it supports, such as relational, XML; or classified according to the type of computer supported, e.g., server cluster, mobile phone; or classified according to the query language used, e.g., SQL, XQuery; or by performance impulse emphasis, e.g., maximum size, maximum operating speed; or other classification schemes. Regardless of the manner of classification used, some DBMSs are capable of supporting multiple query languages across categories, for example, simultaneously.
Fig. 1 is a block diagram of an alternative structure of a container corresponding to an operating system according to an embodiment of the present invention, where, as shown in fig. 1, the container technology employs a light isolation technology, each group shares a share of the operating system, and some basic data structures may be isolated inside the operating system. For example, when 4 CPUs share one os currently, each CPU corresponds to one APP, and every two CPUs form one container, which is equivalent to isolating a large-scale integrated circuit into a plurality of small-scale integrated circuits, thereby relieving data table (data structure) conflicts under a large-scale short connection.
The invention is mainly based on container technology and is optimized for special scenes such as large-scale short connection. Short connections have a short lifetime, so the management operation of the system to the connection per unit time is more than that of a long connection, and the concurrent management operation causes a conflict, which is very serious if a large-scale short connection is deployed on high-integration hardware. The short connection means that when both communication parties have data interaction, a connection is established, and after the data transmission is completed, the connection is disconnected, that is, only one service is transmitted in each connection. The advantages are that: the channel does not need to be occupied for a long time, and the use of the channel can be saved for occasions with low service frequency. The disadvantages are as follows: a connection needs to be established every time a service is sent, and the overhead of the connection establishment process is large. The container technology isolates many of the underlying data structures, but does not isolate data that maintains the connection state and other critical data. In particular, IP resources between containers are isolated, which results in a significant increase in the number of connections that can be made for multiple containers, and further exacerbates and highlights conflicts in managing connection state operations. The embodiment of the invention reduces the conflict in the operation by isolating and maintaining the data structure of the connection state, thereby improving the concurrency of large-scale short connection
Fig. 2 is a schematic diagram of a hardware environment of an alternative data recording method according to an embodiment of the present invention, and as shown in fig. 2, the hardware environment may include, but is not limited to, a user device 102, a network 110, and a server 112, wherein the user device 102 may include, but is not limited to, a memory 104, a processor 106, and a display 108, and the server 112 may include, but is not limited to, a database 114 and a processing engine 116. The user device may be, but not limited to, a terminal device such as a smart phone (e.g., an Android phone, an iOS phone, etc.), a tablet computer, a palm computer, and a Mobile Internet Device (MID), a PAD, etc. In the hardware scenario shown in fig. 2, the user equipment 102 may include APPs, each user equipment 102 corresponds to a CPU, after a container corresponding to the CPU is started, the server 112 allocates a data structure in an idle state to the container, and then after the container is bound to the allocated data structure, connection data or interaction data generated by the CPU may be recorded in the bound data structure (which may also be understood as connection data or interaction data generated by the APPs in the user equipment 102). An optional implementation step of the data recording method of the embodiment of the invention is as follows:
step S102, starting a container corresponding to the user equipment 102 and generating a client request;
step S104, the generated client request is sent to the network terminal 110, and the network terminal 110 is instructed to send the client request to the server 112;
step S106, the network terminal 110 forwards the client request to the server 112;
step S108, when the container corresponding to the user device 102 is started, the server 112 finds an idle data structure from the operating system corresponding to the container and allocates the idle data structure to the container, and then after receiving the client request, the server 112 may allocate a network port to the client request, and store information of the network port and/or a network transport protocol in the container-bound data structure, and at the same time, may also store interactive data generated by the APP in the user device 102 in the container-bound data structure.
Optionally, in the embodiment of the present invention, the data recording method may be, but is not limited to, applied to the server 112, and the user equipment 102 may be, but is not limited to, a mobile phone, a tablet computer, a notebook computer, a PC, and other terminal equipment that supports running an application client. The server 112 and the user device 102 may implement data interaction through a network, which may include but is not limited to a wireless network or a wired network. Wherein, this wireless network includes: bluetooth, WIFI, and other networks that enable wireless communication. Such wired networks may include, but are not limited to: wide area networks, metropolitan area networks, and local area networks. The above is merely an example, and this is not limited in this embodiment.
Alternatively, step S106 may be implemented by: determining a data structure in an idle state from N data structures of an operating system as a target data structure, wherein N is an integer greater than 1; binding the target data structure with a target container, wherein the operating system corresponds to M containers, each container comprises a preset number of Central Processing Units (CPUs), M is an integer greater than 1, and the target container is a container in a running state in the M containers; and recording the data information of the target container in the target data structure.
Alternatively, the operating system and the container technology involved in the embodiment of the present invention may be a Linux operating system and a Linux container technology. Data collision among the containers is avoided by setting a respective transmission control protocol-hash (tcp _ hashinfo) data structure for each container.
Fig. 3 is a flow chart of an alternative data recording method according to an embodiment of the present invention. As shown in fig. 3, the method includes:
step S302, selecting a data structure in an idle state from N data structures of an operating system to determine the data structure as a target data structure, wherein N is an integer greater than 1;
step S304, binding a target data structure with a target container, wherein the operating system corresponds to M containers, each container comprises a predetermined number of Central Processing Units (CPUs), M is an integer greater than 1, and the target container is a container in a running state in the M containers;
step S306, recording data information of the target container in the target data structure.
Optionally, N data structures may be allocated in an operating system, where the N data structures may include both a data structure in an idle state and a data structure in a non-idle state, and the N data structures may be both in an idle state or both in a non-idle state. The idle-state data structure according to the embodiment of the present invention may be understood as a data structure that is not bound to any container, and the non-idle-state data structure may be understood as a data structure that is bound to a certain container. And when the data structure in the idle state exists in the operating system, selecting one data structure determined as a target data structure from the data structures to be bound with the target container after starting. The selection of the target data structure may be determined randomly, or may be selected according to the type of APP contained in the container and the size of the capacity of the data structure, or according to a specific configuration in an actual application scenario, which is not limited in this embodiment of the present invention.
Alternatively, recording the data information of the target container in the target data structure may be implemented by:
s1, acquiring connection state data of the target CPU in the target container, wherein the connection state data at least comprises one of the following data: a network protocol corresponding to the target CPU, port data corresponding to the target CPU and a connection state parameter of the target CPU;
s2, recording the connection state data of the target CPU in the target data structure.
It should be noted that each target container may include a predetermined number of CPUs, and data information corresponding to each CPU is stored in the target data structure. The data information corresponding to the CPU may be connection state data, interactive data, or basic data or key data generated in any other CPU. The connection state data of the CPU related in the embodiment of the present invention may include a network transmission protocol corresponding to the CPU, may include a network port identifier corresponding to the CPU, and may also include connection state parameters of the CPU, where the connection state parameters may include at least one of the following: the connected state is connected and in the connected state, and the disconnected state after the connection.
When data conflict occurs between at least two CPUs in the target container, polling can be performed according to the time sequence, and queuing and waiting processing can be performed, or data generated by a CPU with a high priority can be preferentially processed according to the preset CPU priority and according to the priority of the CPU. Or when a conflict occurs, temporarily storing the data with lower priority into a buffer device, and taking out and storing the data into the target data structure when no data conflict exists.
Alternatively, determining one data structure in an idle state from the N data structures of the operating system as the target data structure may be implemented by:
and selecting one data structure which is not bound with any container from the N data structures, and determining the data structure as a target data structure.
The idle-state data structure referred to in the embodiments of the present invention may be understood as a data structure that is not bound to any container. And when the data structure in the idle state exists in the operating system, selecting one data structure determined as a target data structure from the data structures to be bound with the target container after starting. The selection of the target data structure may be determined randomly, or may be selected according to the type of APP contained in the container and the size of the capacity of the data structure, or according to a specific configuration in an actual application scenario, which is not limited in this embodiment of the present invention.
Optionally, after assigning the target data structure to the target container, the method further comprises:
and after the target container is logged off, the binding between the target container and the target data structure is released, so that the target data structure is restored to an idle state.
It should be noted that, the establishment and cancellation of the container are directly related according to the data interaction requirement of the CPU, or the container may be manually configured by the user, set and started according to the requirement of the service scenario, or the container may be cancelled after the data interaction is finished or when the CPU does not need to be isolated by the container. When the container is started, independent data recording needs to be performed on the container, so that a target data structure can be allocated to a target container in a running state in an operating system, then the target container and the target data structure are bound, and data generated in the target container are all recorded in the target data structure. When the target container is logged off, the service requirement no longer exists at this time, and the data structure is not needed to record data, so that the binding between the target container and the target data structure can be released, and the target data structure is restored to an idle state, so that the binding can be performed with the next started container. The related information recorded in the target data structure can be transferred to other servers, or directly discarded, or discarded after being preserved for a period of time in other servers.
Optionally, before determining one data structure in an idle state from the N data structures of the operating system as the target data structure, the method further includes:
in the process of starting an operating system, distributing N data structures in a database corresponding to the operating system, wherein the data structures comprise: and (6) a hash data table.
It should be noted that, the allocation or creation of the data structure may be performed during the starting process of the operating system, or may be performed after the starting of the operating system is completed, and because the memory space or resource occupied by the data structure is more, it is preferable that the memory space occupied by the N data structures is allocated during the starting process of the operating system, so as to avoid that the reallocation may occupy a large amount of time or cause insufficient memory space after the starting of the operating system is completed.
Optionally, after allocating N data structures in the database corresponding to the operating system, the method further includes:
when the target container is started and no data structure in an idle state exists in the N data structures, determining a host corresponding to the operating system;
and determining the data structure corresponding to the host as a target data structure.
Fig. 4 is a flowchart of an alternative data recording method according to an embodiment of the present invention, as shown in fig. 4, taking a tcp _ hashinfo data structure corresponding to each container in a Linux system as an example, the data recording method includes the following steps:
s1, distributing a certain number of tcp _ hashinfo data structures in the starting process of the Linux system, wherein the number can be set by starting parameters;
s2, after the container is started, acquiring a tcp _ hashinfo serving as the container from the idle tcp _ hashinfo, and if no idle tcp _ hashinfo exists, using a mode before optimization, namely sharing the tcp _ hashinfo with the host;
s3, in the container operation process, the connection states generated by the container are all recorded in the tcp _ hashinfo bound by the container;
s4, releasing the tcp _ hashinfo bound by itself after the container is logged out.
the data recorded in tcp _ hashinfo further includes that after the client request of the CPU in the container is sent to the server, the server allocates a network port to the CPU, then the CPU establishes a connection with the network port, and performs data interaction, and connection parameter information, port information, network transmission protocol information, and the like of the device can be recorded in tcp _ hashinfo. When the interaction between the CPU and the network port is finished and the connection is disconnected, the corresponding disconnected state data may also be recorded in the tcp _ hashinfo data structure.
FIG. 5 is an interactive flowchart of an alternative container and data structure binding method according to an embodiment of the present invention, as shown in FIG. 5, including the following steps:
s501, starting a container 502;
s502, the server 506 allocates a data structure 504 in an idle state to the container 502;
s503, the container 502 is bound with the data structure 506;
s504, the data information generated by the container 502 is recorded in the data structure 504;
s505, the container 502 is logged out;
s506, the container 502 is unbound from the data structure 504.
Fig. 6 is an interaction flowchart of an alternative data recording method according to an embodiment of the present invention, as shown in fig. 6, including the following steps:
s601, setting that the container 502 comprises a CPU5022 and a CPU5024, and sending the client request generated by the CPU5022 to the server 506;
s602, the server 506 allocates a connected network port 508 for the CPU 5022;
s603, the CPU5022 establishes a port connection with the network port 508;
s604, the connection data and the interaction data of the CPU5022 are recorded on the data structure 504 bound with the container 502.
Under the condition of not implementing optimization, if the 48CPU machines are divided into 6 groups of containers to respectively carry out large-scale short connection tests, wherein the idle CPU consumption accounts for more than 50% of the total CPU consumption due to data protection (the CPUs carry out queuing polling when conflicts occur), the consumption proportion of the idle CPU can be greatly reduced through the technical scheme in the embodiment of the invention. The embodiment of the invention reduces the conflict in the operation by isolating and maintaining the data structure of the connection state, thereby improving the concurrency of large-scale short connection.
It should be noted that, for simplicity of description, the above-mentioned method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present invention is not limited by the order of acts, as some steps may occur in other orders or concurrently in accordance with the invention. Further, those skilled in the art should also appreciate that the embodiments described in the specification are preferred embodiments and that the acts and modules referred to are not necessarily required by the invention.
Through the above description of the embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but the former is a better implementation mode in many cases. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (e.g., ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal device (e.g., a mobile phone, a computer, a server, or a network device) to execute the method according to the embodiments of the present invention.
According to another aspect of the embodiments of the present invention, there is also provided a data recording apparatus for implementing the above data recording method. Fig. 7 is a block diagram of an alternative structure of a data recording apparatus according to an embodiment of the present invention, as shown in fig. 7, the apparatus including:
a first determining module 702, configured to determine, as a target data structure, one data structure in an idle state from N data structures of an operating system, where N is an integer greater than 1;
a binding module 704, configured to bind the target data structure with a target container, where the operating system corresponds to M containers, each container includes a predetermined number of Central Processing Units (CPUs), M is an integer greater than 1, and the target container is a container in a running state in the M containers;
a recording module 706, configured to record the data information of the target container in the target data structure.
Optionally, the recording module 706 includes:
an obtaining unit, configured to obtain connection status data of a target CPU in the target container, where the connection status data at least includes one of: a network protocol corresponding to the target CPU, port data corresponding to the target CPU, and a connection state parameter of the target CPU;
a recording unit for recording the connection state data of the target CPU in the target data structure.
Optionally, the first determining module 702 includes:
and the determining unit is used for selecting one data structure which is not bound with any container from the N data structures and determining the data structure as the target data structure.
Optionally, the apparatus further comprises:
and the releasing module is used for releasing the binding between the target container and the target data structure after the target container is cancelled so as to enable the target data structure to recover the idle state.
Optionally, the apparatus further comprises:
an allocation module, configured to allocate N data structures in a database corresponding to the operating system in a process of starting the operating system, where the data structures include: and (6) a hash data table.
Optionally, the apparatus further comprises:
a second determining module, configured to determine a host corresponding to the operating system when the target container is started and there is no data structure in an idle state in the N data structures;
and a third determining module, configured to determine a data structure corresponding to the host as the target data structure.
According to another aspect of the embodiment of the present invention, there is also provided an electronic device for implementing the data recording method, where the electronic device can be applied to, but is not limited to, the server 112 shown in fig. 2. As shown in fig. 8, the electronic device comprises a memory 902 and a processor 904, the memory 902 having a computer program stored therein, the processor 904 being arranged to perform the steps of any of the above-described method embodiments by means of the computer program.
Optionally, in this embodiment, the electronic apparatus may be located in at least one network device of a plurality of network devices of a computer network.
Optionally, in this embodiment, the processor may be configured to execute the following steps by a computer program:
s1, determining a data structure in an idle state from N data structures of an operating system as a target data structure, wherein N is an integer greater than 1;
s2, binding a target data structure with a target container, wherein the operating system corresponds to M containers, each container comprises a predetermined number of Central Processing Units (CPUs), M is an integer greater than 1, and the target container is a container in a running state in the M containers;
s3, the data information of the target container is recorded in the target data structure.
Alternatively, it can be understood by those skilled in the art that the structure shown in fig. 8 is only an illustration, and the electronic device may also be a terminal device such as a smart phone (e.g., an Android phone, an iOS phone, etc.), a tablet computer, a palm computer, a Mobile Internet Device (MID), a PAD, and the like. Fig. 8 is a diagram illustrating a structure of the electronic device. For example, the electronic device may also include more or fewer components (e.g., network interfaces, etc.) than shown in FIG. 8, or have a different configuration than shown in FIG. 8.
The memory 902 may be configured to store software programs and modules, such as program instructions/modules corresponding to the data recording method and apparatus in the embodiments of the present invention, and the processor 904 executes various functional applications and data processing by running the software programs and modules stored in the memory 902, that is, implements the data recording method. The memory 902 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 902 may further include memory located remotely from the processor 904, which may be connected to the terminal over a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof. The memory 902 may be, but is not limited to, a program step for storing a data recording method. As an example, as shown in fig. 6, the memory 902 may include, but is not limited to, the first determining module 702, the binding module 704, and the recording module 706 in the data recording apparatus. In addition, the data recording device may further include, but is not limited to, other module units in the data recording device, which is not described in detail in this example.
Optionally, the transmitting device 906 is used for receiving or sending data via a network. Examples of the network may include a wired network and a wireless network. In one example, the transmission device 906 includes a Network adapter (NIC) that can be connected to a router via a Network cable and other Network devices to communicate with the internet or a local area Network. In one example, the transmission device 906 is a Radio Frequency (RF) module, which is used for communicating with the internet in a wireless manner.
In addition, the electronic device further includes: a display 908 for displaying a binding of the target container to the target data structure; and a connection bus 910 for connecting the respective module parts in the above-described electronic apparatus.
Embodiments of the present invention also provide a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to perform the steps of any of the above method embodiments when executed.
Alternatively, in the present embodiment, the storage medium may be configured to store a computer program for executing the steps of:
s1, determining a data structure in an idle state from N data structures of an operating system as a target data structure, wherein N is an integer greater than 1;
s2, binding a target data structure with a target container, wherein the operating system corresponds to M containers, each container comprises a predetermined number of Central Processing Units (CPUs), M is an integer greater than 1, and the target container is a container in a running state in the M containers;
s3, the data information of the target container is recorded in the target data structure.
Optionally, the storage medium is further configured to store a computer program for executing the steps included in the method in the foregoing embodiment, which is not described in detail in this embodiment.
Alternatively, in this embodiment, a person skilled in the art may understand that all or part of the steps in the methods of the foregoing embodiments may be implemented by a program instructing hardware associated with the terminal device, where the program may be stored in a computer-readable storage medium, and the storage medium may include: flash disks, Read-Only memories (ROMs), Random Access Memories (RAMs), magnetic or optical disks, and the like.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
The integrated unit in the above embodiments, if implemented in the form of a software functional unit and sold or used as a separate product, may be stored in the above computer-readable storage medium. Based on such understanding, the technical solution of the present application may be substantially implemented or a part of or all or part of the technical solution contributing to the prior art may be embodied in the form of a software product stored in a storage medium, and including instructions for causing one or more computer devices (which may be personal computers, servers, network devices, or the like) to execute all or part of the steps of the method described in the embodiments of the present application.
In the above embodiments of the present application, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
In the several embodiments provided in the present application, it should be understood that the disclosed client may be implemented in other manners. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one type of division of logical functions, and there may be other divisions when actually implemented, for example, a plurality of units or components may be combined or may be integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, units or modules, and may be in an electrical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The foregoing is only a preferred embodiment of the present application and it should be noted that those skilled in the art can make several improvements and modifications without departing from the principle of the present application, and these improvements and modifications should also be considered as the protection scope of the present application.
Claims (9)
1. A data recording method, comprising:
selecting a data structure which is not bound with any container from N data structures of an operating system to be determined as a target data structure, wherein N is an integer greater than 1;
binding the target data structure with a target container, wherein the operating system corresponds to M containers, each container comprises a preset number of Central Processing Units (CPUs), M is an integer greater than 1, and the target container is a container in a running state in the M containers;
and recording the data information of the target container in the target data structure.
2. The method of claim 1, wherein the recording data information of the target container in the target data structure comprises:
acquiring connection state data of a target CPU in the target container, wherein the connection state data at least comprises one of the following data: a network protocol corresponding to the target CPU, port data corresponding to the target CPU, and a connection state parameter of the target CPU;
recording the connection state data of the target CPU in the target data structure.
3. The method of claim 1, wherein after assigning the target data structure to a target container, the method further comprises:
and after the target container is logged off, removing the binding between the target container and the target data structure so as to restore the target data structure to an idle state.
4. The method of any of claims 1 to 3, wherein before determining a data structure in an idle state from among N data structures of an operating system as a target data structure, the method further comprises:
in the process of starting the operating system, distributing N data structures in a database corresponding to the operating system, wherein the data structures comprise: and (6) a hash data table.
5. The method of claim 4, wherein after allocating N data structures in the database corresponding to the operating system, the method further comprises:
when the target container is started and no data structure in an idle state exists in the N data structures, determining a host corresponding to the operating system;
and determining the data structure corresponding to the host as the target data structure.
6. A data recording apparatus, comprising:
the first determining module is used for selecting one data structure which is not bound with any container from N data structures of an operating system to be determined as a target data structure, wherein N is an integer larger than 1;
a binding module, configured to bind the target data structure with a target container, where the operating system corresponds to M containers, each container includes a predetermined number of Central Processing Units (CPUs), M is an integer greater than 1, and the target container is a container in an operating state among the M containers;
and the recording module is used for recording the data information of the target container in the target data structure.
7. The apparatus of claim 6, wherein the recording module comprises:
an obtaining unit, configured to obtain connection status data of a target CPU in the target container, where the connection status data at least includes one of: a network protocol corresponding to the target CPU, port data corresponding to the target CPU, and a connection state parameter of the target CPU;
a recording unit for recording the connection state data of the target CPU in the target data structure.
8. A computer-readable storage medium comprising a stored program, wherein the program when executed performs the method of any of claims 1 to 5.
9. An electronic device comprising a memory and a processor, characterized in that the memory has stored therein a computer program, the processor being arranged to execute the method of any of claims 1 to 5 by means of the computer program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911003589.XA CN110780817B (en) | 2019-10-18 | 2019-10-18 | Data recording method and apparatus, storage medium, and electronic apparatus |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911003589.XA CN110780817B (en) | 2019-10-18 | 2019-10-18 | Data recording method and apparatus, storage medium, and electronic apparatus |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110780817A CN110780817A (en) | 2020-02-11 |
| CN110780817B true CN110780817B (en) | 2021-12-07 |
Family
ID=69386250
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911003589.XA Active CN110780817B (en) | 2019-10-18 | 2019-10-18 | Data recording method and apparatus, storage medium, and electronic apparatus |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110780817B (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103678203A (en) * | 2013-12-13 | 2014-03-26 | 国家计算机网络与信息安全管理中心 | Method and device for achieving zero copy of network card |
| CN105190545A (en) * | 2014-01-27 | 2015-12-23 | 华为技术有限公司 | Virtualization method and apparatus, and computer device |
| CN106484886A (en) * | 2016-10-17 | 2017-03-08 | 金蝶软件（中国）有限公司 | A kind of method of data acquisition and its relevant device |
| CN106874102A (en) * | 2015-12-18 | 2017-06-20 | 北京奇虎科技有限公司 | Resource regulating method and device based on container work property |
| CN107179955A (en) * | 2017-05-08 | 2017-09-19 | 北京海杭通讯科技有限公司 | Equipment blocking method and its terminal based on Android containers |
| CN108140014A (en) * | 2015-11-02 | 2018-06-08 | 华为技术有限公司 | Create and use the system and method for the data structure for multiple programming |
| CN108255598A (en) * | 2016-12-28 | 2018-07-06 | 华耀(中国)科技有限公司 | The virtual management platform resource distribution system and method for performance guarantee |
| CN109284169A (en) * | 2018-09-10 | 2019-01-29 | 福建星瑞格软件有限公司 | Big data platform process management method and computer equipment based on process virtual |
| CN110119308A (en) * | 2018-02-07 | 2019-08-13 | 北京第一视角科技有限公司 | The system for managing extensive container application |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9652178B2 (en) * | 2013-09-10 | 2017-05-16 | Veritas Technologies | Systems and methods for protecting virtual machine data |
| CN107463402B (en) * | 2017-07-31 | 2018-09-14 | 腾讯科技（深圳）有限公司 | The operation method and device of virtual opetrating system |
| US10956593B2 (en) * | 2018-02-15 | 2021-03-23 | International Business Machines Corporation | Sharing of data among containers running on virtualized operating systems |
-
2019
- 2019-10-18 CN CN201911003589.XA patent/CN110780817B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103678203A (en) * | 2013-12-13 | 2014-03-26 | 国家计算机网络与信息安全管理中心 | Method and device for achieving zero copy of network card |
| CN105190545A (en) * | 2014-01-27 | 2015-12-23 | 华为技术有限公司 | Virtualization method and apparatus, and computer device |
| CN108140014A (en) * | 2015-11-02 | 2018-06-08 | 华为技术有限公司 | Create and use the system and method for the data structure for multiple programming |
| CN106874102A (en) * | 2015-12-18 | 2017-06-20 | 北京奇虎科技有限公司 | Resource regulating method and device based on container work property |
| CN106484886A (en) * | 2016-10-17 | 2017-03-08 | 金蝶软件（中国）有限公司 | A kind of method of data acquisition and its relevant device |
| CN108255598A (en) * | 2016-12-28 | 2018-07-06 | 华耀(中国)科技有限公司 | The virtual management platform resource distribution system and method for performance guarantee |
| CN107179955A (en) * | 2017-05-08 | 2017-09-19 | 北京海杭通讯科技有限公司 | Equipment blocking method and its terminal based on Android containers |
| CN110119308A (en) * | 2018-02-07 | 2019-08-13 | 北京第一视角科技有限公司 | The system for managing extensive container application |
| CN109284169A (en) * | 2018-09-10 | 2019-01-29 | 福建星瑞格软件有限公司 | Big data platform process management method and computer equipment based on process virtual |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110780817A (en) | 2020-02-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109343963B (en) | Application access method and device for container cluster and related equipment | |
| US20200065010A1 (en) | Cluster system with calculation and storage converged | |
| JP5510556B2 (en) | Method and system for managing virtual machine storage space and physical hosts | |
| CN109379448B (en) | File distributed deployment method and device, electronic equipment and storage medium | |
| CN106326226B (en) | The method and system of log-on data library service in a kind of public cloud | |
| CN103677858A (en) | Method, system and device for managing virtual machine software in cloud environment | |
| CN106354559A (en) | Method and device for processing cloud desktop resources | |
| CN113965560A (en) | Data transmission method, proxy server, storage medium, and electronic device | |
| CN115454636A (en) | Container cloud platform GPU resource scheduling method, device and application | |
| CN112084173A (en) | Data migration method and device and storage medium | |
| CN111225003B (en) | NFS node configuration method and device | |
| US7970882B2 (en) | Management apparatus and management method | |
| CN104517067A (en) | Method, device and system for data access | |
| CN112035062B (en) | Migration method of local storage of cloud computing, computer equipment and storage medium | |
| CN116339927B (en) | Equipment determining method, device, storage medium and electronic device | |
| CN110780817B (en) | Data recording method and apparatus, storage medium, and electronic apparatus | |
| CN108829340B (en) | Storage processing method, device, storage medium and processor | |
| CN104735109A (en) | Medical image data storing system | |
| CN110716690B (en) | Data recovery method and system | |
| US9432476B1 (en) | Proxy data storage system monitoring aggregator for a geographically-distributed environment | |
| CN107493181B (en) | Indication method and device of virtual expansion port | |
| CN115826845A (en) | Storage resource allocation method and device, storage medium and electronic device | |
| CN112347036B (en) | Inter-cloud migration method and device of cloud storage system | |
| CN111722783B (en) | Data storage method and device | |
| CN112398668B (en) | IaaS cluster-based cloud platform and node switching method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| REG | Reference to a national code |
Ref country code: HK Ref legal event code: DE Ref document number: 40022174 Country of ref document: HK |
|
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |