CN115774701A

CN115774701A - Data sharing method and device, electronic equipment and storage medium

Info

Publication number: CN115774701A
Application number: CN202211538950.0A
Authority: CN
Inventors: 赵二城
Original assignee: Capitalonline Data Service Co ltd
Current assignee: Capitalonline Data Service Co ltd
Priority date: 2022-12-01
Filing date: 2022-12-01
Publication date: 2023-03-10
Anticipated expiration: 2042-12-01
Also published as: CN115774701B

Abstract

The application provides a data sharing method, a data sharing device, electronic equipment and a storage medium. The method comprises the following steps: creating a mirror image file and a first folder; mounting the mirror image file to a first folder; starting a virtual machine, and adding an image file serving as a virtual storage device into the virtual machine, wherein the virtual storage device stores first data; and sharing the first data to the storage server through the file sharing client. Through the process, the host machine can share the data of the users in the virtual machines to the storage server by using more resources than the virtual machines, and the data sharing efficiency of the virtual machines is improved.

Description

Data sharing method and device, electronic equipment and storage medium

Technical Field

The present application relates to the field of data sharing technologies, and in particular, to a data sharing method and apparatus, an electronic device, and a storage medium.

Background

Due to the limited storage space in the virtual machine, data of a user in the virtual machine on the cloud platform needs to be stored in the storage server, and in the related art, the virtual machine transmits data to the storage server by establishing an NFS (Network File System) client. Because computing resources, I/O (Input/Output) resources, and network transmission resources of the virtual machine are limited, sharing data to the storage server on the virtual machine is inefficient, and mass data storage generated when a user performs GPU (Graphics Processing Unit) rendering or big data Processing cannot be far satisfied, which affects data storage efficiency and user experience of using a cloud platform.

Disclosure of Invention

The embodiment of the application provides a data sharing method and device, electronic equipment and a storage medium, so as to solve the problems in the related art.

In a first aspect, an embodiment of the present application provides a data sharing method, which is applied to a host, where the host includes a file sharing client and a virtual machine, and the host is in communication connection with a storage server, and the data sharing method includes: creating a mirror image file and a first folder; mounting the mirror image file to the first folder; starting the virtual machine, and adding the image file into the virtual machine as a virtual storage device, wherein the virtual storage device stores first data; and sharing the first data to the storage server through the file sharing client.

In a second aspect, an embodiment of the present application provides a data sharing apparatus, which is deployed on a host, where the host includes a file sharing client and a virtual machine, and the host is in communication connection with a storage server, and the data sharing apparatus includes: a creating unit configured to create an image file and a first folder; a mounting unit configured to mount the mirror image file to the first folder; the starting unit is configured to start the virtual machine and add the image file to the virtual machine as a virtual storage device, wherein the virtual storage device stores first data; a sharing unit configured to share the first data to a storage server through the file sharing client.

In a third aspect, an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored on the memory, where the processor implements the method described in any one of the above when executing the computer program.

In a fourth aspect, the present application provides a computer-readable storage medium, in which a computer program is stored, and the computer program, when executed by a processor, implements the method of any one of the above.

This application has following advantage:

according to the embodiment of the application, the host machine can add the virtual storage device to the virtual machine, the data generated by the user running program in the virtual machine is obtained through the virtual storage device, the data of the user in the virtual machine is shared to the storage server by establishing the file sharing client on the host machine and using more computing resources, I/O resources and network transmission resources than the virtual machine, the problems of low efficiency and low speed of the virtual machine in directly sharing the data to the storage server can be effectively solved, the speed of sharing the data by the virtual machine is improved, and better experience is provided for the user stored data of the virtual machine.

The foregoing description is only an overview of the technical solutions of the present application, and the following detailed description of the present application is given to enable the technical means of the present application to be more clearly understood and to enable the above and other objects, features, and advantages of the present application to be more clearly understood.

Drawings

In the drawings, like reference numerals refer to the same or similar parts or elements throughout the several views unless otherwise specified. The figures are not necessarily to scale. It is appreciated that these drawings depict only some embodiments in accordance with the disclosure and are not to be considered limiting of its scope.

FIG. 1 is a schematic diagram illustrating one approach to data sharing implemented by the related art;

FIG. 2 is a schematic diagram illustrating a data sharing scheme according to an embodiment of the present application;

FIG. 3 is a flow chart illustrating a data sharing method according to an embodiment of the present application;

fig. 4 is a block diagram showing a configuration of a data sharing apparatus according to an embodiment of the present application; and

fig. 5 is a block diagram illustrating an electronic device according to an embodiment of the present application.

Detailed Description

In the following, only certain exemplary embodiments are briefly described. As those skilled in the art will recognize, the described embodiments may be modified in various different ways, without departing from the spirit or scope of the present application. Accordingly, the drawings and description are to be regarded as illustrative in nature, and not as restrictive.

In order to facilitate understanding of the technical solutions of the embodiments of the present application, the following description is made of related art of the embodiments of the present application. The following related arts as alternatives can be arbitrarily combined with the technical solutions of the embodiments of the present application, and all of them belong to the scope of the embodiments of the present application.

In a related technology before the present application, in order to store data in a virtual machine in a storage server, the virtual machine shares data with the storage server by establishing an NFS (Network File System) client. Fig. 1 is a schematic diagram illustrating one scheme of implementing data sharing in the related art. As shown in fig. 1, the virtual machine runs in a host machine, and the host machine allocates part of resources, such as computing resources, network transmission resources, and I/O (Input/Output) resources, for the virtual machine to use. Because the resources used by the virtual machine are limited, the data sharing efficiency on the virtual machine is low, and particularly when a user of the virtual machine runs a GPU (Graphics Processing Unit) for rendering or large data Processing to generate a large amount of data, the speed and efficiency for sharing the large amount of data are low, which affects the data storage experience of the user of the virtual machine.

In view of the above, embodiments of the present application provide a new data sharing scheme to solve all or part of the above problems in the related art. In order to more clearly show the data sharing scheme provided in the embodiments of the present application, specific application examples of the schemes of the embodiments of the present application are given as follows. Fig. 2 is a schematic diagram illustrating a data sharing scheme according to an embodiment of the present application. As shown in fig. 2, the host adds virtual storage to the virtual machines, where the virtual machines store data. The host machine acquires data in the virtual machine by sharing the virtual storage device with the virtual machine. The host machine shares the data of the virtual users to the storage server by establishing the file sharing client side through higher computing resources, I/O resources and network transmission resources. Compared with the related technical scheme, the technical scheme of the embodiment of the application can share the data in the virtual machine to the storage server with higher efficiency and higher speed.

An embodiment of the present application provides a data sharing method 300, which is applied to a host machine, where the host machine includes a file sharing client and a virtual machine. The host machine is in communication connection with the storage server. The host machine can be a cloud server, a personal computer or a physical server. The virtual machine can be a complete computer system which is created by a host machine through a virtualization technology, has complete hardware system functions and runs in a completely isolated environment. The data sharing method 300 of the present application will be described with reference to fig. 3. Fig. 3 is a flow chart illustrating a data sharing method 300 according to an embodiment of the present application. As shown in fig. 3, the data sharing method 300 may include the following steps S301 to S304. The following describes steps S301 to S304 in detail with reference to specific embodiments.

The process first proceeds to step S301. In step S301, an image file and a first folder are created.

In some embodiments, the host creates the image file through a command line command. For example, the host may create the image file using a qemu-img create command, or may create the image file using a dd (copy and convert file) command. The image file may be used to encapsulate the contents of the entire disk, such as system files, boot files, partition table information, and the like. The image file may also be used to represent a fixed size storage device. The image file may be an image file such as img image, raw image, qcow2 image. For example, the host may create a 10G size image file in qcow2 format named test via the qemu-img create-f qcow2test. Qcow 2G command.

In some embodiments, the host formats the mirror file to build the file system. The file system of computer is a method for storing and organizing computer data, and is responsible for storing user's files in disk hardware and persistently storing the files. Because the created image file has no file system, cannot be mounted by an operating system of a host, and cannot be stored, the created image file needs to be formatted to construct the file system. The File system may be a File system such as an ext3 (Third extended File system) format, an ext4 (Fourth extended File system) format, and an exFAT (extended File Allocation Table) format. For example, the host may use an mkfs.ext4share.img command to construct a file system in ext4 format on the share.img image file, so that the image file may be mounted by the host, and may also store data.

In some embodiments, the host creates the first folder via an mkdir command. For example, if the host uses the mkdir first directory command, a folder named first directory is created. One or more subfolders may be included in the first folder.

The process then proceeds to step S302. In step S302, mount the image file to the first folder.

The mount mainly functions to associate a file system on a storage device with a file system driver in an operating system, where the storage device is a logical device and may also be a virtual storage device. For example, an image file may be viewed as a virtual storage device. The mount may be to associate a top level directory in the image file to a directory below the root directory of the host, and accessing the directory in the host is equivalent to accessing the image file.

In one embodiment, the host mounts the image file of the built file system under the first folder using mount command, such that accessing the first folder is equivalent to accessing the image file. Thus, storing data into the first folder is also equivalent to storing data into the image file.

The process then proceeds to step S303. In step S303, the virtual machine is started, and the image file is added to the virtual machine as a virtual storage device, where the virtual storage device stores first data.

In some embodiments, the host may create and start Virtual machines using KVM (Kernel-based Virtual Machine) virtualization technology. In other embodiments, the host may create and start a virtual machine using Xen virtualization technology. The above is merely an example of the virtualization technology used by the host, and those skilled in the art can apply the principles of the present application to a host using any virtualization technology according to the actual situation.

In some embodiments, the host adds the image file of the built file system to the virtual machine after the virtual machine is started. For example, the host may add the image file as a piece of virtual storage device to the virtual machine through a virsh attach-disk (add disk) command, and after the virtual machine acquires the virtual storage device, the host may store the first data in the virtual machine in the virtual storage device.

In one embodiment, the virtual machine creates a second folder and mounts the virtual storage device under the second folder. For example, the virtual machine creates a second folder through an mkdir command, and mounts the virtual storage device under the second folder using a mount command, so as to realize the association of the second folder with the virtual storage device.

In an embodiment, the virtual machine may store the first data in the second folder, and then the first folder in the host may obtain the first data in the virtual machine. Since the first folder in the host mounts the image file of the file system and adds the image file to the virtual machine as a virtual storage device, and the virtual machine mounts the virtual storage device to the second folder, the host shares the virtual storage device with the virtual machine, that is, the first folder shares the virtual storage device with the second folder, so that storing the first data to the second folder by the virtual machine is equal to storing the first data to the first folder.

The first data in the virtual machine may be data generated by a user of the virtual machine running a program. For example, users of virtual machines generate large amounts of data when training machine learning models or performing big data analysis. The user of the virtual machine can set the data output path as a second folder before the program is executed, and all data generated by the program is stored in the second folder.

The process then proceeds to step S304. In step S304, the first data is shared to the storage server by the file sharing client.

In the method 300, the storage server may be a storage server deployed on a cloud platform, or may be a storage server deployed locally. The host machine can share and store the first data in the server by establishing a file sharing client.

In some embodiments, the File sharing client established by the host is any one of a NFS (Network File System) client, an SMB (Server Message Block) client, and an FTP (File Transfer Protocol) client.

In an embodiment, the host shares the first data with the storage server by establishing an NFS client. NFS allows computers in the network to share data, and in NFS applications, client applications of the local NFS can transparently read and write files located on a remote NFS server as if they had access to local files. The NFS adopts a C/S (Client/Server) architecture, the storage Server as a Server needs to start the NFS Server first, and the host as a Client needs to start the NFS Client. After the storage server starts the NFS server, the storage server may set the shared directory to implement operations (e.g., data reading and data storing) of the shared directory in the storage server by the host, thereby completing sharing and storing the first data from the NFS client to the NFS server.

In some embodiments, after the host starts the file sharing client, the host acquires a shared directory of the storage server, and shares the first data in the first folder to the shared directory. In one embodiment, the host may obtain the shared directory of the storage server by querying the relevant information of the storage server. For example, the ip address of the storage server is 10.0.0.101, and the host may use a showmount (query about NFS server related information) command to obtain the shared directory set by the server through a showmount-e 10.0.0.101 command.

In one embodiment, the host mounts the shared directory obtained from the storage server under the first folder for associating the shared directory with the first folder. The data read-write operation of the host machine on the first folder is equal to the data read-write operation of the shared directory of the storage server. Therefore, the host can share the first data to the shared directory of the server by storing the first data to the first folder. The function and implementation of the mount of the shared directory by the host may refer to the corresponding description of the method for mounting the mirror image file to the first folder, which is not described herein again.

Corresponding to the application scenario and method of the method provided by the embodiment of the present application, the embodiment of the present application further provides a data sharing apparatus 400 deployed on a host, where the host includes a file sharing client and a virtual machine, and the host is in communication connection with a storage server. Fig. 4 is a block diagram showing a configuration of a data sharing apparatus 400 according to an embodiment of the present application. As shown in fig. 4, the data sharing apparatus 400 may include: a creation unit 401, a mount unit 402, an activation unit 403, and a sharing unit 404.

The creating unit 401 is configured to create an image file and a first folder.

The creating unit 401 may be a component in the host, through which the host creates the image file and the first folder. The host machine can be a cloud server, a personal computer or a physical server.

For the description of the features such as the image file, the first folder, etc., reference may be made to the above description of the data sharing method 300, and a detailed description of such features will be omitted here for the sake of simplicity of the description.

A mounting unit 402 configured to mount the image file to the first folder.

The mounting unit 402 may be a component in the host, through which the host mounts the image file to the first folder.

In one embodiment, the host mounts the image file of the file system under the first folder through a mount command, so that accessing the first folder is equal to accessing the image file. Thus, storing data into the first folder is also equivalent to storing data into the image file.

A starting unit 403, configured to start the virtual machine and add the image file to the virtual machine as a virtual storage device, where the virtual storage device stores the first data.

The starting unit 403 may be a component in the host, through which the host starts the virtual machine and adds the image file as a virtual storage to the virtual machine.

In some embodiments, the host may create and start a Virtual Machine using KVM (Kernel-based Virtual Machine) virtualization technology. In other embodiments, the host may create and start a virtual machine using Xen virtualization technology. The virtualization technology used by the host machine is only exemplified by way of example, and those skilled in the art can apply the principles of the present application to a host machine using any virtual machine technology according to actual situations.

In some embodiments, the host adds the image file of the built file system to the virtual machine after the virtual machine is started. For example, the host may add the image file as a block of virtual storage to the virtual machine via a virsh attach-disk command.

The startup unit 403 is further configured to mount, by the virtual machine, the virtual storage to a second folder in the virtual machine.

A sharing unit 404 configured to share the first data to a storage server through the file sharing client.

The sharing unit 404 may be a component in the host, and the host shares the first data to the storage server through the component via the file sharing client. The storage server may be a storage server deployed on a cloud platform, or may be a storage server deployed locally. The host machine can share and store the first data in the server by establishing a file sharing client.

In some embodiments, the File sharing client established by the host is any one of an NFS (Network File System) client, an SMB (Server Message Block) client, and an FTP (File Transfer Protocol) client.

The functions of each module in each device in the embodiment of the present application can be referred to the corresponding description in the above method, and have corresponding beneficial effects, which are not described herein again.

FIG. 5 is a block diagram of an electronic device used to implement embodiments of the present application. As shown in fig. 5, the electronic apparatus includes: a memory 501 and a processor 502, the memory 501 having stored therein a computer program operable on the processor 502. The processor 502, when executing the computer program, implements the method in the above embodiments. The number of the memory 501 and the processor 502 may be one or more.

The electronic device further includes:

the communication interface 503 is used for communicating with an external device and performing data interactive transmission.

If the memory 501, the processor 502 and the communication interface 503 are implemented independently, the memory 501, the processor 502 and the communication interface 503 may be connected to each other through a bus and perform communication with each other. The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an Extended ISA (EISA) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown in FIG. 5, but this is not intended to represent only one bus or type of bus.

Optionally, in a specific implementation, if the memory 501, the processor 502, and the communication interface 503 are integrated on a chip, the memory 501, the processor 502, and the communication interface 503 may complete communication with each other through an internal interface.

Embodiments of the present application provide a computer-readable storage medium, which stores a computer program, and when the program is executed by a processor, the computer program implements the method provided in the embodiments of the present application.

The embodiment of the present application further provides a chip, where the chip includes a processor, and is configured to call and run an instruction stored in a memory from the memory, so that a communication device in which the chip is installed executes the method provided in the embodiment of the present application.

An embodiment of the present application further provides a chip, including: the system comprises an input interface, an output interface, a processor and a memory, wherein the input interface, the output interface, the processor and the memory are connected through an internal connection path, the processor is used for executing codes in the memory, and when the codes are executed, the processor is used for executing the method provided by the embodiment of the application.

It should be understood that the Processor may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic, discrete hardware components, etc. The general purpose processor may be a microprocessor or any conventional processor or the like. It is noted that the processor may be a processor supporting an Advanced reduced instruction set machine (ARM) architecture.

Further, the memory may optionally include a read only memory and a random access memory. The memory may be either volatile memory or nonvolatile memory, or may include both volatile and nonvolatile memory. The non-volatile Memory may include a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically Erasable PROM (EEPROM), or a flash Memory. Volatile Memory can include Random Access Memory (RAM), which acts as external cache Memory. By way of example, and not limitation, many forms of RAM may be used. For example, static Random Access Memory (Static RAM, SRAM), dynamic Random Access Memory (DRAM), synchronous Dynamic Random Access Memory (SDRAM), double Data Rate Synchronous Dynamic Random Access Memory (DDR SDRAM), enhanced Synchronous SDRAM (ESDRAM), synclink DRAM (SLDRAM), and Direct bus RAM (DR RAM).

In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. The procedures or functions according to the present application are generated in whole or in part when the computer program instructions are loaded and executed on a computer. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another computer readable storage medium.

In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the application. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Moreover, various embodiments or examples and features of various embodiments or examples described in this specification can be combined and combined by one skilled in the art without being mutually inconsistent.

Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present application, "a plurality" means two or more unless specifically limited otherwise.

Any process or method described in a flow diagram or otherwise herein may be understood as representing a module, segment, or portion of code, which includes one or more executable instructions for implementing specific logical functions or steps of the process. And the scope of the preferred embodiments of the present application includes other implementations in which functions may be performed out of the order shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved.

The logic and/or steps described in the flowcharts or otherwise described herein, such as an ordered listing of executable instructions that can be considered to implement logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions.

It should be understood that portions of the present application may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. All or part of the steps of the method of the above embodiments may be implemented by hardware that is configured to be instructed to perform the relevant steps by a program, which may be stored in a computer-readable storage medium, and which, when executed, includes one or a combination of the steps of the method embodiments.

In addition, functional units in the embodiments of the present application may be integrated into one processing module, or each unit may exist alone physically, or two or more units are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. The integrated module may also be stored in a computer-readable storage medium if it is implemented in the form of a software functional module and sold or used as a separate product. The storage medium may be a read-only memory, a magnetic or optical disk, or the like.

The above description is only an exemplary embodiment of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive various changes or substitutions within the technical scope described in the present application, and these should be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A data sharing method is applied to a host machine, the host machine comprises a file sharing client and a virtual machine, the host machine is in communication connection with a storage server, and the data sharing method comprises the following steps:

creating a mirror image file and a first folder;

mounting the mirror image file to the first folder;

starting the virtual machine, and adding the image file into the virtual machine as a virtual storage device, wherein the virtual storage device stores first data;

and sharing the first data to the storage server through the file sharing client.

2. The data sharing method of claim 1, further comprising:

the virtual machine mounts the virtual storage device to a second folder in the virtual machine.

3. The data sharing method according to claim 1 or 2, wherein the sharing the first data to the storage server by the file sharing client includes:

acquiring a shared directory of the storage server;

sharing the first data in the first folder to the shared directory.

4. The data sharing method according to claim 1 or 2, wherein after the creating of the image file, the data sharing method further comprises:

and formatting the image file to construct a file system.

5. The data sharing method according to claim 1 or 2, wherein the file sharing system client is any one of an NFS client, an FTP client, and an SMB client.

6. The data sharing method of claim 3, further comprising:

and mounting the shared directory to the first folder.

7. A data sharing device is deployed on a host machine, the host machine comprises a file sharing client and a virtual machine, the host machine is in communication connection with a storage server, and the data sharing device comprises:

a creating unit configured to create an image file and a first folder;

a mounting unit configured to mount the mirror image file to the first folder;

the starting unit is configured to start the virtual machine and add the image file to the virtual machine as a virtual storage device, wherein the virtual storage device stores first data;

a sharing unit configured to share the first data to the storage server through the file sharing client.

8. The data sharing device of claim 7, wherein the startup unit is further configured to mount, by the virtual machine, the virtual storage device to a second folder in the virtual machine.

9. An electronic device comprising a memory, a processor and a computer program stored on the memory, the processor implementing the method of any one of claims 1-6 when executing the computer program.

10. A computer-readable storage medium, having stored therein a computer program which, when executed by a processor, implements the method of any of claims 1-6.