CN117056031B - Cross-container dynamic library sharing method based on virtual process - Google Patents

Cross-container dynamic library sharing method based on virtual process Download PDF

Info

Publication number
CN117056031B
CN117056031B CN202311310584.8A CN202311310584A CN117056031B CN 117056031 B CN117056031 B CN 117056031B CN 202311310584 A CN202311310584 A CN 202311310584A CN 117056031 B CN117056031 B CN 117056031B
Authority
CN
China
Prior art keywords
virtual
dynamic library
page table
page
dynamic
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202311310584.8A
Other languages
Chinese (zh)
Other versions
CN117056031A (en
Inventor
温研
谷天阳
袁利
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Linzhuo Information Technology Co Ltd
Original Assignee
Beijing Linzhuo Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Linzhuo Information Technology Co Ltd filed Critical Beijing Linzhuo Information Technology Co Ltd
Priority to CN202311310584.8A priority Critical patent/CN117056031B/en
Publication of CN117056031A publication Critical patent/CN117056031A/en
Application granted granted Critical
Publication of CN117056031B publication Critical patent/CN117056031B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44Arrangements for executing specific programs
    • G06F9/455Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533Hypervisors; Virtual machine monitors
    • G06F9/45558Hypervisor-specific management and integration aspects
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44Arrangements for executing specific programs
    • G06F9/455Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533Hypervisors; Virtual machine monitors
    • G06F9/45558Hypervisor-specific management and integration aspects
    • G06F2009/45562Creating, deleting, cloning virtual machine instances
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44Arrangements for executing specific programs
    • G06F9/455Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533Hypervisors; Virtual machine monitors
    • G06F9/45558Hypervisor-specific management and integration aspects
    • G06F2009/45595Network integration; Enabling network access in virtual machine instances

Abstract

The invention discloses a cross-container dynamic library sharing method based on virtual process, which establishes a virtual process which is not actually executed for the process when the process is created for the first time in the running environment, the address space layout of the virtual process is the same as that of the process, and when other running environments of an operating system are started, a system virtual page table consisting of virtual page tables of all virtual processes in the current operating system and virtual dynamic library page tables is copied into the running environment, so that the sharing of the process to the dynamic library among different running environments is realized through the constructed virtual process, and the memory overhead of the loading process of the dynamic library is obviously reduced.

Description

Cross-container dynamic library sharing method based on virtual process
Technical Field
The invention belongs to the technical field of cross-running environment development, and particularly relates to a cross-container dynamic library sharing method based on a virtual process.
Background
Within the operating system's running environment, the same library file will be loaded by default for a dynamic library operating system with the same full path, and it will belong to a certain dynamic library by default for a certain symbolic operating system. Wherein the operating environment includes a container or the like. In a certain running environment of the operating system, multiple processes all depend on the dynamic library a, and the dynamic library a depends on the other dynamic libraries B, C and D, so when the processes are started, the operating system loads the dynamic library a and simultaneously loads the dynamic libraries B, C and D on which the dynamic library a depends, and at this time, the dynamic libraries B, C and D loaded by different processes must be the same. However, in different operating environments, when different processes load the dynamic library a, the dynamic libraries B, C and D loaded by the operating system may be different, which may result in different external symbol addresses used when the external symbols are called in the Code of the dynamic library a, so that Code pages (Code Memory pages) used by the dynamic library a to call the external symbols need to be shared between different operating environments. For a symbol, if multiple processes all depend on the symbol f in a certain running environment, the symbol is a function or a variable, then different processes will search for the symbol f in each dynamic library when starting, where the symbol f that these processes depend on is necessarily located in the same dynamic library, but when different processes all depend on the symbol f in different running environments, the symbol f may be located in different dynamic libraries. This is caused by the different paths of the same dynamic library within different operating environments and the different dependent dynamic libraries, so that for different processes created by multiple operating environments using the same dynamic library, existing dynamic library sharing mechanisms cannot realize such cross-operating environment process-related dynamic library sharing.
Disclosure of Invention
In view of the above, the present invention provides a cross-container dynamic library sharing method based on virtual processes, which can realize dynamic library sharing between the same processes running in different running environments.
The invention provides a cross-container dynamic library sharing method based on a virtual process, which specifically comprises the following steps:
step 1, before a first running environment is started, acquiring a dynamic library loading sequence of a current process, virtual page tables of all virtual processes and virtual dynamic library page tables as system virtual page tables, wherein the system virtual page tables are used for storing address space layout of the processes; starting a first running environment, and copying a system virtual page table into the first running environment;
step 2, when a first running environment creates a first process, if the same virtual process exists in the system virtual page table, the virtual process is used as the first virtual process to acquire a virtual page table and a virtual dynamic library page table corresponding to the virtual process, and step 3 is executed, otherwise, a standard flow creation process is executed, a virtual process corresponding to the standard flow creation process is created, the virtual page table and the virtual dynamic library page table of the virtual process are adopted to update the system virtual page tables in all running environments, and step 4 is executed;
step 3, establishing mapping from virtual memory to physical memory pages of the process according to a virtual page table of the first virtual process, and adding the first virtual process into a process scheduling queue of an operating system;
step 4, when the first process loads the dynamic library, if the same dynamic library exists in the loaded dynamic library, loading the dynamic library according to the virtual address of the dynamic library, and ending the process; otherwise, loading the dynamic library to establish mapping between the virtual memory and the dynamic library function address, storing the mapping between the dynamic library function address and the virtual memory into a virtual dynamic library page table of the first virtual process, and updating a system virtual page table in the started running environment by using the virtual dynamic library page table to finish the process.
Further, the judging manner that the same virtual process exists in the system virtual page table in the step 2 is as follows: and adopting the hash value of the executable file to identify the process and the corresponding virtual process, calculating the hash value of the executable file corresponding to the first process, and searching the virtual process with the same hash value in the system virtual page table.
Further, the judging manner of the same dynamic library exists in the loaded dynamic library in the step 4 is as follows: and identifying the dynamic library by adopting the hash value of the dynamic library, calculating the hash value of the dynamic library, and searching the same dynamic library as the loaded dynamic library in the operating system according to the hash value.
Further, the loading the dynamic library according to the virtual address of the dynamic library in the step 4 is as follows: and obtaining the virtual address of the dynamic library from the virtual dynamic library page table of the first virtual process, and loading the dynamic library according to the virtual address.
Further, the process of creating the virtual process corresponding to the virtual process in the step 2 is:
copying a page table of a first process to serve as a virtual page table of a virtual process and not distributing physical memory pages;
copying the relevant sections of the dynamic library in the page table of the first process into virtual dynamic library page tables of the dynamic library according to the loading sequence of the relevant dynamic library of the first process;
and obtaining a subprocess of the first process according to other standard programs called by the fork system, taking the subprocess as a virtual process, and not adding the virtual process to a process scheduling queue of an operating system.
Further, the way to copy the page table of the first process as the virtual page table of the virtual process and not allocate the physical memory page is: the kernel do_mmap function implementation is modified.
Advantageous effects
The invention establishes the virtual process which is not actually executed for the process when the running environment creates the process for the first time, the address space layout of the virtual process is the same as that of the process, and when other running environments of the operating system are started, the system virtual page table formed by the virtual page tables of all the virtual processes in the current operating system and the virtual dynamic library page table is copied into the running environment, thereby realizing the sharing of the dynamic library by the processes among different running environments through the constructed virtual process and obviously reducing the memory overhead of the loading process of the dynamic library.
Detailed Description
The present invention will be described in detail with reference to the following examples.
The invention provides a cross-container dynamic library sharing method based on a virtual process, which has the following core ideas: when the running environment of the operating system creates a process for the first time, a virtual process which is not actually executed is created for the process, the address space layout of the virtual process is the same as that of the process, and when other running environments of the operating system are started, a system virtual page table consisting of virtual page tables of all virtual processes in the current operating system and virtual dynamic library page tables is copied into the running environment, so that sharing of dynamic libraries among different running environments is realized through the constructed virtual process.
The invention provides a cross-container dynamic library sharing method based on a virtual process, which specifically comprises the following steps:
step 1, before a new running environment is started in an operating system, the running environment is recorded as a first running environment, and a dynamic library loading sequence of a process in the operating system, a virtual page table and a virtual dynamic library page table corresponding to all virtual processes in each current running environment are obtained; after a new running environment is started, copying virtual page tables corresponding to all virtual processes and virtual dynamic library page tables serving as current system virtual page tables into a first running environment.
The virtual process is a mapping constructed by the running environment for the created process, and a virtual page table and a virtual dynamic library page table are created for the virtual process to store the address space layout of the process, wherein the address space layout refers to the mapping relationship between the process and the virtual memory and the physical memory page of the related dynamic library. The virtual process and the corresponding process correspond to the same executable file and have the same hash value of the executable file.
And 2, when the first running environment creates the first process, calculating a hash value of an executable file corresponding to the first process, recording the hash value as the first hash value, searching whether a virtual process with the same hash value exists in a system virtual page table stored in the first running environment according to the first hash value, if so, acquiring a virtual page table and a virtual dynamic library page table corresponding to the virtual process by taking the virtual process as the first virtual process, executing the step 3, otherwise, executing a standard flow creation process of an operating system, creating the virtual process for the process, copying the virtual page table and the virtual dynamic library page table of the virtual process to the started running environment, and updating the system virtual page table, and executing the step 4.
Specifically, if a corresponding virtual process exists in the system virtual page table stored in the first operating environment, it is indicated that other operating environments in the current operating system have created a process corresponding to the executable file, and the operating system establishes a page table for the process and allocates a physical memory page for the process. The page table is a special data structure provided by the operating system, is located in a page table area of the operating system space, and is used for recording the mapping relation between the virtual memory page and the physical memory page. Each process has its own page table with pointers to the page table in the process control block table (PCB table).
The process of creating the virtual process for the first process is realized in a mode of adopting the modified fork system call, and specifically comprises the following steps:
step 2.1, virtually loading the mirror image of the executable file corresponding to the first process into the memory, namely: the page table of the first process is copied as a virtual page table of the virtual process, and only virtual memory is allocated for the virtual process, but physical memory pages are not allocated.
The page table of the first process is copied by modifying the kernel do_mmap function, virtual memory is only allocated for the virtual process, and an actual physical memory page is not allocated for the virtual process.
In the process of loading an executable file of a process into virtual memory by an operating system kernel, the executable file is typically divided into segments, including code segments, data segments, BSS segments, etc., which are typically discontinuous in virtual memory and mapped by page tables, the actual execution of the present invention is a copy of the page tables containing these segments of the first process.
Step 2.2, completing virtual loading of the dynamic library according to the dynamic library loaded by the first process and the loading sequence of the dynamic library, namely: copying a segment related to the dynamic library in the page table of the first process to establish a virtual dynamic library page table for the dynamic library to be loaded, wherein the virtual dynamic library page table is used for storing the address space layout of the dynamic library in the memory, namely the mapping between the virtual memory and the physical memory page of the dynamic library.
The method can ensure that the address space layout of the dynamic library in the virtual process is completely consistent with that of the first process under the same kernel by completely loading the dynamic library according to the loading sequence identical to that of the first process.
And 2.3, obtaining a subprocess of the first process according to other standard programs called by the fork system, taking the subprocess as a virtual process, and not adding the virtual process to a process scheduling queue of an operating system so as to ensure that the operating system cannot actually schedule and execute the virtual process.
And step 3, establishing mapping from the virtual memory of the process to the physical memory page according to the virtual page table of the first virtual process, realizing that the first process shares the physical memory page with the corresponding established process, and adding the first virtual process into a process scheduling queue of the operating system.
Step 4, when the first process loads the dynamic library, calculating a hash value of the dynamic library and recording the hash value as a second hash value, searching the dynamic library which is the same as the first hash value in the dynamic library loaded by the operating system according to the second hash value, if the dynamic library exists, acquiring a virtual memory of the dynamic library from a virtual dynamic library page table of the first virtual process, loading the dynamic library according to the virtual memory to realize that the first process shares the dynamic library with the corresponding established process, and ending the process; if the virtual memory function address does not exist, loading the dynamic library by adopting the standard flow of the operating system, establishing mapping between the virtual memory and the dynamic library function address for the dynamic library, storing a second hash value of the dynamic library, storing the second hash value and the mapping between the dynamic library function address and the virtual memory into a virtual dynamic library page table of the first virtual process, updating a system virtual page table in the started running environment by using the virtual dynamic library page table, and ending the flow.
The invention is implemented by modifying an executable file loader of an operating system, such as a dynamic linker LD of Linux.
Examples
The cross-container dynamic library sharing method based on the virtual process provided by the invention is adopted in the example, and the dynamic library sharing of the processes among different containers in the Linux system is realized by modifying the dynamic linker LD of Linux, and the method specifically comprises the following steps:
s1, when any container newly builds a process, the process is recorded as a main process, a corresponding virtual process is created for the main process, and the address space layout of the virtual process is consistent with the maintenance of the main process, including the initial address layout of an executable file and a dynamic library. The method specifically comprises the following steps:
s1.1, searching whether the same virtual process exists in the current operating system environment according to the hash value of the current executable file, if so, selecting the corresponding virtual process, constructing a mapping from a virtual memory to a physical memory page by using a virtual page table of the virtual process, wherein in practice, the mirror image of the main process and an actual physical memory page loaded by a dynamic library are stored physical memory pages so as to realize sharing of the physical memory pages, and the stored physical memory pages are created by processes in other containers and added into an operating system process scheduling queue; if not, a process is created according to the standard process creation flow, and a virtual process is created for the process.
S1.2, when the main process loads the dynamic library, calculating a hash value of the dynamic library, searching the same dynamic library in the loaded dynamic library of the operating system kernel according to the hash value, if the same dynamic library exists, using a page table corresponding to the dynamic library, if the same dynamic library does not exist, establishing mapping between a physical memory page and a virtual memory for the dynamic library, and recording the hash value of the dynamic library.
S1.3, acquiring a dynamic library loaded by a main process, a loading sequence of the dynamic library and an address space layout of an executable file, and searching whether the same process exists in the current environment according to a hash value of the executable file in all running environments, if not, creating a corresponding virtual process by calling a newly added system call virtual fork of the embodiment, wherein compared with a standard fork system call, the virtual fork is modified by the following three places, and other processing procedures are the same as those of the fork system call:
a. copying a page table corresponding to a main process by modifying a kernel do_mmap function, but not actually distributing physical memory pages, wherein virtual memory is distributed only in the process, and actual physical memory is not distributed to the process;
b. copying a virtual dynamic library page table for each dynamic library, and virtually loading the dynamic libraries according to the sequence of loading the dynamic libraries by the main process;
c. the created new process is not added to the process scheduling queue of the operating system, i.e., the operating system does not schedule this virtual process to actually execute, which is used only to create the page table.
S2, when a dynamic library is dynamically loaded by calling a dlopen function after a process is started, calculating a hash value of the dynamic library, searching whether the same dynamic library exists in all running environments, if so, using a virtual dynamic library page table of the existing same dynamic library, otherwise, virtually loading the same dynamic library in a corresponding virtual process.
S3, when a newly built running environment is created, copying page tables, namely address space layout, of all processes which are currently active into the newly created running environment.
In summary, the above embodiments are only preferred embodiments of the present invention, and are not intended to limit the scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (5)

1. A cross-container dynamic library sharing method based on a virtual process is characterized by comprising the following steps:
step 1, before a first running environment is started, acquiring a dynamic library loading sequence of a current process, virtual page tables of all virtual processes and virtual dynamic library page tables as system virtual page tables, wherein the system virtual page tables are used for storing address space layout of the processes; starting a first running environment, and copying a system virtual page table into the first running environment;
step 2, modifying an executable file loader of an operating system, when a first running environment creates a first process, taking a virtual process as a first virtual process to acquire a virtual page table and a virtual dynamic library page table corresponding to the virtual process when the same virtual process exists in a system virtual page table, executing step 3, otherwise, executing a standard flow creation process, creating a virtual process corresponding to the standard flow creation process, and updating system virtual page tables in all running environments by adopting the virtual page table and the virtual dynamic library page table of the virtual process, and executing step 4;
step 3, establishing mapping from virtual memory to physical memory pages of the process according to a virtual page table of the first virtual process, and adding the first virtual process into a process scheduling queue of an operating system;
step 4, when the first process loads the dynamic library, if the same dynamic library exists in the loaded dynamic library, loading the dynamic library according to the virtual address of the dynamic library, and ending the process; otherwise, loading the dynamic library to establish mapping between the virtual memory and the dynamic library function address, storing the mapping between the dynamic library function address and the virtual memory into a virtual dynamic library page table of the first virtual process, updating a system virtual page table in the started running environment by using the virtual dynamic library page table, and ending the flow;
the process of creating the virtual process corresponding to the step 2 is as follows:
copying a page table of a first process to serve as a virtual page table of a virtual process and not distributing physical memory pages;
copying the relevant sections of the dynamic library in the page table of the first process into virtual dynamic library page tables of the dynamic library according to the loading sequence of the relevant dynamic library of the first process;
and obtaining a subprocess of the first process according to other standard programs called by the fork system, taking the subprocess as a virtual process, and not adding the virtual process to a process scheduling queue of an operating system.
2. The method for sharing a cross-container dynamic library according to claim 1, wherein the judging manner that the same virtual process exists in the system virtual page table in the step 2 is: and adopting the hash value of the executable file to identify the process and the corresponding virtual process, calculating the hash value of the executable file corresponding to the first process, and searching the virtual process with the same hash value in the system virtual page table.
3. The method for sharing a cross-container dynamic library according to claim 1, wherein the judging manner that the same dynamic library exists in the loaded dynamic library in the step 4 is: and identifying the dynamic library by adopting the hash value of the dynamic library, calculating the hash value of the dynamic library, and searching the same dynamic library as the loaded dynamic library in the operating system according to the hash value.
4. The method for sharing a dynamic library across containers according to claim 1, wherein the loading the dynamic library according to the virtual address of the dynamic library in step 4 is: and obtaining the virtual address of the dynamic library from the virtual dynamic library page table of the first virtual process, and loading the dynamic library according to the virtual address.
5. The method of claim 1, wherein the copying the page table of the first process as the virtual page table of the virtual process and not allocating the physical memory page is: the kernel do_mmap function implementation is modified.
CN202311310584.8A 2023-10-11 2023-10-11 Cross-container dynamic library sharing method based on virtual process Active CN117056031B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311310584.8A CN117056031B (en) 2023-10-11 2023-10-11 Cross-container dynamic library sharing method based on virtual process

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311310584.8A CN117056031B (en) 2023-10-11 2023-10-11 Cross-container dynamic library sharing method based on virtual process

Publications (2)

Publication Number Publication Date
CN117056031A CN117056031A (en) 2023-11-14
CN117056031B true CN117056031B (en) 2023-12-29

Family

ID=88664795

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311310584.8A Active CN117056031B (en) 2023-10-11 2023-10-11 Cross-container dynamic library sharing method based on virtual process

Country Status (1)

Country Link
CN (1) CN117056031B (en)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6349355B1 (en) * 1997-02-06 2002-02-19 Microsoft Corporation Sharing executable modules between user and kernel threads
CN1725216A (en) * 2004-07-22 2006-01-25 国际商业机器公司 Method and apparatus for supporting shared library text replication across a fork system call
CN116126472A (en) * 2023-01-20 2023-05-16 北京麟卓信息科技有限公司 Method for rapidly sharing files in cross-running environment based on dynamic memory virtual disk

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6349355B1 (en) * 1997-02-06 2002-02-19 Microsoft Corporation Sharing executable modules between user and kernel threads
CN1725216A (en) * 2004-07-22 2006-01-25 国际商业机器公司 Method and apparatus for supporting shared library text replication across a fork system call
CN116126472A (en) * 2023-01-20 2023-05-16 北京麟卓信息科技有限公司 Method for rapidly sharing files in cross-running environment based on dynamic memory virtual disk

Also Published As

Publication number Publication date
CN117056031A (en) 2023-11-14

Similar Documents

Publication Publication Date Title
CA2275970C (en) Object and method for providing efficient multi-user access to shared operating system kernal code using instancing
US5990907A (en) Automatic font management within an operating system environment
US8352938B2 (en) System, method and program to migrate a virtual machine
US5539899A (en) System and method for handling a segmented program in a memory for a multitasking data processing system utilizing paged virtual storage
US7543126B2 (en) Apparatus, system, and method for implementing protected virtual memory subcontexts
EP0700000A2 (en) System and method combining a global object identifier with a local object address in a single object pointer
US9063805B2 (en) Method and system for enabling access to functionality provided by resources outside of an operating system environment
Bensoussan et al. The Multics virtual memory
CN106445656B (en) A kind of method and device for realizing thread-local storage
JP5678133B2 (en) Sharing multiple operating system subprocesses across multiple tasks
EP0403124A2 (en) Overlay swapping
US20080301719A1 (en) Mapping Of Dynamic Link Libraries In Computing Devices
CN117056031B (en) Cross-container dynamic library sharing method based on virtual process
US7386702B2 (en) Systems and methods for accessing thread private data
US20060242375A1 (en) Program stack handling
CN106293963B (en) Method and system for communication between application layer and drive layer in windows system
CN102662857B (en) For carrying out virtualized equipment and method for storage
US6772259B2 (en) Interrupt handlers used in different modes of operations
KR20100110710A (en) Method for simplifying interfaces having dynamic libraries
US5450587A (en) Expanded memory addressing scheme
McGee On dynamic program relocation
JP7087150B1 (en) Memory control system
JPH10320203A (en) Shared memory system
KR100884926B1 (en) Method for utilizing legacy shared libraries on VM-less embedded systems
CN113569231B (en) Multiprocess MPU protection method and device and electronic equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant