CN116069683A - Media file memory management method, device, computer equipment and readable medium - Google Patents

Media file memory management method, device, computer equipment and readable medium Download PDF

Info

Publication number
CN116069683A
CN116069683A CN202111300555.4A CN202111300555A CN116069683A CN 116069683 A CN116069683 A CN 116069683A CN 202111300555 A CN202111300555 A CN 202111300555A CN 116069683 A CN116069683 A CN 116069683A
Authority
CN
China
Prior art keywords
memory
memory page
pointer
media file
media
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202111300555.4A
Other languages
Chinese (zh)
Inventor
骆俊
王日红
张晓渠
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
ZTE Corp
Original Assignee
ZTE Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by ZTE Corp filed Critical ZTE Corp
Priority to CN202111300555.4A priority Critical patent/CN116069683A/en
Priority to PCT/CN2022/128907 priority patent/WO2023078228A1/en
Publication of CN116069683A publication Critical patent/CN116069683A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02—Addressing or allocation; Relocation
    • G06F12/06—Addressing a physical block of locations, e.g. base addressing, module addressing, memory dedication
    • G06F12/0646—Configuration or reconfiguration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02—Addressing or allocation; Relocation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02—Addressing or allocation; Relocation
    • G06F12/0223—User address space allocation, e.g. contiguous or non contiguous base addressing
    • G06F12/023—Free address space management
    • G06F12/0253—Garbage collection, i.e. reclamation of unreferenced memory
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02—Addressing or allocation; Relocation
    • G06F12/06—Addressing a physical block of locations, e.g. base addressing, module addressing, memory dedication
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5011—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals
    • G06F9/5016—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals the resource being the memory
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The present disclosure provides a method for managing a memory of a media file, the method comprising: responding to the corresponding media file obtained according to the media file playing request, and generating an annular memory page structure for the media file; the memory page structure comprises a pushing pointer, a recycling pointer and a plurality of memory pages, wherein the pushing pointer and the recycling pointer point to different memory pages, the memory pages are loaded with media data of media files, and each memory page forms a ring shape and can flow according to a preset flowing direction; pushing the media data mounted on the memory page pointed by the pushing pointer in the process of flowing the memory page, and replacing the mounted media data with the media data which is not mounted in the media file in the memory page pointed by the recycling pointer until the last media data in the media file is mounted in the memory structure; the problem of insufficient performance of the CDN server can be improved and solved. The disclosure also provides a media file memory management apparatus, a computer device and a readable medium.

Description

Media file memory management method, device, computer equipment and readable medium
Technical Field
The present disclosure relates to the field of computer technologies, and in particular, to a method and apparatus for managing a memory of a media file, a computer device, and a readable medium.
Background
With the rapid development of the internet, people spend more and more time in video, and the rising of short video changes people's life and entertainment modes. As the bandwidth of media services increases, in order to avoid the phenomenon of jamming when viewers watch videos, more and more video service manufacturers open CDN (Content Delivery Network ) acceleration services.
The CDN can hierarchically send the video resources from the source station to the nearest service nodes around the client, so that the physical service distance is shortened, and the client can access the video resources faster. When a conventional CDN server supports a large-traffic media service, its memory management adopts an LRU (Least recently used ) structure, and its memory management structure appears as a linked list. After the CDN server acquires the media data, the media data is read into the LRU linked list, and for the media data which is not accessed for a long time, the CDN server recovers the memory from the tail end of the LRU linked list, so that the memory management is realized.
For a CDN server carrying a large-flow media service, the media file is generally larger, about the number G, so that the LRU linked list is overlong, the time for the CDN server to traverse the LRU linked list is longer, the problem of insufficient performance of the CDN server is caused, and the problems of CPU (Central processing Unit) high flushing and excessive memory utilization rate caused by untimely memory recovery are solved.
Disclosure of Invention
The present disclosure provides a media file memory management method, apparatus, computer device and readable medium.
In a first aspect, an embodiment of the present disclosure provides a method for managing a memory of a media file, where the method includes:
responding to a corresponding media file obtained according to a media file playing request, and generating an annular memory page structure for the media file; the memory page structure comprises a pushing pointer, a recycling pointer and a plurality of memory pages, wherein the pushing pointer and the recycling pointer point to different memory pages, the memory pages are mounted with media data of the media files, and each memory page forms a ring shape and can flow according to a preset flowing direction;
and in the flowing process of the memory page, pushing the media data mounted on the memory page pointed by the pushing pointer, and replacing the mounted media data with the media data which is not mounted in the media file in the memory page pointed by the recycling pointer until the last media data in the media file is mounted in the memory structure.
In some embodiments, the method further comprises:
and setting the position of the recovery pointer according to the size of the idle memory of the operating system in the process of generating the annular memory page structure for the media file.
In some embodiments, the memory page structure includes a first region in which the flow direction is a direction from the reclamation pointer to the push pointer and a second region in which the flow direction is a direction from the push pointer to the reclamation pointer;
setting the position of the recovery pointer according to the size of the idle memory of the operating system, including:
setting the reclamation pointer in the first area in response to the idle memory of the operating system being greater than a preset first threshold;
and setting the reclamation pointer in the second area in response to the idle memory of the operating system being smaller than or equal to a preset first threshold value.
In some embodiments, before generating the circular memory page structure for the media file, the method further comprises: aiming at the media file, applying a first memory page to an operating system, and establishing a memory pool according to the first memory page;
The generating a ring-shaped memory page structure for the media file includes: generating a ring-shaped memory page structure for the media file at least according to the first memory page in the memory pool.
In some embodiments, the generating a ring-shaped memory page structure for the media file at least according to the first memory page in the memory pool includes:
responding to the shortage of memory resources in the memory pool, and applying for a second memory page to the operating system;
and generating a ring-shaped memory page structure for the media file according to the first memory page and the second memory page in the memory pool.
In some embodiments, after establishing the memory pool from the first memory page, the method further comprises: and not recording the occupation of the first memory page.
In some embodiments, after establishing the memory pool from the first memory page, the method further comprises:
and responding to the satisfaction of a preset memory page recovery condition, and releasing the first memory page in the memory pool to the operating system.
In some embodiments, the memory page reclamation condition includes at least one of: a first recovery condition of a memory page corresponding to an xfs file system, a second recovery condition of the memory page corresponding to an ext file system, and a third recovery condition of the memory page corresponding to a fuse file system;
The preset memory page recovery condition is satisfied, including: at least one of the first recovery condition of the memory page, the second recovery condition of the memory page and the third recovery condition of the memory page is satisfied.
In some embodiments, the method further comprises:
and releasing the memory pages in the memory page structure into the memory pool in response to completion of pushing all the media data in the media file.
In yet another aspect, an embodiment of the present disclosure further provides a media file memory management device, including a memory page structure generating module, a media data pushing module, and a memory page recycling module, where the memory page structure generating module is configured to generate an annular memory page structure for a media file in response to acquiring a corresponding media file according to a media file playing request; the memory page structure comprises a pushing pointer, a recycling pointer and a plurality of memory pages, wherein the pushing pointer and the recycling pointer point to different memory pages, the memory pages are mounted with media data of the media files, and each memory page forms a ring shape and can flow according to a preset flowing direction;
the media data pushing module is used for pushing media data mounted on the memory page pointed by the pushing pointer in the memory page flowing process;
And the memory page recycling module is used for replacing the mounted media data with the media data which is not mounted in the media file in the memory page pointed by the recycling pointer in the process of flowing the memory page until the last media data in the media file is mounted in the memory structure.
In yet another aspect, the disclosed embodiments also provide a computer device, comprising: one or more processors; a storage device having one or more programs stored thereon; the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the media file memory management method as described above.
In yet another aspect, embodiments of the present disclosure further provide a computer readable medium having a computer program stored thereon, wherein the program when executed implements a media file memory management method as described above.
The embodiment of the disclosure provides a media file memory management method, which comprises the following steps: responding to the corresponding media file obtained according to the media file playing request, and generating an annular memory page structure for the media file; the memory page structure comprises a pushing pointer, a recycling pointer and a plurality of memory pages, wherein the pushing pointer and the recycling pointer point to different memory pages, the memory pages are loaded with media data of media files, and each memory page forms a ring shape and can flow according to a preset flowing direction; pushing the media data mounted on the memory page pointed by the pushing pointer in the process of flowing the memory page, and replacing the mounted media data with the media data which is not mounted in the media file in the memory page pointed by the recycling pointer until the last media data in the media file is mounted in the memory structure; according to the embodiment of the disclosure, the memory use efficiency of the CDN server in a high-bandwidth high-flow scene can be improved, the consumption of an operating system and the time delay caused by memory recovery are greatly reduced, and the media file playing service with the size of a plurality of G can be provided by using the memory with the size of 2-5M, so that the problem of insufficient performance of the CDN server facing the high-flow media service is solved.
Drawings
FIG. 1 is a schematic diagram of an operating system architecture according to an embodiment of the present disclosure;
FIG. 2 is a flowchart illustrating a method for managing a memory of a media file according to an embodiment of the present disclosure;
FIG. 3 is a schematic diagram of a ring-shaped memory page according to an embodiment of the disclosure;
FIG. 4 is a second flowchart illustrating a method for managing a memory of a media file according to an embodiment of the disclosure;
FIG. 5 is a flowchart illustrating a method for generating a ring-shaped memory page structure according to an embodiment of the present disclosure;
FIG. 6 is a flowchart illustrating a third embodiment of a method for managing a memory of a media file according to the present disclosure;
fig. 7 is a schematic structural diagram of a media file memory management device according to an embodiment of the disclosure;
FIG. 8 is a second schematic structural diagram of a memory management device for media files according to an embodiment of the present disclosure;
fig. 9 is a schematic diagram of a third structure of a memory management device for media files according to an embodiment of the disclosure.
Detailed Description
Example embodiments will be described more fully hereinafter with reference to the accompanying drawings, but may be embodied in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. As used herein, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
Embodiments described herein may be described with reference to plan and/or cross-sectional views with the aid of idealized schematic diagrams of the present disclosure. Accordingly, the example illustrations may be modified in accordance with manufacturing techniques and/or tolerances. Thus, the embodiments are not limited to the embodiments shown in the drawings, but include modifications of the configuration formed based on the manufacturing process. Thus, the regions illustrated in the figures have schematic properties and the shapes of the regions illustrated in the figures illustrate the particular shapes of the regions of the elements, but are not intended to be limiting.
Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and the present disclosure, and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.
The embodiment of the disclosure provides a media file memory management method, which is applied to a media file memory management device, wherein the media file memory management device can be a module in a CDN server, the CDN server runs a Linux/Unix operating system, and after the media file memory management device is loaded into the operating system, memory management service can be provided for media service. Embodiments of the present disclosure support xfs (log file system), ext (Extended file system ) and fuse (Filesystem in Userspace, user space file system), the services of the application layer have no impact on the content of the present invention.
Fig. 1 is a schematic diagram of an operating system architecture according to an embodiment of the present disclosure, as shown in fig. 1, where a media file memory management device belongs to a Linux/Unix kernel module in an operating system, and runs in a Linux/Unix kernel space, and is responsible for assisting the operating system to manage memory pages, so as to improve system performance. The media file memory management device operates above an xfs file system and below a virtual file system vfs, and realizes the media file memory management function by replacing part of functions in the sendfile (zero copy) flow of xfs.
As shown in fig. 2, the method for managing the memory of the media file provided in the embodiment of the disclosure includes the following steps:
step 21, responding to the corresponding media file obtained according to the media file playing request, and generating an annular memory page structure for the media file; the memory page structure comprises a pushing pointer, a recycling pointer and a plurality of memory pages, wherein the pushing pointer and the recycling pointer point to different memory pages, the memory pages are loaded with media data of media files, and each memory page forms a ring shape and can flow according to a preset flowing direction.
In this step, the client sends a media file playing request to the CDN server, and the CDN server obtains a corresponding media file according to the media file playing request. After the media file is acquired, the media file memory management apparatus generates a ring-shaped memory page structure for the media file.
Fig. 3 is a schematic diagram of a ring-shaped memory page structure provided in an embodiment of the disclosure, where, as shown in fig. 3, the ring-shaped memory page structure includes a plurality of memory pages, different memory pages mount different media data in a media file, and each memory page is connected end to form a ring. The memory page structure also includes a push pointer and a recycle pointer that point to different memory pages. The media data mounted on the memory page pointed by the pushing pointer is media data pushed to the client, and is media data being watched by the user. The memory page pointed by the recovery pointer is the memory page which is recovered and reused at present, and the recovery and reuse of the memory page can be realized by replacing the media data mounted on the memory page. The memory pages in the memory page structure can flow circularly to form a data stream, the flow direction (shown by an arrow in fig. 3) can be preset, and once the annular memory page structure is generated, the flow direction of the memory pages is fixed.
Step 22, in the process of flowing the memory page, pushing the media data mounted on the memory page pointed by the pushing pointer, and replacing the mounted media data with the media data which is not mounted in the media file in the memory page pointed by the recycling pointer until the last media data in the media file is mounted in the memory structure.
In the step, in the process of flowing a memory page in a memory page structure, on one hand, media data mounted on a corresponding memory page is pushed to a client side initiating a media file playing request at the position of a pushing pointer, so that a pushing service is realized, and a user can watch the media data; since the user views the media file from front to back, the disclosed embodiments treat the playing of one media file as one data stream flowing from front to back. The push service sequentially transmits each media data of the media file to the user in the order from front to back. On the other hand, at the position of the recycling pointer, recycling and reutilizing the corresponding memory page, namely deleting the media data mounted in the memory page (the deleted media data is pushed media data), and mounting the media data which is not mounted on the memory page.
After pushing media data mounted on a memory page pointed by a pushing pointer, the pushed memory page flows according to the flowing direction of the memory page, when the pushed memory page flows to the position of a recycling pointer, the media data which is not mounted in the media file and is pushed in the pushed memory page is replaced by the media data which is not mounted in the media file, so that recycling and reutilization of the memory page are realized, the memory page on which new media data are mounted again continues to flow according to the flowing direction of the memory page, and when the memory page flows to the position of the pushing pointer, the media data mounted on the memory page are pushed, and the cycle is repeated, so that annular data flow is formed.
The embodiment of the disclosure provides a media file memory management method, which comprises the following steps: responding to the corresponding media file obtained according to the media file playing request, and generating an annular memory page structure for the media file; the memory page structure comprises a pushing pointer, a recycling pointer and a plurality of memory pages, wherein the pushing pointer and the recycling pointer point to different memory pages, the memory pages are loaded with media data of media files, and each memory page forms a ring shape and can flow according to a preset flowing direction; pushing the media data mounted on the memory page pointed by the pushing pointer in the process of flowing the memory page, and replacing the mounted media data with the media data which is not mounted in the media file in the memory page pointed by the recycling pointer until the last media data in the media file is mounted in the memory structure; according to the embodiment of the disclosure, the memory use efficiency of the CDN server in a high-bandwidth high-flow scene can be improved, the consumption of an operating system and the time delay caused by memory recovery are greatly reduced, and the media file playing service with the size of a plurality of G can be provided by using the memory with the size of 2-5M, so that the problem of insufficient performance of the CDN server facing the high-flow media service is solved.
The media file service has a characteristic that a user views the media file from front to back, wherein the media data which is already seen cannot be watched again with high probability, so that memory pages for mounting the pushed media data are not required to enter a memory management device of an operating system, and the memory pages can be directly used for mounting the media data which the user views, so that a complicated memory management flow in the operating system can be bypassed, the time consumption of the media service in acquiring the memory pages can be greatly reduced, and the performance of the media file service is improved. Based on the above characteristics of the media file service, the embodiments of the present disclosure provide an annular memory page structure, where the memory page structure can carry a push service of a media file, and after the memory page of the pushed media data reaches the position of the recovery pointer, media data that has not been pushed can be mounted again, and by flowing the media data stream and the memory page, playing of a larger media file can be achieved by using a smaller memory.
For high-volume media file services, in the related art, the LRU memory structure has a relatively long linked list, and the operating system consumes relatively long time when traversing the linked list, resulting in a CPU flush and excessive memory usage. In the embodiment of the disclosure, a memory page structure is generated for a media file to mount a media data stream, and compared with the LRU linked list, the memory page required for the ring-shaped media data stream is smaller, which is only about 2-5M. According to the embodiment of the disclosure, the characteristic that the media data watched by the user cannot be watched repeatedly again is considered, the annular memory page structure is designed so as to form annular media data streams, and the memory page on which the media data pushed by the user is mounted can be quickly mounted with the media data to be pushed, so that the media files with the size of G can be played only by the memory size of about 2-5M, and the utilization rate of the memory and the consumption of an operating system and a CPU (Central processing unit) are greatly reduced.
In some embodiments, the method for managing the memory of the media file further comprises the following steps: in the process of generating the annular memory page structure for the media file, the position of the recovery pointer is set according to the size of the idle memory of the operating system. The position of the pushing pointer is the position of the memory page to which the media data being pushed belongs (i.e. the position of the memory page to which the media data being watched by the user belongs), and the position of the recycling pointer is the position of the memory page to which recycling begins.
In some embodiments, as shown in fig. 3, the memory page structure includes a first area a in which a flow direction is from the recycle pointer to the push pointer, and a second area B in which a flow direction is from the push pointer to the recycle pointer. The media data mounted on the memory page in the first area A is media data to be pushed, the media data mounted on the memory page in the second area B is media data completed by pushing, and the position of the recovery pointer can be adjusted relative to the position of the pushing pointer.
Correspondingly, the setting the position of the recovery pointer according to the size of the idle memory of the operating system comprises the following steps: setting a recycle pointer in a first area in response to the idle memory of the operating system being greater than a preset first threshold; and setting the reclamation pointer in the second area in response to the idle memory of the operating system being less than or equal to a preset first threshold. That is, under the condition that the free memory of the operating system is larger, the recovery pointer is relatively far away from the pushing pointer along the flowing direction, so that more memory pages for mounting the pushed media data between the recovery pointer and the pushing pointer (namely, the second area B is larger than the first area A), and the memory pages can be mounted with new media data after a longer time, thereby reducing the recovery and reuse times of the memory pages on the whole and improving the performance of the CDN server. Under the condition that the free memory of the operating system is smaller, the recovery pointer is relatively close to the pushing pointer along the flowing direction, so that memory pages for mounting pushed media data between the recovery pointer and the pushing pointer are fewer (namely, the second area B is smaller than the first area A), new media data can be mounted on the memory pages in a short time, the number of times of recovery and reuse of the memory pages is increased as a whole, and correspondingly, compared with the condition that the free memory of the operating system is larger, the performance of the CDN server is reduced.
In the related art, an operating system performs memory management based on a memory page, and does not particularly distinguish between the type and the use of data mounted on the memory page, because the memory management of the operating system is not specific to a user or a specific service, the memory management of the operating system must have universality to adapt to various complex memory management scenarios. However, this general memory management manner may be insufficient when processing the media service of the large-flow large-volume file, because the memory pages on which the media file is mounted are distributed on the LRU linked list, which may result in the LRU linked list being long, and the operating system needs to traverse the LRU linked list when retrieving the memory pages, which may result in more time consumption of the CPU, thereby resulting in the problem of insufficient performance of the CDN server.
To solve the above-mentioned problems, embodiments of the present disclosure provide a separate memory management service for a mass media service by establishing a memory pool.
Thus, in some embodiments, as shown in fig. 4, before generating the ring-shaped memory page structure for the media file (i.e., step 21), the media file memory management method may further include the steps of:
step 20, applying for the first memory page to the operating system for the media file, and establishing a memory pool according to the first memory page.
It should be noted that this step may be performed in the initialization stage, i.e. the memory pool of the media file is pre-established. In the initialization phase, the following initialization operations may also be performed: determining the size of a reserved memory according to the current performance of a CDN server, replacing partial functions such as sendfile and the like, generating a debugging interface file under a proc catalog of an operating system, initializing stream management, initializing a memory pool, creating a memory management and stream management kernel thread and the like.
After the initialization of the memory pool is completed, a memory (i.e. a first memory page) with a certain size is applied to the operating system, the memory pool is formally formed after the memory application is successful, and the memory management of the media file is started.
Accordingly, the generating a ring-shaped memory page structure for the media file (i.e. step 21) includes: generating a ring-shaped memory page structure for the media file at least according to the first memory page in the memory pool. That is, the memory pages in the memory page structure are preferentially retrieved from the memory pool of the corresponding media file.
According to the embodiment of the disclosure, a memory pool technology is adopted, a certain number of first memory pages are applied to an operating system in advance to form a memory pool, and independent memories can be obtained from the operating system of the CDN server to provide independent services for the package of the media files. The method comprises the steps of establishing a memory pool for each media file, and preferentially acquiring memory pages required by generating an annular memory page structure from the memory pool, so that the memory pages of the media file can be concentrated together for unified management instead of being distributed for management by an operating system, the memory pages can be rapidly operated without being distributed into the operating system and processed by a memory management device of the operating system, a complex memory management flow is avoided, the memory pages are not required to be frequently applied from the operating system, the time consumption of memory management can be reduced, and the performance of memory service of the media file is improved.
In some embodiments, as shown in fig. 5, the generating a ring-shaped memory page structure for the media file at least according to the first memory page in the memory pool includes the following steps:
step 211, in response to the insufficient memory resources in the memory pool, applying for a second memory page to the operating system.
Step 212, generating a ring-shaped memory page structure for the media file according to the first memory page and the second memory page in the memory pool.
In the embodiment of the disclosure, the amount of memory allocated to the media file is not fixed, and in the process of performing memory management, if the memory resources in the memory pool are insufficient, the memory can be acquired again through system allocation.
In some embodiments, after the memory pool is established according to the first memory page, the method for managing the memory of the media file may further include the steps of: occupancy of the first memory page is not recorded.
The occupation of the first memory page means that, once the first memory page is successfully applied to the operating system, the first memory page is occupied to establish a memory pool, and the first memory page allocated by the operating system for the memory service of the media file is then occupied.
In the related art, the operating system records occupied memory pages, i.e. adds 1 to the count of occupied memory pages, and the memory reclamation program kwsap of the operating system only reclaims memory pages with the count of 0.
In the embodiment of the disclosure, even if the first memory page is successfully applied to the operating system, and the memory pool is built according to the first memory page, occupation of the first memory page is truly realized, the count of the occupied first memory page is not increased, i.e. the count of the first memory page is not added with 1, so that the first memory page is completely transparent to the operating system, the operating system can not sense occupation of the first memory page, normal recovery of memory through kswap by the operating system can not be influenced, and performance of the operating system can not be influenced. That is, when the operating system is deficient in memory resources and triggers kswap to recover the memory resources, the operating system can recover the first memory page in the memory pool for other programs needing the memory resources, so that the performance stability of the CDN server under the complex scene of multi-service combined installation is improved. According to the embodiment of the disclosure, the memory size of the managed media file can be dynamically adjusted, the memory management elasticity of the media file is enhanced, and the problem of overhigh memory utilization rate of the CDN server is solved without causing great influence on an operating system.
In some embodiments, as shown in fig. 6, after the memory pool is established according to the first memory page (i.e., step 20), the media file memory management method may further include the steps of:
And step 21', releasing the first memory page in the memory pool to the operating system in response to the preset memory page recovery condition being met.
The embodiment of the disclosure can actively release the memory resources to the operating system, and avoid the excessive memory resources from being occupied to cause the excessive memory utilization rate of the CDN server.
The embodiment of the disclosure can simultaneously support xfs, ext, fuse file systems and flexibly set memory page recycling conditions for different types of file systems. The ext file system and the fuse file system are systems for managing files in an operating system, wherein the fuse file system is a file system which is completely realized in a user mode, the spending of a server for switching the user mode and the system mode back and forth is avoided, and the ext file system is a Linux/Unix standard expansion file system.
In some embodiments, the memory page reclamation conditions include at least one of: the method comprises the steps of a first recovery condition of a memory page corresponding to an xfs file system, a second recovery condition of the memory page corresponding to an ext file system and a third recovery condition of the memory page corresponding to a fuse file system. Correspondingly, the preset memory page recovery condition is satisfied and includes: at least one of the first recovery condition of the memory page, the second recovery condition of the memory page and the third recovery condition of the memory page is satisfied.
Under complex scenes such as multi-service combined setting, the embodiment of the disclosure can actively release the memory to the operating system according to the preset memory page recovery condition, so that the memory utilization rate of the CDN server is reduced, and the overall performance of the CDN server is improved.
In some embodiments, the first reclamation condition for the memory page corresponding to the xfs file system may include: the residual memory of the operating system is larger than a preset minimum memory and smaller than a self-defined memory waterline value; the second recycling condition of the memory page corresponding to the ext file system may include: the periodically set memory recovery time arrives, and for an ext file system, K can recover memory pages at 4 am every day; the third recycling condition of the memory page corresponding to the fuse file system may include: media data is not viewed for more than a preset period of time, illustratively, for a fuse file system, one hour of unviewed memory pages are reclaimed.
Aiming at the problems of CPU flushing and excessive memory utilization rate of a CDN server caused by untimely memory recycling in the related technology, the embodiment of the disclosure sets a memory page structure recycling mechanism, namely, all memory pages in a memory page structure are recycled. Thus, in some embodiments, the method for managing memory of media files may further comprise the steps of: and releasing the memory pages in the memory page structure into the memory pool in response to completion of pushing all the media data in the media file. In this step, the memory page structure may be scanned periodically, and it is determined whether all media data in the corresponding media file is pushed, and if all pushing is completed, each memory page in the memory page structure may be recovered to the memory pool.
It should be noted that, the CDN server may provide a plurality of media files, and generate an annular memory page structure for each media file.
The CDN server in the embodiments of the present disclosure may be applicable to RTSP (Real Time Streaming Protocol, real-time streaming protocol) on-demand service scenarios, HLS (HTTP Live Streaming, HTTP-based adaptive bitrate streaming protocol) on-demand service scenarios, DASH (Dynamic Adaptive Streaming over HTTP, HTTP-based dynamic adaptive streaming) on-demand service scenarios, HPD (HTTP Progressive Download, progressive download) on-demand service scenarios. The RTSP on-demand service, HLS on-demand service, DASH on-demand service, and HPD on-demand service are different only in the protocols used by the application layer and the formats of the media files, and the bottom layers are all services related to the package of the media files, so that the method for managing the memory of the media files provided by the embodiments of the present disclosure can be applied. The application scenario of the embodiments of the present disclosure is not limited to the above scenario, and the application scenario involving the CDN server media parcel mechanism may be applicable.
Aiming at the problem of insufficient performance of a CDN server under the condition of large flow, the embodiment of the disclosure provides a scheme for performing memory management on media files, and the independent memory management is performed on the media files by adopting the ideas of a memory pool and data flow.
Independent memory can be obtained from the operating system of the CDN server through the memory pool to provide independent service for the package of the media files. The memory page used for mounting the data stream can be obtained from the memory pool, if the memory resources of the memory pool are insufficient, the memory page can be applied to the operating system, in addition, the memory resources in the memory pool can be released to the operating system, and the excessive memory resources are prevented from being occupied, so that the memory utilization rate of the CDN server is prevented from being too high.
Through the data stream form, a ring-shaped memory page structure can be provided for each media file, and when a user plays the media file, the memory page of the watched media data can be directly used as the memory page for mounting the media data to be played, so that a ring-shaped flowable media data stream is formed. Therefore, the files with the playing number G only occupy 2-5M of extremely small memory, the consumption of the memory management of an operating system can be reduced under a high-flow scene, the utilization rate of a CPU (Central processing Unit) is reduced, and the overall performance of the CDN server is improved.
According to the media file memory management method, the memory of the media file is independently maintained, and the performance of the CDN server is improved by adopting a memory pool, an annular memory page structure and a data flow mechanism. After a media file playing request of a user is obtained, an annular memory page structure is generated for the requested media file, media data is mounted on memory pages in the memory page structure, and a data stream is formed through the circulation flow of the memory pages. The memory pages in the memory pool can be acquired from the operating system, and can be actively released and recovered to the operating system. When the memory pages applied to the operating system enter the memory pool queue, the count is not increased any more, so that the kswap of the operating system can recover the memory pages normally, that is, the memory management device for the media file in the embodiment of the disclosure does not reserve the memory any more, and if the operating system needs to recover the memory, the memory can be recovered normally through the kwap.
The implementation basis of the embodiment of the disclosure is to replace part of functions in the sendfile flow of the xfs file system, and the operation set and functions of the xfs file system can be replaced in the initialization stage, so that all memory pages of the service application by calling sendfile are obtained from a memory pool, and the management function of the memory pages of the media file is realized. In addition, the embodiment of the disclosure also expands the support of ext and fuse file systems, and can actively release the memory to the operating system.
Based on the same technical concept, the embodiment of the present disclosure further provides a media file memory management device, as shown in fig. 7, where the media file memory management device includes a memory page structure generating module 101, a media data pushing module 102, and a memory page recycling module 103, where the memory page structure generating module 101 is configured to generate an annular memory page structure for a media file in response to acquiring a corresponding media file according to a media file playing request; the memory page structure comprises a pushing pointer, a recycling pointer and a plurality of memory pages, wherein the pushing pointer and the recycling pointer point to different memory pages, the memory pages are mounted with media data of the media files, and each memory page forms a ring shape and can flow according to a preset flowing direction.
The media data pushing module 102 is configured to push, during the memory page flowing process, media data mounted on a memory page pointed by the pushing pointer.
The memory page reclamation module 103 is configured to replace, in the memory page pointed by the reclamation pointer, media data that is mounted in the media file with media data that is not mounted in the media file until the last media data in the media file is mounted in the memory structure in the memory page flowing process.
In some embodiments, the memory page structure generating module 101 is further configured to set the location of the reclamation pointer according to the size of the free memory of the operating system in the process of generating the ring-shaped memory page structure for the media file.
In some embodiments, the memory page structure includes a first region in which the flow direction is from the reclamation pointer to the push pointer and a second region in which the flow direction is from the push pointer to the reclamation pointer.
The memory page structure generation module 101 is further configured to set the reclamation pointer in the first area in response to the idle memory of the operating system being greater than a preset first threshold; and setting the reclamation pointer in the second area in response to the idle memory of the operating system being smaller than or equal to a preset first threshold value.
In some embodiments, as shown in fig. 8, the memory management device for a media file further includes a memory pool management module 104, where the memory pool management module 104 is configured to apply a first memory page to an operating system for the media file before the memory page structure generating module 101 generates a ring-shaped memory page structure for the media file, and establish a memory pool according to the first memory page.
The memory page structure generating module 101 is configured to generate a ring-shaped memory page structure for the media file according to at least the first memory page in the memory pool.
In some embodiments, the memory page structure generating module 101 is configured to apply for a second memory page to the operating system in response to the memory resource in the memory pool being insufficient; and generating a ring-shaped memory page structure for the media file according to the first memory page and the second memory page in the memory pool.
In some embodiments, the memory pool management module 104 is further configured to, after the memory pool is established according to the first memory page, not record occupancy of the first memory page.
In some embodiments, the memory pool management module 104 is further configured to, after the memory pool is established according to the first memory page, release the first memory page in the memory pool to the operating system in response to a preset memory page reclamation condition being met.
In some embodiments, the memory page reclamation condition includes at least one of: a first recovery condition of a memory page corresponding to an xfs file system, a second recovery condition of the memory page corresponding to an ext file system, and a third recovery condition of the memory page corresponding to a fuse file system;
The preset memory page recovery condition is satisfied, including: at least one of the first recovery condition of the memory page, the second recovery condition of the memory page and the third recovery condition of the memory page is satisfied.
In some embodiments, as shown in fig. 9, the media file memory management apparatus further includes a stream management module 105, where the stream management module 105 is configured to release the memory pages in the memory page structure into the memory pool in response to completion of pushing all media data in the media file.
The disclosed embodiments also provide a computer device comprising: one or more processors and a storage device; the storage device stores one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors implement the media file memory management method provided in the foregoing embodiments.
The disclosed embodiments also provide a computer readable medium having a computer program stored thereon, wherein the computer program when executed implements the media file memory management method as provided by the foregoing embodiments.
Those of ordinary skill in the art will appreciate that all or some of the steps of the methods, functional modules/units in the apparatus disclosed above may be implemented as software, firmware, hardware, and suitable combinations thereof. In a hardware implementation, the division between the functional modules/units mentioned in the above description does not necessarily correspond to the division of physical components; for example, one physical component may have multiple functions, or one function or step may be performed cooperatively by several physical components. Some or all of the physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on computer readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media). The term computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data, as known to those skilled in the art. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital Versatile Disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by a computer. Furthermore, as is well known to those of ordinary skill in the art, communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media.
Example embodiments have been disclosed herein, and although specific terms are employed, they are used and should be interpreted in a generic and descriptive sense only and not for purpose of limitation. In some instances, it will be apparent to one skilled in the art that features, characteristics, and/or elements described in connection with a particular embodiment may be used alone or in combination with other embodiments unless explicitly stated otherwise. It will therefore be understood by those of ordinary skill in the art that various changes in form and details may be made therein without departing from the scope of the present invention as set forth in the following claims.

Claims (12)

1. A method for managing memory of a media file, the method comprising:
responding to a corresponding media file obtained according to a media file playing request, and generating an annular memory page structure for the media file; the memory page structure comprises a pushing pointer, a recycling pointer and a plurality of memory pages, wherein the pushing pointer and the recycling pointer point to different memory pages, the memory pages are mounted with media data of the media files, and each memory page forms a ring shape and can flow according to a preset flowing direction;
And in the flowing process of the memory page, pushing the media data mounted on the memory page pointed by the pushing pointer, and replacing the mounted media data with the media data which is not mounted in the media file in the memory page pointed by the recycling pointer until the last media data in the media file is mounted in the memory structure.
2. The method of claim 1, wherein the method further comprises:
and setting the position of the recovery pointer according to the size of the idle memory of the operating system in the process of generating the annular memory page structure for the media file.
3. The method of claim 2, wherein the memory page structure includes a first region in which the flow direction is from the recycle pointer to the push pointer and a second region in which the flow direction is from the push pointer to the recycle pointer;
setting the position of the recovery pointer according to the size of the idle memory of the operating system, including:
setting the reclamation pointer in the first area in response to the idle memory of the operating system being greater than a preset first threshold;
And setting the reclamation pointer in the second area in response to the idle memory of the operating system being smaller than or equal to a preset first threshold value.
4. The method of claim 1, wherein prior to generating a circular memory page structure for the media file, the method further comprises: aiming at the media file, applying a first memory page to an operating system, and establishing a memory pool according to the first memory page;
the generating a ring-shaped memory page structure for the media file includes: generating a ring-shaped memory page structure for the media file at least according to the first memory page in the memory pool.
5. The method of claim 4, wherein generating a circular memory page structure for the media file based at least on the first memory page in the memory pool comprises:
responding to the shortage of memory resources in the memory pool, and applying for a second memory page to the operating system;
and generating a ring-shaped memory page structure for the media file according to the first memory page and the second memory page in the memory pool.
6. The method of claim 4, wherein after establishing a memory pool from the first memory page, the method further comprises: and not recording the occupation of the first memory page.
7. The method of claim 4, wherein after establishing a memory pool from the first memory page, the method further comprises:
and responding to the satisfaction of a preset memory page recovery condition, and releasing the first memory page in the memory pool to the operating system.
8. The method of claim 7, wherein the memory page reclamation conditions include at least one of: a first recovery condition of a memory page corresponding to an xfs file system, a second recovery condition of the memory page corresponding to an ext file system, and a third recovery condition of the memory page corresponding to a fuse file system;
the preset memory page recovery condition is satisfied, including: at least one of the first recovery condition of the memory page, the second recovery condition of the memory page and the third recovery condition of the memory page is satisfied.
9. The method of any one of claims 4-8, wherein the method further comprises:
and releasing the memory pages in the memory page structure into the memory pool in response to completion of pushing all the media data in the media file.
10. The memory management device for the media files is characterized by comprising a memory page structure generation module, a media data pushing module and a memory page recovery module, wherein the memory page structure generation module is used for responding to the acquisition of the corresponding media files according to the media file playing request to generate an annular memory page structure for the media files; the memory page structure comprises a pushing pointer, a recycling pointer and a plurality of memory pages, wherein the pushing pointer and the recycling pointer point to different memory pages, the memory pages are mounted with media data of the media files, and each memory page forms a ring shape and can flow according to a preset flowing direction;
The media data pushing module is used for pushing media data mounted on the memory page pointed by the pushing pointer in the memory page flowing process;
and the memory page recycling module is used for replacing the mounted media data with the media data which is not mounted in the media file in the memory page pointed by the recycling pointer in the process of flowing the memory page until the last media data in the media file is mounted in the memory structure.
11. A computer device, comprising:
one or more processors;
a storage device having one or more programs stored thereon;
the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the media file memory management method of any of claims 1-9.
12. A computer readable medium having stored thereon a computer program, wherein the program when executed implements the media file memory management method of any of claims 1-9.
CN202111300555.4A 2021-11-04 2021-11-04 Media file memory management method, device, computer equipment and readable medium Pending CN116069683A (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN202111300555.4A CN116069683A (en) 2021-11-04 2021-11-04 Media file memory management method, device, computer equipment and readable medium
PCT/CN2022/128907 WO2023078228A1 (en) 2021-11-04 2022-11-01 Memory management method and apparatus for media file, and computer device and readable medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111300555.4A CN116069683A (en) 2021-11-04 2021-11-04 Media file memory management method, device, computer equipment and readable medium

Publications (1)

Publication Number Publication Date
CN116069683A true CN116069683A (en) 2023-05-05

Family

ID=86177538

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111300555.4A Pending CN116069683A (en) 2021-11-04 2021-11-04 Media file memory management method, device, computer equipment and readable medium

Country Status (2)

Country Link
CN (1) CN116069683A (en)
WO (1) WO2023078228A1 (en)

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102521279A (en) * 2011-11-25 2012-06-27 Tcl集团股份有限公司 Playing method, playing system and player of streaming media files
CN104683861A (en) * 2013-11-27 2015-06-03 北京大唐高鸿软件技术有限公司 Method for playing media stream based on UDP cache in local area network
CN106534889B (en) * 2016-11-15 2019-05-07 北京工商大学 A kind of real time flow medium multichannel retransmission method based on loop buffer
EP3661216A1 (en) * 2018-11-30 2020-06-03 InterDigital CE Patent Holdings A method and apparatus for loop-playing video content
CN112911376A (en) * 2021-02-01 2021-06-04 华录智达科技股份有限公司 Playing method based on real-time video playing fluency

Also Published As

Publication number Publication date
WO2023078228A1 (en) 2023-05-11

Similar Documents

Publication Publication Date Title
US10390055B1 (en) Video file processing
US9781486B2 (en) RS-DVR systems and methods for unavailable bitrate signaling and edge recording
US9049484B2 (en) Efficient assignment of program copies in a network digital video recorder
US8738736B2 (en) Scalable content streaming system with server-side archiving
CN106301865B (en) Data processing method and device applied to service providing device
CN101753936B (en) Configuration method of storage space and video server for video monitoring system
CN109766174B (en) Resource scheduling method, resource scheduling apparatus, and computer-readable storage medium
CN111694791A (en) Data access method and device in distributed basic framework
CN105786539B (en) File downloading method and device
US10782888B2 (en) Method and device for improving file system write bandwidth through hard disk track management
KR20130082685A (en) System and method for providing content service with virtual machine
US20190069046A1 (en) Method for creation and distribution of segmented video over distributed multicast-aware sparse networks with low latency
CN103220587A (en) Method and device for obtaining time shifting contents
CN116069683A (en) Media file memory management method, device, computer equipment and readable medium
CN112269763A (en) File aggregation method and related device
CN111435299B (en) Application processing method and device
CN112738525B (en) Video processing method, apparatus and computer readable storage medium
CN106302617B (en) Data processing method and device applied to computing device
CN102438171A (en) Method and device for processing program source files as well as network television system
KR102445589B1 (en) Systems, methods, and devices for managing segmented media content
CN109743343B (en) Passenger service system for rail transit
CN114520832A (en) Data stream processing method, device, equipment and computer storage medium
JP2017143475A (en) Distribution management apparatus, reception apparatus, distributed processing system, and program
CN111736758A (en) Setting method, device, equipment and medium of persistent cache
CN116939132A (en) Video storage method, terminal and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination