CN115858081B - UI (user interface) media resource display method, device, equipment and storage medium - Google Patents
UI (user interface) media resource display method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN115858081B CN115858081B CN202310160259.1A CN202310160259A CN115858081B CN 115858081 B CN115858081 B CN 115858081B CN 202310160259 A CN202310160259 A CN 202310160259A CN 115858081 B CN115858081 B CN 115858081B
- Authority
- CN
- China
- Prior art keywords
- displayed
- display
- interface
- decoding
- media
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
- Digital Computer Display Output (AREA)
Abstract
The invention belongs to the technical field of Internet and discloses a method, a device, equipment and a storage medium for displaying media resources of a UI (user interface); the method comprises the following steps: when a plurality of display requests of the UI interface are received, creating a plurality of virtual screens corresponding to the display requests; when receiving the coding request, obtaining media resources to be displayed according to the coding request; calling a decoder to decode the media resources to be displayed to obtain decoded media resources to be displayed; writing the decoded media resources to be displayed into the corresponding virtual screens so that the UI interface can simultaneously display the decoded media resources to be displayed in the plurality of virtual screens; according to the invention, the virtual screens are respectively created when the display requests of the plurality of applications are received, the codec is called to decode the media resources of the plurality of applications, and the decoded media resources are written into the corresponding virtual screens for the UI interface to display the media resources of the plurality of applications at the same time, so that the media resources of the plurality of applications are displayed at the same time, and the service displayed by the multi-application interface can be developed efficiently.
Description
Technical Field
The present invention relates to the field of internet technologies, and in particular, to a method, an apparatus, a device, and a storage medium for displaying UI media resources.
Background
A User Interface (UI) is a medium for interaction and information exchange between a system and a User, and the UI implements conversion between an internal form of information and a human-acceptable form. The user interface is designed to be related to the interaction between the user and the hardware, and aims to enable the user to conveniently and effectively operate the hardware to achieve bidirectional interaction, complete the work hoped to be completed by the hardware, and the user interface is widely defined and comprises a man-machine interaction and a graphic user interface, wherein the user interface exists in the field of information exchange between human beings and machines.
The UI interface comprises a plurality of applications for facilitating the life and work of users, the functions of each application are different, the content displayed for the users and the generated actions and effects are different, but because of the limitation of hardware and platform specifications on the market at present, encoding and decoding do not support multiple processes, and the UI interface resource display is that the media decoding and rendering preview are taken as an integral body, the function, namely the decoding and the media rendering, are usually completed in the same process, namely an instruction is completed, the components of all links in the mode use a unified or compatible technical stack, and the media resource display requirements of a plurality of applications cannot be responded at the same time, so that the media resource display requirements of a plurality of applications are excessively restrained when more services need to be developed, and the realization is difficult.
The foregoing is provided merely for the purpose of facilitating understanding of the technical solutions of the present invention and is not intended to represent an admission that the foregoing is prior art.
Disclosure of Invention
The invention mainly aims to provide a UI media resource display and aims to solve the technical problem that the prior art cannot display multimedia pictures of a plurality of applications at the same time and seriously influences the development of multi-interface business.
In order to achieve the above object, the present invention provides a method for displaying media resources of a UI interface, the method comprising the steps of:
when a plurality of display requests of a UI interface are received, creating a plurality of virtual screens corresponding to the display requests;
when a coding request corresponding to the display request is received, obtaining a media resource to be displayed according to the coding request;
calling a decoder to decode the media resources to be displayed to obtain decoded media resources to be displayed;
and writing the decoded media resources to be displayed into the corresponding virtual screen so that the UI interface can simultaneously display the decoded media resources to be displayed in the plurality of virtual screens.
Optionally, when receiving a plurality of display requests of the UI interface, creating a virtual screen corresponding to the display requests includes:
when a plurality of display requests of the UI interface are received, creating a corresponding virtual screen for each display request;
and generating access parameters of the virtual screen according to the creation address of the virtual screen, and feeding back the access parameters to the UI.
Optionally, when receiving the coding request corresponding to the display request, obtaining the media resource to be displayed according to the coding request, and further including:
if the coding request corresponding to the display request is not received within the preset time, the corresponding relation between the virtual screen and the display request is relieved;
and when a new display request is received, taking the virtual screen as a virtual screen corresponding to the new display request.
Optionally, when receiving the coding request corresponding to the display request, obtaining the media resource to be displayed according to the coding request includes:
obtaining reference position information of the media resource to be displayed according to the decoding request;
and accessing the media resource to be displayed according to the reference position information of the media resource to be displayed to obtain the media resource to be displayed.
Optionally, when receiving the encoding request corresponding to the display request, after obtaining the media resource to be displayed according to the encoding request, the method includes:
acquiring the decoding requirement of the media resource to be displayed;
acquiring current coding and decoding parameters, and calculating current coding and decoding resources according to the current coding and decoding parameters;
judging whether the current coding and decoding resource meets the decoding requirement of the media resource to be displayed or not;
if the current coding and decoding resources do not meet the decoding requirements of the media resources to be displayed, feeding back decoding failure to the UI;
and if the current encoding and decoding resources meet the decoding requirements of the media resources to be displayed, executing the step of calling a decoder to decode the media resources to be displayed to obtain decoded media resources to be displayed.
Optionally, writing the decoded media resource to be displayed into a corresponding virtual screen includes:
writing the decoded media resources to be displayed into the corresponding virtual screens, generating decoding marks in each virtual screen, and feeding back the decoding marks to the UI interface so that the UI interface simultaneously displays the decoded media resources to be displayed in the plurality of virtual screens.
Optionally, after writing the decoded media resource to be displayed into the corresponding virtual screen, so that the UI interface displays the decoded media resource to be displayed in the multiple virtual screens simultaneously, the method further includes:
clearing the decoded media resources to be displayed in the virtual screen to obtain a blank virtual screen;
and when a plurality of display requests of the UI interface are received, taking the blank virtual screen as a virtual screen corresponding to the display requests.
In addition, in order to achieve the above object, the present invention also provides a UI interface media resource display device, which includes:
the system comprises a request acquisition module, a display module and a display module, wherein the request acquisition module is used for creating a plurality of virtual screens corresponding to a plurality of display requests of a UI (user interface) when receiving the display requests;
the request acquisition module is further used for acquiring media resources to be displayed according to the coding request when the coding request corresponding to the display request is received;
the multimedia display module is used for calling a decoder to decode the media resources to be displayed to obtain decoded media resources to be displayed;
the multimedia display module is further configured to write the decoded media resources to be displayed into a corresponding virtual screen, so that the UI interface displays the decoded media resources to be displayed in the multiple virtual screens simultaneously.
In addition, to achieve the above object, the present invention also proposes a UI interface media resource display device, including: the system comprises a memory, a processor and a UI interface media resource display program stored on the memory and executable on the processor, the UI interface media resource display program configured to implement the steps of the UI interface media resource display method as described above.
In addition, in order to achieve the above object, the present invention also proposes a storage medium having stored thereon a UI interface media resource display program which, when executed by a processor, implements the steps of the UI interface media resource display method as described above.
According to the invention, the virtual screens are respectively created when the display requests of the plurality of applications are received, the codec is called to decode the media resources of the plurality of applications, and the decoded media resources are written into the corresponding virtual screens for the UI interface to display the media resources of the plurality of applications at the same time, so that the media resources of the plurality of applications are displayed at the same time, and the service displayed by the multi-application interface can be developed efficiently.
Drawings
FIG. 1 is a schematic diagram of a UI media resource display device of a hardware runtime environment according to an embodiment of the present invention;
FIG. 2 is a flowchart of a first embodiment of a method for displaying UI media resources according to the present invention;
FIG. 3 is a schematic diagram illustrating a media resource display among a call controller, a virtual screen, and a UI according to an embodiment of the UI media resource display method of the present invention;
FIG. 4 is a flowchart illustrating a second embodiment of a method for displaying UI media resources according to the present invention;
FIG. 5 is a schematic diagram illustrating information interaction between a call controller and a UI according to an embodiment of a method for displaying UI media resources of the present invention;
FIG. 6 is a flowchart illustrating a method for displaying a UI media resource according to a third embodiment of the invention;
fig. 7 is a block diagram illustrating a first embodiment of a UI media resource display device according to the present invention.
The achievement of the objects, functional features and advantages of the present invention will be further described with reference to the accompanying drawings, in conjunction with the embodiments.
Detailed Description
It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
Referring to fig. 1, fig. 1 is a schematic structural diagram of a UI interface media resource display device of a hardware running environment according to an embodiment of the present invention.
As shown in fig. 1, the UI interface media resource display device may include: a processor 1001, such as a central processing unit (CentralProcessing Unit, CPU), a communication bus 1002, a user interface 1003, a network interface 1004, a memory 1005. Wherein the communication bus 1002 is used to enable connected communication between these components. The user interface 1003 may include a Display, an input unit such as a Keyboard (Keyboard), and the optional user interface 1003 may further include a standard wired interface, a wireless interface. The network interface 1004 may optionally include a standard wired interface, a WIreless interface (e.g., a WIreless-FIdelity (WI-FI) interface). The Memory 1005 may be a high-speed random access Memory (RandomAccess Memory, RAM) Memory or a stable nonvolatile Memory (NVM), such as a disk Memory. The memory 1005 may also optionally be a storage device separate from the processor 1001 described above.
Those skilled in the art will appreciate that the structure shown in fig. 1 does not constitute a limitation of the UI interface media asset display device, and may include more or fewer components than shown, or may combine certain components, or a different arrangement of components.
As shown in fig. 1, an operating system, a network communication module, a user interface module, and a UI interface media resource display program may be included in the memory 1005 as one type of storage medium.
In the UI interface media resource display device shown in fig. 1, the network interface 1004 is mainly used for data communication with a network server; the user interface 1003 is mainly used for data interaction with a user; the processor 1001 and the memory 1005 in the UI interface media resource display device of the present invention may be disposed in the UI interface media resource display device, where the UI interface media resource display device invokes the UI interface media resource display program stored in the memory 1005 through the processor 1001, and executes the UI interface media resource display method provided by the embodiment of the present invention.
The embodiment of the invention provides a method for displaying media resources of a UI interface, and referring to fig. 2, fig. 2 is a schematic flow chart of a first embodiment of the method for displaying media resources of the UI interface.
In this embodiment, the UI interface media resource display method includes the following steps:
step S10: when a plurality of display requests of a UI interface are received, a plurality of virtual screens corresponding to the display requests are created.
It can be understood that the display request may be that after the user has triggered the operation interface, the triggering operation may be a signal that is sent to the call controller and that some frames will be displayed, so that the triggering operation may be similar to that of clicking on the intelligent home management interface in the intelligent home control system and clicking on the sub-computer control interface in the seat system for easy understanding, but the user does not determine the specific media resource to be displayed at this time.
It should be understood that, the execution body of the present invention is a call controller between the UI interface and the codec, and when the call controller receives a plurality of display requests, the call controller may create a virtual screen corresponding to each display request for a plurality of display requests of a plurality of applications and a plurality of display requests of the same application, which is simply that one display request corresponds to one virtual screen.
It should be noted that the virtual screen may be a description of a storage space that implements a step of storing the decoded media resource between the UI interface rendering and decoding actions, and one or more virtual screens may exist, which is determined according to the memory size of the call controller. In short, the virtual screen may be a storage space storing the media resources decoded by the codec, and the UI reads the decoded media resources from the virtual screen after the codec decodes the media resources.
It is worth further emphasizing that when multiple display requests of the UI interface are received, a corresponding virtual screen is created for each display request, then an access parameter of each virtual screen is generated according to the creation address of each virtual screen, the access parameter is fed back to the UI interface, the UI interface associates the access parameter with each display request, the access parameter is that the controller is called to feed back to the UI interface to successfully receive the display request of the UI interface, the virtual screen is successfully created, a signal of the coding request can be sent, and if the UI interface wants to view media resources to be displayed after decoding in the virtual screen, the media resources to be displayed can be accessed through the access parameter.
In a specific implementation, if the UI interface operated by the current user is an intelligent home control system, when the user clicks on the management interface of the intelligent home system at the UI interface, the UI interface may send a display request to the call controller, to indicate that the user wants to view the media resource situation of a certain intelligent home appliance at this time, specifically, whether to view the media resource of the television, the media resource of the monitor, or the media resource of the sweeping robot is uncertain at this time, and the display request may be a signal that the user wants to display an intention of a certain media resource.
It should be understood that the media resource may be information such as a real-time video, a real-time motion track, a staged image, etc., and for convenience of understanding, the real-time video may be understood as a real-time monitoring video of a monitor in the smart home system, or may be a real-time computer interface operation of a sub-computer in the agent system; the real-time track can be a cleaning track of an intelligent sweeping robot in the intelligent home system and can be a real-time driving route of an intelligent automobile.
Step S20: and when receiving a coding request corresponding to the display request, obtaining the media resource to be displayed according to the coding request.
It is understood that each display request corresponds to a virtual screen one by one, and each display request corresponds to an encoding request one by one; the encoding request may be understood as that after the user selects the media resource to be displayed through the UI interface, the user sends related information such as the network location of the media resource to be displayed as the encoding request to the call controller.
It should be appreciated that the media asset to be displayed may be a media asset that the user has selected to be displayed on the UI interface.
It should be noted that, if the corresponding encoding request is not received continuously after the display request is received, a time may be preset, which may be 30 seconds, 1 minute or 2 minutes, and when the decoding request corresponding to the display request is not received within a preset time after the display request, the virtual screen corresponding to the display request may be cleared for the next display request.
In the implementation, when a user enters the seat system, the user enters a sub-computer display management interface of the seat system through a UI interface, the UI interface sends a display request to a calling controller, and the calling controller creates a virtual screen corresponding to the display request; when the user selects to display the sub-computer A, the UI interface sends a coding request comprising the network address of the sub-computer A to the calling controller, and the calling controller accesses the media resource of the sub-computer A through the network address of the sub-computer A.
Step S30: and calling a decoder to decode the media resources to be displayed to obtain decoded media resources to be displayed.
It is understood that the decoder may be a hardware decoder such as a GPU/CPU, and the call controller may invoke the hardware codec to decode the media asset to be displayed through a software development kit (SDK, which may also be understood as invoking a hardware interface between the decoder and the codec).
It should be appreciated that a software development kit is typically a collection of development tools that some software engineers create application software for a particular software package, software framework, hardware platform, operating system, or the like.
It should be noted that, the calling decoder may decode the plurality of media resources to be displayed to obtain a plurality of decoded media resources to be displayed. In short, the same encoder in hardware can only decode the resources of one application at one time point, but can decode the multimedia pictures of multiple applications at the same time by calling the controller.
It is emphasized that the UI interface sends the media resources to be decoded to the call controller, and the call controller uniformly calls the codec to decode the media resources to be decoded for each application of the UI interface.
It is worth emphasizing that the codec may decode the media resources to be decoded for each application, and then perform media rendering on the decoded media resources through the UI interface, so as to separate the codec and the rendering, because the products on the market currently use more codecs as an integral task, but use unified or compatible technical stacks as a whole, which has a limitation on the technical options of each component. Media coding and decoding are focused on functions and performance specifications, a UI (user interface) is focused on user interaction, each technical stack also has the own emphasis, and the final requirements are different, but the media coding and decoding are realized by using the same technology, so that various contradictions exist. The whole system is more dependent on a hardware platform, so that the coupling performance is high; if the platform is replaced, all dependent components must be changed accordingly, which is very complex and cumbersome.
It should be appreciated that decoding and rendering are separated to enable decoding of multiple media assets at the same time, avoiding the problem that other media assets can only be decoded after a single media asset is decoded, rendered and displayed.
In the implementation, a user opens a management interface of the application A and the application B at the UI interface point at the same time, and the user wants to display an a video in the application A and a B video in the application B at the same time, then sends an a decoding request and a B decoding request at the same time, and calls a controller to call a codec to decode the a and the B at the same time.
Step S40: and writing the decoded media resources to be displayed into the corresponding virtual screen so that the UI interface can simultaneously display the decoded media resources to be displayed in the plurality of virtual screens.
It can be understood that the virtual screen is in a writable state at this time, and the virtual screen and the display request of the media resource to be displayed correspond to each other.
It should be understood that, unlike the existing method that decodes a media resource and then directly sends the decoded resource to a UI interface for rendering and displaying, the call controller, after calling the codec to decode a plurality of media resources, stores the decoded media resource in a virtual screen corresponding to each display request, and if the UI interface is to display the media resource of that application display request, calls the decoded media resource from the virtual screen corresponding to the display request for rendering and displaying.
The UI interface can read media resources in the multiple virtual screens simultaneously, and can display media resources of multiple applications simultaneously or display multiple media resources of the same application simultaneously.
It should be noted that, writing the decoded media resources to be displayed into the corresponding virtual screen, generating a decoding flag in each virtual screen, and feeding back the decoding flag to the UI interface, so that when the UI interface receives the decoding flag, rendering and displaying the decoded media resources to be displayed in the multiple virtual screens at the same time, where the decoding flag may be understood as a signal sent by the calling controller to the UI interface and decoded successfully, and when the UI interface receives the signal of successful decoding, the UI interface may continuously read the decoded media resources in the virtual screen and then perform rendering and displaying.
It is further noted that, based on the decoding flag, the UI interface does not need to repeatedly access the virtual screen to see whether decoding is completed, and when the flag is received, the UI interface can read the media resource to be displayed after decoding in the virtual screen, so that it is more convenient for the UI interface to determine whether decoding is completed and whether rendering display can be performed.
It should be noted that, the relationship between the call controller, the virtual screen and the UI interface may refer to fig. 3, where the codec service may be understood as the call controller (because the codec service implicitly calls the codec to perform the codec), the virtual screen is an intermediate layer between the codec and the UI rendering, so may be the virtual screen as shown in fig. 3, or may be an intermediate layer, and the UI interface is the UI interaction service in fig. 3, because it may be understood that the UI interface is a service interface for performing information interaction with the user; the encoding and decoding server side writes data into the middle layer/virtual screen, and the data in the middle layer/virtual screen is read by the UI interaction service interface.
According to the embodiment, the virtual screen is respectively created when the display requests of the plurality of applications are received, the codec is called to decode the media resources of the plurality of applications, and the decoded media resources are written into the corresponding virtual screen for the UI interface to display the media resources of the plurality of applications at the same time, so that the media resources of the plurality of applications are displayed at the same time, and the service displayed by the multi-application interface can be efficiently developed.
Referring to fig. 4, fig. 4 is a flowchart illustrating a UI media resource display method according to a second embodiment of the invention.
Based on the above first embodiment, the UI media resource display method of this embodiment in step S20 includes:
step S21: and obtaining the reference position information of the media resource to be displayed according to the decoding request.
It is understood that the reference location information may be a stored network address of the media asset to be displayed, may be an address link, may be a network address.
It should be understood that when the call controller receives the coding request, the reference position information of the media resource which the user wants to display is obtained according to the coding request, the corresponding media resource can be accessed according to the reference position information, the access is successful, and the information of the successful access is fed back to the UI interface; it can be understood that the controller is invoked to obtain the media resources to be decoded after the access is successful.
In the implementation, the UI interface opens a sub-computer management interface of the seat system, and the user selects and displays an interface of the sub-computer N, at this time, the UI interface sends the network address information of the sub-computer N interface to the call controller in a coding request manner, and the call controller accesses according to the network address of the sub-computer N interface, and then calls the codec to decode the sub-computer N interface.
Step S22: and accessing the media resource to be displayed according to the reference position information of the media resource to be displayed to obtain the media resource to be displayed.
It can be understood that, after the call controller accesses the media resource to be displayed according to the reference position, the call controller cannot directly feed back the information of successful access to the UI interface, and it is also required to determine whether the codec can decode the media resource to be displayed.
It should be noted that, according to the decoding request, the relevant information of the media resource that the user wants to display on the UI may be obtained, where the relevant information may be definition, frame rate, resolution, definition, etc., for example, the user wants to display the media resource to be displayed on the UI with 1080p definition.
It should be appreciated that the decoding capability of the codec is limited, limited by the total memory, total frame rate, resolution, and sharpness; and a plurality of media resources may be decoded at the same time, the decoding resources that can be used by the current codec need to be calculated first, and the decoding resources of the current codec and the decoding requirements of the media resources to be displayed are compared to determine whether decoding can be performed.
It should be emphasized that, when receiving the encoding request corresponding to the display request, obtaining the decoding requirement of the media resource to be displayed after obtaining the media resource to be displayed according to the encoding request, where the decoding requirement may be occupied memory, definition, resolution and the like of the media resource to be displayed;
acquiring a current coding and decoding parameter, calculating current coding and decoding resources according to the current coding and decoding parameter, wherein the current coding and decoding parameter can be the decoding resources occupied by the current coder and decoder, and the current coding and decoding resources can be the decoding resources usable by the current coder and decoder, wherein the maximum decoding resources of the coder and decoder are known, such as the maximum memory, the maximum frame rate, the maximum resolution and the maximum value of the number of decoding media resources of the coder and decoder, and checking to obtain the decoding resources occupied by the current coder, such as 10.8G of the currently occupied memory, 40 of the current total frame rate, 4k of the current maximum resolution and 2 of the number of currently decoding media resources;
judging whether the current coding and decoding resources meet the decoding requirements of the media resources to be displayed, namely, judging whether the available decoding resources of the current coder and decoder can support the decoding requirements of the media resources to be displayed;
if the current encoding and decoding resources do not meet the decoding requirements of the media resources to be displayed, feeding back decoding failure to the UI interface, and feeding back the decoding failure reason to the UI interface, wherein the decoding failure reason can be insufficient memory, unsupported definition, excessive current decoding resources, excessive current decoding total frame rate and the like.
And if the current encoding and decoding resources meet the decoding requirements of the media resources to be displayed, executing the step of calling a decoder to decode the media resources to be displayed to obtain decoded media resources to be displayed.
In specific implementation, the interaction between the call controller and the UI may refer to fig. 5, where the codec on the left side in the figure is equivalent to the call controller (actually implying that the call controller calls the codec to perform codec), and the UI interface for performing information interaction with the user is on the right side, firstly, the UI sends a request to the call controller to create a virtual screen, the call controller creates the virtual screen, and after the call controller creates the virtual screen, the call controller feeds back the information of successful creation to the virtual screen; the UI sends the information of the media resource to be displayed to the calling controller, and the requirement of decoding the media resource is described, and the media resource is decoded to a specific position, the decoding definition, the decoding memory and the like; the calling controller receives the decoding requirement, and then feeds back the information received and ready to start decoding to the UI interface; and at the moment, calling the controller to write the decoded media resources into the virtual screen, and circularly reading the decoded media resources from the virtual screen by the UI interface, and rendering and displaying.
According to the embodiment, the position information of the media resources to be displayed is reflected through the coding request, the media resources to be displayed are accessed according to the position information, each display screen is in one-to-one correspondence with the media resources to be displayed, whether the decoding capability of current decoding can support decoding of the resources to be displayed or not is judged before a decoder is called, feedback is timely carried out, and the situation that the midway resources are insufficient and cannot be decoded and stuck is avoided.
Referring to fig. 6, fig. 6 is a flowchart illustrating a UI media resource display method according to a third embodiment of the invention.
Based on the above-mentioned first embodiment, the UI media resource display method of this embodiment further includes, after the step S40:
step S50: and clearing the decoded media resources to be displayed in the virtual screen to obtain a blank virtual screen.
It can be understood that the virtual screen is a storage space for storing the decoded media resource to be displayed, and when the user finishes displaying the decoded media resource on the UI interface, the decoded media resource to be displayed in the virtual screen can be clearly displayed.
It should be appreciated that when the user is not in need of displaying the decoded media asset in the current virtual screen, the UI interface will send the display completed information to the call controller, because the codec will continue to decode the media asset to be decoded, the decoded media asset will be written continuously in the virtual screen,
it should be noted that, when the call controller receives the display completion information sent by the UI interface, the call codec is stopped to decode the media resource to be displayed.
It should be noted that, in order to avoid that the user closes the display of the media resource to be displayed due to misoperation, the calling controller may clear the decoded media resource to be displayed in the virtual screen to obtain a blank virtual screen after a preset time (1 minute).
It should be noted that, the blank virtual screen does not completely delete the content in the current virtual screen, but releases the correspondence between the current virtual screen and the display request and decoding request, and may associate and correspond the virtual screen with the new display request.
And when a new display request corresponds to the blank virtual screen, writing the decoded new media resource corresponding to the new display request into the blank virtual screen, and covering the decoded media resource corresponding to the previous display request.
Step S60: and when a plurality of display requests of the UI interface are received, taking the blank virtual screen as a virtual screen corresponding to the display requests.
It is understood that when multiple display requests of the UI interface are received, it may be understood that the call controller receives a new display request, and associates and corresponds the new display request with the blank virtual screen.
According to the embodiment, after the information of display completion is received, the corresponding relation between the virtual screen and the display request and the decoding request is cleared, and after a new display request is received, the virtual screen and the new display request are associated again, and media resources in the virtual screen are covered, so that the effect of recycling the virtual screen is achieved, the memory occupied by the virtual screen is reduced, and the operation efficiency is improved.
In addition, the embodiment of the invention also provides a storage medium, wherein the storage medium is stored with a UI interface media resource display program, and the UI interface media resource display program realizes the steps of the UI interface media resource display method when being executed by a processor.
Referring to fig. 7, fig. 7 is a block diagram illustrating a first embodiment of a UI media resource display device according to the present invention.
As shown in fig. 7, the UI media resource display device provided by the embodiment of the invention includes:
a request acquisition module 10, configured to, when receiving a plurality of display requests of a UI interface, create a plurality of virtual screens corresponding to the display requests;
the request acquisition module 10 is further configured to, when receiving a coding request corresponding to the display request, obtain a media resource to be displayed according to the coding request;
the multimedia display module 20 is configured to invoke a decoder to decode the media resource to be displayed to obtain a decoded media resource to be displayed;
the multimedia display module 20 is further configured to write the decoded media resources to be displayed into a corresponding virtual screen, so that the UI interface displays the decoded media resources to be displayed in the multiple virtual screens simultaneously.
According to the embodiment, the virtual screen is respectively created when the display requests of the plurality of applications are received, the codec is called to decode the media resources of the plurality of applications, and the decoded media resources are written into the corresponding virtual screen for the UI interface to display the media resources of the plurality of applications at the same time, so that the media resources of the plurality of applications are displayed at the same time, and the service displayed by the multi-application interface can be efficiently developed.
In an embodiment, the request obtaining module 10 is further configured to, when receiving a plurality of display requests of the UI interface, create a corresponding virtual screen for each display request;
and generating access parameters of the virtual screen according to the creation address of the virtual screen, and feeding back the access parameters to the UI.
In an embodiment, the request obtaining module 10 is further configured to obtain reference location information of the media resource to be displayed according to the decoding request;
and accessing the media resource to be displayed according to the reference position information of the media resource to be displayed to obtain the media resource to be displayed.
In an embodiment, the request obtaining module 10 is further configured to, if the encoding request corresponding to the display request is not received within a preset time, release the corresponding relationship between the virtual screen and the display request;
and when a new display request is received, taking the virtual screen as a virtual screen corresponding to the new display request.
In an embodiment, the request obtaining module 10 is further configured to obtain a decoding requirement of the media resource to be displayed;
acquiring current coding and decoding parameters, and calculating current coding and decoding resources according to the current coding and decoding parameters;
judging whether the current coding and decoding resource meets the decoding requirement of the media resource to be displayed or not;
if the current coding and decoding resources do not meet the decoding requirements of the media resources to be displayed, feeding back decoding failure to the UI;
and if the current encoding and decoding resources meet the decoding requirements of the media resources to be displayed, executing the step of calling a decoder to decode the media resources to be displayed to obtain decoded media resources to be displayed.
In an embodiment, the multimedia display module 20 is further configured to write the decoded media resources to be displayed into a corresponding virtual screen, generate a decoding flag in each virtual screen, and feed back the decoding flag to the UI interface, so that the UI interface displays the decoded media resources to be displayed in the multiple virtual screens simultaneously.
In an embodiment, the multimedia display module 20 is further configured to clear the decoded media resources to be displayed in the virtual screen to obtain a blank virtual screen;
and when a plurality of display requests of the UI interface are received, taking the blank virtual screen as a virtual screen corresponding to the display requests.
It should be understood that the foregoing is illustrative only and is not limiting, and that in specific applications, those skilled in the art may set the invention as desired, and the invention is not limited thereto.
It should be noted that the above-described working procedure is merely illustrative, and does not limit the scope of the present invention, and in practical application, a person skilled in the art may select part or all of them according to actual needs to achieve the purpose of the embodiment, which is not limited herein.
Furthermore, it should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or system that comprises the element.
The foregoing embodiment numbers of the present invention are merely for the purpose of description, and do not represent the advantages or disadvantages of the embodiments.
From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. Read Only Memory)/RAM, magnetic disk, optical disk) and including several instructions for causing a terminal device (which may be a mobile phone, a computer, a server, or a network device, etc.) to perform the method according to the embodiments of the present invention.
The foregoing description is only of the preferred embodiments of the present invention, and is not intended to limit the scope of the invention, but rather is intended to cover any equivalents of the structures or equivalent processes disclosed herein or in the alternative, which may be employed directly or indirectly in other related arts.
Claims (8)
1. The UI media resource display method is characterized by comprising the following steps:
when a call controller receives a plurality of display requests of a UI interface, creating a plurality of virtual screens corresponding to the display requests, wherein the plurality of display requests are display requests of different application devices;
when the call controller receives a coding request corresponding to the display request, obtaining media resources to be displayed of different application devices according to the coding request;
simultaneously decoding the media resources to be displayed of the different application devices through the calling controller to obtain decoded media resources to be displayed;
writing the decoded media resources to be displayed into the corresponding virtual screens through the call controller so that the UI interface can render and display the decoded media resources to be displayed in the plurality of virtual screens simultaneously;
when the call controller receives the coding request corresponding to the display request, obtaining media resources to be displayed of different application devices according to the coding request, the call controller comprises:
acquiring the decoding requirement of the media resource to be displayed, wherein the decoding requirement can be the occupied memory, definition and resolution of the media resource to be displayed; acquiring current coding and decoding parameters, and calculating current coding and decoding resources according to the current coding and decoding parameters; judging whether the current coding and decoding resource meets the decoding requirement of the media resource to be displayed or not; if the current coding and decoding resources do not meet the decoding requirements of the media resources to be displayed, feeding back decoding failure to the UI; if the current coding and decoding resources meet the decoding requirements of the media resources to be displayed, executing the step of calling decoders corresponding to the UI interfaces to decode the media resources to be displayed of the different application devices to obtain decoded media resources to be displayed;
when the calling controller receives the coding request corresponding to the display request, obtaining media resources to be displayed of different application devices according to the coding request, and further comprising:
if the coding request corresponding to the display request is not received within the preset time, the corresponding relation between the virtual screen and the display request is relieved; and when a new display request is received, taking the virtual screen as a virtual screen corresponding to the new display request.
2. The UI interface media resource display method of claim 1, wherein when the call controller receives a plurality of display requests of the UI interface, creating a virtual screen corresponding to the display requests comprises:
when the call controller receives a plurality of display requests of the UI interface, creating a corresponding virtual screen for each display request;
and generating access parameters of the virtual screen according to the creation address of the virtual screen, and feeding back the access parameters to the UI.
3. The UI interface media resource display method of claim 1, wherein when the call controller receives the encoding request corresponding to the display request, obtaining media resources to be displayed of different application devices according to the encoding request, comprises:
when the calling controller receives a coding request corresponding to the display request, obtaining reference position information of media resources to be displayed of different application devices according to the coding request;
and accessing the media resources to be displayed according to the reference position information of the media resources to be displayed to obtain the media resources to be displayed of different application devices.
4. The UI interface media resource display method of claim 1, wherein writing the decoded media resource to be displayed to a corresponding virtual screen comprises:
writing the decoded media resources to be displayed into the corresponding virtual screens, generating decoding marks in each virtual screen, and feeding back the decoding marks to the UI interface so that the UI interface simultaneously displays the decoded media resources to be displayed in the plurality of virtual screens.
5. The UI interface media resource display method of any of claims 1-4, wherein after writing the decoded media resource to be displayed into the corresponding virtual screen, so that the UI interface renders and displays the decoded media resource to be displayed in the plurality of virtual screens at the same time, the method further comprises:
clearing the decoded media resources to be displayed in the virtual screen to obtain a blank virtual screen;
and when a plurality of display requests of the UI interface are received, taking the blank virtual screen as a virtual screen corresponding to the display requests.
6. A UI interface media asset display device, the UI interface media asset display device comprising:
the request acquisition module is used for creating a plurality of virtual screens corresponding to the display requests when the call controller receives the display requests of the UI interface, wherein the display requests are display requests of different application devices;
the request acquisition module is further used for acquiring media resources to be displayed of different application devices according to the coding request when the calling controller receives the coding request corresponding to the display request; if the coding request corresponding to the display request is not received within the preset time, the corresponding relation between the virtual screen and the display request is relieved; when a new display request is received, the virtual screen is used as a virtual screen corresponding to the new display request; acquiring the decoding requirement of the media resource to be displayed, wherein the decoding requirement can be the occupied memory, definition and resolution of the media resource to be displayed; acquiring current coding and decoding parameters, and calculating current coding and decoding resources according to the current coding and decoding parameters; judging whether the current coding and decoding resource meets the decoding requirement of the media resource to be displayed or not; if the current coding and decoding resources do not meet the decoding requirements of the media resources to be displayed, feeding back decoding failure to the UI; if the current coding and decoding resources meet the decoding requirements of the media resources to be displayed, executing the step of calling decoders corresponding to the UI interfaces to decode the media resources to be displayed of the different application devices to obtain decoded media resources to be displayed;
the multimedia display module is used for simultaneously decoding the media resources to be displayed of the different application devices through the calling controller to obtain decoded media resources to be displayed;
the multimedia display module is further configured to write the decoded media resource to be displayed into a corresponding virtual screen through the call controller, so that the UI interface renders and displays the decoded media resource to be displayed in the multiple virtual screens at the same time.
7. A UI interface media asset display device, the device comprising: a memory, a processor, and a UI interface media asset display program stored on the memory and executable on the processor, the UI interface media asset display program configured to implement the UI interface media asset display method of any one of claims 1 to 5.
8. A storage medium having stored thereon a UI interface media asset display program which when executed by a processor implements the UI interface media asset display method according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310160259.1A CN115858081B (en) | 2023-02-24 | 2023-02-24 | UI (user interface) media resource display method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310160259.1A CN115858081B (en) | 2023-02-24 | 2023-02-24 | UI (user interface) media resource display method, device, equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115858081A CN115858081A (en) | 2023-03-28 |
| CN115858081B true CN115858081B (en) | 2023-06-16 |
Family
ID=85658814
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310160259.1A Active CN115858081B (en) | 2023-02-24 | 2023-02-24 | UI (user interface) media resource display method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115858081B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114356264A (en) * | 2021-12-30 | 2022-04-15 | 威创集团股份有限公司 | Signal generation method, device, equipment and readable storage medium |
| CN114727070A (en) * | 2022-05-10 | 2022-07-08 | 中科院广州电子技术有限公司 | Camera real-time video data processing method and device |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP4476261B2 (en) * | 2006-09-13 | 2010-06-09 | 株式会社ソニー・コンピュータエンタテインメント | Decoding device and decoding method |
| US10200725B2 (en) * | 2015-06-15 | 2019-02-05 | Intel Corporation | Adaptive data streaming based on virtual screen size |
| CN106792104A (en) * | 2017-01-19 | 2017-05-31 | 北京行云时空科技有限公司 | It is a kind of while supporting the method and system that shows of multiwindow image |
| CN111953992A (en) * | 2020-07-07 | 2020-11-17 | 西安万像电子科技有限公司 | Decoding method and device |
-
2023
- 2023-02-24 CN CN202310160259.1A patent/CN115858081B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114356264A (en) * | 2021-12-30 | 2022-04-15 | 威创集团股份有限公司 | Signal generation method, device, equipment and readable storage medium |
| CN114727070A (en) * | 2022-05-10 | 2022-07-08 | 中科院广州电子技术有限公司 | Camera real-time video data processing method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115858081A (en) | 2023-03-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP1986369B1 (en) | End user control configuration system with dynamic user interface | |
| US6904597B2 (en) | Inter-thread communications between different components using double buffer | |
| CN110457143B (en) | Micro-service calling method and device | |
| CN102855142B (en) | A kind of client framework structure based on RIA and its implementation | |
| US20130091417A1 (en) | Method and apparatus for using proxies to interact with webpage analytics | |
| CN112770188B (en) | Video playing method and device | |
| CN113032080B (en) | Page implementation method, application program, electronic device and storage medium | |
| WO2020143159A1 (en) | User interface processing method and device | |
| CN105630461A (en) | Managing method of android application interface | |
| CN102404633B (en) | Monitoring module, intelligent television system and television | |
| WO2022068671A1 (en) | Cloud desktop display method and system | |
| CN107862810A (en) | A kind of self-aided terminal control system based on browser | |
| CN112925463A (en) | Information flow publishing processing method, device, medium and electronic equipment | |
| CN104951287A (en) | Method and device for generating webpage | |
| CN107172259B (en) | Terminal and information interaction method and storage device based on two-dimensional code | |
| CN111681157A (en) | Image rendering method and system | |
| CN111523065B (en) | Rich text data processing method and device, electronic equipment and computer storage medium | |
| CN115858081B (en) | UI (user interface) media resource display method, device, equipment and storage medium | |
| WO2023246486A1 (en) | Method and apparatus for creating connector | |
| JP2006277771A (en) | Terminal device and operation history recording method for computer network | |
| CN108563485B (en) | Input panel display method and device | |
| US20070073902A1 (en) | Information processor, information processing method and program | |
| CN115237481A (en) | Method, device and equipment for driving external equipment and storage medium | |
| CN112749033B (en) | Display device and system notification calling method | |
| CN110660463A (en) | Report editing method, device and equipment based on ultrasonic system and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |