CN118377561A - Display method, device, system and electronic equipment - Google Patents
Display method, device, system and electronic equipment Download PDFInfo
- Publication number
- CN118377561A CN118377561A CN202410371284.9A CN202410371284A CN118377561A CN 118377561 A CN118377561 A CN 118377561A CN 202410371284 A CN202410371284 A CN 202410371284A CN 118377561 A CN118377561 A CN 118377561A
- Authority
- CN
- China
- Prior art keywords
- display
- virtual
- application
- window
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 71
- 230000015572 biosynthetic process Effects 0.000 claims description 7
- 238000003786 synthesis reaction Methods 0.000 claims description 7
- 238000002156 mixing Methods 0.000 claims description 5
- 230000008569 process Effects 0.000 description 38
- 239000010410 layer Substances 0.000 description 19
- 238000010586 diagram Methods 0.000 description 17
- 238000009877 rendering Methods 0.000 description 12
- 230000006870 function Effects 0.000 description 11
- 239000003795 chemical substances by application Substances 0.000 description 9
- 239000000203 mixture Substances 0.000 description 6
- 230000006978 adaptation Effects 0.000 description 5
- 230000001360 synchronised effect Effects 0.000 description 5
- 238000004891 communication Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 4
- 239000011159 matrix material Substances 0.000 description 4
- 239000002344 surface layer Substances 0.000 description 4
- 230000008859 change Effects 0.000 description 3
- 230000003190 augmentative effect Effects 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 230000014509 gene expression Effects 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 238000001514 detection method Methods 0.000 description 1
- 238000001125 extrusion Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 239000000725 suspension Substances 0.000 description 1
Landscapes
- Digital Computer Display Output (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The application discloses a display method, a display device, a display system and electronic equipment. The display method comprises the following steps: under the condition that a program interface of an application program is not matched with a first display area, loading display information of the application program into a virtual window, wherein the first display area is a display area which can be provided by electronic equipment; and displaying an application window in the first display area, wherein the application window is an interface obtained by scaling the virtual window.
Description
Technical Field
The application belongs to the technical field of electronic equipment, and particularly relates to a display method, a display device, a display system and electronic equipment.
Background
With the trend of large screens of mobile electronic devices, the electronic devices are increasingly popular in performing multi-window display functions such as multi-window display, split-screen display, small-window display, picture-in-picture display and the like on multiple applications;
In the related art, the process of realizing the multi-window display function needs to adjust the user interface and the function compatibility in real time, so that the application compatibility is poor, and the problems of incomplete display and inconsistent display proportion frequently occur.
Disclosure of Invention
The embodiment of the application aims to provide a display method, a display device, electronic equipment and a readable storage medium, which improve the display compatibility of application programs and solve the problems of incomplete display and inconsistent display proportion of a display screen on a plurality of application windows.
In a first aspect, an embodiment of the present application provides a display method, including: under the condition that a program interface of an application program is not matched with a first display area, loading display information of the application program into a virtual window, wherein the first display area is a display area which can be provided by electronic equipment; and displaying an application window in the first display area, wherein the application window is an interface obtained by scaling the virtual window.
In a second aspect, an embodiment of the present application provides a display apparatus, including: the loading module is used for loading the display information of the application program into the virtual window under the condition that the program interface of the application program is not matched with the first display area, and the first display area is a display area which can be provided by the electronic equipment; and the display module is used for displaying an application window in the first display area, wherein the application window is an interface obtained by scaling the virtual window.
In a third aspect, an embodiment of the present application provides a display system, including: the framework layer is used for creating a virtual window corresponding to the application program at the system side and determining display configuration information of the virtual window; the local library is used for drawing the display content of the virtual window to the target cache region based on the display configuration information; and the system kernel is used for adding the display content in the target cache region into the video memory.
In a fourth aspect, embodiments of the present application provide an electronic device comprising a processor and a memory storing a program or instructions executable on the processor, the program or instructions implementing the steps of the method as in the first aspect when executed by the processor.
In a fifth aspect, embodiments of the present application provide a readable storage medium having stored thereon a program or instructions which when executed by a processor perform the steps of the method as in the first aspect.
In a sixth aspect, embodiments of the present application provide a chip comprising a processor and a communication interface coupled to the processor for running a program or instructions implementing the steps of the method as in the first aspect.
In a seventh aspect, embodiments of the present application provide a computer program product stored in a storage medium, the program product being executable by at least one processor to implement a method as in the first aspect.
In the embodiment of the application, the electronic equipment constructs the virtual window corresponding to the running application program, loads the display information of the application program into the virtual window, performs scaling processing on the virtual window, and displays the application window obtained by the scaling processing in the first display area, so that the application program does not need to do additional compatible processing because the application program does not sense the scaling process of the virtual window, thereby greatly reducing the adaptation workload of the application program, simultaneously, the application program does not perform operations such as real-time destroying, re-layout, drawing and the like according to the size, and the like, and realizes seamless switching and connection in a multi-window scene through scaling of the virtual window because the application cannot sense the switching of window modes, thereby improving the display compatibility of the application program, and solving the problems of incomplete display and inconsistent display scale of the application window in the display screen.
Drawings
FIG. 1 is a flow chart of a display method according to some embodiments of the present application;
FIG. 2 illustrates a schematic diagram of an application interacting with a virtual window in some embodiments of the application;
FIG. 3 illustrates a schematic diagram of a virtual window synthesis process provided in some embodiments of the application;
FIG. 4 illustrates one of the schematic diagrams of a second interface provided in some embodiments of the application;
FIG. 5 illustrates a second schematic diagram of a second interface provided in some embodiments of the application;
FIG. 6 illustrates a third schematic view of a second interface provided in some embodiments of the application;
FIG. 7 illustrates a schematic block diagram of a display device provided by some embodiments of the application;
FIG. 8 illustrates a schematic block diagram of a display system provided by some embodiments of the application;
FIG. 9 illustrates a block diagram of an electronic device provided by some embodiments of the application;
Fig. 10 is a schematic diagram of a hardware structure of an electronic device according to some embodiments of the present application.
Detailed Description
The technical solutions of the embodiments of the present application will be clearly described below with reference to the drawings in the embodiments of the present application, and it is apparent that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which are obtained by a person skilled in the art based on the embodiments of the present application, fall within the scope of protection of the present application.
The terms first, second and the like in the description and in the claims, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the terms so used are interchangeable under appropriate circumstances such that the embodiments of the application are capable of operation in sequences other than those illustrated or otherwise described herein, and that the objects identified by "first," "second," etc. are generally of a type not limited to the number of objects, for example, the first object may be one or more. Furthermore, in the description and claims, "and/or" means at least one of the connected objects, and the character "/", generally means that the associated object is an "or" relationship.
The display method, the display device, the display system, the electronic device and the readable storage medium provided by the embodiment of the application are described in detail below with reference to fig. 1 to 10 by means of specific embodiments and application scenes thereof.
In some embodiments of the present application, a display method is provided, and fig. 1 shows a schematic flow chart of the display method provided in some embodiments of the present application. As shown in fig. 1, the display method includes:
Step 102, loading display information of an application program into a virtual window under the condition that a program interface of the application program is not matched with a first display area, wherein the first display area is a display area which can be provided by electronic equipment;
In the embodiment of the application, the program interface is an interface which is required to be displayed in the electronic equipment in the running process of the application program, and the first display area is a display area provided by the electronic equipment for the application program. And when the first display area is not matched with the program interface, loading the display information of the application program into a virtual window matched with the program interface, namely, the virtual window is matched with the program interface of the application program.
When the electronic device runs a plurality of application programs and needs to perform split-screen display on the program interfaces of the application programs, the first display area provided by the electronic device to each application program is a partial display screen, and at this time, the program interfaces of the application programs are not matched with the first display area. The electronic device constructs a plurality of virtual windows matched with the display screen, and loads display information of each application program into the corresponding virtual window.
When the electronic device runs an application program and needs to display a small window on a program interface of the application program, the first display area provided by the electronic device to the application program is a partial display screen, and at this time, the program interface of the application program is not matched with the first display area. The electronic equipment constructs a virtual window matched with the application program, and loads display information of the application program into the virtual window.
FIG. 2 is a schematic diagram illustrating interaction of an application program with a virtual window in some embodiments of the application, as shown in FIG. 2, where the virtual window issues a conventional screen size reference configuration message to the application program, causing a perceived application of the application program to run in the virtual window.
And 104, displaying an application window in the first display area, wherein the application window is an interface obtained by scaling the virtual window.
In the embodiment of the application, the first display area is a display area for displaying an application window on a display screen of the electronic device, and the application window is a window obtained by zooming a virtual window. The application windows displayed by the electronic equipment correspond to the running application programs one by one, and the display content in the application windows is the same as the display content in the virtual windows.
In the embodiment of the application, the electronic equipment constructs the virtual window corresponding to the running application program, loads the display information of the application program into the virtual window, performs scaling processing on the virtual window, and displays the application window obtained by the scaling processing in the first display area, so that the application program does not need to do additional compatible processing because the application program does not sense the scaling process of the virtual window, thereby greatly reducing the adaptation workload of the application program, simultaneously, the application program does not perform operations such as real-time destroying, re-layout, drawing and the like according to the size, and the like, and realizes seamless switching and connection in a multi-window scene through scaling of the virtual window because the application cannot sense the switching of window modes, thereby improving the compatibility of the application program and solving the problems of incomplete display and inconsistent display scale of the application window of a display screen.
In some embodiments of the application, the number of applications is at least two;
loading display information of an application program into a virtual window, comprising:
loading at least two display information corresponding to at least two application programs into at least two first virtual windows, wherein the at least two first virtual windows correspond to the at least two application programs one by one;
and mixing at least two first virtual windows to obtain a second virtual window.
In the embodiment of the application, the number of the application programs running in the electronic equipment is at least two, so that a first virtual window corresponding to at least two application programs one by one is required to be constructed, at least two display information corresponding to at least two application programs is also required to be acquired, and the at least two display information is respectively loaded into the corresponding first virtual window.
At least two application programs correspond to different program interfaces, and different display information is displayed in the different program interfaces. When the electronic device runs at least two application programs, the electronic device can construct first virtual windows corresponding to the at least two application programs, and the sizes of the at least two first virtual windows are matched with the size of the display screen. In the process of respectively loading the display information of at least two application programs into the corresponding first virtual windows, the application programs are not required to carry out additional adaptation on the display size.
After the at least two first virtual windows are loaded with the display information, the at least two first virtual windows are required to be subjected to mixing processing (Wm Compos) to obtain a second virtual window, and the display position relationship of the at least two first selected windows in the second virtual window is matched with the display position relationship of the application windows finally displayed on the upper screen.
The second virtual window is a virtual window generated by splicing the plurality of first virtual windows, the second virtual window comprises a plurality of first virtual windows, and the size of the second virtual window is not matched with the size of the display screen of the electronic equipment, so that the second virtual window needs to be scaled before being displayed, and the scaled second virtual window is matched with the size of the display screen of the electronic equipment.
Specifically, a second virtual window obtained by splicing the plurality of first virtual windows is scaled to form a real screen which is finally displayed, so that the second virtual window is displayed on the display screen. In the process of zooming a second virtual window formed by a plurality of first virtual windows, the running application program does not sense, so that the display size and the display position of the application window change, but the application program does not sense the display change of the application window.
Fig. 3 is a schematic diagram of a virtual window synthesis process provided in some embodiments of the present application, where, as shown in fig. 3, a first virtual window 1, a first virtual window 2, and a first virtual window 3 perform virtual window mixing to form a second virtual window, where the second virtual window scales at a window display Layer (Surface Layer), and drives to transmit content displayed by a plurality of application windows that are finally displayed to a display screen, where the display screen displays the plurality of application windows in a corresponding first display area.
In the embodiment of the application, when the electronic equipment runs a plurality of application programs, the display information of the application programs can be loaded into the corresponding first virtual windows, and then at least two first virtual windows are mixed to obtain the second virtual window, so that the display screen of the electronic equipment can carry out split-screen display on the windows of the application programs, the application programs cannot perceive the window scaling process, and the problems that the display screen is not fully displayed on the application windows displayed by the split-screen display and the display proportion of the application windows is not consistent are solved. In some embodiments of the present application, before loading at least two display information corresponding to at least two application programs into at least two first virtual windows, the display method further includes:
constructing at least two first virtual windows and at least two cache controllers;
The at least two cache controllers determine at least two display cache information according to the display configuration information of the at least two first virtual windows;
based on the at least two display cache information, physical memory of the electronic device is allocated to the at least two first virtual windows;
Wherein the display configuration information includes at least one of: display position information, display size information, screen resolution information, screen density information, window mode information, and input map information.
In the embodiment of the application, the display configuration information is the display configuration information issued to the application programs by the virtual window, and physical memory can be allocated to the application windows corresponding to at least two application programs according to the display configuration information, so that when the display sizes and the display positions of the application windows corresponding to different application programs are different, the occupied physical memory is different, and the problem that a plurality of application windows are equally allocated with larger physical memory to bring larger load to an image processor of the electronic equipment is avoided, and the influence on terminal performance is reduced. And creating a display cache controller (Buffer Controler) corresponding to the virtual window while the electronic device creates the virtual window corresponding to each application program, and inputting display configuration information into the cache controller, wherein the cache controller can determine the display cache information corresponding to each application program.
Specifically, the display cache controller can generate corresponding display cache information according to the display configuration information, and the display cache controller can transmit the determined display cache information to the display synthesis system to adjust the display hierarchy, so that corresponding physical memory is allocated to each application window.
After the electronic device determines the display configuration information corresponding to each virtual window, parameters such as a display position, a display size and the like of the application window corresponding to the virtual window on the display screen can be determined according to the display configuration information, so that display Buffer information (Buffer) of each application program can be calculated, and the display Buffer information is virtual display Buffer information. After determining the display Buffer information, the display Buffer information is distributed to a virtual memory agent (Buffer proxy) of the application. After the electronic device determines the display configuration information, the display composition system (SurfaceFlinger) is notified to adjust the display Layer (Surface Layer), so that physical memory is allocated to each application window according to the display cache information, the application drawing resolution is reduced, and the system memory is saved. When the display buffer controller distributes display buffer information to a virtual display agent of an application program, the application program receives a vertical synchronous signal (VSYNC, vertical Vync) to draw a next frame, and the View highest level (ViewRootImpl) traverses the whole View (View) tree to draw and render actual display content on a display screen. The proxy policy may act on the rendering nodes (RenderNode) of each view, i.e., each rendering node is a mapped region that is drawn in the corresponding virtual memory. The application program uses the graphics processor to render through the programming graphics program interface (OpenGL ES/Vilkan) library, and at this time, the application program draws logically in the virtual memory buffer, compared to the drawing and rendering of the application program, so that the drawing resolution is reduced.
In the embodiment of the application, the display cache information of each application program can be accurately calculated by constructing a plurality of virtual windows and constructing the corresponding display cache controllers at the same time, so that the accuracy of distributing physical memory to each application window in the follow-up process is improved, and the workload of graphic processing is further reduced.
In the embodiment of the application, the electronic equipment can determine the display cache information of each running application program by acquiring the display configuration information corresponding to the virtual window, so that the physical memory is distributed to the application window corresponding to each application program based on the display cache information, and the drawing and rendering resolution of each application program is reduced, thereby reducing the workload of graphic processing, greatly reducing the system power consumption and providing the performance smoothness.
In some embodiments of the present application, loading display information of an application into a virtual window includes:
scaling the virtual window;
and loading the scaled virtual window into the cache region.
In the embodiment of the application, the buffer area (buffer) loaded by the scaled virtual window is a buffer area for scaling the buffer area corresponding to the virtual window before scaling. Specifically, during the process of loading display information into at least two virtual windows by the electronic device, the system of the electronic device needs to create a corresponding buffer area for loading the display information, and the buffer area does not need to be matched with the size of the virtual windows. After matrix scaling transformation is carried out on the buffer area corresponding to the virtual window before scaling, the buffer area loaded by the scaled virtual window with the actual needed display size can be obtained. After the buffer area loaded by the scaled virtual window is generated, the application program can layout the display interface and the display control according to the service of the application program, and draw the display information to the buffer area loaded by the scaled virtual window.
Illustratively, the rotation matrix includes 9 elements, each of which changes to cause a change in the image, and the corresponding relational expressions (1), (2), (3) of the rotation matrix are as follows:
wherein MSCALE _ X, MSCALE _y is a scaling parameter, MSKEW _ X, MSKEW _y is a chamfering parameter, MPERSP _0, MPERSP _1, MPERSP _2 are perspective parameters, MTRANS _ X, MTRANS _y is an offset parameter, t x、ty is a position offset value, S x、Sy is a chamfering value, and θ is a rotation angle.
Illustratively, for a single coordinate, the scaling process requires multiplying its coordinate value by a scaling value, scaling expressions (4), (5), (6) are as follows:
x＝k1×x0； (4)
y＝k2×y0； (5)
Where x is the scaled x-axis coordinate, y is the scaled y-axis coordinate, x 0 is the pre-scaled x-axis coordinate, y 0 is the pre-scaled y-axis coordinate, and k 1、k2 is the scaling factor.
In the embodiment of the application, the virtual window is scaled, the buffer corresponding to the application window actually displayed on the display screen is obtained, and the scaled virtual window is loaded into the matched buffer for on-screen display, so that the scaling process of the application program can not be perceived, and the scaling display compatibility of the application program is improved.
In some embodiments of the application, displaying the second interface includes at least one of:
At least two application windows are tiled on the second interface;
at least one application window of the at least two application windows is displayed in suspension on the second interface;
at least two application windows are displayed on the second interface in the form of dynamic controls.
In the embodiment of the application, the plurality of application windows can be tiled in the second interface, so that split-screen display of the plurality of application windows is realized, mutual extrusion among the plurality of application windows is avoided, the display aesthetic property is improved, and the interactive operation of a user on the plurality of application windows is not influenced when the application windows are tiled.
Fig. 4 illustrates one of the schematic diagrams of the second interface provided in some embodiments of the present application, where a plurality of application windows 402 are tiled on the second interface, and 3 application windows 402 are displayed in the left split screen of the second interface, and 1 application window 402 is displayed in the right split screen of the second interface, as illustrated in fig. 4.
In the embodiment of the application, at least one application window in the plurality of application windows can be displayed in the second interface in the form of a floating window, namely, one application window is displayed in the second interface in a full screen mode, the other application windows are displayed in the form of the floating window, and the operation convenience of the user is improved by displaying the application window of the application program required to be operated by the user in the full screen mode and displaying the application window required to be watched by the user in the floating window mode.
FIG. 5 illustrates a second schematic diagram of a second interface provided in some embodiments of the present application, as shown in FIG. 5, in which a first application window 502 is displayed full screen, and a second application window 504 is displayed in the second interface in the form of a floating window and is floating displayed on the first application window 502. Wherein the display position of the second application window 504 is adjustable.
In the embodiment of the application, the plurality of application windows can be displayed in the second interface in the form of dynamic control, so that the display area of the application windows in the second interface is further reduced.
Fig. 6 illustrates a third schematic diagram of a second interface provided in some embodiments of the present application, as shown in fig. 6, in which two application windows 602 are respectively displayed in the form of two dynamic controls, and an application icon 604 is further displayed in the second interface, where the display positions of the application windows 602 are adjustable.
According to the display method provided by the embodiment of the application, the execution main body can be a display device. In the embodiment of the present application, a display device is used as an example to execute a display method, and the display method provided in the embodiment of the present application is described.
In some embodiments of the present application, a display device is provided, and fig. 7 shows a schematic block diagram of the display device provided in some embodiments of the present application. As shown in fig. 7, the display device 700 includes:
the loading module 702 is configured to load display information of an application program into a virtual window when a program interface of the application program is not matched with a first display area, where the first display area is a display area that can be provided by an electronic device;
And the display module 704 is configured to display an application window in the first display area, where the application window is an interface obtained by scaling a virtual window.
In the embodiment of the application, the electronic equipment constructs the virtual window corresponding to the running application program, loads the display information of the application program into the virtual window, performs scaling processing on the virtual window, and displays the application window obtained by the scaling processing in the first display area, so that the application program does not need to do additional compatible processing because the application program does not sense the scaling process of the virtual window, thereby greatly reducing the adaptation workload of the application program, simultaneously, the application program does not perform operations such as real-time destroying, re-layout, drawing and the like according to the size, and the like, and realizes seamless switching and connection in a multi-window scene through scaling of the virtual window because the application cannot sense the switching of window modes, thereby improving the compatibility of the application program and solving the problems of incomplete display and inconsistent display scale of the application window of a display screen.
In some embodiments of the application, the number of applications is at least two;
The loading module 702 is configured to load at least two display information corresponding to at least two application programs into at least two first virtual windows, where the at least two first virtual windows correspond to the at least two application programs one by one;
The display device 700 further includes:
And the processing module is used for carrying out mixed processing on at least two first virtual windows to obtain a second virtual window.
In the embodiment of the application, when the electronic equipment runs a plurality of application programs, the display information of the application programs can be loaded into the corresponding first virtual windows, and then at least two first virtual windows are mixed to obtain the second virtual window, so that the display screen of the electronic equipment can carry out split-screen display on the windows of the application programs, the application programs cannot perceive the window scaling process, and the problems that the display screen is not fully displayed on the application windows displayed by the split-screen display and the display proportion of the application windows is not consistent are solved.
In some embodiments of the present application, the display device 700 further includes:
the building module is used for building at least two first virtual windows and at least two cache controllers;
The determining module is used for determining at least two display cache information according to the display configuration information of at least two first virtual windows by the at least two cache controllers;
the allocation module is used for allocating physical memory of the electronic equipment to the at least two first virtual windows based on the at least two display cache information;
Wherein the display configuration information includes at least one of: display position information, display size information, screen resolution information, screen density information, and window mode information.
In the embodiment of the application, the display cache information of each application program can be accurately calculated by constructing a plurality of virtual windows and constructing the corresponding display cache controllers at the same time, so that the accuracy of distributing physical memory to each application window in the follow-up process is improved, and the workload of graphic processing is further reduced.
In the embodiment of the application, the electronic equipment can determine the display cache information of each running application program by acquiring the display configuration information corresponding to the virtual window, so that the physical memory is distributed to the application window corresponding to each application program based on the display cache information, and the drawing and rendering resolution of each application program is reduced, thereby reducing the workload of graphic processing, greatly reducing the system power consumption and providing the performance smoothness.
In some embodiments of the present application, the display device 700 further includes:
The scaling module is used for scaling the virtual window;
the loading module 702 loads the scaled virtual window into the buffer.
In the embodiment of the application, the virtual window is scaled, the buffer corresponding to the application window actually displayed on the display screen is obtained, and the scaled virtual window is loaded into the matched buffer for on-screen display, so that the scaling process of the application program can not be perceived, and the scaling display compatibility of the application program is improved.
The display device in the embodiment of the application can be an electronic device or a component in the electronic device, such as an integrated circuit or a chip. The electronic device may be a terminal, or may be other devices than a terminal. The electronic device may be a Mobile phone, a tablet computer, a notebook computer, a palm computer, a vehicle-mounted electronic device, a Mobile internet appliance (Mobile INTERNET DEVICE, MID), an augmented reality (augmented reality, AR)/Virtual Reality (VR) device, a robot, a wearable device, an ultra-Mobile personal computer (UMPC), a netbook or a Personal Digital Assistant (PDA), etc., and may also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a Television (TV), a teller machine, a self-service machine, etc., which are not particularly limited in the embodiments of the present application.
The display device in the embodiment of the application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, and the embodiment of the present application is not limited specifically.
The display device provided by the embodiment of the application can realize each process realized by the embodiment of the method, and in order to avoid repetition, the description is omitted here.
In some embodiments of the present application, a display system is provided, and fig. 8 shows a schematic block diagram of the display system provided by some embodiments of the present application. As shown in fig. 8, the display system 800 includes:
A framework layer 820 for creating a virtual window corresponding to the application program at the system side and determining display configuration information of the virtual window;
a local library 840 for drawing display contents of the virtual window to the target buffer area based on the display configuration information;
the system kernel 860 is configured to add the display content in the target buffer to the video memory.
In the embodiment of the present application, the display system includes a framework layer 820, a local library 840 (Native), and a system Kernel 860 (Kernel), where the framework layer 820 is capable of creating virtual windows on a system side of the electronic device, and if the number of running applications is at least two, constructing at least two virtual windows, and determining display configuration information of each virtual window. The local library 840 can obtain the display configuration information determined by the framework layer 820 and draw the display content of the virtual window into the target buffer. The system kernel 860 can add the display content in the target buffer to the memory, and display the display content through the application window.
In the embodiment of the present application, a virtual window corresponding to an operating application program is built in the framework layer 820, display information of the application program is loaded into the virtual window, display content in the virtual window is drawn to a target buffer area based on real configuration information through the local library 840, and display content in the target buffer area is added to a video memory through the system kernel 860, so that the corresponding display content is displayed in the application window. Because the application program does not perceive the scaling process of the virtual window, the application program does not need to do additional compatible processing, the adapting workload of the application program is greatly reduced, meanwhile, the application program does not perform operations such as real-time destroying, re-layout and drawing according to the size, and the like, in a multi-window scene, seamless switching connection can be realized because the application program cannot perceive the switching of window modes, the compatibility of the application program is improved, and the problems of incomplete display of the application window and inconsistent display proportion of the display screen are solved.
In some embodiments of the present application, the framework layer 820 is further configured to create a target buffer on the system side, where the target buffer is used to buffer the scaled virtual window.
In the embodiment of the present application, the framework layer 820 can also create a target buffer area on the system side of the electronic device, where the target buffer area can buffer the scaled virtual window. The buffer area (buffer) loaded by the scaled virtual window is a buffer area for scaling the buffer area corresponding to the virtual window before scaling. Specifically, during the process of loading display information into at least two virtual windows by the electronic device, the system of the electronic device needs to create a corresponding buffer area for loading the display information, and the buffer area does not need to be matched with the size of the virtual windows. After matrix scaling transformation is carried out on the buffer area corresponding to the virtual window before scaling, the buffer area loaded by the scaled virtual window with the actual needed display size can be obtained. After the buffer area loaded by the scaled virtual window is generated, the application program can layout the display interface and the display control according to the service of the application program, and draw the display information to the buffer area loaded by the scaled virtual window.
In the embodiment of the application, the virtual window is scaled, the buffer corresponding to the application window actually displayed on the display screen is obtained, and the scaled virtual window is loaded into the matched buffer for on-screen display, so that the scaling process of the application program can not be perceived, and the scaling display compatibility of the application program is improved.
In some embodiments of the application, the frame layer 820 includes: a cache controller 822 for transmitting display configuration information to the local repository 840; the local library 840 includes: the display composition system 842 is configured to allocate physical memory to the virtual window based on the display configuration information.
In an embodiment of the present application, the framework layer 820 includes a cache controller 822 (Buffer Controler), where the cache controller 822 is capable of transmitting display configuration information to the local library 840. The local library 840 is configured with a display composition system 842 (Surface Flinger), the display composition system 842 is capable of retrieving display configuration information, and the display composition system 842 is capable of allocating physical memory for a virtual window based on the display configuration information. The display cache controller 822 can generate corresponding display cache information according to the display configuration information, and the display cache controller 822 can transmit the determined display cache information to the display composition system 842 to adjust the display hierarchy, so as to allocate corresponding physical memory for each application window.
In the embodiment of the application, by constructing a plurality of virtual windows and constructing the corresponding display buffer controller 822 at the same time, the display buffer information of each application program can be accurately calculated, so that the accuracy of distributing physical memory to each application window in the follow-up process is improved, and the workload of graphic processing is further reduced.
In some embodiments of the application, the frame layer 820 further comprises:
the virtual memory agent 824 is configured to configure a drawing policy of the application program according to the display configuration information, and transmit the drawing policy to the local library 840;
The local library 840 further includes:
Graphics engine 844 for rendering the display content of the virtual window to the target buffer by programming the graphics program interface to invoke the graphics processor based on the rendering policy.
In the embodiment of the present application, the framework layer 820 further includes a virtual memory agent 824 (Buffer proxy), and the virtual memory agent 824 can configure a drawing policy of an application program based on the display configuration information after obtaining the display configuration information, and transmit the drawing policy to the local repository 840. Also included in the local library 840 is a graphics engine 844 (HWUI), the graphics engine 844 invoking a Graphics Processor (GPU) via a programmed graphics program interface (OpenGL ES/Vilkan) to render the display of the virtual window to the target buffer based on rendering policies.
Illustratively, as shown in FIG. 8, the display system is capable of executing a display application window, the method of displaying an application window comprising:
step 201, creating a virtual window for each application, and simultaneously creating a virtual video memory controller corresponding to the virtual window;
In the embodiment of the application, virtual windows corresponding to the application programs one by one are created in the frame layer, and virtual video memory controllers corresponding to the virtual windows one by one are arranged in the frame layer, and under different window modes, the positions and the sizes of the application programs and the display screen are different, and the virtual video memory controllers can calculate proper cache information of the virtual video memory according to the actual display positions, the virtual window sizes, the resolution and the display density of the display screen and the current window mode, and package the cache information of the virtual video memory into real configuration information to be sent to a virtual video memory agent of an application end.
Step 202, after the virtual video memory controller calculates the display configuration information, notifying the display synthesis system to allocate physical memory according to the virtual video memory information;
In the embodiment of the application, the virtual video memory controller in the frame Layer can transmit the display configuration information to the display synthesis system, and the display synthesis system adjusts the display Surface Layer (Surface Layer) to allocate the actual physical memory according to the virtual video memory information, thereby saving the system memory.
Step 203, distributing the real configuration information to a virtual video memory agent of an application side, and configuring a drawing strategy of the application side;
in the embodiment of the application, the framework layer can transmit the display configuration information to the virtual video memory agent, and View RootImpl (the highest level of View) can traverse the whole View tree to draw and render the actual display content on the display screen when the application layer receives the Vsync (Vertical Synchronization ) signal of the system to draw the next frame of image. The drawing policy of the virtual video memory agent acts on the rendering node of each view, that is, each rendering node draws in the mapping area of the corresponding virtual video memory.
Step 204, the application uses the graphics processor to render through the programming graphics program interface library;
in step 205, the display content buffer drives the display content delivery.
In the embodiment of the application, the graphic engine in the local library calls the graphic processor to render through the programming graphic program interface library, and the graphic engine is logically drawn in the buffer area of the virtual video memory instead of the actual size of the virtual window in comparison with the drawing and rendering of the application program, so that the resolution is reduced, the workload of the graphic processor is reduced, the system power consumption is greatly reduced, and the smoothness of the performance is provided.
Optionally, the embodiment of the present application further provides an electronic device, which includes the display device in any of the embodiments, so that the electronic device has all the advantages of the display device in any of the embodiments, and will not be described in detail herein.
Optionally, an electronic device is further provided in the embodiment of the present application, fig. 9 shows a block diagram of a structure of the electronic device according to the embodiment of the present application, as shown in fig. 9, the electronic device 900 includes a processor 902, a memory 904, and a program or an instruction stored in the memory 904 and capable of running on the processor 902, where the program or the instruction implements each process of the foregoing display method embodiment when executed by the processor 902, and the process can achieve the same technical effect, and is not repeated herein.
The electronic device in the embodiment of the application includes the mobile electronic device and the non-mobile electronic device.
Fig. 10 is a schematic diagram of a hardware structure of an electronic device implementing an embodiment of the present application.
The electronic device 1000 includes, but is not limited to: radio frequency unit 1001, network module 1002, audio output unit 1003, input unit 1004, sensor 1005, display unit 1006, user input unit 1007, interface unit 1008, memory 1009, and processor 1010.
Those skilled in the art will appreciate that the electronic device 1000 may also include a power source (e.g., a battery) for powering the various components, which may be logically connected to the processor 1010 by a power management system to perform functions such as managing charge, discharge, and power consumption by the power management system. The electronic device structure shown in fig. 10 does not constitute a limitation of the electronic device, and the electronic device may include more or less components than shown, or may combine certain components, or may be arranged in different components, which are not described in detail herein.
The processor 1010 is configured to load display information of an application program into the virtual window when a program interface of the application program is not matched with a first display area, where the first display area is a display area that can be provided by the electronic device;
And the processor 1010 is configured to display an application window in the first display area, where the application window is an interface obtained by scaling a virtual window.
In the embodiment of the application, the electronic equipment constructs the virtual window corresponding to the running application program, loads the display information of the application program into the virtual window, performs scaling processing on the virtual window, and displays the application window obtained by the scaling processing in the first display area, so that the application program does not need to do additional compatible processing because the application program does not sense the scaling process of the virtual window, thereby greatly reducing the adaptation workload of the application program, simultaneously, the application program does not perform operations such as real-time destroying, re-layout, drawing and the like according to the size, and the like, and realizes seamless switching and connection in a multi-window scene through scaling of the virtual window because the application cannot sense the switching of window modes, thereby improving the compatibility of the application program and solving the problems of incomplete display and inconsistent display scale of the application window of a display screen.
Further, the number of applications is at least two; the processor 1010 is configured to load at least two display information corresponding to at least two application programs into at least two first virtual windows, where the at least two first virtual windows correspond to the at least two application programs one by one;
And a processor 1010, configured to perform a mixing process on at least two first virtual windows to obtain a second virtual window.
In the embodiment of the application, when the electronic equipment runs a plurality of application programs, the display information of the application programs can be loaded into the corresponding first virtual windows, and then at least two first virtual windows are mixed to obtain the second virtual window, so that the display screen of the electronic equipment can carry out split-screen display on the windows of the application programs, the application programs cannot perceive the window scaling process, and the problems that the display screen is not fully displayed on the application windows displayed by the split-screen display and the display proportion of the application windows is not consistent are solved.
Further, a processor 1010 configured to construct at least two first virtual windows, and at least two cache controllers;
A processor 1010, configured to determine at least two display buffer information according to display configuration information of at least two first virtual windows by using at least two buffer controllers;
a processor 1010 configured to allocate physical memory of the electronic device to at least two first virtual windows based on at least two display cache information;
Wherein the display configuration information includes at least one of: display position information, display size information, screen resolution information, screen density information, and window mode information.
In the embodiment of the application, the display cache information of each application program can be accurately calculated by constructing a plurality of virtual windows and constructing the corresponding display cache controllers at the same time, so that the accuracy of distributing physical memory to each application window in the follow-up process is improved, and the workload of graphic processing is further reduced. In the embodiment of the application, the electronic equipment can determine the display cache information of each running application program by acquiring the display configuration information corresponding to the virtual window, so that the physical memory is distributed to the application window corresponding to each application program based on the display cache information, and the drawing and rendering resolution of each application program is reduced, thereby reducing the workload of graphic processing, greatly reducing the system power consumption and providing the performance smoothness.
Further, the processor 1010 is configured to perform scaling processing on the first buffer area of the virtual window, and determine a second buffer area;
A processor 1010 for loading display information into the second buffer.
In the embodiment of the application, the virtual window is scaled, the buffer corresponding to the application window actually displayed on the display screen is obtained, and the scaled virtual window is loaded into the matched buffer for on-screen display, so that the scaling process of the application program can not be perceived, and the scaling display compatibility of the application program is improved.
It should be understood that in an embodiment of the present application, the input unit 1004 may include a graphics processor (Graphics Processing Unit, GPU) 10041 and a microphone 10042, and the graphics processor 10041 processes image data of a still picture or a motion file obtained by an image capturing device (e.g., a camera) in a motion file capturing mode or an image capturing mode. The display unit 1006 may include a display panel 10061, and the display panel 10061 may be configured in the form of a liquid crystal display, an organic light emitting diode, or the like. The user input unit 1007 includes at least one of a touch panel 10071 and other input devices 10072. The touch panel 10071 is also referred to as a touch screen. The touch panel 10071 can include two portions, a touch detection device and a touch controller. Other input devices 10072 may include, but are not limited to, a physical keyboard, function keys (e.g., volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, and so forth, which are not described in detail herein.
The memory 1009 may be used to store software programs as well as various data. The memory 1009 may mainly include a first memory area storing programs or instructions and a second memory area storing data, wherein the first memory area may store an operating system, application programs or instructions (such as a sound playing function, an image playing function, etc.) required for at least one function, and the like. Further, the memory 1009 may include volatile memory or nonvolatile memory, or the memory 1009 may include both volatile and nonvolatile memory. The nonvolatile Memory may be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically Erasable EPROM (EEPROM), or a flash Memory. The volatile memory may be random access memory (Random Access Memory, RAM), static random access memory (STATIC RAM, SRAM), dynamic random access memory (DYNAMIC RAM, DRAM), synchronous Dynamic Random Access Memory (SDRAM), double data rate Synchronous dynamic random access memory (Double DATA RATE SDRAM, DDRSDRAM), enhanced Synchronous dynamic random access memory (ENHANCED SDRAM, ESDRAM), synchronous link dynamic random access memory (SYNCH LINK DRAM, SLDRAM), and Direct random access memory (DRRAM). Memory 1009 in embodiments of the application includes, but is not limited to, these and any other suitable types of memory.
The processor 1010 may include one or more processing units; optionally, the processor 1010 integrates an application processor that primarily processes operations involving an operating system, user interface, application programs, and the like, and a modem processor that primarily processes wireless communication signals, such as a baseband processor. It will be appreciated that the modem processor described above may not be integrated into the processor 1010.
The embodiment of the application also provides a readable storage medium, and the readable storage medium stores a program or an instruction, which when executed by a processor, implements each process of the above method embodiment, and can achieve the same technical effects, so that repetition is avoided, and no further description is provided herein.
The processor is a processor in the electronic device in the above embodiment. Readable storage media include computer readable storage media such as computer readable memory ROM, random access memory RAM, magnetic or optical disks, and the like.
The embodiment of the application further provides a chip, the chip comprises a processor and a communication interface, the communication interface is coupled with the processor, the processor is used for running programs or instructions, the processes of the embodiment of the display method can be realized, the same technical effects can be achieved, and the repetition is avoided, and the description is omitted here.
It should be understood that the chips referred to in the embodiments of the present application may also be referred to as system-on-chip chips, chip systems, or system-on-chip chips, etc.
Embodiments of the present application provide a computer program product stored in a storage medium, where the program product is executed by at least one processor to implement the respective processes of the embodiments of the display method described above, and achieve the same technical effects, and for avoiding repetition, a detailed description is omitted herein.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, apparatus, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, apparatus, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, apparatus, article or device that comprises the element. Furthermore, it should be noted that the apparatus and the scope of the apparatus in the embodiments of the present application are not limited to performing the functions in the order shown or discussed, but may also include performing the functions in a substantially simultaneous manner or in an opposite order depending on the functions involved, e.g., the described apparatus may be performed in an order different from that described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.
From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment device may be implemented by means of software plus a necessary general hardware platform, or may be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present application may be embodied essentially or in part in the form of a computer software product stored on a storage medium (e.g. ROM/RAM, magnetic disk, optical disk) comprising instructions for causing a terminal (which may be a mobile phone, a computer, a server, or a network device, etc.) to perform the means of the embodiments of the present application.
The embodiments of the present application have been described above with reference to the accompanying drawings, but the present application is not limited to the above-described embodiments, which are merely illustrative and not restrictive, and many forms may be made by those having ordinary skill in the art without departing from the spirit of the present application and the scope of the claims, which are to be protected by the present application.
Claims (13)
1. A display method, characterized in that the display method comprises:
Under the condition that a program interface of an application program is not matched with a first display area, loading display information of the application program into a virtual window, wherein the first display area is a display area which can be provided by electronic equipment;
And displaying an application window in the first display area, wherein the application window is an interface obtained by scaling the virtual window.
2. The display method according to claim 1, wherein the number of the application programs is at least two;
The loading the display information of the application program into the virtual window comprises the following steps:
Loading at least two display information corresponding to at least two application programs into at least two first virtual windows, wherein the at least two first virtual windows are in one-to-one correspondence with the at least two application programs;
and mixing the at least two first virtual windows to obtain a second virtual window.
3. The display method according to claim 2, wherein before loading at least two display information corresponding to at least two of the application programs into at least two first virtual windows, respectively, the method further comprises:
constructing the at least two first virtual windows and at least two cache controllers;
the at least two cache controllers determine at least two display cache information according to the display configuration information of the at least two first virtual windows;
Based on the at least two display cache information, distributing physical memory of the electronic equipment to the at least two first virtual windows;
Wherein the display configuration information includes at least one of: display position information, display size information, screen resolution information, screen density information, window mode information, and input map information.
4. A display method according to any one of claims 1 to 3, wherein loading display information of the application program into a virtual window comprises:
scaling the virtual window;
and loading the scaled virtual window into a cache region.
5. A display device, characterized in that the display device comprises:
The loading module is used for loading the display information of the application program into the virtual window under the condition that the program interface of the application program is not matched with a first display area, wherein the first display area is a display area which can be provided by the electronic equipment;
and the display module is used for displaying an application window in the first display area, wherein the application window is an interface obtained by scaling the virtual window.
6. The display device of claim 5, wherein the number of applications is at least two;
The loading module is used for loading at least two display information corresponding to at least two application programs into at least two first virtual windows respectively, wherein the at least two first virtual windows are in one-to-one correspondence with the at least two application programs;
The display device further includes:
And the processing module is used for carrying out mixed processing on the at least two first virtual windows to obtain a second virtual window.
7. The display device according to claim 6, further comprising:
The building module is used for building the at least two first virtual windows and the at least two cache controllers;
the determining module is used for determining at least two display cache information according to the display configuration information of the at least two first virtual windows by the at least two cache controllers;
The allocation module is used for allocating physical memory of the electronic equipment to the at least two first virtual windows based on the at least two display cache information;
Wherein the display configuration information includes at least one of: display position information, display size information, screen resolution information, screen density information, and window mode information.
8. The display device according to any one of claims 5 to 7, further comprising:
the scaling module is used for scaling the virtual window;
And the loading module is used for loading the scaled virtual window into the cache region.
9. A display system, the display system comprising:
The framework layer is used for creating a virtual window corresponding to the application program at the system side and determining display configuration information of the virtual window;
the local library is used for drawing the display content of the virtual window to a target cache region based on the display configuration information;
and the system kernel is used for adding the display content in the target cache region into a video memory.
10. The display system of claim 9, wherein the display system is configured to display the plurality of images,
The frame layer is further configured to create the target cache area on the system side, where the target cache area is used to cache the scaled virtual window.
11. The display system of claim 9, wherein the frame layer comprises:
the cache controller is used for transmitting the display configuration information to the local library;
the local library includes:
And the display synthesis system is used for distributing physical memory to the virtual window based on the display configuration information.
12. The display system of any one of claims 9 to 11, wherein the frame layer further comprises:
The virtual video memory agent is used for configuring the drawing strategy of the application program according to the display configuration information and transmitting the drawing strategy to the local library;
the local library further comprises:
And the graphic engine is used for calling a graphic processor to draw the display content of the virtual window to the target cache region through a programming graphic program interface based on the drawing strategy.
13. An electronic device, comprising:
A processor and a memory storing a program or instructions executable on the processor, which when executed by the processor, implement the steps of the method of any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410371284.9A CN118377561A (en) | 2024-03-29 | 2024-03-29 | Display method, device, system and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410371284.9A CN118377561A (en) | 2024-03-29 | 2024-03-29 | Display method, device, system and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN118377561A true CN118377561A (en) | 2024-07-23 |
Family
ID=91909077
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410371284.9A Pending CN118377561A (en) | 2024-03-29 | 2024-03-29 | Display method, device, system and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN118377561A (en) |
-
2024
- 2024-03-29 CN CN202410371284.9A patent/CN118377561A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2022110903A1 (en) | Method and system for rendering panoramic video | |
| WO2018161957A1 (en) | Method and device for layer drawing control, and mobile terminal | |
| US9489165B2 (en) | System and method for virtual displays | |
| US7002604B1 (en) | Screen rotation | |
| EP2245598B1 (en) | Multi-buffer support for off-screen surfaces in a graphics processing system | |
| US9009595B2 (en) | User manipulation of video feed to computer screen regions | |
| CN1329870C (en) | Block-based rotation of arbitrary-shaped images | |
| KR102304891B1 (en) | Method and system for generating mask overlay for display panel corresponding to touch path | |
| CN110865758B (en) | Display method and electronic equipment | |
| EP2997547B1 (en) | Primitive-based composition | |
| US11024257B2 (en) | Android platform based display device and image display method thereof | |
| CN116821040B (en) | Display acceleration method, device and medium based on GPU direct memory access | |
| CN112035877A (en) | Information hiding method and device, electronic equipment and readable storage medium | |
| US20180260930A1 (en) | Graphics Processing using Multiple Primitives | |
| CN116107531A (en) | Interface display method and device | |
| CN114416269A (en) | Interface display method and display device | |
| CN114339072A (en) | Image processing circuit, method and electronic device | |
| US20060187239A1 (en) | System and method for improving visual appearance of efficient rotation algorithm | |
| CN116347166A (en) | Display device and window display method | |
| CN115729544A (en) | Desktop component generation method and device, electronic equipment and readable storage medium | |
| CN118377561A (en) | Display method, device, system and electronic equipment | |
| CN111813404B (en) | Application method, medium and client based on mixed graphic display | |
| CN116048238A (en) | Display control method, display control device, electronic equipment and storage medium | |
| CN117931034A (en) | Display device and image generation method | |
| CN117372597A (en) | Animation rendering method and device, electronic equipment and readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |