CN117453082A - Display method of selected area on virtual desktop and cloud desktop system - Google Patents
Display method of selected area on virtual desktop and cloud desktop system Download PDFInfo
- Publication number
- CN117453082A CN117453082A CN202311583196.7A CN202311583196A CN117453082A CN 117453082 A CN117453082 A CN 117453082A CN 202311583196 A CN202311583196 A CN 202311583196A CN 117453082 A CN117453082 A CN 117453082A
- Authority
- CN
- China
- Prior art keywords
- pointer
- cloud server
- cloud
- operation instruction
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 53
- 238000004891 communication Methods 0.000 claims abstract description 7
- 230000002457 bidirectional effect Effects 0.000 claims abstract description 6
- 238000013139 quantization Methods 0.000 claims description 2
- 230000002194 synthesizing effect Effects 0.000 claims description 2
- 238000005457 optimization Methods 0.000 description 4
- 230000005540 biological transmission Effects 0.000 description 3
- 230000006835 compression Effects 0.000 description 3
- 238000007906 compression Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 108091026890 Coding region Proteins 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
Abstract
The invention discloses a display method of a selected area on a virtual desktop, which can be applied to a cloud desktop system. The cloud desktop system comprises a cloud server and a cloud terminal which can be in bidirectional data communication with the cloud server. The display method of the selected area on the virtual desktop comprises the following steps: and the cloud server receives the pointer operation instruction and pointer position information from the cloud terminal. And obtaining the current display area according to the pointer operation instruction and the pointer position information. And obtaining an area image capable of being displayed on the cloud terminal through the display area. According to the method and the device for displaying the page by the cloud terminal, the cloud terminal receives the pointer position information and the pointer operation instruction from the cloud terminal, so that browsing of a selected area of a user of the cloud terminal on a display interface can be guaranteed, high usability of the displayed page of the cloud terminal is improved, and browsing of the user is guaranteed to be an implementation operation.
Description
Technical Field
The present invention relates to the technical field of user interfaces. The method is particularly applied to a cloud desktop system and a cloud platform, and particularly relates to a display method of a selected area on a virtual desktop and the cloud desktop system.
Background
The display content on the local desktop is connected with the host through the interface of the display screen, and the host pushes the display picture to the display, so that the image quality is not lost. And the content displayed on the cloud desktop is compressed and decompressed through video coding, so that the image quality is lost. And as the bandwidth of the network transmission changes, the video compression ratio becomes higher and the display quality becomes worse as the bandwidth becomes smaller. Especially when the user needs to view the local area of the user interface, if the user cannot clearly display the local area, the user's viewing and subsequent operations are affected.
Disclosure of Invention
In order to at least partially solve the technical problems mentioned in the background art, an object of the present invention is to provide a display method of a selected area on a virtual desktop and a cloud desktop system. By using the method disclosed by the invention, the menu display area of the selected area can be formed according to the pointer and the selection instruction of the pointer, so that the display of the user interface is optimized in real time.
According to the first aspect of the invention, a display method of a selected area on a virtual desktop is provided, and the display method can be applied to a cloud desktop system. The cloud desktop system comprises a cloud server and a cloud terminal which can be in bidirectional data communication with the cloud server.
The display method of the selected area on the virtual desktop comprises the following steps: and the cloud server receives the pointer operation instruction and pointer position information from the cloud terminal. And obtaining the current display area according to the pointer operation instruction and the pointer position information. And obtaining an area image capable of being displayed on the cloud terminal through the display area.
The second aspect of the invention provides a cloud desktop system, which comprises a cloud server and a cloud terminal capable of two-way data communication with the cloud server.
The cloud server is configured to receive pointer position information and pointer operation instructions from the cloud terminal. And obtaining the current display area according to the pointer position information and the pointer operation instruction. And obtaining an area image capable of being displayed on the cloud terminal through the display area.
The cloud server configures a video encoder, and the cloud terminal configures a video decoder.
The cloud terminal is further configured to be capable of displaying the virtual desktop image acquired from the cloud server.
The cloud server is further configured to receive pointer position information and pointer operation instructions from the cloud terminal, wherein the pointer position information and the pointer operation instructions can be displayed on the virtual desktop image. And obtaining the current display area according to the shortcut function menu area obtained by the pointer position information and the pointer operation instruction.
The video encoder is configured to obtain region encoding parameters for a current display region at a set resolution. And removing partial codes of the current display area from the virtual desktop image to obtain desktop coding parameters.
The cloud server is further configured to synthesize the virtual desktop image to be displayed into a video stream through the region encoding parameters and the desktop encoding parameters.
And a video decoder configured to identify the video stream as a virtual desktop image to be displayed.
The cloud terminal is further configured to locally display the virtual desktop image to be displayed.
By utilizing the scheme disclosed in the aspects of the disclosure, the pointer operation instruction and the pointer position information are received from the cloud terminal through the cloud server. And obtaining the current display area according to the pointer operation instruction and the pointer position information. The area image which can be displayed on the cloud terminal is obtained through the display area, so that when a user browses a user interface, the area selected by the user is highlighted, and then the current display area can be encoded according to different resolutions through video encoding parameters, so that the cloud terminal can be ensured to normally browse, particularly browse the user attention area, while the network transmission flow is limited, the usability of the cloud terminal for displaying the page is improved, the image quality of the display area is ensured, and the user's viewing and subsequent use of the user interface are ensured.
Drawings
Fig. 1 is a flow chart for explaining a display method of a selected area on a virtual desktop in an embodiment of the present invention.
Fig. 2 is a schematic diagram for explaining the composition of a cloud desktop system in one embodiment of the present invention.
FIG. 3 is a schematic diagram illustrating a selected area of a pointer on a virtual desktop in one embodiment of the present invention.
FIG. 4 is a schematic diagram illustrating a selected area of a pointer on a virtual desktop in another embodiment of the present invention.
Detailed Description
For a clearer understanding of the technical features, objects and effects of the present invention, embodiments of the present invention will now be described with reference to the drawings, in which like reference numerals refer to identical or structurally similar but functionally identical components throughout the separate views.
In this document, "schematic" means "serving as an example, instance, or illustration," and any illustrations, embodiments described herein as "schematic" should not be construed as a more preferred or advantageous solution. For simplicity of the drawing, only the portions related to the present exemplary embodiment are schematically shown in the drawings, and they do not represent actual structures and actual proportions thereof as products.
According to the first aspect of the invention, a display method of a selected area on a virtual desktop is provided, and the display method can be applied to a cloud desktop system. The cloud desktop system comprises a cloud server 101 and a cloud terminal 201 which can be in bidirectional data communication with the cloud server.
As shown in fig. 1, the method for displaying the selected area on the virtual desktop includes:
in step S101, the cloud server 101 receives the pointer operation instruction and pointer position information from the cloud terminal 201.
Step S102, the current display area is obtained according to the pointer operation instruction and the pointer position information. As shown in fig. 3, when the pointer 10 is positioned at the position 11 of the screen 20, after the left key is pressed, the pointer is dragged to the pointer position 12 along the locus 13, and a rectangular setting display area 30 is formed between the pointer positions 11 and 12. Thereby setting the display area as the current display area.
In step S103, an area image that can be displayed on the cloud terminal 201 is obtained from the display area.
The display method of the selected area on the virtual desktop is applied to a cloud desktop system, a user terminal is arranged on a cloud terminal 201, display content is generated by a display card at one end of a cloud server 101101, and unlike a traditional computer, the display content is not directly transmitted to a client through a display interface. This is because clients tend to be on the other side of the network, transmitting very far away, and transmitting display content directly would take up excessive network bandwidth.
The cloud server 101 encodes the video before transmitting the display content to achieve the goal of compressing the data amount, and transmits the encoded video to the client through the network. After receiving the video, the client decodes the video first and then displays the decoded screen content on a local display screen through a display interface.
When it is determined that the user uses the computer display through the cloud terminal 201, if an instruction or operation to move to another point after clicking the left button of the pointer on the currently operated desktop is performed, the stay position of the pointer of the user and the current display area are perceived. According to the invention, the operating system and software of the cloud desktop system can obtain the position coordinates of the pointer in the current screen through the software interface and the hardware interface.
The method is realized by a setting algorithm, and the coordinate information of the current display area selected by the pointer, namely the coordinate information of the current display area, is obtained by taking the pointer position and the moved pointer information, namely the coordinate information of two stopped pointers, as references.
The current display area range information is converted into coding parameters acceptable by a video encoder through a setting algorithm. When the video encoder is invoked, perceptual coding parameters are added in addition to conventional video coding parameters. Therefore, the high-resolution coding of the sensitive area, namely the current display area, is realized, the bandwidth requirement of network transmission is adapted to the maximum extent, and the continuous high-network bandwidth support is not needed while the range resolution of the current display area is ensured.
In another embodiment of the present invention, a method for displaying a selected area on a virtual desktop further includes.
The cloud server 101 obtains the bottom layer user application and the bottom layer user application interface information which are called by the current cloud terminal 201 in the cloud server 101. The bottom layer user application interface information is interface identification information which is used by a user, can be displayed by the bottom layer application in the cloud server 101.
The bottom layer user application interface information is interface identification information which is used by a user, can be displayed by the bottom layer application in the cloud server 101.
For example, the user operating system configured in the cloud server 101 is a Windows10 operating system or a macOS operating system, which corresponds to an underlying user application, and an interface of the Windows10 operating system or the macOS operating system corresponds to an underlying user application interface. The information of the underlying user application interface is equivalent to the system configuration information of the Windows10 operating system or the macOS operating system, so that the cloud server 101 can learn what the operating system is currently used by the user.
In one embodiment of the present invention, the method for displaying the selected area on the virtual desktop further includes:
the cloud server 101 receives a pointer operation instruction set of an underlying user application from the cloud terminal 201. The pointer operation instruction set includes a bottom layer interface region selection instruction. The bottom layer interface area selection instruction comprises the following steps: and generating a selection area in the first click operation instruction and the pointer movement operation instruction of the pointer.
And calling a corresponding bottom layer user application interface according to the bottom layer user application interface information.
The cloud server 101 receives the pointer first click operation instruction from the cloud terminal 201 on the underlying user application interface, and obtains pointer position information of the first click from the pointer first click operation instruction.
The cloud server 101 receives the pointer movement operation instruction from the cloud terminal 201 and obtains the pointer position information after movement from the pointer movement operation instruction. The cloud server 101 generates a selection area on the bottom layer user interface according to the pointer position information of the first click and the pointer position information after movement. The cloud server 101 obtains the current display area according to the selected area.
For example, after clicking the left button of the mouse on the desktop of the Windows10 operating system or the macOS operating system, it is slid from the first position to the second position, and the two opposite points are taken as a rectangle according to the first position and the second position. A selection area, i.e., a current display area, is generated on the underlying user interface. Therefore, the user side can clearly see the current display area, and the user can conveniently browse subsequent operations.
In another embodiment of the present invention, a method for displaying a selected area on a virtual desktop includes:
the bottom layer interface region selection instruction comprises a first click release operation instruction, so that the bottom layer interface region selection instruction is ended. After receiving the pointer movement operation instruction from the cloud terminal 201, if the pointer first clicks to release the operation instruction, the cloud server 101 ends the bottom layer interface region selection instruction and releases the current display region. Therefore, the selection of the current display area is finished through the lifting of the left button of the mouse, and the processing resources of the system are saved.
In another embodiment of the present invention, a method for displaying a selected area on a virtual desktop includes:
the cloud server 101 obtains the desktop icon position from the underlying user application interface. The cloud server 101 determines whether the selected area includes the desktop icon according to the desktop icon position, and if so, obtains the current display area according to the area where the desktop icon is located. Thereby improving the display of icon content on the desktop.
In another embodiment of the present invention, a method for displaying a selected area on a virtual desktop includes:
the cloud server 101 obtains the word processing application and word processing application interface information loaded by the user on the underlying user application in the cloud server 101 at the current cloud terminal 201. Word processing applications include word, windows self-contained text editors, WPS, and other word processing applications, for example.
The cloud server 101 receives a pointer operation instruction set of a word processing application from the cloud terminal 201. The pointer operation instruction set of the word processing application includes a word area selection instruction. The text region selection instruction includes: and generating a text line selection area in the text field clicking operation instruction and the text field clicking release operation instruction.
And calling a corresponding word processing application interface according to the word processing application interface information.
The cloud server 101 receives a word segment clicking operation instruction from the cloud terminal 201 on the word processing application interface, and obtains pointer position information of a word segment start from the word segment clicking operation instruction.
The cloud server 101 receives the text field click release operation instruction from the cloud terminal 201 and obtains pointer position information of the end of the text field from the text field click release operation instruction.
The cloud server 101 generates a text line selection area on the user interface according to pointer position information of a text field and pointer position information of a text segment. The cloud server 101 obtains the current display area according to the text line selection area. For example, a few lines of text content are selected by a cursor. Thereby facilitating the display of key paragraphs and text contents.
In another embodiment of the present invention, the method for displaying a selected area on a virtual desktop further includes:
the cloud server 101 obtains the image processing application and the image processing application interface information loaded on the underlying user application in the cloud server 101 by the current cloud terminal 201. Graphics processing applications include, for example, image processing applications such as CorelDRAW, photoshop, CAD, UG.
The cloud server 101 receives a pointer operation instruction set of an image processing application from the cloud terminal 201. The pointer operation instruction set of the image processing application includes an image region selection instruction. The image region selection instruction comprises the following steps: clicking operation instructions at the image initial selection position and operation instructions for moving and staying at the image end position, and generating an image selection area at the image initial selection position and the operation instructions for moving and staying at the image end position.
And calling a corresponding image processing application interface according to the image processing application interface information.
The cloud server 101 receives an image start selection position clicking operation instruction from the cloud terminal 201 on the image processing application interface, and obtains pointer position information of an image start from the image start selection position clicking operation instruction.
The cloud server 101 receives the move-and-stay-to-image-end position operation instruction from the cloud terminal 201 and obtains pointer position information of the end of the image from the move-and-stay-to-image-end position operation instruction.
The cloud server 101 generates an image selection area on the user interface according to the pointer position information of the image start and the pointer position information of the image end. The cloud server 101 obtains a current display area according to the image selection area. Thereby forming a selected current display area on the interface of the current graphics processing software.
In another embodiment of the present invention, the method for displaying a selected area on a virtual desktop further includes:
the cloud server 101 obtains the display line segment position from the image processing application interface. The cloud server 101 determines whether the selected area includes a display line segment according to the display line segment position, and if yes, obtains a current display area according to the area where the display line segment is located. Therefore, the definition of the lines in the graphic processing software or the playing software is ensured.
In another embodiment of the present invention, the method for displaying a selected area on a virtual desktop further includes: the pointer operation instruction comprises a left key click instruction, a right key click instruction, a left key continuous click instruction and a right key continuous click instruction of the mouse.
The method comprises a pen point single click instruction, a pen point continuous click instruction, a pen side case single click instruction, a set function key single click instruction and a continuous click instruction on a keyboard, a left key single click instruction, a right key single click instruction, a left key continuous click instruction and a right key continuous click instruction on a notebook computer touch pad of a handwriting pen.
In another embodiment of the present invention, the method for displaying a selected area on a virtual desktop further includes: the cloud terminal 201 can display a virtual desktop image acquired from the cloud server 101.
The display method of the selected area on the virtual desktop comprises the following steps:
the cloud server 101 receives a pointer operation instruction and pointer position information that can be displayed on a virtual desktop image from the cloud terminal 201.
And obtaining the current display area according to the pointer operation instruction and the pointer position information.
The region coding parameters of the current display region at the set resolution are obtained. And removing partial codes of the current display area from the virtual desktop image to obtain desktop coding parameters. And synthesizing the virtual desktop image to be displayed into a video stream through the region coding parameters and the desktop coding parameters.
The video decoder of the cloud terminal 201 can recognize the video stream as a virtual desktop image to be displayed. The cloud terminal 201 locally displays a virtual desktop image to be displayed.
Setting the resolution includes, more than the cloud terminal 201 displaying the page resolution and less than the cloud terminal 201 displaying the page resolution. And obtaining the coding parameters of the current display area under the set resolution by using an ROI method or a QP MAP quantization parameter adjustment method.
In a second aspect of the present invention, as shown in fig. 2, a cloud desktop system is provided, which includes a cloud server 101 and a cloud terminal 201 capable of bidirectional data communication with the cloud server.
The cloud server 101 is configured to receive pointer position information and pointer operation instructions from the cloud terminal 201. And obtaining the current display area according to the pointer position information and the pointer operation instruction. An area image that can be displayed on the cloud terminal 201 is obtained by displaying the area.
Cloud server 101 configures the video encoder and cloud terminal 201 configures the video decoder.
The cloud terminal 201 is also configured to be able to display a virtual desktop image acquired from the cloud server 101.
The cloud server 101 is further configured to receive pointer position information and pointer operation instructions that can be displayed on the virtual desktop image from the cloud terminal 201. And obtaining the current display area according to the shortcut function menu area obtained by the pointer position information and the pointer operation instruction.
The video encoder is configured to obtain region encoding parameters for a current display region at a set resolution. And removing partial codes of the current display area from the virtual desktop image to obtain desktop coding parameters.
The cloud server 101 is further configured to synthesize the virtual desktop image to be displayed into a video stream through the region encoding parameter and the desktop encoding parameter.
And a video decoder configured to identify the video stream as a virtual desktop image to be displayed.
The cloud terminal 201 is further configured to locally display a virtual desktop image to be displayed.
In another embodiment of the present invention, a user need first needs to specify a user requirement, that is, if the user clicks the left button to do a selection operation without releasing the left button when using and watching the screen content of the display, and the user needs to pay attention to the display content in the selection range of the mouse, and the interest range of the attention will be in the selection content area of the mouse.
The scheme details are embodied in that in a conventional cloud desktop system, an additional technical scheme is added:
the operating system and software can obtain the position coordinates of the mouse in the current screen through the software interface and the hardware interface. When the left key is pressed, the position coordinates at that time are saved. If the system and the software continue to monitor that the left button of the mouse is not released, the current mouse position can be continuously recorded in real time.
The software is realized through a specified algorithm, and the coordinate information of the range of the sensitive area selected by the mouse is calculated by taking the coordinate information of the position of the mouse as a reference. The sensitive area is generally rectangular.
The software is realized through a specified algorithm, and the range information of the sensitive area selected by the mouse is converted into coding parameters which can be accepted by the video encoder. When the video encoder is invoked, perceptual coding parameters are added in addition to conventional video coding parameters.
The cloud desktop service provider can determine whether the current sensing coding requirement of the selected content is not loosened after the left mouse button is pressed by using a scene or an algorithm of the background system. When the demand is judged to occur, the technical scheme is started.
Let the screen resolution be SW x SH.
The software presets the left mouse button to select the content sensitive area as a rectangle. The method specifically comprises the following steps:
step S201, a user at a client operates a mouse, and mouse control information is returned to a server through a network;
in step S202, the server system software obtains control information and obtains the position information { X1, Y1} of the mouse in the screen when the left button of the mouse is pressed.
Step S203, if the server system software detects that the left mouse button is not released, the user is considered to be performing a selection operation, the starting points { X1, Y1} of the selection operation are saved, and the current mouse position { X2, Y2} is recorded in real time;
in step S204, the server software calculates the range of the sensitive area based on the mouse selection start point and the current position according to the mouse position information. Taking a rectangle and the selection operation is from top left to bottom right as an example:
the sensitive area is a rectangle with the vertexes of the four coordinate points:
the upper left dot coordinates lt= { X1, Y1}.
The upper right dot coordinates rt= { X2, Y1}.
Lower left dot coordinates lb= { X1, Y2}.
The bottom right dot coordinates rb= { X2, Y2}.
As shown in FIG. 4, if the selection operation is performed in other directions, the coordinates of each vertex of the rectangle can be simply obtained by using { X1, Y1} and { X2, Y2} as diagonal vertices of the rectangle, and the calculation process is not described again. In one embodiment of the method for displaying a selected area on a virtual desktop of the present invention, the method includes:
in step S301, the software converts the sensitive area range coordinates into the perceptually encoded optimized area location information. For a common video encoder, the ROI or QP map aware coding region is usually set to be the minimum unit ctu _size of a specific number of pixels, and may be generally 8, 16 or 32.
The unit is used for recalculating to obtain the coordinates of the sensing coding region as follows:
the upper left dot coordinates clt=lt/ctu _size.
Upper right dot coordinates crl=rt/ctu _size.
The lower left dot coordinates clb=lb/ctu _size.
Lower right dot coordinates crb=rb/ctu _size.
In step S302, the software converts the perceptual coding optimization region location information into coding parameters and applies the coding parameters to the video encoder.
Taking the optimization method of adjusting QP as an example, if the value of QP is qp_delta.
If ROI, the encoding parameters require the upper left and lower right corners of a given ROI region. The addition parameters are as follows: roi_config= (CLT, CRB, qp_delta)
If the QP map is used, the added encoding parameter is the QP matrix of the whole frame, and for the point in the matrix whose coordinates are within CLT, CRL, CLB, CRB, the QP value is qp+qp_delta, and the other points are unchanged.
Step S303, the video encoder applies the coding parameters to code the desktop content into a video code stream;
step S304, the server transmits the video code stream to the client through the network;
in step S305, the client decodes the video, displays the decoded video on the screen, and obtains an optimization effect on the region of interest of the mouse.
And step S306, when the user releases the left button of the mouse, the optimized coding parameters of the current interest region are canceled, and the optimized effect is closed.
The invention enables the region of interest to be automatically formed according to the left click of the mouse in the desktop video coding application, and the real-time optimization coding becomes possible.
In one embodiment of the method for displaying a selected area on a virtual desktop in the present invention, when a left mouse button is selected in different operating systems or software interfaces, the selected object sometimes extends beyond the rectangular range of the mouse starting point. According to different selection rules, the selected regions of interest in different ranges are generated, and additional algorithms can be introduced, but the method belongs to the protection scope of the invention.
The video decoder referred to in this disclosure is a specific component that decodes a video bitstream into a visual data stream (YUV or RGB format) according to a specific video compression standard. Video coding may be implemented by software or by a chip. Typically in a chip SOC there is a dedicated video decoder IP.
The video encoder referred to in this disclosure is a specific component that encodes a visual data stream (in YUV or RGB format) into a video bitstream according to a specific video compression standard, and configuration parameters. Video decoding may be implemented by software or by a chip. Typically in a chip SOC, there is a dedicated video encoder IP.
It should be understood that although the present disclosure has been described in terms of various embodiments, not every embodiment is intended to include only a single embodiment, and that such descriptions are provided for clarity only, and that the disclosure is not limited to the embodiments shown and described herein, as such, may be suitably combined in any number of embodiments, as would be apparent to one of ordinary skill in the art.
The above list of detailed descriptions is only specific to practical embodiments of the present invention, and they are not intended to limit the scope of the present invention, and all equivalent embodiments or modifications that do not depart from the spirit of the present invention should be included in the scope of the present invention.
Claims (10)
1. The display method of the selected area on the virtual desktop is characterized by being applicable to a cloud desktop system; the cloud desktop system comprises a cloud server and a cloud terminal which can carry out bidirectional data communication with the cloud server;
the display method of the selected area on the virtual desktop comprises the following steps:
the cloud server receives a pointer operation instruction and pointer position information from the cloud terminal;
obtaining a current display area according to the pointer operation instruction and the pointer position information;
and obtaining an area image capable of being displayed on the cloud terminal through the display area.
2. The method of displaying a selected area on a virtual desktop of claim 1, further comprising;
the cloud server obtains the bottom layer user application and bottom layer user application interface information which are called by the current cloud terminal in the cloud server; the bottom layer user application interface information is interface identification information which is used by a user, is applied to the bottom layer in the cloud server and can be displayed to the user;
the method for displaying the selected area on the virtual desktop further comprises the following steps:
the cloud server receives a pointer operation instruction set of the bottom layer user application from the cloud terminal; the pointer operation instruction set comprises a bottom interface region selection instruction; the bottom layer interface region selection instruction comprises the following steps: the method comprises the steps that a pointer first clicking operation instruction and a pointer moving operation instruction generate a selection area;
calling a corresponding bottom layer user application interface according to the bottom layer user application interface information;
the cloud server receives a first click operation instruction of the pointer from the cloud terminal on the bottom layer user application interface and obtains pointer position information of the first click from the first click operation instruction of the pointer;
the cloud server receives the pointer movement operation instruction from the cloud terminal and obtains the moved pointer position information from the pointer movement operation instruction;
the cloud server generates the selection area on the bottom layer user interface according to the pointer position information of the first click and the pointer position information after movement;
and the cloud server obtains the current display area according to the selected area.
3. The method for displaying a selected area on a virtual desktop as claimed in claim 2, comprising:
the bottom interface region selection instruction comprises a first click release operation instruction, so that the bottom interface region selection instruction is ended;
and after receiving the pointer movement operation instruction from the cloud terminal, if the pointer first click release operation instruction is received, ending the bottom layer interface region selection instruction and releasing the current display region by the cloud server.
4. A method of displaying a selected area on a virtual desktop as claimed in claim 2 or 3, comprising:
the cloud server obtains a desktop icon position from the bottom layer user application interface;
and the cloud server judges whether the desktop icons are included in the selected area according to the desktop icon positions, and if yes, the current display area is obtained according to the area where the desktop icons are located.
5. A method of displaying a selected area on a virtual desktop as claimed in claim 2 or 3, comprising:
the cloud server obtains word processing application and word processing application interface information loaded on the bottom user application in the cloud server by the current cloud terminal;
the cloud server receives a pointer operation instruction set of the word processing application from the cloud terminal; the pointer operation instruction set of the word processing application comprises a word area selection instruction; the text region selection instruction comprises the following steps: a text field clicking operation instruction and a text field clicking release operation instruction, wherein a text line selection area is generated in the text field clicking operation instruction and the text field clicking release operation instruction;
calling a corresponding word processing application interface according to the word processing application interface information;
the cloud server receives the word segment clicking operation instruction from the cloud terminal on the word processing application interface, and obtains pointer position information of a word segment from the word segment clicking operation instruction;
the cloud server receives the text segment click release operation instruction from the cloud terminal and obtains pointer position information of the end of the text field from the text segment click release operation instruction;
the cloud server generates the text line selection area on the user interface according to the pointer position information of the beginning of the text segment and the pointer position information of the ending of the text segment;
and the cloud server obtains the current display area according to the text line selection area.
6. A method of displaying a selected area on a virtual desktop as claimed in claim 2 or 3, further comprising:
the cloud server obtains image processing application and image processing application interface information loaded on the bottom user application in the cloud server by the current cloud terminal;
the cloud server receives a pointer operation instruction set of the image processing application from the cloud terminal; the pointer operation instruction set of the image processing application comprises an image area selection instruction; the image area selection instruction comprises the following steps: clicking an operation instruction at an image initial selection position, moving and staying to an image end position, and generating an image selection area at the image initial selection position;
calling a corresponding image processing application interface according to the image processing application interface information;
the cloud server receives a click operation instruction of an image initial selection position from the cloud terminal on the image processing application interface, and obtains pointer position information of an image start from the click operation instruction of the image initial selection position;
the cloud server receives the moving and stopping to image end position operation instruction from the cloud terminal and obtains pointer position information of image end from the moving and stopping to image end position operation instruction;
the cloud server generates the image selection area on the user interface according to the pointer position information of the image start and the pointer position information of the image end;
and the cloud server obtains the current display area according to the image selection area.
7. The method for displaying a selected area on a virtual desktop as recited in claim 6, comprising:
the cloud server obtains the position of a display line segment from the image processing application interface;
and the cloud server judges whether the selected area comprises a display line segment or not according to the position of the display line segment, and if so, the current display area is obtained according to the area where the display line segment is located.
8. The method of claim 1, wherein the pointer operation command includes a left click command, a right click command, a left click command, a right click command of the mouse;
the pen point clicking instruction, the pen point continuous clicking instruction, the pen side case clicking instruction, the set function key clicking instruction and the continuous clicking instruction on the keyboard, the left key clicking instruction, the right key clicking instruction, the left key continuous clicking instruction and the right key continuous clicking instruction on the notebook computer touch pad of the handwriting pen.
9. The method for displaying a selected area on a virtual desktop according to claim 1, comprising displaying the virtual desktop image acquired from the cloud server on the cloud terminal;
the display method of the selected area on the virtual desktop comprises the following steps:
the cloud server receives a pointer operation instruction and pointer position information which can be displayed on the virtual desktop image from the cloud terminal;
obtaining a current display area according to the pointer operation instruction and the pointer position information;
obtaining the region coding parameters of the current display region under the set resolution; removing partial codes of the current display area from the virtual desktop image to obtain desktop coding parameters; synthesizing the virtual desktop image to be displayed into a video stream through the region coding parameters and the desktop coding parameters;
the video decoder of the cloud terminal can identify the video stream as the virtual desktop image to be displayed; the cloud terminal locally displays the virtual desktop image to be displayed;
the set resolution comprises a resolution larger than the resolution of the cloud terminal display page and a resolution smaller than the resolution of the cloud terminal display page; and obtaining the coding parameters of the current display area under the set resolution by using an ROI method or a QP MAP quantization parameter adjustment method.
10. The cloud desktop system is characterized by comprising a cloud server and a cloud terminal which can be in bidirectional data communication with the cloud server;
the cloud server is configured to receive pointer position information and pointer operation instructions from the cloud terminal; obtaining a current display area according to the pointer position information and the pointer operation instruction; obtaining an area image capable of being displayed on the cloud terminal through the display area;
the cloud server configures a video encoder, and the cloud terminal configures a video decoder;
the cloud terminal is further configured to be capable of displaying the virtual desktop image acquired from the cloud server;
the cloud server is further configured to receive pointer position information and pointer operation instructions from the cloud terminal, wherein the pointer position information and the pointer operation instructions can be displayed on the virtual desktop image; obtaining a current display area according to the pointer position information and the shortcut function menu area obtained by the pointer operation instruction;
the video encoder is configured to obtain a region coding parameter of the current display region under a set resolution; removing partial codes of the current display area from the virtual desktop image to obtain desktop coding parameters;
the cloud server is further configured to synthesize a virtual desktop image to be displayed into a video stream through the region coding parameters and the desktop coding parameters;
the video decoder is configured to identify the video stream as the virtual desktop image to be displayed;
the cloud terminal is further configured to locally display the virtual desktop image to be displayed.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311583196.7A CN117453082A (en) | 2023-11-24 | 2023-11-24 | Display method of selected area on virtual desktop and cloud desktop system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311583196.7A CN117453082A (en) | 2023-11-24 | 2023-11-24 | Display method of selected area on virtual desktop and cloud desktop system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117453082A true CN117453082A (en) | 2024-01-26 |
Family
ID=89594879
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311583196.7A Pending CN117453082A (en) | 2023-11-24 | 2023-11-24 | Display method of selected area on virtual desktop and cloud desktop system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117453082A (en) |
-
2023
- 2023-11-24 CN CN202311583196.7A patent/CN117453082A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP5306429B2 (en) | Method for digital document processing | |
| CN112533021B (en) | Display method and display equipment | |
| CN110166792B (en) | Image display method, device, system and storage medium | |
| US20090238405A1 (en) | Method and system for enabling a user to play a large screen game by means of a mobile device | |
| US9485290B1 (en) | Method and system for controlling local display and remote virtual desktop from a mobile device | |
| US20120005630A1 (en) | Highly Responsive Screen Output Device, Screen Output System, and Screen Output Method | |
| CN102053867B (en) | Method and device for dynamically switching pictures | |
| JP6089454B2 (en) | Image distribution apparatus, display apparatus, and image distribution system | |
| US20060053233A1 (en) | Method and system for implementing a remote overlay cursor | |
| US10432681B1 (en) | Method and system for controlling local display and remote virtual desktop from a mobile device | |
| JP2009080573A (en) | Display method | |
| CN112055256B (en) | Image processing method and display device for panoramic image | |
| CN111899175A (en) | Image conversion method and display device | |
| CN111930233B (en) | Panoramic video image display method and display device | |
| US20140089812A1 (en) | System, terminal apparatus, and image processing method | |
| US20030184549A1 (en) | Image processing apparatus, and apparatus for and method of receiving processed image | |
| CN117453082A (en) | Display method of selected area on virtual desktop and cloud desktop system | |
| CN115988265A (en) | Rendering method and device of display picture and terminal equipment | |
| CN114710633A (en) | Display apparatus and video signal display method | |
| CN111417007B (en) | Image transmission method, device, terminal and storage medium | |
| CN117453083A (en) | Method for displaying shortcut function list on virtual desktop and cloud desktop system | |
| CN113259716A (en) | Video issuing method, video acquiring method, server, terminal and system | |
| CN112926420A (en) | Display device and menu character recognition method | |
| EP4296962A1 (en) | Information processing device, information processing method, and program | |
| CN112419988B (en) | Backlight adjusting method and display device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |