CN116996726A - Processing method and electronic equipment - Google Patents

Processing method and electronic equipment Download PDF

Info

Publication number
CN116996726A
CN116996726A CN202310953217.3A CN202310953217A CN116996726A CN 116996726 A CN116996726 A CN 116996726A CN 202310953217 A CN202310953217 A CN 202310953217A CN 116996726 A CN116996726 A CN 116996726A
Authority
CN
China
Prior art keywords
display
display data
frame
electronic device
data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310953217.3A
Other languages
Chinese (zh)
Inventor
李凡智
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Lenovo Beijing Ltd
Original Assignee
Lenovo Beijing Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Lenovo Beijing Ltd filed Critical Lenovo Beijing Ltd
Priority to CN202310953217.3A priority Critical patent/CN116996726A/en
Publication of CN116996726A publication Critical patent/CN116996726A/en
Pending legal-status Critical Current

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
    • H04N21/4307—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen
    • H04N21/43078—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen for seamlessly watching content streams when changing device, e.g. when watching the same program sequentially on a TV and then on a tablet
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • G06F3/1454—Digital output to display device ; Cooperation and interconnection of the display device with other functional units involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/436—Interfacing a local distribution network, e.g. communicating with another STB or one or more peripheral devices inside the home
    • H04N21/4363—Adapting the video or multiplex stream to a specific local network, e.g. a IEEE 1394 or Bluetooth® network
    • H04N21/43632—Adapting the video or multiplex stream to a specific local network, e.g. a IEEE 1394 or Bluetooth® network involving a wired protocol, e.g. IEEE 1394
    • H04N21/43635—HDMI
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
    • H04N21/44004—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving video buffer management, e.g. video decoder buffer or video display buffer
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
    • H04N21/47217—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for controlling playback functions for recorded or on-demand content, e.g. using progress bars, mode or play-point indicators or bookmarks

Abstract

The application discloses a processing method and electronic equipment, wherein the processing method comprises the following steps: determining that the first electronic equipment is in a first mode, and caching preset frame display data processed by a display card; transmitting first display data of each frame in the preset frame display data to a first display for display through a first channel at a first moment according to a display sequence; transmitting one or more frames of second display data of the preset frame display data to second electronic equipment for display at a second moment through a second channel; the first channel and the second channel are different, the second time is earlier than the first time, and the first display data and the second display data are the same or belong to the same frame of display data.

Description

Processing method and electronic equipment
Technical Field
The application belongs to the technical field of computers, and particularly relates to a processing method and electronic equipment.
Background
In the existing screen-throwing application, compression and transmission of images/videos by a screen-throwing source terminal device and image/video playback by a destination terminal device inevitably bring delay to screen-throwing display, and how to reduce the screen-throwing display delay becomes a technical problem in the field.
Disclosure of Invention
Therefore, the application discloses the following technical scheme:
a processing method applied to a first electronic device, the method comprising:
determining that the first electronic equipment is in a first mode, and caching preset frame display data processed by a display card;
transmitting first display data of each frame of the preset frame display data to a first display for display through a first channel at a first moment according to a display sequence;
transmitting one or more frames of second display data of the preset frame display data to second electronic equipment for display at a second moment through a second channel;
the first channel and the second channel are different, the second time is earlier than the first time, and the first display data is the same as or belongs to the same frame of display data as the corresponding second display data.
Optionally, the method further comprises:
and determining one or more frames of display data with the latest display sequence in the preset frame of display data as or processed into one or more frames of second display data.
Optionally, the method further comprises: and determining display delay time of the second electronic device and the first electronic device, and determining the second display data and/or the second moment based on the display delay time.
Optionally, the method further comprises: and determining display delay time of the second electronic equipment and the first electronic equipment, and adjusting the cached preset frame display data length based on the delay time.
Optionally, the method further comprises:
determining the information variation of the current frame data processed by the display card compared with the previous frame data of the current frame data;
adjusting the rendering frame rate according to the information variation;
and rendering the current frame data based on the adjusted rendering frame rate.
Optionally, the method further comprises:
and compressing the second display data to send the compressed second display data to the second electronic equipment through the second channel at the second moment.
A processing method applied to a second electronic device, the method comprising:
receiving second display data sent by the first electronic device at a second moment through a second channel;
performing display processing on the second display data so as to be used for synchronously displaying the second display data with corresponding first display data displayed by the first display;
the first display data are display data of a preset frame processed by the cached display card and are sent to a first display to be displayed through a first channel at a first moment according to a display sequence by the first electronic equipment;
The first channel and the second channel are different, the second time is earlier than the first time, and the first display data is the same as or belongs to the same frame of display data as the corresponding second display data.
Optionally, the method further comprises feeding back the receiving time and/or the receiving state information to the first electronic device.
Optionally, the method further comprises:
and if the second display data is in a compressed format, decompressing the second display data before performing display processing on the second display data.
An electronic device, comprising:
a memory for storing at least one set of computer instructions;
a processor for implementing the processing method according to any of claims 1-6 or implementing the processing method according to any of the preceding claims by executing the set of instructions stored in the memory.
As can be seen from the above scheme, the present application discloses a processing method and an electronic device, wherein the processing method includes: determining that the first electronic equipment is in a first mode, and caching preset frame display data processed by a display card; transmitting first display data of each frame in the preset frame display data to a first display for display through a first channel at a first moment according to a display sequence; transmitting one or more frames of second display data of the preset frame display data to second electronic equipment for display at a second moment through a second channel; the first channel and the second channel are different, the second time is earlier than the first time, and the first display data and the second display data are the same or belong to the same frame of display data.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the related art, the drawings that are required to be used in the embodiments or the related technical descriptions will be briefly described, and it is apparent that the drawings in the following description are only embodiments of the present application, and other drawings may be obtained according to the provided drawings without inventive effort for those skilled in the art.
Fig. 1 is a schematic flow chart of a processing method applied to a first electronic device according to the present application;
fig. 2 is a schematic flow chart of another processing method applied to the first electronic device provided by the present application;
FIG. 3 is a schematic illustration of an application of a screen-casting application based on a frame buffer mechanism provided by the present application;
fig. 4 is a schematic flow chart of a processing method applied to a second electronic device according to the present application;
fig. 5 is a component configuration diagram of an electronic device provided by the present application.
Detailed Description
The following description of the embodiments of the present application will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
The embodiment of the application discloses a processing method and electronic equipment, which are used for reducing screen-throwing display delay in screen-throwing application and enabling a screen-throwing destination end and a screen-throwing source end to synchronously display. The disclosed processing method includes a processing method applied to a first electronic device and a processing method applied to a second electronic device, where the first electronic device and the second electronic device are a device used as a source end of screen projection and a device used as a destination end of screen projection respectively, and may be a plurality of general or special computing device environments or electronic devices under configuration, for example: a dedicated screen-casting device, a personal computer, a server computer, a hand-held or portable device, a tablet device, a multiprocessor apparatus, etc.
The first electronic device and the second electronic device are respectively provided with a display or can be externally connected with the display for information display.
Referring to fig. 1, a flowchart of a processing method applied to a first electronic device disclosed in an embodiment of the present application is shown, where the processing method applied to the first electronic device at least includes:
step 101, determining that the first electronic device is in a first mode, and caching preset frame display data processed by the display card.
The first electronic device may operate in a first mode or a second mode. The first mode is a corresponding mode in which information needs to be synchronously displayed at a screen-throwing source end and a screen-throwing destination end in a screen-throwing mode, and specifically can be any one of a mirror screen-throwing mode and an extended screen-throwing mode, which is determined according to actual requirements; the second mode is a corresponding mode in which information does not need to be synchronously displayed at the screen-throwing source end and the screen-throwing destination end in the non-screen-throwing mode or the screen-throwing mode, such as a display mode in the non-screen-throwing mode or the screen-throwing mode.
In the embodiment of the application, the mirror image screen projection mode specifically refers to that the first electronic device sends the same data to be displayed to the first channel and the second channel so as to synchronously display the same data on a first display of the first electronic device end and a second display of the second electronic device end; the extended screen-throwing mode refers to that the first electronic device sends different data to be displayed to the first channel and the second channel so as to synchronously display different data (completely different data or partial data) on the first display of the first electronic device end and the second display of the second electronic device end, thereby achieving the effect of taking the second display of the second electronic device end as an extended screen of the first electronic device; the display mode in the screen-throwing mode means that the first electronic device only sends data to be displayed to the second channel and does not send the data to the first channel, and correspondingly, only the second display at the second electronic device end is displayed, so that the effect that the second display at the second electronic device end is used as the display of the first electronic device is achieved.
The first channel is a multimedia data (audio/video data) transmission channel between the first electronic device and a first display at the first electronic device end, the first display can be a self-contained display or an external display of the first electronic device, and the first channel can be a multimedia data transmission channel inside the first electronic device or a multimedia data transmission channel outside the first electronic device correspondingly, and can be a channel formed based on wireless connection or wired connection without limitation. For example, the first channel is specifically an audio/video data transmission channel formed between the first electronic device and the first display externally connected with the first electronic device through an HDMI (High Definition Multimedia Interfac, high-definition multimedia interface) interface and a connection line or through a DP (DisplayPort) interface and a connection line.
The second channel is a multimedia data (audio/video data) transmission channel between the first electronic device and the second electronic device, and corresponds to a screen projection application, the channel is essentially a screen projection channel, and the channel can be formed based on wireless connection or wired connection without limitation.
In the embodiment of the application, the first mode is any one mode of a mirror image screen-throwing mode, an extended screen-throwing mode and the like which need to synchronously display information at a screen-throwing source end and a screen-throwing destination end, so that the screen-throwing display delay of the screen-throwing destination end relative to the screen-throwing source end is reduced based on the processing method of the application, and the high synchronism of double-end information display is ensured.
In this step, specifically, an application mode detection may be performed on the first electronic device, where the mode of the first electronic device is determined, and in the case where it is determined that the first electronic device is in the first mode, the display data of the preset frame after the display card is processed is cached.
The display card is a display card of the first electronic equipment end, and can be an internal display card or an external display card of the first electronic equipment. The display data of the preset frame processed by the display card is at least the display data of the preset frame processed by the rendering process of the display card, and specifically can be but not limited to the display data of the preset frame number (preset frame length) obtained by rendering the objects to be displayed, such as live photo, a group of images, videos, PPT and the like, by the frame by the display card.
The embodiment of the application provides a concept of frame buffering at a screen throwing source end, namely a first electronic equipment end, and particularly relates to a concept of setting a frame buffering mechanism at the first electronic equipment end to pre-render display data of at least one frame (1 frame+), namely, a certain time delay exists between the time when a display card finishes rendering one frame of data and the time when the frame of data after rendering is sent to a first display of the first electronic equipment end, after the display card renders data to be displayed according to the rendering progress, a rendering completion result is stored in a buffer area, which is equivalent to that the display data after the display card renders is output to the buffer and is not directly output to a display module of the first display of the first electronic equipment end, and then the rendered display data is output to a corresponding display end in the screen throwing application from the buffer area according to the need.
The number of frames of the display data stored in the buffer area (i.e., the number of frames corresponding to the preset frame display data) is at least one frame, and specifically, the number of frames of the display data stored in the buffer area may be set as required in frame buffering, where the number of frames of the display data stored in the buffer area may be, but is not limited to, set according to a display delay time of the second electronic device relative to the first electronic device, caused by compression and transmission of an image/video and playback of the image/video of the second electronic device in a screen-throwing application.
Step 102, sending the first display data of each frame of the preset frame display data to a first display for display at a first moment through a first channel according to a display sequence.
Each frame of first display data of the preset frame display data may be each frame of display data in the preset frame display data, or may be data obtained by processing each frame of display data in the preset frame display data, such as partial data extracted from each frame of display data, that is, each frame of first display data corresponds to one frame of display data in the preset frame display data in a one-to-one manner, specifically may be the corresponding one frame of display data itself, or may be data obtained by processing the corresponding one frame of display data (such as partial picture extracted from one frame of picture), which is not limited and depends on application requirements. For example, in the mirror screen projection mode, each frame of first display data in the preset frame of display data is specifically each frame of display data in the preset frame of display data, and in the extended screen projection mode, each frame of first display data in the preset frame of display data may be specifically part of each frame of display data in the preset frame of display data.
Based on a frame buffer mechanism set at the first electronic device side, in the embodiment of the application, each frame of display data after rendering processing of the display card at the first electronic device side is not sent to the first display at the first electronic device side in real time, but is sent to the buffer area, and when the sending time to the first display arrives, that is, the corresponding first moment is reached, each frame of first display data of the preset frame of display data is sent from the buffer area to the first display for display. The sending time of the first display data to the first display is delayed by a certain time compared with the output time of the corresponding display data at the display card.
The buffer area is buffered with preset frame display data processed by the display card, wherein each frame of display data corresponds to a respective first moment, and each frame of display data corresponds to a respective first display data (the frame display data or the processed frame display data), and the display data is sent to the first display through the first channel at the respective first moment in sequence according to the display sequence.
The first delay time of the time of sending the first display data of the preset frame display data to the first display compared with the time of outputting the display data corresponding to the first display data by the display card can be set in advance in a frame buffer mechanism of the first electronic device, the first delay time can be set to be in a form of a duration (time length) or a frame length (frame number), so that the time corresponding to a certain duration or a certain frame number after the display card outputs the rendered display data of each frame is determined to be the sending time of the first display data corresponding to the display data of each frame, namely, the first time, and each frame of the first display data of the cached preset frame display data is sent to the first display through a first channel according to the display sequence at the corresponding first time.
Preferably, the first delay time may be set with reference to a display delay time of the second electronic device compared with the first electronic device in the screen-projection application, and preferably, the first delay time is not less than a display delay time of the second electronic device compared with the first electronic device, so as to ensure high synchronization of double-end information display of the second electronic device and the first electronic device in the screen-projection application.
When the first display data is data obtained by processing a corresponding frame of display data, the frame of display data may be processed in real time at a corresponding first time, and part of the data of the frame of display data may be actually extracted as the first display data and sent to the first display through the first channel for display, or the frame of display data may be processed in advance before the corresponding first time, and when the corresponding first time is reached, the first display data obtained by processing may be sent to the first display through the first channel for display.
Step 103, sending one or more frames of second display data of the preset frame display data to the second electronic device for display through the second channel at a second moment.
The first channel and the second channel are different, the second time is earlier than the first time, and the first display data is the same as or belongs to the same frame of display data as the corresponding second display data.
Similarly, the second display data of each frame in the preset frame display data may be each frame display data in the preset frame display data, or may be data obtained by processing each frame display data in the preset frame display data, that is, each frame of second display data corresponds to one frame display data in the preset frame display data in a one-to-one manner, and may specifically be the corresponding one frame display data itself, or data obtained by processing the corresponding one frame display data (such as a part of pictures extracted from one frame of pictures).
For example, in the mirror screen projection mode, the second display data of each frame is specifically each frame of display data in the preset frame of display data, and in this mode, the first display data of each frame is also each frame of display data in the preset frame of display data, so as to support synchronous display of the same display data in the first electronic device and the second electronic device.
In the extended screen-throwing mode, the second display data of each frame is specifically data obtained after each frame of display data in the preset frame of display data is processed, such as partial data in each frame of display data, and in the mode, the first display data and the second display data corresponding to the same display data in the preset frame of display data are different, and may respectively belong to the same corresponding display data and respectively be different partial data of the same corresponding display data, and the first display data and the second display data may be completely different, or the partial data in the first display data and the second display data are different, for example, the first display data and the second display data are respectively a left half picture and a right half picture in the same corresponding display picture, or respectively are all pictures in the same corresponding display picture and a middle area picture, etc.
From the above description, it is easy to understand that each frame of display data in the preset frame of display data corresponds to one frame of first display data and one frame of second display data, so that the first display data and the second display data are in a one-to-one correspondence, and the corresponding first display data and second display data are respectively used for synchronous display at the first electronic device end and the second electronic device end.
In the first mode, in addition to sending the first display data of each frame of the preset frame display data to the first display for display at a first time through the first channel according to the display sequence, the embodiment of the application further sends one or more frames of second display data of the preset frame display data to the second electronic device for display at a second time through the second channel.
Specifically, for each frame of first display data, the first display data is sent to the first display at the first electronic device end through the first channel at the corresponding first moment, and the second display data corresponding to the frame of first display data or the multi-frame second display data (i.e. a group of second display data) starting with the second display data corresponding to the frame of first display data is sent to the second electronic device at the second moment through the second channel for displaying on the second display at the second electronic device end. The second moment is earlier than the first moment, the second display data corresponding to the first display data of the frame or a group of second display data starting from the second display data corresponding to the first display data of the frame is sent to the second electronic equipment before the first moment corresponding to the first display data of the frame, so that the second electronic equipment can receive the second display data corresponding to the first display data of the frame before the first display displays the first display data of the frame, and further the second electronic equipment can synchronously display the second display data corresponding to the first display data of the frame and the first display data of the frame of the first electronic equipment at the same time, and the information synchronous display requirement of a screen-throwing source end and a destination end in a first mode is met.
The second display data corresponding to the first display data of each frame corresponds to corresponding second time respectively, so that a matching relationship exists between the first time and the second time, and the existing matching relationship specifically means that the second time corresponding to the second display data corresponding to the first display data of each frame is matched with the first time corresponding to the first display data of the frame, and the second time matched with the first time is earlier than the first time.
It is readily understood that in embodiments in which a set of second display data is transmitted to the second electronic device at a corresponding second time instant, the set of second display data corresponds to the same second time instant.
In summary, in the processing method applied to the first electronic device (on-screen source end) provided by the embodiment of the application, when the first electronic device is in the first mode, the preset frame display data processed by the display card is buffered, and each frame of first display data in the preset frame display data is sent to the first display through the first channel at the first moment, and one or more frames of second display data of the preset frame display data is sent to the second electronic device (on-screen destination end) for display through the second channel at the second moment earlier than the first moment, so that the pre-rendering and buffering of at least one frame of display data based on the frame buffer mechanism is realized, and before the on-screen source end displays based on the buffered data, the data to be synchronously displayed at the on-screen destination end is transmitted to the on-screen destination end based on the buffered data, so that the received data is synchronously displayed with the on-screen source end at the on-screen destination end, the on-screen display delay is at least reduced, and the high synchronicity of the on-screen display of information between the on-screen destination end and the on-screen source end is ensured.
In an alternative embodiment, referring to the flowchart of the processing method shown in fig. 2, the processing method applied to the first electronic device provided in the present application may further include, before step 103:
step 201, determining one or more frames of display data with the latest display sequence in the preset frame of display data as or processing the one or more frames of second display data.
The step 201 is specifically executed between the step 101 and the step 103, and the execution sequence of the step 102 depends on the processing time node requirement of the corresponding frame display data in the preset frame display data in the actual processing process.
In an embodiment, it may be determined that one frame of display data, which is the latest in display order, of the preset frame of display data is the one frame of second display data or is processed as the one frame of second display data. For example, the display card outputs and stores the last frame of display data in the buffer in a streaming manner as the one frame of second display data, or processes the last frame of display data into the one frame of second display data.
In this embodiment, optionally, each time the display card at the first electronic device outputs a frame of rendered display data to the buffer, the rendered display data may be used or processed into a frame of second display data, and the obtained second display data is sent to the second electronic device through the second channel, where the second time is preferably a time when the frame of second display data is obtained, and the first time may be determined based on the first delay time set in the frame buffer mechanism. Because the second moment corresponding to the second display data of each frame is always earlier than the first moment corresponding to the first display data corresponding to the second display data of the frame, the screen-throwing display delay of the second electronic equipment can be at least reduced, and the high synchronism of the information display of the second electronic equipment and the first electronic equipment is ensured.
In another embodiment, it may be determined that the multi-frame display data with the latest display sequence in the preset frame display data is used as or processed into the multi-frame second display data, and at the corresponding second moment, a set of second display data formed by the multi-frame second display data is sent to the second electronic device.
In this embodiment, the number of frames of the second display data in the set of second display data may be preset, where the set number of frames is less than or equal to the buffered number of frames corresponding to the preset frame display data.
Optionally, the display card at the first electronic device outputs rendered display data up to the set frame number (and not sent) to the buffer memory every time, that is, the rendered display data is used as or processed into multi-frame second display data, and the obtained multi-frame second display data is sent to the second electronic device together through the second channel. In this embodiment, preferably, the second time is preferably a time when the multi-frame second display data is obtained, the first time may be determined based on the first delay time set in the frame buffer mechanism, specifically, the first time of the first display data corresponding to the first frame second display data in the multi-frame second display data may be determined based on the set first delay time and the second time, and the first time of the first display data corresponding to the subsequent frame second display data may be sequentially determined according to a display sequence and a display frame rate of the first display. In this embodiment, the second time corresponding to the second display data of each frame is always earlier than the first time corresponding to the first display data corresponding to the second display data of the frame, so that at least the screen-throwing display delay of the second electronic device can be reduced, and high synchronism of information display between the second electronic device and the first electronic device is ensured.
In an optional embodiment, the processing method applied to the first electronic device provided by the present application may further include: and determining display delay time of the second electronic device and the first electronic device, and determining the second display data and/or the second moment based on the display delay time.
Wherein, but not limited to, when the first mode is started, the display delay time of the second electronic device relative to the first electronic device can be calibrated by the first electronic device automatically through information interaction with the second electronic device. For example, first, the first electronic device sends test data to the second electronic device, after the second electronic device receives the test data, the second electronic device feeds back receiving time and/or receiving state information to the first electronic device, and the first electronic device determines display delay time of the second electronic device relative to the first electronic device according to a feedback result and information such as a transmission path state between the first electronic device and the second electronic device.
In the case that the test data is compressed data, the time information fed back by the second electronic device may specifically be the time when the decompression is received and completed.
The transmission path status information between the first electronic device and the second electronic device may include, but is not limited to, transmission path length, transmission delay, network stability, and the like.
Thereafter, the second display data and/or the second time may be determined based on the display delay time. Specifically, based on the display delay time and the predicted display time of each first display data in the preset frame display data determined based on the frame buffer mechanism, the second display data which should be sent in advance at the corresponding time and the specific time required for sending the second display data to the second electronic device, namely, the second time, are determined, each second time is ensured to be earlier than the first time matched with the second display data, the time of sending each frame of the second display data to the second electronic device is ensured correspondingly, and the time of sending the second display data to the first display of the first electronic device end corresponding to the second display data of the frame is earlier than the time of sending the first display data of the frame of the second display data of the second electronic device end, so that the screen throwing display delay of the second electronic device can be at least reduced, and the high synchronism of information display of the second electronic device and the first electronic device is ensured.
In practical application, the display delay time between the second electronic device and the first electronic device may change along with the state change of the transmission path between the two devices, and the display delay time may be detected in real time or periodically through the data receiving time and/or the receiving state information interaction between the first electronic device and the second electronic device in the operation process of the first mode, and the second display data and/or the second time may be adaptively adjusted based on the change of the display delay time, for example, the originally determined second time is advanced, so as to adapt to the situation that the display delay time increases, and the like.
In an optional embodiment, the processing method applied to the first electronic device provided by the present application may further include: and determining display delay time of the second electronic equipment and the first electronic equipment, and adjusting the cached preset frame display data length based on the delay time.
The manner of determining the display delay time can be found in the description of the previous embodiment.
After determining the display delay time of the second electronic device and the first electronic device, the embodiment adjusts the buffered display data length of the preset frame based on the display delay time, and the adjustment of the display data length of the preset frame refers to adjusting the frame number of the display data in the buffer.
Specifically, the length of the display data of the preset frame can be adjusted based on the change of the display delay time.
In practical application, in the running process of the first mode, through the data receiving time and/or receiving state information interaction between the first electronic device and the second electronic device, the display delay time of the second electronic device and the first electronic device is detected in real time or periodically, whether the display delay time is changed or not is judged, and in the case of the change, the length of the cached preset frame display data is adaptively adjusted based on the change of the display delay time, for example, when the display delay time is increased, the frame number of the preset frame display data in the cache is increased, so that a larger time difference can be supported between the second moment and the first moment matched with the second moment, the high synchronism of the double-end device information display in the first mode is ensured as much as possible, otherwise, if the display delay time is reduced, the frame number of the preset frame display data in the cache can be reduced.
In an optional embodiment, the processing method applied to the first electronic device provided by the application may further include the following processing:
determining the information variation of the current frame data processed by the display card compared with the previous frame data of the current frame data; adjusting the rendering frame rate according to the information variation; and rendering the current frame data based on the adjusted rendering frame rate.
The display card is specifically a display card of the first electronic equipment terminal.
The embodiment controls the frame rate when the display card at the first electronic equipment end performs display data rendering. In the frame rate control, the information change amount of the current frame data processed by the display card compared with the previous frame data of the current frame data is specifically determined, and the rendering frame rate is adjusted according to the determined information change amount.
The method includes, but is not limited to, determining an object with a change (such as a new increase or decrease) and/or an object with a state change (such as a change from a static state to a motion state or a change in a position or a change in a duty ratio in an image frame) by analyzing an image corresponding to the current frame data and an image corresponding to the previous frame data, further determining an information change amount corresponding to the changed object and/or the object with the state change, and adjusting the current rendering frame rate of the display card according to the information change amount and a preset adjustment strategy.
Preferably, the height of the adjusted rendering frame rate and the size of the information change amount are in positive correlation, namely, the larger the information change amount is, the higher the adjusted rendering frame rate is, and otherwise, the lower the adjusted rendering frame rate is.
In practical application, a plurality of information variation intervals can be set, and the frame rate corresponding to each interval is set, wherein the larger the information variation represented by the information variation interval is, the higher the frame rate corresponding to the interval is, after the information variation of the current frame data processed by the display card compared with the previous frame data is obtained, the corresponding target frame rate is determined according to the interval where the information variation is located, so that the rendering frame rate of the display card is adjusted to be the target frame rate, and the current frame data is rendered according to the adjusted frame rate so as to adapt to the information variation of the current frame data compared with the previous frame data.
But not limited to the above implementation manner, for example, a relational function expression between the information variable quantity and the frame rate may be set, where the information variable quantity is an independent variable, the frame rate is an independent variable, and the function design follows the principle that the level of the adjusted rendering frame rate and the size of the information variable quantity are in positive correlation, so that after the information variable quantity of the current frame data processed by the display card compared with the previous frame data is obtained, the target frame rate corresponding to the information variable quantity may be obtained according to the set function, the rendering frame rate of the display card is adjusted to the target frame rate, and the current frame data is rendered according to the adjusted frame rate.
The frame rate control modes in the prior art mainly have two modes, namely a vertical synchronous mode, namely rendering according to a set frame rate and rendering according to display performance, but the two modes lack flexibility, and are difficult to better meet the data display requirement. In this embodiment, the rendering frame rate of the graphics card is adjusted according to the information variation between the front frame data and the rear frame data as required, the rendering frame rate is improved when the information variation is high, and the rendering frame rate is reduced when the information variation is low, so that the high frame rate experience under the condition of high information variation is ensured, smooth display of the pictures is ensured, no blocking is caused, and the power consumption can be greatly reduced under the condition of low information variation.
In an optional embodiment, the processing method applied to the first electronic device provided by the present application may further include: and compressing the second display data to send the compressed second display data to the second electronic equipment through the second channel at the second moment.
In this embodiment, after obtaining the one or more frames of second display data to be sent based on the buffered preset frame display data, the one or more frames of second display data are compressed according to a required compression format, and the obtained one or more frames of second display data in a corresponding compression format are sent to the second electronic device through the second channel, so as to display the second electronic device.
Referring to the example of fig. 3, in this example, the device 1 is a first electronic device, the device 2 is a second electronic device, the number of frames of display data buffered by the device 1 (corresponding to a "buffer frame" in the figure) is n+1, where the device 1 intercepts the latest N frame data (i.e. the buffer frame 1-buffer frame N) from the buffer as a set of second display data, compresses the set of second display data, and sends the compressed set of second display data to the device 2 through a second channel at a corresponding second moment, and then, the device 2 may implement synchronous display of information on the second display at the device 2 side (the display device of the device 2 in the figure) and the first display at the device 1 side (the display device of the device 1 in the figure) by decompressing and playing back the received set of second display data.
Referring to fig. 4, a flowchart of a processing method applied to a second electronic device provided by the present application is shown, where the processing method applied to the second electronic device at least includes:
step 401, receiving second display data sent by the first electronic device at a second moment through the second channel.
The second electronic device may specifically receive one or more frames of second display data sent by the first electronic device through the second channel in a first mode, such as a mirror screen projection mode, an extension screen projection mode, and the like.
Step 402, performing display processing on the second display data, so as to display the second display data synchronously with corresponding first display data displayed by the first display;
the first display data are display data of preset frames processed by the cached display card and are sent to a first display to be displayed through a first channel at a first moment according to a display sequence by the first electronic device.
The first channel and the second channel are different, the second time is earlier than the first time, and the first display data is the same as or belongs to the same frame of display data as the corresponding second display data.
And for the received one or more frames of second display data, the second electronic equipment sends each frame of second display data to a second display of the second electronic equipment end at a corresponding first time according to the display sequence so as to display the second display synchronously with the first display data sent to the first display by the first electronic equipment end at the first time.
If the received second display data is in a compressed format, the second electronic device further performs decompression processing on the received second display data before performing display processing on the second display data.
The corresponding first time refers to a time when a frame of first display data corresponding to the frame of second display data is sent to the first display at the first electronic device end. Specifically, a time synchronization mechanism between the second electronic device and the first electronic device can be set, so that the second electronic device is controlled to send each frame of received second display data to a second display at the second electronic device end at a corresponding first time according to a display sequence to display, and synchronous sending and displaying of a frame of first display data corresponding to the frame of second display data to the first display at the first electronic device end can be realized.
The time synchronization mechanism may include, but is not limited to, setting a second delay time from receiving a frame of second display data or a group of second display data to displaying the frame of second display data or a group of first frame of second display data in the group of second display data, correspondingly, after receiving a frame of second display data or a group of second display data, timing the second delay time, taking a time when the second delay time is timed as a first time corresponding to the frame of second display data or the first frame of second display data (i.e. a time when the frame of second display data or the first frame of second display data is sent to the first display at the first electronic device end), sending the frame of second display data to the second display at the second electronic device end, sending other frames except the first frame of data in the group of second display data to the second display at a first time corresponding to the frame of second display data according to a corresponding display sequence and a frame rate, so as to ensure that the second display device end and the first electronic device end meet the requirement of the first electronic device end of the first display data are synchronized.
In summary, the processing method applied to the second electronic device provided in the embodiment of the present application receives the second display data sent by the first electronic device at the second moment through the second channel, and performs display processing on the received second display data, where the second moment is earlier than the sending moment of the first display data corresponding to the second display data (i.e., the data that is used by the first electronic device end to be displayed in synchronization with the second display data) to the first display at the first electronic device end, so that the second electronic device can better support the second electronic device to synchronously display the received second display data and the corresponding first display data at the first electronic device end, thereby at least reducing the screen-throwing display delay between the second electronic device and the first electronic device, and ensuring the high synchronicity of the information display between the second electronic device and the first electronic device.
In an optional embodiment, the processing method applied to the second electronic device provided by the present application may further include: and feeding back the receiving moment and/or the receiving state information to the first electronic equipment.
Optionally, the second electronic device may feed back at least one of the receiving time and the corresponding receiving status information to the first electronic device after receiving the second display data (one or more frames of second display data) sent by the first electronic device each time.
When the transmitted second display data is compressed data, the time information fed back by the second electronic device may specifically be the time when the decompression is received and completed.
The receiving status information may include, but is not limited to, whether the second display data is received, whether the received second display data is complete, whether the received second display data is damaged, and the like, so that the receiving condition of the second display data by the second electronic device is timely known by the first electronic device, and further, abnormality relieving processing is conveniently performed by timely diagnosing an abnormality cause (such as abnormal screen throwing function, overlarge network delay, poor network stability, and the like) under the condition that the second electronic device receives an abnormality.
The embodiment of the application also discloses an electronic device, the composition structure of which is shown in fig. 5, at least comprising:
a memory 10 for storing a set of computer instructions;
the set of computer instructions may be implemented in the form of a computer program.
The processor 20 is configured to implement a processing method as disclosed in any of the above method embodiments as being applied to a first electronic device or as being applied to a second electronic device by executing a set of computer instructions.
The processor 20 may be a central processing unit (Central Processing Unit, CPU), application-specific integrated circuit (ASIC), digital Signal Processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic device, etc.
The electronic device is provided with a display device and/or a display interface, and can be externally connected with the display device.
Optionally, the electronic device further includes a camera assembly, and/or an external camera assembly is connected thereto.
In addition, the electronic device may include communication interfaces, communication buses, and the like. The memory, processor and communication interface communicate with each other via a communication bus.
The communication interface is used for communication between the electronic device and other devices. The communication bus may be a peripheral component interconnect standard (Peripheral Component Interconnect, PCI) bus or an extended industry standard architecture (Extended Industry Standard Architecture, EISA) bus or the like, and may be classified as an address bus, a data bus, a control bus, or the like.
It should be noted that, in the present specification, each embodiment is described in a progressive manner, and each embodiment is mainly described as different from other embodiments, and identical and similar parts between the embodiments are all enough to be referred to each other.
For convenience of description, the above system or apparatus is described as being functionally divided into various modules or units, respectively. Of course, the functions of each element may be implemented in the same piece or pieces of software and/or hardware when implementing the present application.
From the above description of embodiments, it will be apparent to those skilled in the art that the present application may be implemented in software plus a necessary general hardware platform. Based on such understanding, the technical solution of the present application may be embodied essentially or inventive contributing portions thereof in the form of a software product, which may be stored in a storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the embodiments or portions of the embodiments of the present application.
Finally, it is further noted that relational terms such as first, second, third, fourth, and the like are used herein to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises an element.
The foregoing is merely a preferred embodiment of the present application and it should be noted that modifications and adaptations to those skilled in the art may be made without departing from the principles of the present application, which are intended to be comprehended within the scope of the present application.

Claims (10)

1. A processing method applied to a first electronic device, the method comprising:
determining that the first electronic equipment is in a first mode, and caching preset frame display data processed by a display card;
transmitting first display data of each frame of the preset frame display data to a first display for display through a first channel at a first moment according to a display sequence;
transmitting one or more frames of second display data of the preset frame display data to second electronic equipment for display at a second moment through a second channel;
the first channel and the second channel are different, the second time is earlier than the first time, and the first display data is the same as or belongs to the same frame of display data as the corresponding second display data.
2. The method of claim 1, further comprising:
and determining one or more frames of display data with the latest display sequence in the preset frame of display data as or processed into one or more frames of second display data.
3. The method of claim 1, further comprising: and determining display delay time of the second electronic device and the first electronic device, and determining the second display data and/or the second moment based on the display delay time.
4. The method of claim 1, further comprising: and determining display delay time of the second electronic equipment and the first electronic equipment, and adjusting the cached preset frame display data length based on the delay time.
5. The method of claim 1, further comprising:
determining the information variation of the current frame data processed by the display card compared with the previous frame data of the current frame data;
adjusting the rendering frame rate according to the information variation;
and rendering the current frame data based on the adjusted rendering frame rate.
6. The method of claim 1, further comprising:
and compressing the second display data to send the compressed second display data to the second electronic equipment through the second channel at the second moment.
7. A processing method applied to a second electronic device, the method comprising:
receiving second display data sent by the first electronic device at a second moment through a second channel;
Performing display processing on the second display data so as to be used for synchronously displaying the second display data with corresponding first display data displayed by the first display;
the first display data are display data of a preset frame processed by the cached display card and are sent to a first display to be displayed through a first channel at a first moment according to a display sequence by the first electronic equipment;
the first channel and the second channel are different, the second time is earlier than the first time, and the first display data is the same as or belongs to the same frame of display data as the corresponding second display data.
8. The method of claim 1, further comprising feeding back a reception time and/or reception status information to the first electronic device.
9. The method of claim 1, further comprising:
and if the second display data is in a compressed format, decompressing the second display data before performing display processing on the second display data.
10. An electronic device, comprising:
a memory for storing at least one set of computer instructions;
a processor for implementing the processing method according to any of claims 1-6 or the processing method according to any of claims 7-9 by executing the set of instructions stored in the memory.
CN202310953217.3A 2023-07-31 2023-07-31 Processing method and electronic equipment Pending CN116996726A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310953217.3A CN116996726A (en) 2023-07-31 2023-07-31 Processing method and electronic equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310953217.3A CN116996726A (en) 2023-07-31 2023-07-31 Processing method and electronic equipment

Publications (1)

Publication Number Publication Date
CN116996726A true CN116996726A (en) 2023-11-03

Family

ID=88526023

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310953217.3A Pending CN116996726A (en) 2023-07-31 2023-07-31 Processing method and electronic equipment

Country Status (1)

Country Link
CN (1) CN116996726A (en)

Similar Documents

Publication Publication Date Title
US10049642B2 (en) Sending frames using adjustable vertical blanking intervals
AU2014339383B2 (en) Method and apparatus for controlling transmission of compressed picture according to transmission synchronization events
CN106528025B (en) Multi-screen image projection method, terminal, server and system
CN109379619B (en) Sound and picture synchronization method and device
US9832421B2 (en) Apparatus and method for converting a frame rate
CN111479154B (en) Equipment and method for realizing sound and picture synchronization and computer readable storage medium
CN104918137A (en) Method enabling spliced screen system to play videos
US20130187832A1 (en) Display apparatus and method for controlling the same
CN111510773A (en) Resolution adjustment method, display screen, computer storage medium and equipment
CN109509424B (en) Display driving device, control method thereof and display device
US11646001B2 (en) Display device, data processor, data processing method, driving method, storage medium, and electronic device
CN113625981A (en) Multi-screen display method and device
US10412320B1 (en) Method and system for switching display from first video source to second video source
CN114302092A (en) One-display frame insertion circuit, method, device, chip, electronic device and medium
CN111083496A (en) Data processing method and related product
TW202203198A (en) Scalar, display device and associated data processing method
CN111083547A (en) Method, apparatus, and medium for balancing video frame rate error based on trigger mode
CN112040284B (en) Synchronous display control method and device of multiple display screens and storage medium
CN113691862B (en) Video processing method, electronic device for video playing and video playing system
CN111741343B (en) Video processing method and device and electronic equipment
CN116996726A (en) Processing method and electronic equipment
WO2023040592A1 (en) Image data transmission method and apparatus, terminal, and medium
CN114173054B (en) Multi-frame frequency spliced video source display control method and system and LED display system thereof
KR100651543B1 (en) Wireless terminal for reducing moving picture screen distortion
US10893229B1 (en) Dynamic pixel rate-based video

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination