WO2022111672A1 - 数据处理方法、装置及电子设备 - Google Patents

数据处理方法、装置及电子设备 Download PDF

Info

Publication number
WO2022111672A1
WO2022111672A1 PCT/CN2021/133831 CN2021133831W WO2022111672A1 WO 2022111672 A1 WO2022111672 A1 WO 2022111672A1 CN 2021133831 W CN2021133831 W CN 2021133831W WO 2022111672 A1 WO2022111672 A1 WO 2022111672A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
receiving end
transmitted
network transmission
parameter
Prior art date
Application number
PCT/CN2021/133831
Other languages
English (en)
French (fr)
Inventor
林新伟
Original Assignee
维沃移动通信有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 维沃移动通信有限公司 filed Critical 维沃移动通信有限公司
Publication of WO2022111672A1 publication Critical patent/WO2022111672A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/45—Management operations performed by the client for facilitating the reception of or the interaction with the content or administrating data related to the end-user or to the client device itself, e.g. learning user preferences for recommending movies, resolving scheduling conflicts
    • H04N21/462—Content or additional data management, e.g. creating a master electronic program guide from data received from the Internet and a Head-end, controlling the complexity of a video stream by scaling the resolution or bit-rate based on the client capabilities
    • H04N21/4621—Controlling the complexity of the content stream or additional data, e.g. lowering the resolution or bit-rate of the video stream for a mobile client with a small screen
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • G06F3/1454—Digital output to display device ; Cooperation and interconnection of the display device with other functional units involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/436—Interfacing a local distribution network, e.g. communicating with another STB or one or more peripheral devices inside the home
    • H04N21/4363—Adapting the video stream to a specific local network, e.g. a Bluetooth® network
    • H04N21/43637—Adapting the video stream to a specific local network, e.g. a Bluetooth® network involving a wireless protocol, e.g. Bluetooth, RF or wireless LAN [IEEE 802.11]

Definitions

  • the present application belongs to the field of communication technologies, and in particular relates to a data processing method, device and electronic device.
  • a user can synchronously display a screen of a device with a small screen to a screen of a device with a large screen through wireless screen projection technology.
  • the resolution and frame rate of the output screen image of the source end of the small screen device are set to a lower maximum value, for example, the maximum frame rate is 30fps and the maximum resolution is 1080p.
  • the small screen device can only use the frame rate of 30fps and the resolution of 1080p.
  • Performing screen projection limits the display capability of a large-screen device, which results in poor display effect of the projected video resources during the screen projection process.
  • the purpose of the embodiments of the present application is to provide a data processing method, apparatus, and electronic device, which can solve the problem of poor display effect of the projected video resources during the projection process.
  • an embodiment of the present application provides a data processing method, the method includes: adjusting a data parameter of the data to be transmitted according to at least one of a network transmission state and a playback parameter of a receiving end device; adjusting the data parameter
  • the data to be transmitted is encoded to obtain encoded data, and the encoded data is transmitted to the receiving end device.
  • an embodiment of the present application provides a data processing apparatus, the data processing apparatus includes: an adjustment module, an encoding module, and a transmission module; and an adjustment module for at least one of the network transmission status and playback parameters of the receiving end device.
  • One item is to adjust the data parameters of the data to be transmitted; the encoding module is used to encode the data to be transmitted after adjusting the data parameters to obtain encoded data; the transmission module is used to transmit the encoded data to the receiving end device.
  • an embodiment of the present application provides an electronic device, the electronic device includes: an external image chip, an encoder, and a radio frequency unit; the external image chip is used for at least one of the network transmission state and the playback parameters of the receiving end device.
  • the item is used to adjust the data parameters of the data to be transmitted;
  • the encoder is used to encode the data to be transmitted after adjusting the data parameters to obtain the coded data;
  • the radio frequency unit is used to transmit the coded data to the receiving end device.
  • an embodiment of the present application provides an electronic device, the electronic device includes a processor, a memory, and a program or instruction stored on the memory and executable on the processor, the program or instruction being The processor implements the steps of the method according to the first aspect when executed.
  • an embodiment of the present application provides a readable storage medium, where a program or an instruction is stored on the readable storage medium, and when the program or instruction is executed by a processor, the steps of the method according to the first aspect are implemented .
  • an embodiment of the present application provides a chip, the chip includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is configured to run a program or an instruction to implement the first aspect the method described.
  • an embodiment of the present application provides a computer program product, where the program product is executed by at least one processor to implement the method described in the first aspect.
  • the transmitting end device adjusts the data parameters of the data to be transmitted according to at least one of the network transmission state and the playback parameters of the receiving end device; then, the transmitting end device adjusts the data parameters to be transmitted after adjusting the data parameters.
  • the transmission data is encoded to obtain the encoded data, and finally the transmitting end device transmits the encoded data to the receiving end device. That is to say, in data transmission, the transmitting end device can flexibly adjust the data parameters of the data to be transmitted according to the network transmission state and the playback capability of the receiving end device before transmitting the data.
  • the picture quality can be improved by adjusting the data parameters of the data to be transmitted, so that the sending end device can determine
  • the receiving end device has strong playback capability, it can improve the screen projection experience on the screen of the receiving end device with strong playback capability
  • the specified data parameters reduce the amount of transmitted data, which can avoid the problem that the display screen of the receiving end is stuck due to the blocking of the network card when the network becomes poor.
  • FIG. 1 is a schematic flowchart of a data processing method provided by an embodiment of the present application.
  • FIG. 2 is a schematic structural diagram of a possible data processing apparatus according to an embodiment of the present application.
  • FIG. 3 is a schematic structural diagram of a possible electronic device according to an embodiment of the present application.
  • FIG. 4 is a schematic hardware diagram of an electronic device according to an embodiment of the present application.
  • first, second and the like in the description and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It is to be understood that the data so used are interchangeable under appropriate circumstances so that the embodiments of the present application can be practiced in sequences other than those illustrated or described herein, and distinguish between “first”, “second”, etc.
  • the objects are usually of one type, and the number of objects is not limited.
  • the first object may be one or more than one.
  • “and/or” in the description and claims indicates at least one of the connected objects, and the character “/" generally indicates that the associated objects are in an "or” relationship.
  • FIG. 1 is a schematic flowchart of a data processing method provided by an embodiment of the present application. As shown in FIG. 1, the method includes the following S101 and S102:
  • the transmitting end device adjusts the data parameter of the data to be transmitted according to at least one of the network transmission state and the playing parameter of the receiving end device.
  • the data to be transmitted may be text data, image data, or audio data, which is not specifically limited in the embodiment of the present application.
  • the sending end device may be an electronic device with a smaller screen size, such as a mobile phone, a tablet computer, a notebook computer, etc.; the receiving end device may be an electronic device with a larger screen size, such as a tablet computer. , computers, televisions, projectors, etc.
  • the network transmission status indicates whether the transmission network used by the transmitting end device and the receiving end device to transmit data is stable during the data transmission process.
  • the transmission network may be a WiFi network, such as a dual-band wireless WiFi network.
  • the sending end device when the sending end device starts to project the screen to the receiving end device, or in the process of projecting the screen, the sending end device can obtain the network transmission status in real time.
  • the playback parameters of the receiving end device include a resolution, a frame rate, and the like of a supported video to be played.
  • a TV terminal can support video playback with a resolution of 4k and a frame rate of 60fps, which can be recorded as a playback capability of 4k@60fps.
  • the data parameters of the data to be transmitted may include at least one of the following: frame rate, resolution, contrast, and chromaticity.
  • the data parameters of the data to be transmitted include the picture parameters when the transmitting end device transmits the picture during the screen projection process.
  • parameters or parameter values of the data parameters of the data to be transmitted and the original data parameters of the screen projection may be different or the same, which are not specifically limited in this embodiment of the present application.
  • the sending end device can adjust the transmission parameter value of the picture to be transmitted according to the playback capability of the receiving end device, so as to reduce the amount of data to be transmitted, such as reducing the resolution of the picture to be transmitted. at least one of rate and frame rate, thereby increasing the playback smoothness of the picture to be transmitted.
  • the transmission data volume of the picture after adjusting the data parameters of the data to be transmitted is larger than the transmission data volume of the picture before the adjustment.
  • the sender device In the process of screen projection, it is assumed that the sender device is transmitting video at 1080p@30fps, and the receiver device has a strong playback capability, such as supporting 4k@60fps. If the sender device detects that the network environment is very stable, it can To support 4k@60fps video transmission, the sending end device can adjust the 1080p@30fps picture data to 4k@60fps through technologies such as super-subtraction and frame insertion, and then encode and transmit it to the receiving end device. In this way, the receiving end device can watch the video. The projection effect of 4k@60fps improves the projection experience.
  • the sending end device can adjust the transmission parameter value of the picture to be transmitted according to the playback capability of the receiving end device, and can increase the amount of data to be transmitted, such as increasing the resolution of the picture to be transmitted. at least one of the rate and the frame rate, thereby improving the picture quality of the picture to be transmitted.
  • the transmission data amount of the picture after adjusting the data parameter of the data to be transmitted is smaller than the transmission data amount of the picture before the adjustment.
  • the sender device In the process of screen projection, assuming that the sender device is transmitting video at 4k@60fps, if the sender device detects that the network environment is unstable, for example, the sender device detects that the network rate cannot meet the current screen projection transmission, then the receiver It is prone to screen freezes and affects the viewing experience after screen projection.
  • the sender device can reduce the resolution and frame rate of the 4k@60fps picture data, such as adjusting to 1080p@30fps, to reduce the amount of data transmitted, and then encode and package it for transmission, so as to avoid network congestion when the network becomes poor. The resulting lag at the receiving end.
  • the transmitting end device encodes the data to be transmitted after adjusting the data parameters to obtain encoded data, and transmits the encoded data to the receiving end device.
  • the data parameter of the data received by the receiving end device is the adjusted data parameter of the data to be transmitted.
  • the receiving end device displays the screen projection data on the screen with the adjusted data parameters.
  • An embodiment of the present application provides a data processing method.
  • the transmitting end device adjusts the data parameters of the data to be transmitted according to at least one of the network transmission state and the playback parameters of the receiving end device; then, the transmitting end device adjusts the data The data to be transmitted after the parameters is encoded to obtain encoded data, and finally the transmitting end device transmits the encoded data to the receiving end device. That is to say, in data transmission, the transmitting end device can flexibly adjust the data parameters of the data to be transmitted according to the network transmission state and the playback capability of the receiving end device before transmitting the data.
  • the picture quality can be improved by adjusting the data parameters of the data to be transmitted, so that the sending end device can determine
  • the receiving end device has strong playback capability, it can improve the screen projection experience on the screen of the receiving end device with strong playback capability
  • the specified data parameters reduce the amount of transmitted data, which can avoid the problem that the display screen of the receiving end is stuck due to the blocking of the network card when the network becomes poor.
  • the above-mentioned S101 may be specifically performed by S101a:
  • the sending end device adjusts the value of the data parameter of the data to be transmitted from the first value to the second value according to at least one of the network transmission state and the playback parameter of the receiving end device.
  • the second numerical value when at least one of the network transmission state and the playback parameter of the receiving end satisfies the first condition, the second numerical value is greater than the first numerical value. In the case that at least one of the network transmission state and the playback parameter of the receiving end satisfies the second condition, the second numerical value is smaller than the first numerical value.
  • At least one of the network transmission state and the playback parameters of the receiver satisfies the first condition, including at least one of the following (1) to (3):
  • the network transmission rate is greater than the first threshold
  • the first threshold may be set by default, or may be manually set by the user as required.
  • the network transmission rate is greater than the first threshold, it may be indicated that the network transmission state is better.
  • the network transmission rate is greater than the code rate of the data to be transmitted
  • the network transmission rate is greater than the code rate of the data to be transmitted, it can be indicated that the network can support data transmission with a higher code rate.
  • the playing capability of the receiving end can support the playing of data higher than the first numerical value of the data parameter. That is, the receiving end has a strong playback capability.
  • At least one of the network transmission state and the playback parameter of the receiver satisfies the second condition, including at least one of the following (4) to (6):
  • the network transmission rate is less than the second threshold
  • the second threshold may be set by default, or may be manually set by the user as required.
  • the network transmission rate is less than the second threshold, it may be indicated that the network transmission state is poor.
  • the first threshold is greater than or equal to the second threshold. If the network transmission rate is greater than the first threshold, it can indicate that the current network status is good and more data can be transmitted. Therefore, the picture quality can be improved first, and the resolution of the picture and the frame rate of the video can be increased; the network transmission rate is lower than the second threshold.
  • the threshold value can indicate that the current network status is poor, and there may be a delay in screen projection, so it can reduce the picture quality, the resolution of the picture and the frame rate of the video.
  • the network transmission rate is less than the code rate of the data to be transmitted
  • the network transmission rate is less than the code rate of the data to be transmitted, it can be indicated that the network state is poor, and network congestion may occur.
  • the parameter value of the playback parameter of the receiving end is smaller than the first value.
  • the parameter value of the playback parameter of the receiving end is less than the first value of the data parameter of the data to be transmitted, it may indicate that the playback capability of the receiving end cannot support the playback of data higher than the first value of the data parameter.
  • whether the picture needs to be adjusted can be calculated according to the code rate of the compressed data of the current picture and the current rate of WiFi transmission. If the WiFi rate is lower than the bit rate to be transmitted, it means that the current network transmission will be blocked, and the screencast data may not be transmitted to the receiving end device in time, which will cause the screencast to freeze.
  • the sender device can reduce the resolution of the transmitted picture according to the current network load, for example, if the resolution is determined to be 1080p, the electronic device can adjust The resolution of the picture to be transmitted is 1080p. Assuming that the currently transmitted frame rate is 60fps, if the network status is poor, the sender device can reduce the frame rate according to the current network load. For example, if the frame rate is determined to be 30fps, the sender device can adjust the frame rate of the picture to be transmitted to 30fps .
  • the sender device can increase the resolution of the transmitted picture according to the current network load. For example, if the resolution is determined to be 4k, the sender device can adjust The resolution of the picture to be transmitted is 4k. Assuming that the currently transmitted frame rate is 30fps, if the network status is good, the sender device can increase the frame rate according to the current network load. For example, if the frame rate is determined to be 60fps, the sender device can adjust the frame rate of the picture to be transmitted to 60fps .
  • the electronic device may increase the frame rate of the video by using the frame interpolation technique, and may increase the resolution of the video by using the super-resolution technique.
  • the frame rate and resolution can be adjusted at the same time.
  • the sender device can also adjust the contrast and chroma to ensure a better display effect.
  • the sending end device can judge in real time whether the data parameter of the data to be transmitted needs to be adjusted according to at least one of the network transmission state and the playback parameter of the receiving end device; when it is determined that the first condition is satisfied, it is determined that it needs to be adjusted higher.
  • the data parameters of the transmitted data when it is determined that the second condition is met, it is determined that the data parameters of the data to be transmitted need to be lowered, so that the parameters can be adjusted in time according to the network status, the screen projection display effect of the receiving device can be flexibly controlled, and the picture can be improved.
  • the clarity of the screen can be avoided, or the screen freeze caused by poor network conditions can be avoided in time.
  • the data processing method provided by this embodiment of the present application may further include the following S103 and S104 before the above-mentioned S101:
  • the sending end device acquires at least one of a network transmission state and a playback parameter of the receiving end device.
  • the transmitting end device can first determine the playback capability of the receiving end device, such as the maximum playback frame rate and the maximum supported resolution of the receiving end device.
  • the transmitting end device determines the data parameters of the data to be transmitted according to at least one of the network transmission state, the playback parameters of the receiving end device, and the original data parameters of the transmission screen.
  • the original data parameters of the source picture of the sending end device may include: frame rate, resolution, contrast, chromaticity, and dynamic range.
  • the sending end device after the sending end device establishes a screen projection connection with the receiving end device, before transmitting the screen projection data, based on at least one of the network transmission status and the playback parameters of the receiving end device, and the original data parameters of the transmission screen , determine the data parameters of the data to be transmitted, transmit video based on the data parameters of the data to be transmitted, and flexibly determine a set of data parameters suitable for the current network and achieve the best display effect of the receiving end device according to the current network status , which makes the screen projection experience better.
  • the playback capability of the receiving end device is high, and the original data parameter value is lowered according to the network transmission state.
  • the transmitting end device may adjust the data parameter according to the playback capability of the receiving end device after the transmitting end device establishes the screen projection connection.
  • Example a The source picture parameter is SDR and does not support HDR, then the sender device can convert the source picture from SDR to HDR.
  • Example b If the image quality of the source image is not good, the sender device can adjust the contrast and chromaticity of the source image first.
  • Example c The network environment is very good and can support 4k@60fps video transmission.
  • the playback capability of the receiver device is 4k@60fps, but the sender screen does not support 4k@60fps. It is assumed that the original screen played on the sender screen is 1080p@30fps If, according to the relevant technology, the content displayed on the screen of the sender is projected to the receiver device with a playback capability of 4k@60fps, and the video is still displayed at 1080p@30fps, then the video displayed on the receiver with a playback capability of 4k@60fps is still displayed. Poor viewing on the device's screen.
  • the sender device detects that the image quality of the source image is poor before the screen is projected, the image quality can be adjusted first, and then the data can be transmitted.
  • the image quality of the source picture can be adjusted before sending the picture.
  • you can adjust the picture quality of the source picture such as converting SDR to HDR, which can make the display content of the transmitted picture have richer details and hue; adjust the contrast, chroma, etc. of the picture, More hidden textures in dark areas of the picture can be displayed, and details in bright areas can be preserved, thereby improving the display effect of the receiving end device.
  • the user can control the sending end device to scan through WiFi to search for the receiving end device.
  • the sending end device and the receiving end device first establish a P2P connection through the P2P protocol, and then the P2P After the connection is successfully established, the sender device and the receiver device can negotiate capabilities through the Real Time Streaming Protocol (RTSP), and then establish a (WiFi Display, WFD) connection based on WiFi.
  • RTSP Real Time Streaming Protocol
  • the sender device will capture the source image, package it through encoding, and then transmit the packaged data to the receiver through the WiFi network. After receiving the data transmitted by the sender device, the receiver device first unpacks it, then decodes it, and finally displays the picture on the screen of the receiver device.
  • the sending end device may first determine the value of the data parameter of the data to be transmitted, for example, according to the network transmission state, the playback parameter of the receiving end device , The original data parameters of the screen projected by the sending end device determine the specific values of the data parameters of the data to be transmitted. The value adjusts the transmission picture, and then encodes and packs it for transmission to the receiving end device.
  • the sender device can also perform network transmission status monitoring (or network load detection), and then perform load control according to the network transmission status.
  • the data size of the picture such as reducing the resolution, frame rate, etc., appropriately reduces the picture definition to ensure the smooth projection of the screen; when the network transmission status is good, the resolution and frame rate can be increased, and the receiving end device can be improved.
  • the display effect of the picture displayed on the screen when the network transmission status is good, the resolution and frame rate can be increased, and the receiving end device can be improved.
  • the electronic device may include: an external image chip, an encoder and a radio frequency unit; the external image chip is used to adjust the data parameters of the data to be transmitted according to at least one of the network transmission state and the playback parameters of the receiving end device.
  • the encoder is used to encode the data to be transmitted after adjusting the data parameters to obtain the encoded data; the radio frequency unit is used to transmit the encoded data to the receiving end device.
  • the electronic device further includes an adjustment control module. Specifically, when the adjustment control module detects that the network rate reaches the set threshold condition, the adjustment control module can synchronously adjust the data parameters to the external image chip.
  • the electronic device can send data into the external image chip for processing, and after the external image chip is processed, the processed data can be sent back to other functional modules at the sending end for further processing.
  • the adjustment control module can detect the network status according to the captured data, and then calculate the data parameters of the data to be transmitted according to conditions such as the receiver's capability and network load status, for example, to obtain a resolution frame rate suitable for transmission. Then, the external image chip processes the image to be transmitted according to the data parameters calculated by the adjustment control module, such as adjusting the resolution, frame rate, contrast, brightness, etc. of the original image.
  • the adjustment control module detects that the network status is not good during the screen projection process
  • the resolution and frame rate can be reduced through the external image chip, and the contrast and brightness can be adjusted adaptively, and then encoded and packaged by other functional modules of the electronic device. transmitted over the network to the receiving device.
  • an external graphics chip it is also possible to optimize the original image, such as converting from SDR to HDR, and adjust contrast and brightness, if it is determined that the image quality of the original image itself is poor before the sending end device casts the screen.
  • the plug-in graphics chip sends the adjusted data back to the encoder of the sending end device.
  • the encoder encodes and packages the adjusted data, and then the sending end
  • the device then sends the encoded data packet to the receiving end device, and the receiving end device displays the projection screen on the screen according to the data parameters of the data sent by the sending end device.
  • the execution body may be a data processing apparatus, or a control module in the data processing apparatus for executing the method for screen projection.
  • the method for performing screen projection by a data processing device is used as an example to describe the device for screen projection provided by the embodiments of the present application.
  • FIG. 2 is a data processing apparatus provided by an embodiment of the application.
  • the data processing apparatus 200 includes: an adjustment module 201, an encoding module 202, and a transmission module 203; At least one of the items is to adjust the data parameters of the data to be transmitted; the encoding module 202 is used to encode the data to be transmitted after adjusting the data parameters to obtain encoded data; the transmission module 203 is used to transmit the encoded data to the receiving end device.
  • the adjustment module is specifically configured to: adjust the value of the data parameter of the data to be transmitted from the first value to the second value; wherein, at least one of the network transmission state and the playback parameter of the receiving end satisfies the first condition In the case of , the second value is greater than the first value; when at least one of the network transmission state and the playback parameter of the receiving end satisfies the second condition, the second value is less than the first value.
  • At least one of the network transmission state and the playback parameter of the receiving end satisfies the first condition, including at least one of the following: the network transmission rate is greater than the first threshold; the network transmission rate is greater than the code rate of the data to be transmitted; The parameter value of the parameter is greater than the first numerical value.
  • At least one of the network transmission state and the playback parameter of the receiving end satisfies the second condition, including at least one of the following: the network transmission rate is less than the second threshold; the network transmission rate is less than the code rate of the data to be transmitted; The parameter value of the parameter is less than the first value.
  • the data parameters include at least one of the following: frame rate, resolution, contrast, and chromaticity.
  • An embodiment of the present application provides a data processing apparatus.
  • the data processing apparatus adjusts the data parameters of the data to be transmitted according to at least one of the network transmission status and the playback parameters of the receiving end device; then, the data processing apparatus adjusts the data The data to be transmitted after the parameters is encoded to obtain encoded data, and finally the data processing device transmits the encoded data to the receiving end device. That is to say, in data transmission, the data processing apparatus can flexibly adjust the data parameters of the data to be transmitted according to the network transmission state and the playback capability of the receiving end device before transmitting the data.
  • the picture quality can be improved by adjusting the data parameters of the data to be transmitted, so that the sending end device can determine
  • the receiving end device has strong playback capability, it can improve the screen projection experience on the screen of the receiving end device with strong playback capability
  • the specified data parameters reduce the amount of transmitted data, which can avoid the problem that the display screen of the receiving end is stuck due to the blocking of the network card when the network becomes poor.
  • the data processing apparatus in this embodiment of the present application may be an apparatus, or may be a component, an integrated circuit, or a chip in a terminal.
  • the apparatus may be a mobile electronic device or a non-mobile electronic device.
  • the mobile electronic device may be a mobile phone, a tablet computer, a notebook computer, a palmtop computer, an in-vehicle electronic device, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (personal digital assistant).
  • UMPC ultra-mobile personal computer
  • netbook or a personal digital assistant
  • non-mobile electronic devices can be servers, network attached storage (Network Attached Storage, NAS), personal computer (personal computer, PC), television (television, TV), teller machine or self-service machine, etc., this application Examples are not specifically limited.
  • Network Attached Storage NAS
  • personal computer personal computer, PC
  • television television
  • teller machine or self-service machine etc.
  • the data processing apparatus in this embodiment of the present application may be an apparatus having an operating system.
  • the operating system may be an Android (Android) operating system, an ios operating system, or other possible operating systems, which are not specifically limited in the embodiments of the present application.
  • the data processing apparatus provided in the embodiments of the present application can implement each process implemented by the data processing apparatus in the foregoing method embodiments, and to avoid repetition, details are not repeated here.
  • an embodiment of the present application further provides an electronic device 300, including a processor 301, a memory 302, a program or instruction stored in the memory 302 and executable on the processor 301, When the program or instruction is executed by the processor 301, each process of the above data processing method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, details are not repeated here.
  • the electronic devices in the embodiments of the present application include the aforementioned mobile electronic devices and non-mobile electronic devices.
  • FIG. 4 is a schematic diagram of a hardware structure of an electronic device implementing an embodiment of the present application.
  • the electronic device 1000 includes but is not limited to: a radio frequency unit 1001, a network module 1002, an audio output unit 1003, an input unit 1004, a sensor 1005, a display unit 1006, a user input unit 1007, an interface unit 1008, a memory 1009, and a processor 1010, etc. part.
  • the processor 1010 may be configured to perform the functions of the adjustment control module and the encoder described above.
  • the electronic device 1000 may further include an external image chip, which may be used to implement the function of adjusting the data parameters of the data to be transmitted according to at least one of the network transmission state and the playback parameters of the receiving end device.
  • an external image chip which may be used to implement the function of adjusting the data parameters of the data to be transmitted according to at least one of the network transmission state and the playback parameters of the receiving end device.
  • the processor 1010 may implement the functions of the above-mentioned external image chip and the functions of the adjustment control module and the encoder.
  • the electronic device 1000 may also include a power source (such as a battery) for supplying power to various components, and the power source may be logically connected to the processor 1010 through a power management system, so that the power management system can manage charging, discharging, and power functions. consumption management and other functions.
  • a power source such as a battery
  • the structure of the electronic device shown in FIG. 4 does not constitute a limitation to the electronic device.
  • the electronic device may include more or less components than the one shown, or combine some components, or arrange different components, which will not be repeated here. .
  • the processor 1010 is configured to adjust the data parameters of the data to be transmitted according to at least one of the network transmission state and the playback parameters of the receiving end device; encode the data to be transmitted after adjusting the data parameters to obtain encoded data;
  • the radio frequency unit 1001 is used for transmitting encoded data to the receiving end device.
  • An embodiment of the present application provides an electronic device, where the electronic device is a sending end device.
  • the sending end device adjusts a data parameter of data to be transmitted according to at least one of a network transmission state and a playback parameter of the receiving end device; then , the transmitting end device encodes the data to be transmitted after adjusting the data parameters to obtain encoded data, and finally the transmitting end device transmits the encoded data to the receiving end device. That is to say, in data transmission, the transmitting end device can flexibly adjust the data parameters of the data to be transmitted according to the network transmission state and the playback capability of the receiving end device before transmitting the data.
  • the picture quality can be improved by adjusting the data parameters of the data to be transmitted, so that the sending end device can determine
  • the receiving end device has strong playback capability, it can improve the screen projection experience on the screen of the receiving end device with strong playback capability
  • the specified data parameters reduce the amount of transmitted data, which can avoid the problem that the display screen of the receiving end is stuck due to the blocking of the network card when the network becomes poor.
  • the value of the data parameter of the data to be transmitted is adjusted from the first numerical value to the second numerical value; wherein, when at least one of the network transmission state and the playback parameter of the receiving end satisfies the first condition, the second numerical value is The numerical value is greater than the first numerical value; when at least one of the network transmission state and the playback parameter of the receiving end satisfies the second condition, the second numerical value is less than the first numerical value.
  • the input unit 1004 may include a graphics processor (Graphics Processing Unit, GPU) 1041 and a microphone 1042. Such as camera) to obtain still pictures or video image data for processing.
  • the display unit 1006 may include a display panel 1061, which may be configured in the form of a liquid crystal display, an organic light emitting diode, or the like.
  • the user input unit 1007 includes a touch panel 1071 and other input devices 1072 .
  • the touch panel 1071 is also called a touch screen.
  • the touch panel 1071 may include two parts, a touch detection device and a touch controller.
  • Other input devices 1072 may include, but are not limited to, physical keyboards, function keys (such as volume control keys, switch keys, etc.), trackballs, mice, and joysticks, which are not described herein again.
  • Memory 1009 may be used to store software programs as well as various data, including but not limited to application programs and operating systems.
  • the processor 1010 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the like, and the modem processor mainly processes wireless communication. It can be understood that, the above-mentioned modulation and demodulation processor may not be integrated into the processor 1010.
  • the embodiments of the present application further provide a readable storage medium, where a program or an instruction is stored on the readable storage medium, and when the program or instruction is executed by a processor, each process of the above data processing method embodiment can be implemented, and the same can be achieved. In order to avoid repetition, the technical effect will not be repeated here.
  • the processor is the processor in the electronic device described in the foregoing embodiments.
  • the readable storage medium includes a computer-readable storage medium, such as a computer read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a magnetic disk or an optical disk, and the like.
  • An embodiment of the present application further provides a chip, where the chip includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is configured to run a program or an instruction to implement the above data processing method embodiments.
  • the chip includes a processor and a communication interface
  • the communication interface is coupled to the processor
  • the processor is configured to run a program or an instruction to implement the above data processing method embodiments.
  • the chip mentioned in the embodiments of the present application may also be referred to as a system-on-chip, a system-on-chip, a system-on-a-chip, or a system-on-a-chip, or the like.
  • the embodiment of the present application further provides a computer program product, the program product is stored in a non-volatile storage medium, the program product is executed by at least one processor to implement each process of the foregoing data processing method embodiment, and The same technical effect can be achieved, and in order to avoid repetition, details are not repeated here.

Landscapes

  • Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
  • Controls And Circuits For Display Device (AREA)

Abstract

本申请公开了一种数据处理方法、装置及电子设备，属于通信技术领域。该方法包括：根据网络传输状态和接收端设备的播放参数中的至少一项，对待传输数据的数据参数进行调整；将调整数据参数后的待传输数据进行编码，得到编码数据，并向所述接收端设备传输该编码数据。

Description

数据处理方法、装置及电子设备
相关申请的交叉引用
本申请主张在2020年11月30日在中国提交的中国专利申请号202011380626.1的优先权，其全部内容通过引用包含于此。
技术领域
本申请属于通信技术领域，具体涉及一种数据处理方法、装置及电子设备。
背景技术
随着通信技术的发展，电子设备的功能越来越强大，比如，用户可以将小屏幕设备的画面通过无线投屏技术同步显示到大屏幕的设备的屏幕上显示。
通常，在无线投屏的过程中，小屏幕设备源端输出屏幕画面的分辨率和帧率设置有较低的最大值，例如帧率最大为30fps、分辨率最大为1080p。
然而，在视频源的帧率高于30fps或者分辨率高于1080p，且大屏幕设备支持较高的分辨率或帧率的情况下，小屏幕设备只能以30fps的帧率和1080p的分辨率进行投屏，限制了大屏幕设备的显示能力，如此导致在投屏过程中的投屏的视频资源的显示效果不佳。
发明内容
本申请实施例的目的是提供一种数据处理方法、装置及电子设备，能够解决在投屏过程中的投屏的视频资源的显示效果不佳的问题。
为了解决上述技术问题，本申请是这样实现的：
第一方面，本申请实施例提供了一种数据处理方法，该方法包括：根据网络传输状态和接收端设备的播放参数中的至少一项，对待传输数据的数据参数进行调整；将调整数据参数后的待传输数据进行编码，得到编码数据，并向接收端设备传输该编码数据。
第二方面，本申请实施例提供了一种数据处理装置，该数据处理装置包括：调整模块、编码模块和传输模块；调整模块，用于根据网络传输状态和接收端设备的播放参数中的至少一项，对待传输数据的数据参数进行调整；编码模块，用于将调整数据参数后的待传输数据进行编码，得到编码数据；传输模块，用于向接收端设备传输该编码数据。
第三方面，本申请实施例提供了一种电子设备，电子设备包括：外挂图像芯片、编码器和射频单元；外挂图像芯片，用于根据网络传输状态和接收端设备的播放参数中的至少一项，对待传输数据的数据参数进行调整；编码器，用于将调整数据参数后的待传输数据进行编码，得到编码数据；射频单元，用于向接收端设备传输该编码数据。
第四方面，本申请实施例提供了一种电子设备，该电子设备包括处理器、存储器及存储在所述存储器上并可在所述处理器上运行的程序或指令，所述程序或指令被所述处理器执行时实现如第一方面所述的方法的步骤。
第五方面，本申请实施例提供了一种可读存储介质，所述可读存储介质上存储程序或指令，所述程序或指令被处理器执行时实现如第一方面所述的方法的步骤。
第六方面，本申请实施例提供了一种芯片，所述芯片包括处理器和通信接口，所述通信接口和所述处理器耦合，所述处理器用于运行程序或指令，实现如第一方面所述的方法。
第七方面，本申请实施例提供了一种计算机程序产品，所述程序产品被至少一个处理器执行以实现第一方面所述的方法。
在本申请实施例中，首先，发送端设备根据网络传输状态和接收端设备的播放参数中的至少一项，对待传输数据的数据参数进行调整；然后，发送端设备将调整数据参数后的待传输数据进行编码，得到编码数据，最后发送端设备向接收端设备传输编码数据。也就是说，在数据传输中，发送端设备可以在传输数据之前，根据网络传输状态以及接收端设备的播放能力，灵活调整待传输数据的数据参数。例如，在投屏过程中，一方面，在网络传输状态较好且接收端设备的播放能力强的情况下，可以通过调整待传输数据的数据参数改善画面质量，从而可以使得发送端设备在确定接收端设 备播放能力较强的情况下，可以提升画面在播放能力强的接收端设备的屏幕上的投屏体验；另一方面，在网络传输状态较差的情况下，可以通过调整待传输数据的数据参数减少传输数据量，可以避免在网络变差时因网卡阻塞导致的接收端显示画面卡顿的问题。
附图说明
图1为本申请实施例提供的一种数据处理方法的流程示意图；
图2为本申请实施例提供的一种数据处理装置可能的结构示意图；
图3为本申请实施例提供的一种电子设备可能的结构示意图；
图4为本申请实施例提供的一种电子设备的硬件示意图。
具体实施方式
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
本申请的说明书和权利要求书中的术语“第一”、“第二”等是用于区别类似的对象，而不用于描述特定的顺序或先后次序。应该理解这样使用的数据在适当情况下可以互换，以便本申请的实施例能够以除了在这里图示或描述的那些以外的顺序实施，且“第一”、“第二”等所区分的对象通常为一类，并不限定对象的个数，例如第一对象可以是一个，也可以是多个。此外，说明书以及权利要求中“和/或”表示所连接对象的至少其中之一，字符“/”，一般表示前后关联对象是一种“或”的关系。
下面结合附图，通过具体的实施例及其应用场景对本申请实施例提供的数据处理方法进行详细地说明。
图1为本申请实施例提供的一种数据处理方法的流程示意图。如图1中所示，该方法包括下述的S101和S102：
S101、发送端设备根据网络传输状态和接收端设备的播放参数中的至少一项，对待传输数据的数据参数进行调整。
需要说明的是，在本申请实施例中，待传输数据可以为文本数据、图像数据，也可以为音频数据，本申请实施例对此不作具体限定。
为了便于说明，下面以发送端设备向接收端设备投屏为例进行说明。
示例性地，在本申请实施例中，发送端设备可以为屏幕尺寸较小的电子设备，例如手机、平板电脑、笔记本电脑等；接收端设备可以为屏幕尺寸较大的电子设备，例如平板电脑、电脑、电视机、投影仪等。
在本申请实施例中，网络传输状态指示发送端设备和接收端设备在数据传输过程中，传输数据使用的传输网络是否稳定。
示例性地，传输网络可以为WiFi网络，比如双频无线WiFi网络。
具体的，在发送端设备向接收端设备开始投屏，或者在投屏的过程中，发送端设备可以实时获取网络传输状态。
示例性地，接收端设备的播放参数包括支持播放视频的分辨率、帧率等。
例如，一个电视端可以支持分辨率为4k、帧率为60fps的视频播放，可以记为播放能力为4k@60fps。
其中，待传输数据的数据参数可以包括以下至少一项：帧率、分辨率、对比度、色度。
需要说明的是，待传输数据的数据参数包括发送端设备在投屏过程中传输画面时的画面参数。
可以理解的是，待传输数据的数据参数和投屏的原始数据参数的参数或参数值可以不同，也可以相同，本申请实施例对此不作具体限定。
需要说明的是，待传输数据的数据参数和投屏的原始数据参数的参数相同的情况下，参数的具体值可以相同，也可以不同。
可选地，在网络传输状态不佳的情况下，发送端设备可以根据接收端设备的播放能力，调整待传输画面的传输参数值，以减少传输的数据量，比如调小待传输画面的分辨率和帧率的至少一项，从而增加待传输画面的播放流畅度。
即，在网络状态较好的情况下，调整待传输数据的数据参数后的画面的传输数据量大于调整前的画面的传输数据量。
示例1-1
若在投屏过程中，假设发送端设备正在以1080p@30fps传输视频，在接收端设备的播放能力较强的情况下，比如支持4k@60fps，若发送端设备检测到网络环境非常稳定，可以支持4k@60fps的视频传输，则发送端设备可以将1080p@30fps的画面数据通过超分插帧等技术调整为4k@60fps，然后再进行编码传输给接收端设备，这样，接收端设备可以看到4k@60fps的投屏效果，提升了投屏体验。
可选地，在网络传输状态较佳的情况下，发送端设备可以根据接收端设备的播放能力，调整待传输画面的传输参数值，可以增加传输的数据量，比如调大待传输画面的分辨率和帧率中的至少一项，从而改善待传输画面的画面质量。
即，在网络状态较差的情况下，调整待传输数据的数据参数后的画面的传输数据量小于调整前的画面的传输数据量。
示例1-2
若在投屏过程中，假设发送端设备正在以4k@60fps传输视频，若发送端设备检测到网络环境不稳定，比如发送端设备检测到网络速率无法满足当前的投屏传输，此时接收端容易出现画面卡顿、影响投屏后的观看体验。则发送端设备可以将4k@60fps的画面数据降低分辨率、帧率，比如调整为1080p@30fps，减少了传输的数据量，然后再进行编码打包传输，从而可以避免网络变差时因网络阻塞导致的接收端播放的卡顿。
S102、发送端设备将调整数据参数后的待传输数据进行编码，得到编码数据，并向接收端设备传输编码数据。
需要说明的是，接收端设备接收的数据的数据参数即为调整后的待传输数据的数据参数。
具体的，接收端设备在接收到调整待传输数据的数据参数后的画面，以调整后的数据参数在屏幕上显示投屏数据。
本申请实施例提供一种数据处理方法，首先，发送端设备根据网络传输状态和接收端设备的播放参数中的至少一项，对待传输数据的数据参数进行调整；然后，发送端设备将调整数据参数后的待传输数据进行编码，得到编码数据，最后发送端设备向 接收端设备传输编码数据。也就是说，在数据传输中，发送端设备可以在传输数据之前，根据网络传输状态以及接收端设备的播放能力，灵活调整待传输数据的数据参数。例如，在投屏过程中，一方面，在网络传输状态较好且接收端设备的播放能力强的情况下，可以通过调整待传输数据的数据参数改善画面质量，从而可以使得发送端设备在确定接收端设备播放能力较强的情况下，可以提升画面在播放能力强的接收端设备的屏幕上的投屏体验；另一方面，在网络传输状态较差的情况下，可以通过调整待传输数据的数据参数减少传输数据量，可以避免在网络变差时因网卡阻塞导致的接收端显示画面卡顿的问题。
可选地，本申请实施例提供的数据处理方法，上述的S101具体可以通过S101a执行：
S101a、发送端设备根据网络传输状态和接收端设备的播放参数中的至少一项，将待传输数据的数据参数的取值由第一数值调整至第二数值。
其中，在网络传输状态和接收端的播放参数中的至少一项满足第一条件的情况下，第二数值大于第一数值。在网络传输状态和接收端的播放参数中的至少一项满足第二条件的情况下，第二数值小于第一数值。
可选地，网络传输状态和接收端的播放参数中的至少一项满足第一条件，包括以下(1)至(3)中的至少一项：
(1)网络传输速率大于第一阈值；
其中，第一阈值可以为默认设置的，也可以为用户根据需要手动设置的。
需要说明的是，在网络传输速率大于第一阈值的情况下，可以指示网络传输状态较好。
(2)网络传输速率大于待传输数据的码率；
可以理解的是，在网络传输速率大于待传输数据的码率的情况下，可以指示网络能够支持更高的码率的数据传输。
(3)接收端的播放参数的参数值大于第一数值。
需要说明的是，在接收端的播放参数的参数值大于待传输数据的数据参数的第一数值的情况下，可以表示接收端的播放能力可以支持比数据参数第一数值更高的数据 的播放。即接收端播放能力较强。
可选地，网络传输状态和接收端的播放参数中的至少一项满足第二条件，包括以下(4)至(6)中的至少一项：
(4)网络传输速率小于第二阈值；
其中，第二阈值可以为默认设置的，也可以为用户根据需要手动设置的。
需要说明的是，在网络传输速率小于第二阈值的情况下，可以指示网络传输状态较差。
可以理解，第一阈值大于或等于第二阈值。网络传输速率大于第一阈值，可以指示当前网络状态较好，可传输更多的数据，因此可以先改善画面的画质，调高画面的分辩率和视频的帧率；网络传输速率小于第二阈值，可以指示当前网络状态较差，可能存在投屏延时，因此可以降低画面的画质，降低画面的分辨率和视频的帧率等。
(5)网络传输速率小于待传输数据的码率；
可以理解的是，在网络传输速率小于待传输数据的码率的情况下，可以指示网络状态较差，可能会出现网络阻塞。
(6)接收端的播放参数的参数值小于第一数值。
需要说明的是，在接收端的播放参数的参数值小于待传输数据的数据参数的第一数值的情况下，可以表示接收端的播放能力不能支持比数据参数第一数值的数据的播放。
示例性地，可以根据当前画面压缩后的数据的码率和当前WiFi传输的速率计算是否需要调整画面。如果WiFi速率比要传输的码率小，则表示当前网络传输会阻塞，可能无法及时将投屏数据传输到接收端设备，就会造成投屏卡顿。
示例性地，假设当前传输的画面分辨率为4k，若网络状态不佳，则发送端设备可以根据当前的网络负载降低传输的画面的分辨率，比如确定分辨率为1080p，则电子设备可以调整待传输画面的分辩率为1080p。假设当前传输的帧率为60fps，若网络状态不佳，则发送端设备可以根据当前的网络负载降低帧率，比如确定帧率为30fps，则发送端设备可以调整待传输画面的帧率为30fps。
同样的，若当前传输的画面分辨率为1080p，若网络状态较佳，则发送端设备可 以根据当前的网络负载提高传输的画面的分辨率，比如确定分辨率为4k，则发送端设备可以调整待传输画面的分辩率为4k。假设当前传输的帧率为30fps，若网络状态较佳，则发送端设备可以根据当前的网络负载提高帧率，比如确定帧率为60fps，则发送端设备可以调整待传输画面的帧率为60fps。
示例性地，电子设备可以通过插帧技术调高视频的帧率，可以通过超分技术调高视频的分辨率。
需要说明的是，上述示例以分别调整帧率和分辨率进行说明，在实际应用中，基于网络传输状态和接收端设备的播放能力，可以同时调整帧率和分辨率，在调整帧率和分辨率之后，发送端设备还可以适应的调整对比度和色度，以保证较好的显示效果。
基于该方案，发送端设备可以根据网络传输状态和接收端设备的播放参数中的至少一项实时判断是否需要调整待传输数据的数据参数；在确定满足第一条件时，则确定需要调高待传输数据的数据参数，在确定满足第二条件时，则确定需要调低待传输数据的数据参数，从而可以根据网络状态及时调整参数，能够灵活控制接收端设备的投屏显示效果，可以提高画面的清晰度，或者可以及时避免在网络条件变差的情况下导致的画面卡顿。
可选地，本申请实施例提供的数据处理方法，在上述的S101之前，还可以包括下述的S103和S104：
S103、在发送端设备和接收端设备建立投屏连接的情况下，发送端设备获取网络传输状态和接收端设备的播放参数中的至少一项。
可以理解，发送端设备可以在建立投屏连接之后，可以先确定接收端设备的播放能力，比如接收端设备的最大播放帧率、最大支持的分辨率等。
S104、发送端设备根据网络传输状态和接收端设备的播放参数中的至少一项，以及传输画面的原始数据参数，确定待传输数据的数据参数。
示例性地，发送端设备的源画面的原始数据参数可以包括：帧率、分辨率、对比度、色度、动态范围。
基于该方案，发送端设备可以在和接收端设备建立投屏连接之后，在传输投屏数据之前，基于网络传输状态和接收端设备的播放参数中的至少一项，以及传输画面的 原始数据参数，确定待传输数据的数据参数，基于该待传输数据的数据参数传输视频，可以灵活的根据当前的网络状态，确定一组适合当前网络，并且可以达到接收端设备最佳的显示效果的数据参数，使得投屏过程体验更佳。
可选地，在原始数据参数值较高，网络状况不佳的情况下，接收端设备的播放能力较高，根据网络传输状态，调低原始数据参数值。
可选地，在原始数据参数对应的显示效果本身较差的情况下，发送端设备可以发送端设备在建立投屏连接之后，根据接收端设备的播放能力调整数据参数。
示例a：源画面参数中为SDR，未支持HDR，则发送端设备可以将源画面进行SDR转HDR。
示例b：源画面的画质不佳，则发送端设备可以先调整源画面的对比度、色度。
示例c：网络环境非常好，可以支持4k@60fps的视频传输，接收端设备的播放能力为4k@60fps，发送端屏幕未支持到4k@60fps，假设发送端屏幕播放的原始画面为1080p@30fps的视频，若按照相关技术，将发送端屏幕显示内容投屏到播放能力为4k@60fps的接收端设备后，显示的依然为1080p@30fps的视频，则在播放能力为4k@60fps的接收端设备的屏幕上的观看效果不佳。
需要说明的是，由于对比度、色度的调整以及SDR调整为HDR，画面的数据量增加较少，因此发送端设备可以直接进行调整。
基于该方案，若发送端设备在投屏之前，检测到源画面的画质不佳，可以先进行画质的调整，然后再进行数据的传输。比如在发送端中，若原始的视频、游戏本身未支持HDR，源画面的对比度、色度无法显示暗区域的纹理细节，原始画面的效果就不佳导致投屏到接收端的屏幕的画面效果难以提升。本申请中，可以在发送画面之前，先对源画面进行画质调整，比如进行SDR转HDR，可以使得传输的画面的显示内容具有更丰富的细节和色调；调整画面的对比度、色度等，可以显示更多画面的暗区域中隐藏的纹理，保留明亮区域的细节，从而可以提高接收端设备的显示效果。
示例性地，用户可以控制发送端设备通过WiFi扫描，搜索接收端设备，在用户确定建立投屏连接的接收端设备后，发送端设备和接收端设备先通过P2P协议先建立P2P连接，在P2P连接建立成功后，发送端设备和接收端设备可以通过实时流传输协议 (Real Time Streaming Protocol，RTSP)进行能力协商，然后基于WiFi建立(WiFi Display，WFD)连接。
通常，发送端设备和接收端设备的投屏连接建立成功后，开始音视频传输时，发送端设备会抓取源画面，经过编码处理打包，再通过WiFi网络将打包好的数据传输给接收端设备，接收端设备接收到发送端设备传输的数据后，先解包，然后解码，最终将画面显示在接收端设备的屏幕上。
本申请实施例中，在发送端设备和接收端设备的投屏连接建立成功后，发送端设备可以先确定待传输数据的数据参数的数值，比如可以根据网络传输状态、接收端设备的播放参数、发送端设备投屏的画面的原始数据参数，确定待传输数据的数据参数的具体值，在待传输数据的数据参数值和原始数据参数值不同的情况下，根据确定待传输数据的数据参数值调整传输画面，然后编码打包传输给接收端设备。在投屏过程中，发送端设备还可以进行网络传输状态监测(或称为网络负载检测)，然后根据网络传输状态进行负载控制，在网络传输状态差的情况下，可以根据网络负载情况调低画面的数据大小，比如降低分辨率、帧率等，适当的降低画面清晰度以保证投屏的流畅；在网络传输状态好的情况下，可以提高分辩率和帧率，可以提高接收端设备的屏幕上显示的画面的显示效果。
可选地，电子设备可以包括：外挂图像芯片、编码器和射频单元；外挂图像芯片，用于根据网络传输状态和接收端设备的播放参数中的至少一项，对待传输数据的数据参数进行调整；编码器，用于将调整数据参数后的待传输数据进行编码，得到编码数据；射频单元，用于向接收端设备传输所述编码数据。
可选地，电子设备中还包括调整控制模块。具体的，在调整控制模块检测到网络速率达到设定的阈值条件时，调整控制模块可以将数据参数同步调整给外挂图像芯片。电子设备可以将数据送入该外挂图像芯片进行处理，该外挂图像芯片处理之后，可以将处理后的数据送回给发送端的其他功能模块继续处理。
具体的，调整控制模块可以根据抓取的数据，检测网络状态，然后根据接收端能力、网络负载状态等条件，计算待传输数据的数据参数，比如计算得到一个适合传输的分辨率帧率。然后，外挂图像芯片根据调整控制模块计算得到的数据参数对待传输 画面进行处理，比如调整原始画面的分辨率、帧率、对比度、亮度等。
可选地，当投屏过程中调整控制模块检测到网络状态不佳时，可以通过外挂图像芯片降低分辨率、帧率，并适应调整对比度和亮度，然后由电子设备的其他功能模块编码打包，通过网络传输到接收端设备。
可选地，外挂图形芯片，还可以在发送端设备投屏之前确定原始画面本身画质较差的情况下，先优化原始画面，例如SDR转HDR，调整对比度、亮度等。
需要说明的是，外挂图形芯片在调整待传输数据的数据参数的数值之后，再将调整后的数据回传给发送端设备的编码器，编码器对调整后的数据进行编码打包，然后发送端设备再将编码后的数据包发送给接收端设备，接收端设备根据发送端设备发送的数据的数据参数在屏幕上显示投屏画面。
需要说明的是，本申请实施例提供的数据处理方法，执行主体可以为数据处理装置，或者该数据处理装置中的用于执行投屏的方法的控制模块。本申请实施例中以数据处理装置执行投屏的方法为例，说明本申请实施例提供的投屏的装置。
图2为申请实施例提供的一种数据处理装置，数据处理装置200包括：调整模块201、编码模块202和传输模块203；调整模块201，用于根据网络传输状态和接收端设备的播放参数中的至少一项，对待传输数据的数据参数进行调整；编码模块202，用于将调整数据参数后的待传输数据进行编码，得到编码数据；传输模块203，用于向接收端设备传输编码数据。
可选地，调整模块具体用于：将待传输数据的数据参数的取值由第一数值调整至第二数值；其中，在网络传输状态和接收端的播放参数中的至少一项满足第一条件的情况下，第二数值大于第一数值；在网络传输状态和接收端的播放参数中的至少一项满足第二条件的情况下，第二数值小于第一数值。
可选地，网络传输状态和接收端的播放参数中的至少一项满足第一条件，包括以下至少一项：网络传输速率大于第一阈值；网络传输速率大于待传输数据的码率；接收端的播放参数的参数值大于第一数值。
可选地，网络传输状态和接收端的播放参数中的至少一项满足第二条件，包括以下至少一项：网络传输速率小于第二阈值；网络传输速率小于待传输数据的码率；接 收端的播放参数的参数值小于第一数值。
可选地，数据参数包括以下至少一项：帧率、分辨率、对比度、色度。
本申请实施例提供一种数据处理装置，首先，数据处理装置根据网络传输状态和接收端设备的播放参数中的至少一项，对待传输数据的数据参数进行调整；然后，数据处理装置将调整数据参数后的待传输数据进行编码，得到编码数据，最后数据处理装置向接收端设备传输编码数据。也就是说，在数据传输中，数据处理装置可以在传输数据之前，根据网络传输状态以及接收端设备的播放能力，灵活调整待传输数据的数据参数。例如，在投屏过程中，一方面，在网络传输状态较好且接收端设备的播放能力强的情况下，可以通过调整待传输数据的数据参数改善画面质量，从而可以使得发送端设备在确定接收端设备播放能力较强的情况下，可以提升画面在播放能力强的接收端设备的屏幕上的投屏体验；另一方面，在网络传输状态较差的情况下，可以通过调整待传输数据的数据参数减少传输数据量，可以避免在网络变差时因网卡阻塞导致的接收端显示画面卡顿的问题。
本申请实施例中的数据处理装置可以是装置，也可以是终端中的部件、集成电路、或芯片。该装置可以是移动电子设备，也可以为非移动电子设备。示例性地，移动电子设备可以为手机、平板电脑、笔记本电脑、掌上电脑、车载电子设备、可穿戴设备、超级移动个人计算机(ultra-mobile personal computer，UMPC)、上网本或者个人数字助理(personal digital assistant，PDA)等，非移动电子设备可以为服务器、网络附属存储器(Network Attached Storage，NAS)、个人计算机(personal computer，PC)、电视机(television，TV)、柜员机或者自助机等，本申请实施例不作具体限定。
本申请实施例中的数据处理装置可以为具有操作系统的装置。该操作系统可以为安卓(Android)操作系统，可以为ios操作系统，还可以为其他可能的操作系统，本申请实施例不作具体限定。
本申请实施例提供的数据处理装置能够实现上述方法实施例中数据处理装置实现的各个过程，为避免重复，这里不再赘述。
可选地，如图3所示，本申请实施例还提供一种电子设备300，包括处理器301，存储器302，存储在存储器302上并可在所述处理器301上运行的程序或指令，该程 序或指令被处理器301执行时实现上述数据处理方法实施例的各个过程，且能达到相同的技术效果，为避免重复，这里不再赘述。
需要说明的是，本申请实施例中的电子设备包括上述所述的移动电子设备和非移动电子设备。
图4为实现本申请实施例的一种电子设备的硬件结构示意图。
该电子设备1000包括但不限于：射频单元1001、网络模块1002、音频输出单元1003、输入单元1004、传感器1005、显示单元1006、用户输入单元1007、接口单元1008、存储器1009、以及处理器1010等部件。
可选地，处理器1010可以用于执行上述调整控制模块和编码器的功能。
可选地，电子设备1000还可以包括一个外挂图像芯片，可以用于实现根据网络传输状态和接收端设备的播放参数中的至少一项，对待传输数据的数据参数进行调整的功能。
可选地，处理器1010可以实现上述的外挂图像芯片的功能以及调整控制模块和编码器的功能。
本领域技术人员可以理解，电子设备1000还可以包括给各个部件供电的电源(比如电池)，电源可以通过电源管理系统与处理器1010逻辑相连，从而通过电源管理系统实现管理充电、放电、以及功耗管理等功能。图4中示出的电子设备结构并不构成对电子设备的限定，电子设备可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件布置，在此不再赘述。
其中，处理器1010，用于根据网络传输状态和接收端设备的播放参数中的至少一项，对待传输数据的数据参数进行调整；将调整数据参数后的待传输数据进行编码，得到编码数据；射频单元1001，用于向接收端设备传输编码数据。
本申请实施例提供一种电子设备，该电子设备为发送端设备，首先，发送端设备根据网络传输状态和接收端设备的播放参数中的至少一项，对待传输数据的数据参数进行调整；然后，发送端设备将调整数据参数后的待传输数据进行编码，得到编码数据，最后发送端设备向接收端设备传输编码数据。也就是说，在数据传输中，发送端设备可以在传输数据之前，根据网络传输状态以及接收端设备的播放能力，灵活调整 待传输数据的数据参数。例如，在投屏过程中，一方面，在网络传输状态较好且接收端设备的播放能力强的情况下，可以通过调整待传输数据的数据参数改善画面质量，从而可以使得发送端设备在确定接收端设备播放能力较强的情况下，可以提升画面在播放能力强的接收端设备的屏幕上的投屏体验；另一方面，在网络传输状态较差的情况下，可以通过调整待传输数据的数据参数减少传输数据量，可以避免在网络变差时因网卡阻塞导致的接收端显示画面卡顿的问题。
可选地，将待传输数据的数据参数的取值由第一数值调整至第二数值；其中，在网络传输状态和接收端的播放参数中的至少一项满足第一条件的情况下，第二数值大于第一数值；在网络传输状态和接收端的播放参数中的至少一项满足第二条件的情况下，第二数值小于第一数值。
应理解的是，本申请实施例中，输入单元1004可以包括图形处理器(Graphics Processing Unit，GPU)1041和麦克风1042，图形处理器1041对在视频捕获模式或图像捕获模式中由图像捕获装置(如摄像头)获得的静态图片或视频的图像数据进行处理。显示单元1006可包括显示面板1061，可以采用液晶显示器、有机发光二极管等形式来配置显示面板1061。用户输入单元1007包括触控面板1071以及其他输入设备1072。触控面板1071，也称为触摸屏。触控面板1071可包括触摸检测装置和触摸控制器两个部分。其他输入设备1072可以包括但不限于物理键盘、功能键(比如音量控制按键、开关按键等)、轨迹球、鼠标、操作杆，在此不再赘述。存储器1009可用于存储软件程序以及各种数据，包括但不限于应用程序和操作系统。处理器1010可集成应用处理器和调制解调处理器，其中，应用处理器主要处理操作系统、用户界面和应用程序等，调制解调处理器主要处理无线通信。可以理解的是，上述调制解调处理器也可以不集成到处理器1010中。
本申请实施例还提供一种可读存储介质，所述可读存储介质上存储有程序或指令，该程序或指令被处理器执行时实现上述数据处理方法实施例的各个过程，且能达到相同的技术效果，为避免重复，这里不再赘述。
其中，所述处理器为上述实施例中所述的电子设备中的处理器。所述可读存储介质，包括计算机可读存储介质，如计算机只读存储器(Read-Only Memory，ROM)、 随机存取存储器(Random Access Memory，RAM)、磁碟或者光盘等。
本申请实施例另提供了一种芯片，所述芯片包括处理器和通信接口，所述通信接口和所述处理器耦合，所述处理器用于运行程序或指令，实现上述数据处理方法实施例的各个过程，且能达到相同的技术效果，为避免重复，这里不再赘述。
应理解，本申请实施例提到的芯片还可以称为系统级芯片、系统芯片、芯片系统或片上系统芯片等。
本申请实施例另提供一种计算机程序产品，所述程序产品被存储在非易失的存储介质中，所述程序产品被至少一个处理器执行以实现上述数据处理方法实施例的各个过程，且能达到相同的技术效果，为避免重复，这里不再赘述。
需要说明的是，在本文中，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者装置不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者装置所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括该要素的过程、方法、物品或者装置中还存在另外的相同要素。此外，需要指出的是，本申请实施方式中的方法和装置的范围不限按示出或讨论的顺序来执行功能，还可包括根据所涉及的功能按基本同时的方式或按相反的顺序来执行功能，例如，可以按不同于所描述的次序来执行所描述的方法，并且还可以添加、省去、或组合各种步骤。另外，参照某些示例所描述的特征可在其他示例中被组合。
通过以上的实施方式的描述，本领域的技术人员可以清楚地了解到上述实施例方法可借助软件加必需的通用硬件平台的方式来实现，当然也可以通过硬件，但很多情况下前者是更佳的实施方式。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质(如ROM/RAM、磁碟、光盘)中，包括若干指令用以使得一台终端(可以是手机，计算机，服务器，空调器，或者网络设备等)执行本申请各个实施例所述的方法。
上面结合附图对本申请的实施例进行了描述，但是本申请并不局限于上述的具体实施方式，上述的具体实施方式仅仅是示意性的，而不是限制性的，本领域的普通技 术人员在本申请的启示下，在不脱离本申请宗旨和权利要求所保护的范围情况下，还可做出很多形式，均属于本申请的保护之内。

Claims (16)

  1. 一种数据处理方法，所述方法包括：
    根据网络传输状态和接收端设备的播放参数中的至少一项，对待传输数据的数据参数进行调整；
    将调整数据参数后的待传输数据进行编码，得到编码数据，并向所述接收端设备传输所述编码数据。
  2. 根据权利要求1所述的方法，其中，所述对待传输数据的数据参数进行调整，包括：
    将所述待传输数据的数据参数的取值由第一数值调整至第二数值；
    其中，在所述网络传输状态和接收端的播放参数中的至少一项满足第一条件的情况下，所述第二数值大于所述第一数值；
    在所述网络传输状态和接收端的播放参数中的至少一项满足第二条件的情况下，所述第二数值小于所述第一数值。
  3. 根据权利要求2所述的方法，其中，所述网络传输状态和接收端的播放参数中的至少一项满足第一条件，包括以下至少一项：
    所述网络传输速率大于第一阈值；
    所述网络传输速率大于所述待传输数据的码率；
    所述接收端的播放参数的参数值大于所述第一数值。
  4. 根据权利要求2所述的方法，其中，所述网络传输状态和接收端的播放参数中的至少一项满足第二条件，包括以下至少一项：
    所述网络传输速率小于第二阈值；
    所述网络传输速率小于所述待传输数据的码率；
    所述接收端的播放参数的参数值小于所述第一数值。
  5. 根据权利要求1至4中任一项所述的方法，其中，所述数据参数包括以下至少一项：帧率、分辨率、对比度、色度。
  6. 一种数据处理装置，所述数据处理装置包括：调整模块、编码模块和传输模块；
    所述调整模块，用于根据网络传输状态和接收端设备的播放参数中的至少一项，对待传输数据的数据参数进行调整；
    所述编码模块，用于将调整数据参数后的待传输数据进行编码，得到编码数据；
    所述传输模块，用于向所述接收端设备传输所述编码数据。
  7. 根据权利要求6所述的数据处理装置，其中，所述调整模块具体用于：
    将所述待传输数据的数据参数的取值由第一数值调整至第二数值；
    其中，在所述网络传输状态和所述接收端的播放参数中的至少一项满足第一条件的情况下，所述第二数值大于所述第一数值；
    在所述网络传输状态和接收端的播放参数中的至少一项满足第二条件的情况下，所述第二数值小于所述第一数值。
  8. 根据权利要求6所述的数据处理装置，其中，所述网络传输状态和接收端的播放参数中的至少一项满足第一条件，包括以下至少一项：
    所述网络传输速率大于第一阈值；
    所述网络传输速率大于所述待传输数据的码率；
    所述接收端的播放参数的参数值大于所述第一数值。
  9. 根据权利要求6所述的数据处理装置，其中，所述网络传输状态和接收端的播放参数中的至少一项满足第二条件，包括以下至少一项：
    所述网络传输速率小于第二阈值；
    所述网络传输速率小于所述待传输数据的码率；
    所述接收端的播放参数的参数值小于所述第一数值。
  10. 根据权利要求6至9中任一项所述的数据处理装置，其中，所述数据参数包括以下至少一项：帧率、分辨率、对比度、色度。
  11. 一种电子设备，所述电子设备包括：外挂图像芯片、编码器和射频单元；
    所述外挂图像芯片，用于根据网络传输状态和接收端设备的播放参数中的至少一项，对待传输数据的数据参数进行调整；
    所述编码器，用于将调整数据参数后的待传输数据进行编码，得到编码数据；
    所述射频单元，用于向所述接收端设备传输所述编码数据。
  12. 一种电子设备，包括处理器，存储器及存储在所述存储器上并可在所述处理器上运行的程序或指令，所述程序或指令被所述处理器执行时实现如权利要求1至5中任一项所述的数据处理方法的步骤。
  13. 一种可读存储介质，所述可读存储介质上存储程序或指令，所述程序或指令被处理器执行时实现如权利要求1至5中任一项所述的数据处理方法的步骤。
  14. 一种计算机软件产品，所述计算机软件产品被至少一个处理器执行以实现如权利要求1至5中任一项所述的数据处理方法。
  15. 一种电子设备，包括电子设备被配置成用于执行如权利要求1至5中任一项所述的数据处理方法。
  16. 一种芯片，所述芯片包括处理器和通信接口，所述通信接口和所述处理器耦合，所述处理器用于运行程序或指令，实现如权利要求1至5中任一项所述的数据处理方法。
PCT/CN2021/133831 2020-11-30 2021-11-29 数据处理方法、装置及电子设备 WO2022111672A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202011380626.1 2020-11-30
CN202011380626.1A CN112492395A (zh) 2020-11-30 2020-11-30 数据处理方法、装置及电子设备

Publications (1)

Publication Number Publication Date
WO2022111672A1 true WO2022111672A1 (zh) 2022-06-02

Family

ID=74937882

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/133831 WO2022111672A1 (zh) 2020-11-30 2021-11-29 数据处理方法、装置及电子设备

Country Status (2)

Country Link
CN (1) CN112492395A (zh)
WO (1) WO2022111672A1 (zh)

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114071197B (zh) * 2020-07-30 2024-04-12 华为技术有限公司 投屏数据处理方法和装置
CN112492395A (zh) * 2020-11-30 2021-03-12 维沃移动通信有限公司 数据处理方法、装置及电子设备
CN113160342B (zh) * 2021-05-14 2023-08-25 杭州网易智企科技有限公司 基于反馈的编码方法及装置、存储介质、电子设备
CN113326019B (zh) * 2021-06-28 2023-04-14 Oppo广东移动通信有限公司 音量控制方法、装置、电子设备及存储介质
CN113838470B (zh) * 2021-09-15 2023-10-03 Oppo广东移动通信有限公司 音频处理方法、装置、电子设备及计算机可读介质及产品
CN114286194B (zh) * 2021-12-08 2024-05-03 北京金山云网络技术有限公司 即时通信视频的处理方法、装置、电子设备及存储介质
CN114466228B (zh) * 2021-12-31 2023-09-05 山东信通电子股份有限公司 一种提高投屏显示流畅度的方法、设备及存储介质
CN115134641B (zh) * 2022-07-05 2023-10-27 北京字跳网络技术有限公司 投屏方法、装置和电子设备
CN115103211B (zh) * 2022-07-27 2023-01-10 广州迈聆信息科技有限公司 数据传输方法、电子装置、设备及计算机可读存储介质
CN115278308B (zh) * 2022-07-28 2023-12-01 北京字跳网络技术有限公司 媒体流处理方法、装置、设备及存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104077098A (zh) * 2013-03-25 2014-10-01 三星电子株式会社 用于提高在装置之间共享屏幕的体验质量的方法和设备
CN106155606A (zh) * 2015-04-07 2016-11-23 中国移动通信集团公司 一种多屏互动方法及装置
CN109587560A (zh) * 2018-11-27 2019-04-05 Oppo广东移动通信有限公司 视频处理方法、装置、电子设备以及存储介质
CN111010576A (zh) * 2019-12-18 2020-04-14 腾讯科技（深圳）有限公司 一种数据处理方法及相关设备
CN112492395A (zh) * 2020-11-30 2021-03-12 维沃移动通信有限公司 数据处理方法、装置及电子设备

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104077098A (zh) * 2013-03-25 2014-10-01 三星电子株式会社 用于提高在装置之间共享屏幕的体验质量的方法和设备
CN106155606A (zh) * 2015-04-07 2016-11-23 中国移动通信集团公司 一种多屏互动方法及装置
CN109587560A (zh) * 2018-11-27 2019-04-05 Oppo广东移动通信有限公司 视频处理方法、装置、电子设备以及存储介质
CN111010576A (zh) * 2019-12-18 2020-04-14 腾讯科技（深圳）有限公司 一种数据处理方法及相关设备
CN112492395A (zh) * 2020-11-30 2021-03-12 维沃移动通信有限公司 数据处理方法、装置及电子设备

Also Published As

Publication number Publication date
CN112492395A (zh) 2021-03-12

Similar Documents

Publication Publication Date Title
WO2022111672A1 (zh) 数据处理方法、装置及电子设备
US8983555B2 (en) Wireless communication techniques
WO2015058670A1 (zh) 一种显示内容处理方法及设备
EP3087716B1 (en) Remote rendering for efficient use of wireless bandwidth for wireless docking
JP6404368B2 (ja) 動的フレームレートサポートを用いる電力最適化
WO2019001347A1 (zh) 移动设备的投屏方法、存储介质、终端及投屏系统
US20210117144A1 (en) Implementing wireless displays that handle rotation and/or portrait-first sinks
WO2019114724A1 (zh) 一种相机缩略图生成的方法、移动终端及存储介质
US10085068B2 (en) Information processing apparatus and information processing method
JP2014528209A (ja) 変化領域を使用したメディア符号化
WO2022022019A1 (zh) 投屏数据处理方法和装置
US9392063B2 (en) Information processing apparatus that controls transfer of image, control method therefor, and storage medium
WO2017075970A1 (zh) 手机上WiHD传输的控制方法、装置及手机
CN114501090B (zh) 投屏方法、装置、设备及计算机可读存储介质
CN115209192A (zh) 一种显示设备、智能设备和摄像头共享方法
WO2016065514A1 (zh) 图像显示方法、用户终端和视频接收设备
CN114071169B (zh) 一种媒体内容共享方法和装置
TWI491259B (zh) 遠端影音分享方法及其應用程式
TW201310979A (zh) 播放裝置以及影像播放方法
CN105376617A (zh) 一种无线映射交互的处理方法
TWI523509B (zh) 實施行動高畫質連結技術的方法及其電子裝置
TWI488490B (zh) 自動調整無線顯示品質方法及其顯示裝置與通信裝置
JP2014187491A (ja) 電子機器
CN115623069A (zh) 一种多屏幕投屏方法及装置
CN117472304A (zh) 屏幕采集方法、装置、终端设备及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21897172

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 31-10-2023)