WO2016110169A1

WO2016110169A1 - Display processing method and device

Info

Publication number: WO2016110169A1
Application number: PCT/CN2015/097019
Authority: WO
Inventors: 赵旭
Original assignee: 中国移动通信集团公司
Priority date: 2015-01-07
Filing date: 2015-12-10
Publication date: 2016-07-14
Also published as: CN105828151B; CN105828151A

Abstract

Provided are a display processing method and device, the display processing method for a sink comprising: obtaining a source image separately transmitted by at least two current sources connected to a sink in a WIFI direct connection manner; generating a target image comprising each source image according to target regions without mutual collision and separately selected by each source; the target region corresponding to the source is used for indicating position information of the source image corresponding to the source in the target image; and outputting the target image to a display module so as to display the target image comprising each source image on the display module. An embodiment of the present invention generates the target image according to the source image transmitted by at least two sources, and the target region without mutual collision and corresponding to the source, and displays on the display module, thus addressing the problem of only achieving one-to-one projection in a screen transmission technique between multiple terminals based on the WIFI, such that data of the multiple terminals can be projected on a display device to achieve split-screen display.

Description

Display processing method and device

Cross-reference to related applications

The present application claims priority to Chinese Patent Application No. 201510007583.5, filed on Jan. 7, 2015, the entire content of

Technical field

The present disclosure relates to the field of multimedia communication technologies, and in particular, to a display processing method and apparatus.

Background technique

In the related art, the protocol architecture of the wireless multimedia transmission technology has been basically clear; the wireless display Wi-Fi Display is a standard established by the Wi-Fi Alliance, and the 1.0 version specification was released in September 2012. Miracast is a certification program for the Wi-Fi Alliance to implement products for Wi-Fi Display technology. It is usually used without distinguishing between Wi-Fi Display and Miracast.

Wi-Fi Display technology is based on MAC layer peer-to-peer (Wi-Fi Direct) transmission, which enables wireless transmission of audio and video data. It is similar to the mirror display function in Apple Air Play. In October 2012, the Android version of the Andriod 4.2 platform began to support the Miracast feature.

Compared with the Apple Air Play function, Wi-Fi Display implements the same function as Air Play's mirror display function, and it does not have the ability to transmit video stream files by URL.

There are two types of devices in the Wi-Fi Display protocol architecture:

(1) The sender (Source): Obtain audio and video data, encode, and send it to the peer for display. Commonly used are mobile phones, tablets, notebooks, etc.;

(2) Sink: Receive audio and video data, decode, play, common organic set-top boxes, TVs, speakers and other equipment.

The following describes the protocol architecture of Wi-Fi Display according to the session process. The session process consists of several phases:

(1) Equipment discovery

With the device discovery mechanism of Wi-Fi Direct, both the sender and the receiver can initiate device discovery actions, collect device information with Wi-Fi Display function, and display the device list through the UI. To the user, the user can choose to initiate the subsequent connection establishment process.

(2) Service discovery

Wi-Fi Direct's service discovery mechanism enables you to discover the Wi-Fi Display-related capabilities of your device.

(3) Equipment selection

The user can initiate a connection by selecting a device from the list of discovered devices. The previous selection can also be saved, and when the saved device is found to exist, the connection can be initiated automatically.

(4) Establish device connection

The MAC layer connection is established using the Wi-Fi Direct connection mechanism, and then a TCP connection is established on the port number 7326 for subsequent session establishment and control message transmission. The above four steps are the operating procedures defined by the Wi-Fi Direct specification. Wi-Fi Display technology must establish a connection between devices based on Wi-Fi Direct before starting the subsequent session.

(5) Ability negotiation

The sending and receiving parties negotiate and set the capability parameters through the RTSP message, including the encoding format of the audio and video, the display resolution, and the port number of the audio and video data transmission.

(6) Establish a session connection

It is recommended to maintain and maintain session connections, including start, pause operations, and parameter adjustments during the session.

(7) Audio and video data transmission

The video data is compressed by H.264 encoding, and the audio data supports LPCM, AAC, and AC3 encoding formats. The encoded audio and video data is encapsulated into an MPEG2-TS packet and transmitted using the RTP+UDP protocol.

(8) End of session

Both the sender and the receiver can initiate a session end operation.

However, the screen transmission technology between WIFI-based multi-terminals on the market can only achieve one-to-one projection. The many-to-one projection is still blank.

Summary of the invention

An object of the present disclosure is to provide a display processing method and apparatus, which solves the problem that a screen transmission technology between multiple terminals based on WIFI can only realize one-to-one projection, so that data of multiple terminals Can be projected onto a display device to achieve the effect of split screen display.

In order to achieve the above object, an embodiment of the present disclosure provides a display processing method, which is used by a receiving end, and includes: a first data obtaining step, where at least two sending ends that are currently connected to the receiving end by using a WIFI direct connection manner are respectively sent. a source image; an image generating step, generating a target image including each source image according to a target region that is not conflictingly selected by each of the transmitting ends; and a target region corresponding to the transmitting end is used to indicate that the source image corresponding to the transmitting end is Position information in the target image; an output step of outputting the target image to the display module to display the target image including each source image in the display module.

The method further includes: a first area determining step, and interacting with each of the transmitting ends to determine a target area that is not selected by each of the transmitting ends and that does not conflict with each other before generating the target image.

The first area determining step includes: an area selection requesting step, sending an area selection request message to the first sending end of the at least two sending ends that has not yet determined the target area, so that the first sending end can The area selection request message determines the first target area; the first parsing step receives and parses the area selection request response message that is returned by the first sending end and carries the first target area, and obtains the first target area therefrom.

The area selection requesting step specifically includes: a first receiving step, receiving an inquiry sent by the first sending end of the at least two sending ends that has not yet determined the target area, and asking whether the receiving end supports the split screen mode a first returning step, returning, to the first sending end, an area selection request message carrying a split screen mode for the first sending end, so that the first sending end can determine according to the area selection request message First target area;

The first parsing step specifically includes: a second receiving step, receiving an area selection request response message that is sent by the first sending end and carrying the first target area; and an area parsing step, configured to parse the area selection request response The message acquires the first target area; and the second returning step returns a target area selection success response message to the first transmitting end.

The area selection request response message in the second receiving step includes at least a split screen mode determined by the first sending end and a split screen block selected in the split screen mode, and the first screen block determines the first target area.

The split screen mode in the area selection request message in the first returning step is all the split screen modes currently supported by the receiving end, when the first sending end is the first sending end; When the first transmitting end is the subsequent transmitting end, the split screen mode in the area selection request message in the first returning step is the split screen mode determined by the previous transmitting end.

The area selection request message, the inquiry message, and the inquiry response message are implemented based on an M3 message in a real-time streaming protocol for WIFI display, and the regional selection request response message, the working mode notification message, and the working mode notification response message are based on M4 message implementation in real-time streaming protocol for WIFI display.

The optional target area is a complete set of all target areas, and the display processing method further includes: a first determining step of determining whether the first target area has been selected; and a reselection request step, when the first When the target area has been selected, the second retransmission request message is sent to the second sending end that selects the first target area before the first sending end; and the second parsing step receives and parses the second sending end to return. The area reselection request response message, from which the second target area reselected by the second transmitting end is obtained.

The method further includes: a conflict determining step, after the new sender joins, determining the at least two according to the third target area selected by the newly added sender after being connected to the receiver through the WIFI direct connection manner. a conflicting sending end that conflicts with the third target area in the target area selected by the sending end; and a disconnection step, after disconnecting the connection with the conflicting sending end, returning to the first data obtaining step.

The method further includes: a second determining step, when receiving the probe request message sent by the new sender, determining whether a connection can be established with more senders; rejecting the step, when unable to communicate with more senders When the connection is established, the new sender is rejected in the probe request response message.

The method further includes: a first group long selection step, carrying, in the probe request message or the probe request response message, an indication information indicating that the receiving end needs to be a group leader, so that the receiving end becomes a group leader; or And a second group of selecting a step, after establishing a connection with the at least two sending ends, negotiating with the at least two sending ends to become a group leader.

The resolution of the source image corresponding to each source image data is determined by the corresponding transmitting end according to the resolution of the display module and the display area size of the source image.

The embodiment of the present disclosure further provides a display processing method for a transmitting end, comprising: a second data acquiring step of acquiring a source image and a target area of the source image in the target image to be generated; and a first sending step, transmitting the source image Data and target area to connect directly to the sender in WIFI mode The receiving end enables the receiving end to output the target image data to the display module after the target image data is generated according to the source image data and the corresponding target area sent by the plurality of transmitting ends, so as to be in the display module The target image is displayed.

The second data obtaining step specifically includes: a third receiving step, receiving, by the receiving end, an area selection request message sent by the first sending end of the at least two transmitting ends that has not determined the target area; the area selection response step After the first target area is determined according to the area selection request message, the area selection request response message carrying the first target area is returned to the receiving end, so that the receiving end can receive the area selection request response message, The first target area is obtained by parsing.

The second data obtaining step specifically includes: a second sending step, the first sending end sends a query message to the receiving end to query whether the receiving end supports the split screen mode; and a fourth receiving step, receiving the receiving end to return Carrying an area selection request message for the split mode selected by the first sending end, and determining a first target area according to the area request message; and a third sending step of transmitting the area carrying the first target area to the receiving end And selecting a request response message; and receiving, by the receiving end, the target area selection success response message returned by the receiving end after acquiring the first target area from the area selection request response message.

The area selection request response message in the third sending step includes at least a split screen mode determined by the transmitting end and a split screen block selected in the split screen mode, and the first target area is determined by the split screen block. .

The split screen mode in the area selection request message in the fourth receiving step is all the split screen modes currently supported by the receiving end; when the first sending end is the first sending end; When the transmitting end is the subsequent transmitting end, the split screen mode in the area selection request message in the fourth receiving step is the split screen mode determined by the previous transmitting end.

The method further includes: a split screen mode determining step, when at least two split screen modes are set in the split screen mode determined by the previous sending end, the split screen mode is selected by the previous sending end The split screen mode.

The display target processing area is a complete set of all target areas, and the display processing method further includes: a reselection receiving step, and receiving an area that is sent by the receiving end when the first target area is selected again by another sending end. Reselecting the request message; reselecting the response step, returning to the receiving end an area selection request response message carrying the reselected second target area, so that the receiving end can receive After the area reselection request response message, the second target area is parsed and acquired.

The embodiment of the present disclosure further provides a display processing apparatus, configured to be used by a receiving end, comprising: a first data acquiring module, configured to send a source image respectively sent by at least two transmitting ends currently connected to the receiving end by a WIFI direct connection manner An image generating module, configured to generate a target image including each source image according to a target region that is not conflictingly selected by each of the transmitting ends; and a target region corresponding to the transmitting end is used to indicate that the source image corresponding to the transmitting end is in the Position information in the target image; an output module, configured to output the target image to the display module, to display the target image including each source image in the display module.

The device further includes: a first area determining module, configured to interact with each of the sending ends to determine a target area that is not selected by each of the transmitting ends and that does not conflict with each other before generating the target image.

The first area determining module includes: an area selection requesting module, configured to send an area selection request message to the first sending end of the at least two sending ends that has not yet determined the target area, so that the first sending And determining, by the first parsing module, the first parsing module, configured to receive and parse the region selection request response message that is sent by the first sending end and that carries the first target area, and obtain the first A target area.

The area selection requesting module specifically includes: a first receiving module, configured to receive an inquiry message sent by the first sending end to query whether the receiving end supports a split screen mode; and a first returning module, configured to The first sending end returns an area selection request message carrying a split screen mode for the first sending end, so that the first sending end can determine the first target area according to the area selection request message;

The first parsing module specifically includes: a second receiving module, configured to receive an area selection request response message that is sent by the first sending end and that carries the first target area; and an area parsing module, configured to parse the area The request response message acquires the first target area; and the second returning module is configured to return a target area selection success response message to the first sending end.

The area selection request response message in the second receiving module includes at least a sending end The determined split screen mode and the split screen block selected in the split screen mode, the first target area is determined by the split screen block.

The split screen mode in the area selection request message in the first returning step is all the split screen modes currently supported by the receiving end; when the first sending end is the first sending end; When the transmitting end is the subsequent transmitting end, the split screen mode in the area selection request message in the first returning step is the split screen mode determined by the previous transmitting end.

The display processing device further includes: a first determining module, configured to determine whether the first target area has been selected; and a reselection requesting module, configured to: And when the first target area has been selected, sending a region reselection request message to a second sending end that selects the first target area before the first sending end; and a second parsing module, configured to receive and parse And returning, by the second sending end, an area reselection request response message, and acquiring a second target area that is reselected by the second sending end.

The device further includes: a conflict determining module, configured to: after the new sending end joins, determine the at least the third target area that is selected after the newly added sending end is connected by the WIFI direct connection manner with the receiving end, a conflicting transmitting end that conflicts with the third target area in the target area selected by the two transmitting ends; and a disconnection step, after disconnecting the connection with the conflicting sending end, returning to the first data obtaining step.

The device further includes: a second determining module, configured to: when receiving the probe request message sent by the new sender, determine whether the connection can be established with more senders; reject the step, when the When the sender establishes a connection, the new sender is rejected in the probe request response message.

The device further includes: a first group long selection module, configured to carry, in the probe request message or the probe request response message, an indication information indicating that the receiving end needs to be a group leader, so that the receiving end becomes a group leader Or, the second group long selection module is configured to, after establishing a connection with the at least two sending ends, negotiate with the at least two sending ends to become a group leader.

The embodiment of the present disclosure further provides a display processing apparatus, configured to: a second data acquisition module, configured to acquire a source image and a target area of the source image in the target image to be generated; Transmitting the source image data and the target area to the receiving end connected to the transmitting end in a WIFI direct connection manner, so that the receiving end can generate the target in the source image data and the corresponding target area sent according to the multiple sending ends. After the image data, the target image data is output to the display module to display the target image in the display module.

The second data acquiring module specifically includes: a third receiving module, configured to receive an area selection request message sent by the receiving end to a first sending end of the at least two sending ends that has not determined the target area; a response module, configured to determine, according to the area selection request message, a first target area, and return an area selection request message carrying the first target area to the receiving end, so that the receiving end can receive the area selection request After responding to the message, the first target area is obtained by parsing.

The second data acquiring module specifically includes: a second sending module, configured to send, by the first sending end, an inquiry message to the receiving end to query whether the receiving end supports the split screen mode; and the fourth receiving module is configured to receive the And the area selection request message that is used by the receiving end to carry the split screen mode for the first sending end, and determines the first target area according to the area request message; the third sending module is configured to send the carrying area to the receiving end a region selection request response message of the first target area; a fifth receiving module, configured to receive a target area selection success response message returned by the receiving end after acquiring the first target area from the area selection request response message.

The area selection request response message in the third sending module includes at least a split screen mode determined by the sending end and a split screen block selected in the split screen mode, and the first target area is determined by the split screen block. .

The device further includes: a split screen mode determining module, configured to determine when the previous transmitting end When at least two split screen modes are set in the split screen mode, the split screen mode in the split screen mode is selected by the previous transmit end.

The display target device is a complete set of all the target areas, and the display processing device further includes: a reselection receiving module, configured to receive, when the first target area is selected by another sending end, the receiving end is sent again And a reselection response module, configured to return, to the receiving end, an area selection request response message carrying the reselected second target area, so that the receiving end can receive the area reselection request response message. Afterwards, the second target area is obtained by parsing.

The above-mentioned technical solutions of the present disclosure have at least the following advantageous effects: in the display processing method and apparatus of the embodiment of the present disclosure, the target image is generated according to the source image transmitted by the at least two transmitting ends and the target region that does not conflict with the transmitting end, And display on a display module, solving the problem that the screen transmission technology between the multi-terminals based on WIFI can only realize one-to-one projection, so that the data of multiple terminals can be projected onto a display device to achieve split screen The effect displayed.

DRAWINGS

1 is a schematic diagram showing the basic steps of a display processing method at a receiving end according to an embodiment of the present disclosure;

2 is a schematic diagram showing the basic steps of a display processing method at a transmitting end according to an embodiment of the present disclosure;

3 is a schematic diagram of a device discovery process of a P2P device in a display processing method according to an embodiment of the present disclosure;

4 is a schematic diagram showing a service discovery process of a P2P device in a display processing method according to an embodiment of the present disclosure;

FIG. 5 is a flow chart showing the message of the transmitting end and the receiving end in the split screen mode according to the embodiment of the present disclosure;

6 is a flowchart showing a disconnection of a disconnection between a transmitting end and a receiving end according to an embodiment of the present disclosure;

FIG. 7 is a schematic diagram showing the group length of the GO negotiation method used in the embodiment of the present disclosure;

FIG. 8 is a block diagram showing a display processing apparatus of a receiving end according to an embodiment of the present disclosure;

FIG. 9 is a structural diagram showing a display processing apparatus of a transmitting end according to an embodiment of the present disclosure;

FIG. 10 shows three split screen modes in the screen quad screen mode of the receiving end of the embodiment of the present disclosure.

detailed description

The technical problems, the technical solutions, and the advantages of the present invention will be more clearly described in conjunction with the accompanying drawings and specific embodiments.

The present disclosure is directed to the problem that the screen transmission technology between multiple terminals based on WIFI in the related art can only achieve one-to-one projection, and provides a display processing method and apparatus, according to the source image and the transmitting end sent by at least two transmitting ends. Corresponding target regions that do not conflict with each other generate target images and display them on a display module, which solves the problem that the screen transmission technology between multiple terminals based on WIFI can only realize one-to-one projection, so that multiple terminals are The data can be projected onto a display device to achieve the effect of split screen display.

In the embodiment of the disclosure, the receiving end supports receiving data sent by multiple transmitting ends at the same time, and the receiving end can process the multi-channel data and then splicing them according to the pre-configuration, and obtaining the target image output to the display module to realize the split screen display. effect. The receiving end can set and adjust the split screen effect according to the needs of the user or the use scenario. For example, when supporting a double or four-player game, when multiple mobile phones are simultaneously projected to the TV side through WiMo (mobile terminal wireless multimedia transmission technology), the upper and lower points are realized. Screen / left and right split screen or quad split screen effect.

Specifically, the display processing method provided by the embodiment of the present disclosure is separately described from the processing processes of the receiving end and the transmitting end:

As shown in FIG. 1, the display processing method for the receiving end includes:

The first data obtaining step 11 is performed to acquire a source image that is currently sent by each of the at least two transmitting ends that are connected to the receiving end by the WIFI direct connection manner;

The image generating step 12 is configured to generate a target image including each source image according to a target region that is not mutually conflicted by each of the transmitting ends, and a target region corresponding to the transmitting end is used to indicate that the source image corresponding to the transmitting end is at the target Location information in the image;

Output step 13 is to output the target image to the display module to display the target image including each source image in the display module.

As shown in FIG. 2, the display processing method for the transmitting end includes:

a second data acquisition step 21, acquiring the source image and the source image in the target image to be generated target area;

a first sending step 22, the source image data and the target area are sent to the receiving end connected to the transmitting end in a WIFI direct connection manner, so that the receiving end can be in the source image data and corresponding according to the multiple sending ends. After the target image data is generated in the target area, the target image data is output to the display module to display the target image in the display module.

Simply put, after the sender/receiver establishes the device connection through the WIFI direct connection mode, the user selects the split-screen display function on the sender device, and selects the display of the sender-side screen image on the receiving-end display device when the split-screen display is selected. Position, after the receiving device receives the sending end setting information and confirms, the transmitting end device compresses and encodes the intercepted multimedia data (video and image) and transmits it to the receiving end display device for display.

In the above example, in the above embodiment of the present disclosure, the receiving end must support the ability to connect, decode, splicing, and display; the specific requirements are as follows:

Multi-transmitter connection receiving capability: Support multiple senders to connect to the same receiver, the receiver is GO, and support multiple P2P senders at the same time;

Multi-channel decoding capability: support decoding of media streams from multiple senders;

Video frame splicing capability: The media stream from multiple senders can be decoded and spliced into a target image for display on the receiving device at the receiving end.

And the specific requirements for the sender are as follows:

Supports the negotiation of the split screen display setting with the receiving end;

Support multi-split screen to select the display position on the receiving end side;

The resolution is adapted to the split screen display of the receiving end.

Preferably, in order to reduce the delay between the transmitting end and the receiving end, the synchronous display function is better realized, and the encoding and decoding formats of the transmitting end and the receiving end need to be consistent, thereby reducing the encoding/decoding time and providing display efficiency; for example; The codec formats of the sender and the receiver are consistent with the WIFI direct connection standard, and the H.264 encoding mode is adopted.

Since there is no information interaction between the sending ends in the specific embodiment of the present disclosure, there may be two or more target areas selected by the transmitting end that cause the conflict of the target area, and the embodiment of the present disclosure passes the receiving end and each of the The information exchange between the senders determines the target areas that do not conflict with each other. The specific methods of the receivers include:

The first area determining step 14 interacts with each of the transmitting ends to determine a target area that each of the transmitting ends does not conflict with each other before generating the target image.

Preferably, the first area determining step 14 includes:

The area selection requesting step 141 is configured to send an area selection request message to the first sending end of the at least two sending ends that has not yet determined the target area, so that the first sending end can determine the first according to the area selection request message. target area;

The first parsing step 142 is to receive and parse the area selection request response message that is sent by the first sending end and that carries the first target area, and obtain the first target area from the first target area.

Correspondingly, the method of the sending end further includes:

a third receiving step 23, receiving an area selection request message sent by the receiving end to the first sending end of the at least two transmitting ends that has not determined the target area;

The area selection response step 24, after determining the first target area according to the area selection request message, returning an area selection request response message carrying the first target area to the receiving end, so that the receiving end can receive the area selection After requesting the response message, the first target area is obtained by parsing.

The method for determining a target area that does not conflict with each other includes two types. The first type, the receiving end sends an area that is not selected by the previous sending end to the transmitting end of the undetermined target area; for example, the receiving end adopts a quadrant mode. , including positions 1, 2, 3, and 4, when the sender 1 selects the position 1, when the sender 2 needs to determine the target area, only the positions 2, 3, and 4 are selected as the optional area, and the position 1 is regarded as the non-selectable area. Then, the target areas determined by the multiple transmitting ends do not conflict with each other; secondly, the receiving end sends all the areas to the transmitting end of the undetermined target area (including the area that has been selected by the preceding transmitting end and is not prior) The area selected by the sender is the complete set of all target areas. In this case, in order to achieve mutual conflict between multiple target areas, an anti-collision algorithm needs to be set, for example, the latter takes precedence or the former takes precedence; specifically, when the sender 1 When the location 1 is selected, and the sender 2 selects the location 1 again, if the former priority algorithm is used, the region reselection response is sent to the sender 2, and the correspondence between the sender 1 and the location 1 is maintained; Method, it is disconnected from a transmission end to the position 1, the position of a target position 2 of the transmitting side, the transmitting side to a transmission area reselection response.

Further, because not all the receiving ends support the split screen function, and even the receiving end supporting the split screen function can work in the single screen display mode, as shown in FIG. 3, the device discovery process between the P2P devices. Figure 4 shows the service discovery process between P2P devices, and device discovery and service. The process of the discovery process is before the network connection, and the specific discovery process is a standard process, which is not described in detail herein. The region selection request step 141 in the embodiment of the present disclosure specifically includes:

The first receiving step 143 is configured to receive an inquiry message sent by the first sending end to query whether the receiving end supports the split screen mode;

a first returning step 144, returning, to the first sending end, an area selection request message carrying a split screen mode for the first sending end, so that the first sending end can determine the first according to the area selection request message. a target area;

The first parsing step 142 specifically includes:

The second receiving step 145 is configured to receive the area selection request response message that is sent by the first sending end that carries the first target area, where the area selection request response message in the second receiving step 145 includes at least the first sending And determining a split screen mode and a split screen block selected in the split screen mode, wherein the first target area is determined by the split screen block.

The area parsing step 146 parses the area selection request response message to acquire the first target area.

The second returning step 147 returns a target area selection success response message to the first transmitting end.

The second data obtaining step 21 of the corresponding sending end specifically includes:

a second sending step 211, the first sending end sends an inquiry message asking the receiving end whether the receiving end supports the split screen mode;

a fourth receiving step 212, receiving an area selection request message that is returned by the receiving end and carrying a split screen mode for the first sending end, and determining a first target area according to the area request message;

a third sending step 213, the area selection request response message carrying the first target area is sent to the receiving end, where the area selection request response message in the third sending step includes at least the split screen mode determined by the sending end and a split screen block selected in the split screen mode, and the first target area is determined by the split screen block.

The fifth receiving step 214 is configured to receive a target area selection success response message returned by the receiving end after acquiring the first target area from the area selection request response message.

As shown in FIG. 10, for example, the split screen mode determined by the transmitting end is a 4-split screen, and the split screen block selected in the split screen mode is the position 1, and the position 1 is the first target position corresponding to the transmitting end. . Specifically, the above split screen mode specifically refers to a split screen mode, such as a 2-split screen, a 4-split screen, etc. The split screen block selected in the split screen mode refers to the specific position displayed on the screen of the receiving end; if only one split screen mode is set in a split screen mode, the position of the split screen block is fixed; otherwise the split screen The position of the block is determined by the split screen mode. When the split screen mode has at least two different split screen modes, the method of the sending end of the embodiment of the present disclosure further includes: a split screen mode determining step, when the previous transmitting end determines When at least two split screen modes are set in the split screen mode, the split screen mode in the split screen mode is selected by the previous transmit end. That is, the first transmitting end selects the split screen mode in the split screen mode, and when the first transmitting end selects the split screen mode, the position of the split screen block is fixed. Preferably, in order to clearly describe the split screen mode, the receiving end screen may be placed in a coordinate system to represent the split screen area by using coordinate values; or the corresponding split points may be set in advance for the split screen mode in different split screen modes. In the screen area, the sender selects the split screen area to be selected.

Taking the 4-screen as an example, at least three types of split screens are included, as shown in FIG. 10, the first type: horizontal split screen, as shown in FIG. 10b, can be applied to a racing game; second: vertical split screen, As shown in FIG. 10c, it can be applied to an airplane-like game; the third split-screen mode is as shown in FIG. 10a, and is generally applicable to simultaneously playing pictures or videos of different sources. In the embodiment of the present disclosure, the transmitting end needs to determine its split screen mode and split screen mode according to its actual situation, wherein the split screen mode needs to be a split screen mode supported by the receiving end, and the query response message includes the receiving end support. Split screen mode.

Since at least two transmitting ends and one receiving end are set in the embodiment of the present disclosure, if each transmitting end selects the split screen mode and the split screen mode according to the actual situation, when the receiving end receives the split screen mode or the split screen mode At different times, the receiving end cannot determine which screen is displayed in accordance with the method determined by the transmitting end; therefore, in the practical application of the present disclosure, when the first transmitting end (generally the first transmitting end) determines its split screen mode, When the sending end sends the query, the receiving end only returns the split screen mode determined by the previous transmitting end; specifically, when the first transmitting end is the first sending end, the receiving end side is in the first returning step. The split screen mode in the selection request message is all the split screen modes currently supported by the receiving end;

When the first transmitting end is the subsequent transmitting end, the split screen mode in the area selection request message in the first returning step is the split screen mode determined by the previous transmitting end.

Correspondingly, when the first transmitting end is the first sending end, the split screen mode in the area selection request message in the fourth receiving step is all the split screen modes currently supported by the receiving end;

When the first transmitting end is the subsequent transmitting end, the split screen mode in the area selection request message in the fourth receiving step is the split screen mode determined by the previous transmitting end.

In particular, if the subsequent splitting mode is set to the single-screen projection mode, the latter principle is followed, and the transmitting end currently connected to the receiving end is disconnected, and the receiving end performs the subsequent transmitting end. Single-screen projection; or follow the former priority principle, the transmitting end currently connected to the receiving end is kept connected, the receiving end sends a reselection response to the transmitting end, and the transmitting end initiates a connection to other receiving ends that are not connected by the transmitting end and performs the connection. Single screen display.

The above two principles can be used alternately according to any actual situation or preset time intervals. For example, for the receiving end that delivers real-time news, the latter priority principle should be adopted in order to display the latest information in time; or for a long time In the case of the displayed advertisement image, the former is preferred to prevent the advertisement information from being replaced; or for the billboard that needs to be replaced once in a period of time, for example, the period of time is 2 days, and the billboard takes the former priority within 2 days. The principle of the latter adopts the latter priority principle after 2 days, which not only ensures that the content is not replaced during the display time of the advertisement, but also ensures that the advertisement can be replaced in time when the advertisement expires.

In the practical application of the present disclosure, in order to avoid that the projection content of the receiving end is always replaced, the principle of the former is preferred, but when a product or a scheme is described using a split screen projection in a conference, the latter principle is generally adopted, which is convenient for Participants learned more about the follow-up options.

It should be noted that, if the receiving end does not support the split screen mode, the first returning step 144 is that the sending end returns an inquiry response message indicating that the receiving end does not support the split screen mode, and ends the flow; the embodiment of the embodiment of the present disclosure The mode can be implemented only when the receiving end supports the split screen mode. Therefore, the receiving end that does not support the split screen mode is not described in detail.

At the same time, in order to support the split screen function, the message parameters in the M3 and M4 messages of the real-time streaming protocol RTSP in the WIFI direct connection are expanded. Specifically, the area selection request message, the inquiry message, and the inquiry response message in the display processing method of the receiving end are implemented based on the M3 message in the real-time streaming protocol for WIFI display, and the area selection request response message and the working mode notification message are And the working mode notification response message is implemented based on the M4 message in the real-time streaming protocol for WIFI display; in the display processing method of the transmitting end, the area selection request message, the inquiry message, and the inquiry response message are based on real-time for WIFI display. The M3 message implementation in the streaming protocol, the regional selection request response message, the working mode notification message, and the working mode notification response message are based on The M4 message implementation in the real-time streaming protocol displayed by WIFI.

The specific expansion of the message parameters in the M3 and M4 messages of the real-time streaming protocol RTSP in the WIFI direct connection is shown in the following table:

It should be noted that, among the multiple transmitting ends that send audio files to the receiving end, only one transmitting end sends an audio file to the receiving end, and the other transmitting end only sends the video file; if there are two or more transmitting ends When transmitting an audio file, the receiving end plays two or more synthesized sounds; how to transmit which sound is negotiated by multiple transmitting terminals is not the focus of the present disclosure, and will not be described in detail herein.

In another embodiment of the present disclosure, the second method for determining the target area that does not conflict with each other is described in detail, that is, when the optional target area is a complete set of all target areas, the former is adopted. In the preferred mode, the display processing method further includes:

a first determining step 31, determining whether the first target area has been selected;

a reselection requesting step 32, when the first target area has been selected, sending a region reselection request message to a second sending end that selects the first target area before the first sending end;

The second parsing step 33 is to receive and parse the area reselection request response message returned by the second sending end, and obtain the second target area that is reselected by the second sending end.

Correspondingly, the method of the sending end further includes:

a reselection receiving step 41, receiving an area reselection request message sent by the receiving end when the first target area is selected again by another transmitting end;

The reselection response step 42 is performed to return to the receiving end a regional selection request response message carrying the reselected second target area, so that the receiving end can parse and acquire the second after receiving the area reselection request response message. target area.

When the latter priority mode is adopted, the display processing method further includes:

The conflict determination step 34, after the new sender joins, determines the target area selected by the at least two senders according to the third target area selected by the newly added sender after being connected to the receiver through the WIFI direct connection manner. a conflicting sender that conflicts with the third target area;

The connection disconnection step 35, after disconnecting the connection with the conflicting sender, returns to the first data acquisition step. The choice of the former priority method or the latter priority method is determined according to the actual situation, and is not specifically limited herein, and the specific situation is consistent with the former priority or the latter priority principle.

Specifically, in the split screen mode, if a new sender device is accessed and a location that is not selected by the other terminal is selected, the terminal screen can be correctly projected to the corresponding location of the receiver device; 5, wherein the M3 responds to the location selected in the OK, and the screen position directly connected to the WIFI is the location that is not selected by the transmitting end of the currently connected receiving end; or,

In the split screen mode, if a new sender device is accessed and a location that the other terminal has selected is selected, the terminal screen can be correctly projected to the corresponding location of the receiver device, and the terminal at the location is previously selected. The receiving device disconnects (the latter priority principle); (the process is as shown in Figure 5, where M3 responds to the selected location in the OK. The screen position directly connected to the WIFI is the location that has been selected by the transmitting end of the currently connected receiving end) . The disconnection process of the sending end and the receiving end is as follows: FIG. 6: The receiving end actively sends an M8 message to the sending end, indicating that the sending end disconnects. And if a new sender device is accessed and the split mode is set to single screen projection, the terminal currently connected to the receiver is disconnected. For example, in the GET_PARAMETER message of M3 in FIG. 5, the WIFI is directly connected to the multipoint. Screen support=1, that is, when the single screen is displayed, the receiving end needs to send a message to disconnect from other sending ends.

According to the limitation of the WIFI direct connection technology in the related art, the number of the transmitting ends that can be connected to one receiving end has an upper limit (the upper limit is not specifically limited), and the method further includes:

The second determining step 36, when receiving the probe request message sent by the new sender, determining whether a connection can be established with more senders;

Rejecting step 37, when the connection cannot be established with more senders, the new sender is rejected in the probe request response message.

Specifically, in another embodiment of the present disclosure, there are two methods for group leader negotiation:

The first group of the long selection step 51, the probe request message or the probe request response message carries an indication information indicating that the receiving end needs to be the group leader, so that the receiving end becomes the group leader; that is, the autonomous group leader does not need to negotiate. The receiving end device sets the group Owner field of the P2P capability attribute in the probe request message or the probe request response message to 1 in the device discovery phase (ie, indicates that the receiving end itself is the group leader); The method of no negotiation is adopted, so that when each sender establishes a connection with the receiver, it does not need to negotiate every time, saving time and unnecessary steps.

Or the second group selection step 52, after establishing a connection with the at least two sending ends, negotiating with the at least two sending ends to become a group leader; that is, GO negotiation, the receiving end device is completed with the sending end device. After the device discovery process, the GO negotiation process is performed. The receiving end always sets its own GO intent value to 15 to indicate that it must be the group leader. The principle of the GO negotiation mechanism is shown in Figure 7, where X1 is the P2P device. The leader intent value, X2 is the group intent value of the P2P device 2.

The following is a detailed description of the split screen display process between multiple mobile phones and televisions:

When the connection between the mobile phone 1 and the television is established, the connection establishment process of the mobile phone 2, the mobile phone 3, the mobile phone N and the television set is the same as the above process after the television set is the group leader without negotiation or GO negotiation; When no more phones can be received, the P2P group limit in the P2P IE group capability is set to 1.

During the process of device discovery and connection between the mobile phone 2 and the television set, the side of the television received is received. The P2P device information associated with the GO may be found in the probe request message or the information carried in the probe response message. (The specific process uses P2P protocol 3.2.4, which is not described in detail here)

When the TV side as the leader can no longer receive more device connections, the bit position can be set to 1.

And in order to display the target image better on the display module, the resolution of the source image corresponding to each source image data is determined by the corresponding transmitting end according to the resolution of the display module and the display area size of the source image. . That is, the transmitting device can adjust the resolution of the sending end according to the resolution of the display module in the receiving device and the setting of the multi-screen, so as to achieve the best display effect and transmission efficiency.

The complete process of the split screen display processing method of the embodiment of the present disclosure is specifically described below with reference to FIG. 5:

First, the process of establishing a connection between the first sender (previous sender) and the receiver is as follows (Step 1 - Step 9):

Step 1: The sender device and the receiver device establish a P2P connection through Wi-Fi direct connection.

Step 2: The sending end device sends a selection request, and queries the RTSP method supported by the receiving end.

Step 3: The receiving end sends a selection response, and the supported method is answered;

Step 4: The receiving end device sends a selection request, and queries the RTSP method supported by the sending end.

Step 5: The sending end sends a selection response, and the supported method is answered;

Step 6, the sending end sends a GET_PARAMETER request (M3) to query the capability parameter of the WIFI direct connection of the receiving end concerned; the following parameters are included: wfd-multi-split-screen-support(M): query whether the receiving end supports multi-screen projection ;

Step 7: The receiving end sends a response message to the sending end, indicating a list of projection modes currently supported by the receiving end; wherein, wfd-multi-split-screen-support(M): indicates a projection mode supported by the receiving end (for example: receiving end) Feedback 1, 2, 4, on behalf of the receiver supports single-screen projection, two-screen projection, four-screen projection);

Step 8: The transmitting end determines, according to the response of the receiving end, the optimal parameter set used by the sending and receiving ends in the WIFI direct connection session, and sends the same to the receiving end through the SET_PARAMETER message (M4); that is, notifying the receiving end that the transmitting end will present the current projection. The mode is set to a few screen projections (assuming the currently returned value is n), and which split screen block under the split screen projection, the value should be in the M3 response message. Wfd-multi-split-screen-support provides a list of supported projection modes;

The transmitting end divides the receiving end screen into n divided screen blocks by coordinate values, and expresses the n divided screen blocks in the form of coordinates and sends them to the receiving end:

For example, in the M4 response message returned by the sender, wfd-multi-split-screen-support: 4, that is, set to 4 split screen display;

And wfd-screen-location (WIFI direct connection screen position) is:

Position ID: 1; coordinates: (x1, y1; x1', y1')

Position ID: 2; coordinates: (x2, y2; x2', y2')

Location ID: 3; coordinates: (x3, y3; x3', y3')

Location ID: 4; coordinates: (x4, y4; x4', y4')

The sender selects the location ID. For example, if the sender selects location 1, it returns ID: 1.

Step 9. After receiving the split screen mode sent by the sending end and the position information in the split screen mode, the receiving end returns an OK message, indicating that the setting is successful;

Further, the process of establishing a connection between the second, third, and fourth transmitting ends and the receiving end is as follows (steps 101 to 109):

Step 101: The sending end device and the receiving end device establish a P2P connection through Wi-Fi direct connection;

Step 102: The sending end device sends a selection request, and queries the RTSP method supported by the receiving end.

Step 103: The receiving end sends a selection response, and the supported method is answered.

Step 104: The receiving end device sends a selection request, and queries the RTSP method supported by the sending end.

Step 105: The sending end sends a selection response, and the supported method is answered.

Step 106: The sender sends a GET_PARAMETER request (M3) to query the capability parameter of the WIFI direct connection of the receiving end concerned; the following parameters are included: wfd-multi-split-screen-support(M): query whether the receiving end supports multi-screen projection ;

Step 107: The receiving end sends a response message to the transmitting end, indicating a projection mode currently supported by the receiving end, where the projection mode is a projection mode selected by the first transmitting end, that is, a quadrant mode;

Wfd-screen-location: Since the first sender has selected the position 1, the location information returned by the receiver at this time only includes (position 2, position 3, position 4);

Step 108: The transmitting end selects a projection position according to the receiving end response, and sends the information to the receiving end through the SET_PARAMETER message (M4): wfd-screen-location(M): The sender selects the location 4; if the subsequent sender selects the location that the previously connected sender has selected, the latter covers the former; if the first sender selects the location 1, the third sender also selects the location 1 When the third sender's screen will be projected to position 1, the first sender will be kicked off and not displayed at position 1;

Step 109: After receiving the split screen mode sent by the sending end and the position information in the split screen mode, the receiving end returns an OK message, indicating that the setting is successful.

After the first, second, third, and fourth transmitting ends respectively establish a connection with the receiving end and complete the split screen setting (assuming that the first, second, third, and fourth transmitting ends respectively select the position 1, the position 2, and the position 3 and position 4), the target image composed of the source images transmitted by the four transmitting ends is simultaneously displayed on the screen of the receiving end by dividing by four. If the first transmitting end selects the position 1, the second transmitting end also selects the position 1, the position 1 displays the source image of the second transmitting end, and the receiving end disconnects the connection with the first transmitting end, if the first transmitting end still needs to source the source When the image is projected to the receiving end, the connection with the receiving end and the split screen setting interaction are re-established. As shown in FIG. 10, in the case where the first transmitting end selects the quadrant projection, the manner in which the screen may be divided includes but is not limited to the case shown in the figure, and the four positions are represented by the transmitting end in the form of coordinates. Come out and send it to the receiver.

In order to achieve the above objective, the embodiment of the present disclosure further provides a display processing apparatus, as shown in FIG.

a first data obtaining module 51, configured to be used by each of the at least two transmitting ends that are currently connected to the receiving end by using a WIFI direct connection manner;

The image generating module 52 is configured to generate a target image including each source image according to each target region that is not conflicted by each of the transmitting ends, and the target region corresponding to the transmitting end is used to indicate that the source image corresponding to the transmitting end is in the Position information in the target image;

The output module 53 is configured to output the target image to the display module to display the target image including each source image in the display module.

Specifically, in the foregoing embodiment of the present disclosure, the device further includes:

The first area determining module is configured to, after generating the target image, interact with each of the sending ends to determine a target area that is not selected by each of the transmitting ends.

Specifically, in the foregoing embodiment of the present disclosure, the first area determining module includes:

An area selection requesting module, configured to not directly determine a target area among the at least two sending ends The first sending end of the domain sends an area selection request message, so that the first sending end can determine the first target area according to the area selection request message;

The first parsing module is configured to receive and parse the area selection request response message that is sent by the first sending end and that carries the first target area, and obtain the first target area from the first target area.

Specifically, in the foregoing embodiment of the present disclosure, the area selection request module specifically includes:

a first receiving module, configured to receive, by the first sending end, an inquiry message that asks whether the receiving end supports a split screen mode;

a first returning module, configured to return, to the first sending end, an area selection request message carrying a split screen mode for the first sending end, so that the first sending end can determine according to the area selection request message First target area;

The first parsing module specifically includes:

a second receiving module, configured to receive an area selection request response message that is sent by the first sending end and that carries the first target area;

An area parsing module, configured to parse the area selection request response message to obtain a first target area;

And a second returning module, configured to return a target area selection success response message to the first sending end. Preferably, the area selection request response message in the second receiving module includes at least a split screen mode determined by the transmitting end and a split screen block selected in the split screen mode, and the first screen block determines the first target area.

Specifically, in the foregoing embodiment of the present disclosure, when the first sending end is the first sending end, the split screen mode in the area selection request message in the first returning step is all points currently supported by the receiving end. Screen mode

Specifically, in the foregoing embodiment of the present disclosure, the area selection request message, the inquiry message, and the inquiry response message are implemented based on an M3 message in a real-time streaming protocol for WIFI display, the area selection request response message, and the working mode notification. The message and work mode notification response message is implemented based on the M4 message in the real-time streaming protocol for WIFI display.

Specifically, in the foregoing embodiment of the present disclosure, the optional target area is a complete set of the target area, and the display processing apparatus further includes:

a first determining module, configured to determine whether the first target area has been selected;

a reselection requesting module, configured to: when the first target area has been selected, send a regional reselection request message to a second sending end that selects the first target area before the first sending end;

The second parsing module is configured to receive and parse the area reselection request response message returned by the second sending end, and obtain the second target area that is reselected by the second sending end.

a conflict determination module, configured to: after the new sender joins, determine the target area selected by the at least two senders according to the third target area selected by the newly added sender after being connected to the receiver through the WIFI direct connection manner a conflicting sender that conflicts with the third target area;

The connection disconnection step, after disconnecting the connection with the conflicting sender, returns to the first data acquisition step.

The second judging module is configured to determine whether a connection can be established with more sending ends when receiving the probe request message sent by the new sending end;

The reject step, when the connection cannot be established with more senders, rejects the new sender in the probe request response message.

a first group of long selection modules, configured to carry, in the probe request message or the probe request response message, an indication information indicating that the receiving end needs to be a group leader, so that the receiving end becomes a group leader;

or

And a second group long selection module, configured to negotiate with the at least two sending ends to become a group leader after establishing a connection with the at least two sending ends.

Specifically, in the foregoing embodiment of the present disclosure, the resolution of the source image corresponding to each source image data is determined by the corresponding transmitting end according to the resolution of the display module and the display area size of the source image.

It should be noted that, the display processing apparatus for the receiving end provided in the above embodiment of the present disclosure is the apparatus applying the display processing method of the receiving end, and all the embodiments of the above display processing method and the beneficial effects thereof are applicable to the processing apparatus. .

In order to achieve the above objectives, the embodiment of the present disclosure further provides a display processing apparatus, such as As shown in Figure 9, for the sender, including:

a second data acquisition module 61, configured to acquire a source image and a target area of the source image in the target image to be generated;

The first sending module 62 is configured to send source image data and a target area to a receiving end that is directly connected to the sending end by using a WIFI, so that the receiving end can be in the source image data sent according to multiple sending ends. And generating target image data corresponding to the target area, and outputting the target image data to the display module to display the target image in the display module.

Specifically, in the foregoing embodiment of the present disclosure, the second data acquiring module specifically includes:

a third receiving module, configured to receive an area selection request message sent by the receiving end to a first sending end of the at least two sending ends that has not determined the target area;

An area selection response module, configured to determine, according to the area selection request message, a first target area, and return an area selection request response message carrying the first target area to the receiving end, so that the receiving end can receive the area After the request response message is selected, the first target area is obtained by parsing.

a second sending module, configured to send, by the first sending end, an inquiry message to the receiving end to query whether the receiving end supports the split screen mode;

a fourth receiving module, configured to receive an area selection request message that is returned by the receiving end and that is used for a split mode of the first sending end, and determine a first target area according to the area request message;

a third sending module, configured to send, to the receiving end, an area selection request response message that carries the first target area;

And a fifth receiving module, configured to receive a target area selection success response message returned by the receiving end after obtaining the first target area from the area selection request response message. Preferably, the area selection request response message in the third sending module includes at least a split screen mode determined by the transmitting end and a split screen block selected in the split screen mode, and the first screen block determines the first target area.

Specifically, in the foregoing embodiment of the present disclosure, when the first sending end is the first sending end, the split screen mode in the area selection request message in the fourth receiving step is all points currently supported by the receiving end. Screen mode

When the first transmitting end is the subsequent transmitting end, the area selection in the fourth receiving step The split screen mode in the request message is the split screen mode determined by the previous sender.

The split screen mode determining module is configured to select, when the at least two split screen modes are set in the split screen mode determined by the previous sending end, the split screen mode in the split screen mode is selected by the previous sending end.

Specifically, in the foregoing embodiment of the present disclosure, the optional target area is a complete set of all target areas, and the display processing apparatus further includes:

a reselection receiving module, configured to receive an area reselection request message sent by the receiving end when the first target area is selected again by another sending end;

a reselection response module, configured to return, to the receiving end, an area selection request response message carrying the reselected second target area, so that the receiving end can parse and acquire the first after receiving the area reselection request response message Two target areas.

It should be noted that, the display processing device for the transmitting end provided in the foregoing embodiment of the present disclosure is a device that applies the display processing method of the transmitting end, and all the embodiments of the display processing method and the beneficial effects thereof are applicable to the processing device. .

In the several embodiments provided by the present application, it should be understood that the disclosed apparatus and method may be implemented in other manners. The device embodiments described above are only schematic. For example, the division of the modules is only a logical function division. In actual implementation, there may be another division manner, for example, multiple modules may be combined, or may be integrated. Go to another system, or some features can be ignored or not executed. In addition, the coupling, or direct coupling, or communication connection of the components shown or discussed may be indirect coupling or communication connection through some interfaces, devices or modules, and may be electrical, mechanical or other forms. of.

Each functional module in each embodiment of the present disclosure may be integrated into one processing module, or each module may be separately used as one module, or two or more modules may be integrated into one module; the above integrated module It can be implemented in the form of hardware or in the form of hardware plus software function modules.

It will be understood by those skilled in the art that all or part of the steps of implementing the foregoing method embodiments may be performed by hardware related to program instructions. The foregoing program may be stored in a computer readable storage medium, and when executed, the program includes The foregoing steps of the method embodiment; and the foregoing storage medium includes: a mobile storage device, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk. The medium in which the program code is stored.

Alternatively, the above-described integrated modules of the present disclosure may also be stored in a computer readable storage medium if implemented in the form of software functional modules and sold or used as separate products. Based on such understanding, the technical solution of the embodiments of the present disclosure may be embodied in the form of a software product in essence or in the form of a software product stored in a storage medium, including a plurality of instructions. A computer device (which may be a personal computer, server, or network device, etc.) is caused to perform all or part of the methods described in various embodiments of the present disclosure. The foregoing storage medium includes various media that can store program codes, such as a mobile storage device, a ROM, a RAM, a magnetic disk, or an optical disk.

The above is a preferred embodiment of the present disclosure, and it should be noted that those skilled in the art can also make several improvements and refinements without departing from the principles of the present disclosure. It should also be considered as the scope of protection of the present disclosure.

Claims

A display processing method for a receiving end, comprising:

a first data obtaining step of acquiring a source image that is currently sent by each of the at least two transmitting ends that are directly connected to the receiving end by using a WIFI direct connection manner;

And an image generating step of generating a target image including each source image according to a target region that is not conflictingly selected by each of the transmitting ends; and a target region corresponding to the transmitting end is used to indicate that the source image corresponding to the transmitting end is in the target image Location information in ;

And an output step of outputting the target image to the display module to display the target image including each source image in the display module.
The display processing method according to claim 1, wherein the method further comprises:

And a first area determining step of interacting with each of the transmitting ends to determine a target area that is not selected by each of the transmitting ends and that does not conflict with each other before generating the target image.
The display processing method according to claim 2, wherein the first region determining step comprises:

The area selection requesting step, sending an area selection request message to the first sending end of the at least two sending ends that has not yet determined the target area, so that the first sending end can determine the first target according to the area selection request message region;

And a first parsing step of receiving and parsing the area selection request response message that is sent by the first sending end and carrying the first target area, and acquiring the first target area therefrom.
The display processing method according to claim 3, wherein

The area selection request step specifically includes:

a first receiving step, receiving an inquiry message sent by the first sending end to query whether the receiving end supports a split screen mode;

a first returning step, returning, to the first sending end, an area selection request message carrying a split screen mode for the first sending end, so that the first sending end can determine the first according to the area selection request message target area;

The first parsing step specifically includes:

a second receiving step, receiving an area selection request response message that is sent by the first sending end and carrying the first target area;

a region parsing step of parsing the region selection request response message to obtain a first target region;

The second returning step returns a target area selection success response message to the first transmitting end.
The display processing method according to claim 4, wherein the region selection request response message in the second receiving step includes at least a split screen mode determined by the first transmitting end and a split screen selected in the split screen mode a block, the first target area being determined by the split screen block.
The display processing method according to claim 5, wherein

When the first sending end is the first sending end, the split screen mode in the area selection request message in the first returning step is all the split screen modes currently supported by the receiving end;

When the first transmitting end is the subsequent transmitting end, the split screen mode in the area selection request message in the first returning step is the split screen mode determined by the previous transmitting end.
The display processing method according to claim 4, wherein the area selection request message, the inquiry message, and the inquiry response message are implemented based on an M3 message in a real-time streaming protocol for WIFI display, the area selection request response message, The work mode notification message and the work mode notification response message are implemented based on the M4 message in the real-time streaming protocol for WIFI display.
The display processing method according to claim 3, 4 or 7, wherein the selectable target area is a complete set of all target areas, and the display processing method further comprises:

a first determining step of determining whether the first target area has been selected;

a reselection request step, when the first target area has been selected, sending a region reselection request message to a second sending end that selects the first target area before the first sending end;

And a second parsing step of receiving and parsing the area reselection request response message returned by the second sending end, and acquiring the second target area reselected by the second sending end.
The display processing method according to claim 3, 4 or 7, wherein the method further comprises:

a conflict determining step, when the new sender joins, determining the target area selected in the at least two senders according to the third target area selected by the newly added sender after being connected to the receiver through the WIFI direct connection manner a conflicting sender that conflicts with the third target area;

The connection disconnection step, after disconnecting the connection with the conflicting sender, returns to the first data acquisition step.
The display processing method according to claim 1, wherein the method further comprises:

a second determining step, when receiving a probe request message sent by the new sender, determining whether a connection can be established with more senders;

The reject step, when the connection cannot be established with more senders, rejects the new sender in the probe request response message.
The display processing method according to claim 1, wherein the method further comprises:

The first set of long selection steps, in the probe request message or the probe request response message, carries an indication information indicating that the receiving end needs to be the group leader, so that the receiving end becomes the group leader;

or

The second group of length selection steps, after establishing a connection with the at least two sending ends, negotiates with the at least two sending ends to become a group leader.
The display processing method according to claim 1, wherein the resolution of the source image corresponding to each source image data is determined by the corresponding transmitting end according to the resolution of the display module and the display area size of the source image.
A display processing method for a transmitting end, comprising:

a second data obtaining step of acquiring a source image and a target area of the source image in the target image to be generated;

a first sending step of transmitting source image data and a target area to a receiving end connected to the transmitting end in a WIFI direct connection manner, so that the receiving end is capable of transmitting the source image data according to the plurality of transmitting ends and corresponding After the target area generates the target image data, the target image data is output to the display module to display the target image in the display module.
The display processing method according to claim 13, wherein the second data obtaining step specifically comprises:

a third receiving step, receiving, by the receiving end, an area selection request message sent by the first sending end of the at least two sending ends that has not determined the target area;

And the area selection response step, after determining the first target area according to the area selection request message, returning an area selection request response message carrying the first target area to the receiving end, so that the receiving end can receive the area selection request After responding to the message, the first target area is obtained by parsing.
The display processing method according to claim 14, wherein the second data obtaining step specifically comprises:

a second sending step, the first sending end sends an inquiry message to the receiving end to query whether the receiving end supports the split screen mode;

a fourth receiving step, receiving an area selection request message that is returned by the receiving end and carrying a split screen mode for the first sending end, and determining a first target area according to the area request message;

a third sending step, sending, to the receiving end, an area selection request response message carrying the first target area;

And a fifth receiving step, receiving a target area selection success response message returned by the receiving end after acquiring the first target area from the area selection request response message.
The display processing method according to claim 15, wherein the region selection request response message in the third transmitting step includes at least a split screen mode determined by the transmitting end and a split screen block selected in the split screen mode, A first target area is determined by the split screen block.
The display processing method according to claim 16, wherein

When the first sending end is the first sending end, the split screen mode in the area selection request message in the fourth receiving step is all the split screen modes currently supported by the receiving end;

When the first transmitting end is the subsequent transmitting end, the split screen mode in the area selection request message in the fourth receiving step is the split screen mode determined by the previous transmitting end.
The display processing method according to claim 17, wherein the method further comprises:

In the split screen mode determining step, when at least two split screen modes are set in the split screen mode determined by the previous sender, the previous splitter mode selects the split screen mode in the split screen mode.
The display processing method according to claim 15, wherein the selectable target area is a complete set of all target areas, and the display processing method further comprises:

a reselection receiving step, receiving an area reselection request message sent by the receiving end when the first target area is selected again by another transmitting end;

Reselecting the response step, returning, to the receiving end, an area selection request response message carrying the reselected second target area, so that the receiving end can parse and acquire the second target after receiving the area reselection request response message region.
The display processing method according to claim 19, wherein the area selection request message, the inquiry message, and the inquiry response message are implemented based on an M3 message in a real-time streaming protocol for WIFI display, the area selection request response message, Work mode notification message and work mode notification ring The message should be implemented based on the M4 message in the real-time streaming protocol for WIFI display.
A display processing device for a receiving end, comprising:

a first data obtaining module, configured to be used by each of the at least two transmitting ends that are currently connected to the receiving end by using a WIFI direct connection manner;

An image generating module, configured to generate a target image including each source image according to a target region that is not conflictingly selected by each of the transmitting ends; and a target region corresponding to the transmitting end is used to indicate that the source image corresponding to the transmitting end is in the Location information in the target image;

And an output module, configured to output the target image to the display module, to display the target image including each source image in the display module.
The display processing device according to claim 21, wherein the device further comprises:

The first area determining module is configured to, after generating the target image, interact with each of the sending ends to determine a target area that is not selected by each of the transmitting ends.
The display processing device according to claim 22, wherein the first region determining module comprises:

a region selection requesting module, configured to send a region selection request message to a first sending end of the at least two sending ends that has not yet determined the target region, so that the first sending end can determine the first according to the region selection request message a target area;

The first parsing module is configured to receive and parse the area selection request response message that is sent by the first sending end and that carries the first target area, and obtain the first target area from the first target area.
The display processing device according to claim 23, wherein

The area selection request module specifically includes:

a first receiving module, configured to receive, by the first sending end, an inquiry message that asks whether the receiving end supports a split screen mode;

a first returning module, configured to return, to the first sending end, an area selection request message carrying a split screen mode for the first sending end, so that the first sending end can determine according to the area selection request message First target area;

The first parsing module specifically includes:

a second receiving module, configured to receive an area selection request response message that is sent by the first sending end and that carries the first target area;

An area parsing module, configured to parse the area selection request response message to obtain a first target area;

And a second returning module, configured to return a target area selection success response message to the first sending end.
The display processing device according to claim 24, wherein the region selection request response message in the second receiving module includes at least a split screen mode determined by the transmitting end and a split screen block selected in the split screen mode, A first target area is determined by the split screen block.
The display processing device according to claim 25, wherein

When the first sending end is the first sending end, the split screen mode in the area selection request message in the first returning step is all the split screen modes currently supported by the receiving end;

When the first transmitting end is the subsequent transmitting end, the split screen mode in the area selection request message in the first returning step is the split screen mode determined by the previous transmitting end.
The display processing device according to claim 24, wherein said region selection request message, inquiry message, and inquiry response message are implemented based on an M3 message in a real-time streaming protocol for WIFI display, said region selection request response message, The work mode notification message and the work mode notification response message are implemented based on the M4 message in the real-time streaming protocol for WIFI display.
The display processing device according to claim 18, 19 or 27, wherein the selectable target area is a complete set of the target area, and the display processing device further comprises:

a first determining module, configured to determine whether the first target area has been selected;

a reselection requesting module, configured to: when the first target area has been selected, send a regional reselection request message to a second sending end that selects the first target area before the first sending end;

The second parsing module is configured to receive and parse the area reselection request response message returned by the second sending end, and obtain the second target area that is reselected by the second sending end.
The display processing device according to claim 18, 19 or 27, wherein the device further comprises:

a conflict determination module, configured to: after the new sender joins, determine the target area selected by the at least two senders according to the third target area selected by the newly added sender after being connected to the receiver through the WIFI direct connection manner a conflicting sender that conflicts with the third target area;

The connection disconnection step, after disconnecting the connection with the conflicting sender, returns to the first data acquisition step.
The display processing device according to claim 21, wherein the device further comprises:

The second judging module is configured to determine whether a connection can be established with more sending ends when receiving the probe request message sent by the new sending end;

The reject step, when the connection cannot be established with more senders, rejects the new sender in the probe request response message.
The display processing device according to claim 21, wherein the device further comprises:

a first group of long selection modules, configured to carry, in the probe request message or the probe request response message, an indication information indicating that the receiving end needs to be a group leader, so that the receiving end becomes a group leader;

or

And a second group long selection module, configured to negotiate with the at least two sending ends to become a group leader after establishing a connection with the at least two sending ends.
The display processing device according to claim 21, wherein the resolution of the source image corresponding to each of the source image data is determined by the corresponding transmitting end according to the resolution of the display module and the display area size of the source image.
A display processing device for a transmitting end, comprising:

a second data acquiring module, configured to acquire a source image and a target area of the source image in the target image to be generated;

a first sending module, configured to send source image data and a target area to a receiving end connected to the transmitting end in a WIFI direct connection manner, so that the receiving end can be in the source image data sent according to multiple sending ends, and After the target image data is generated in the corresponding target area, the target image data is output to the display module to display the target image on the display module.
The display processing device of claim 33, wherein the second data acquisition module comprises:

a third receiving module, configured to receive an area selection request message sent by the receiving end to a first sending end of the at least two sending ends that has not determined the target area;

An area selection response module, configured to determine, according to the area selection request message, a first target area, and return an area selection request response message carrying the first target area to the receiving end, so that the receiving end can receive the area After the request response message is selected, the first target area is obtained by parsing.
The display processing device according to claim 34, wherein the second data acquisition module specifically comprises:

a second sending module, configured to send, by the first sending end, an inquiry message to the receiving end to query whether the receiving end supports the split screen mode;

a fourth receiving module, configured to receive an area selection request message that is returned by the receiving end and that is used for a split mode of the first sending end, and determine a first target area according to the area request message;

a third sending module, configured to send, to the receiving end, an area selection request response message that carries the first target area;

And a fifth receiving module, configured to receive a target area selection success response message returned by the receiving end after obtaining the first target area from the area selection request response message.
The display processing device according to claim 35, wherein the region selection request response message in the third transmitting module includes at least a split screen mode determined by the transmitting end and a split screen block selected in the split screen mode, A first target area is determined by the split screen block.
The display processing device according to claim 36, wherein

When the first sending end is the first sending end, the split screen mode in the area selection request message in the fourth receiving step is all the split screen modes currently supported by the receiving end;

When the first transmitting end is the subsequent transmitting end, the split screen mode in the area selection request message in the fourth receiving step is the split screen mode determined by the previous transmitting end.
The display processing device according to claim 37, wherein the device further comprises:

The split screen mode determining module is configured to select, when the at least two split screen modes are set in the split screen mode determined by the previous sending end, the split screen mode in the split screen mode is selected by the previous sending end.
The display processing device according to claim 35, wherein the selectable target area is a complete set of all target areas, and the display processing apparatus further comprises:

a reselection receiving module, configured to receive an area reselection request message sent by the receiving end when the first target area is selected again by another sending end;

a reselection response module, configured to return, to the receiving end, an area selection request response message carrying the reselected second target area, so that the receiving end can parse and acquire the first after receiving the area reselection request response message Two target areas.
The display processing device according to claim 39, wherein said area selection request message, inquiry message, and inquiry response message are based on M3 in a real-time streaming protocol for WIFI display. The message is implemented, and the regional selection request response message, the working mode notification message, and the working mode notification response message are implemented based on the M4 message in the real-time streaming protocol for WIFI display.