WO2017020711A1

WO2017020711A1 - Communication processing method and electronic device

Info

Publication number: WO2017020711A1
Application number: PCT/CN2016/090811
Authority: WO
Inventors: 陈晓波; 向航; 王新亮; 李斌
Original assignee: 腾讯科技（深圳）有限公司
Priority date: 2015-08-05
Filing date: 2016-07-21
Publication date: 2017-02-09
Also published as: KR20170140369A; JP2018515979A; CN105187758A; CN105187758B; KR102110644B1

Abstract

Disclosed are a communication processing method and an electronic device. The method comprises: controlling the first electronic device to be in a first communication status in which the first electronic device performs voice data interaction with at least one second electronic device; in response to a first operation, controlling the switching from the first communication status to a second communication status, and in the second communication status, maintaining voice data interaction with the at least one second electronic device, and sharing a video frame with at least one second electronic device; and in response to a second operation, controlling the switching from the second communication status to the first communication status.

Description

Communication processing method and electronic device

Technical field

The present invention relates to communication management technologies in the field of information processing, and in particular, to a communication processing method and an electronic device.

Background technique

With the development of technology, the existing voice over Internet Protocol (VOIP)-based voice products, the parties to the call either carry simple voice data, or simply video call, and basically turn off the video call. Also closed the entire call at the same time. This video communication method has the following two drawbacks: First, you cannot unilaterally share video. For example, if one of the two parties initiates a video invitation, the other party will automatically open its own camera to capture the video frame if the answering video is received. Inconvenient to answer, generally will directly reject the invitation of the initiator, end the call, but in this case, the initiator may only want to share their video to the recipient, instead of seeing the recipient's video at the same time. Second, there is no control over whether the video is active in the call. For example, you can't control when to turn on or turn off your own video at any time. When you open the video yourself, the other party can see your video. Otherwise, when you turn it off, The other party can't see your video.

It can be seen that the prior art cannot enable the parties to actively control the timing of sharing video frames, and cannot reduce the use of processing resources of the server, and cannot guarantee the smoothness of both parties.

Summary of the invention

In view of this, an object of the embodiments of the present invention is to provide a communication processing method and an electronic device, which can solve at least the above problems in the prior art.

The embodiment of the invention provides a communication processing method, which is applied to a first electronic device, and the method includes:

Controlling that the first electronic device is in a first communication state, and in the first communication state, the first electronic device performs voice data interaction with at least one second electronic device;

Controlling, in response to the first operation, switching from the first communication state to the second communication state, maintaining voice data interaction with the at least one second electronic device in the second communication state, and sharing with the at least one second electronic device Video frame

In response to the second operation, control switches from the second communication state to the first communication state.

An embodiment of the present invention further provides an electronic device, including:

a first processing unit, configured to control the first electronic device to be in a first communication state, in the first communication state, the first electronic device performs voice data interaction with the at least one second electronic device; Controlling switching from the second communication state to the first communication state;

a second processing unit configured to switch from the first communication state to the second communication state in response to the first operation, to maintain voice data interaction with the at least one second electronic device in the second communication state, and At least one second electronic device shares a video frame.

The communication processing method and the electronic device provided by the embodiments of the present invention can only exchange voice data when the user is in the first communication state, and control the electronic device to switch to the second communication state when responding to the first operation, and transmit the voice data. Sharing the currently collected video frame, in response to the second operation, the electronic device switches back to the first communication state. In this way, the user can flexibly control the interaction timing of the video frame during the voice communication interaction process, reduce the processing resources of the server occupied by the video frame and the voice data during the communication, and ensure the smoothness of both parties.

DRAWINGS

1 is a schematic flowchart of a communication processing method according to an embodiment of the present invention;

2 is a schematic diagram of a connection scenario according to an embodiment of the present invention;

3 is a schematic diagram of an interface for transmitting a video frame according to an embodiment of the present invention;

4 is a schematic diagram of a video frame sending and parsing process according to an embodiment of the present invention;

FIG. 5 is a schematic diagram of a video frame interaction process according to an embodiment of the present invention; FIG.

FIG. 6 is a schematic diagram of ending a video frame interaction process according to an embodiment of the present invention;

7 is a schematic diagram of an operation interface of a dual shared video frame according to an embodiment of the present invention;

FIG. 8 is a schematic flowchart of a process for sharing video frames by multiple people according to an embodiment of the present invention; FIG.

9 is a schematic diagram of an operation interface of a multi-person shared video frame according to an embodiment of the present invention;

FIG. 10 is a first schematic structural diagram of an electronic device according to an embodiment of the present invention; FIG.

11 is a second schematic structural diagram of an electronic device according to an embodiment of the present invention;

FIG. 12 is a third schematic structural diagram of an electronic device according to an embodiment of the present invention.

detailed description

The embodiments of the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

Embodiment 1

An embodiment of the present invention provides a communication processing method, which is applied to a first electronic device. As shown in FIG. 1, the method includes:

Step 11: Control the first electronic device to be in a first communication state, in the first communication state, the first electronic device performs voice data interaction with at least one second electronic device;

Step 12: In response to the first operation, controlling to switch from the first communication state to the second communication state, maintaining voice data interaction with the at least one second electronic device in the second communication state, and with at least one second The electronic device shares the video frame;

Step 13: In response to the second operation, controlling switching from the second communication state to the first communication state.

Here, the first electronic device may be a device such as a smart phone, a tablet computer, a notebook computer or the like.

The second electronic device may be a device such as a smart phone, a tablet computer, or a notebook computer.

Preferably, before performing step 11, the first electronic device may be associated with at least one The two electronic devices establish a communication connection. For example, as shown in FIG. 2, the first electronic device 21 establishes a communication connection with the second electronic device 221 and the second electronic device 222.

It can be understood that, in this embodiment, the manner in which the first electronic device establishes a communication connection with the second electronic device may establish a connection through the mobile cellular network, or may establish a connection through the IP network.

Before the first electronic device and the at least one second electronic device perform the voice data interaction, the first electronic device may further enable the voice collection unit to perform voice collection, and use the collected voice to perform voice data to obtain voice data. The voice collection unit may be a microphone.

In the above step 12, in response to the first operation, the control is switched from the first communication state to the second communication state, and the voice data interaction with the at least one second electronic device is maintained in the second communication state, and The at least one second electronic device shares the video frame, including:

Controlling, in response to the first operation, switching from the first communication state to the second communication state, maintaining voice data interaction with the at least one second electronic device in the second communication state;

And acquiring at least one video frame, and transmitting the at least one video frame to the at least one second electronic device.

The at least one video frame may be obtained by combining at least one video information according to an acquisition time to obtain at least one video frame.

Wherein, the acquiring at least one video frame may be to turn on the camera and periodically collect by the camera. The period may be set according to the performance of the camera. For example, the image information may be set to be collected as a video frame every 2 ms.

Preferably, in this embodiment, the first operation or the second operation is detected before responding to the first operation or the second operation, specifically:

Before the responding to the first operation, the method further includes: determining that the first operation is detected if a touch operation for the first touch operation area and the operation duration is greater than the first threshold is detected;

Before the responding to the second operation, the method further includes determining to detect the second operation if it is detected that the touch operation for the first touch operation area is revoked.

The first touch operation area may be a contact for detecting a touch operation set according to an actual situation, or detecting a preset area of the touch operation;

The contact may include at least: an abscissa of the contact and an ordinate; the preset area includes at least a horizontal and vertical coordinate of a lower left corner of the area and a horizontal and vertical coordinate of the upper right corner.

In addition, the presentation form of the first touch operation area may be a virtual button 31 displayed in the first electronic device as shown in FIG. 3.

The operation duration may be a duration when the user clicks on the first touch operation area, that is, when the user clicks on the first touch operation area, the control turns on the camera, collects by the camera, and sends out to the at least one second electronic device. At least one video frame until it is detected that the user cancels the operation of clicking the first touch operation area.

It can be seen that, by adopting the foregoing solution, only the voice data can be exchanged when the user is in the first communication state, and when the first operation is performed, the control electronic device switches to the second communication state, and the voice data is shared and the currently collected image file is shared. The video frame, in response to the second operation, the electronic device switches back to the first communication state. In this way, the user can flexibly control the interaction timing of the image file video frame during the interaction of the voice communication, thereby reducing the processing resources of the server occupied by the image file video frame and the voice data during communication, and ensuring the two sides of the call. The fluency.

Embodiment 2

Step 12: In response to the first operation, controlling to switch from the first communication state to the second communication state, Maintaining voice data interaction with the at least one second electronic device in the second communication state, and sharing a video frame with the at least one second electronic device;

Preferably, before the step 11 is performed, the first electronic device can establish a communication connection with the at least one second electronic device. For example, as shown in FIG. 2, the first electronic device 21 and the second electronic device 221 and the second electronic device 222 establishes a communication connection.

The first embodiment is different from the first embodiment in that, in the above step 12, the control is switched from the first communication state to the second communication state in response to the first operation, and is maintained in the second communication state with the at least one The two electronic devices perform voice data interaction, and share video frames with the at least one second electronic device, including:

Receiving at least one video frame sent by the at least one second electronic device;

Parsing the at least one video frame sent by the at least one second electronic device.

The parsing video frame may be that the first electronic device parses the video frame by using a video parsing function to obtain an image corresponding to one or more video frames.

It can be seen that, by adopting the foregoing solution, only voice data can be exchanged when the user is in the first communication state, and when the first operation is responded, the control electronic device switches to the second communication state, and both the voice data and the currently collected video frame are shared. In response to the second operation, the electronic device switches back to the first communication state. In this way, the user can flexibly control the interaction timing of the video frame during the interaction of the voice communication, thereby reducing the processing resources of the server occupied by the video frame and the voice data during the communication, and ensuring the smoothness of both parties.

Embodiment 3

The difference from the above two embodiments is that the present embodiment provides a scenario in which the first electronic device can not only collect and send video frames, but also receive and display video frames. Specifically,

Controlling, in response to the first operation, switching from the first communication state to the second communication state, maintaining voice data interaction with the at least one second electronic device in the second communication state; and collecting at least one video frame, transmitting The at least one video frame to the at least one second electronic device;

In addition, at least one video frame sent by the at least one second electronic device can be received; and at least one video frame sent by the second electronic device is parsed. The video frame is also rendered and displayed after parsing the video frame sent by the second electronic device.

Preferably, sending the video frame in this embodiment may include:

Sending a video frame sharing request to the signaling server; wherein the signaling server may be a VOIP signaling server or a mobile cellular network signaling server;

Transmitting, by the signaling server, the video frame sharing request to the at least one second electronic device;

Turning on the camera, acquiring at least one image information, and encoding the image information to obtain a video frame;

Sending a video frame to a data server; wherein the data server can be a VOIP data service Server, or may be a data server in a mobile cellular network;

Forwarding the video frame to the at least one second electronic device by the data server.

Preferably, in this embodiment, a corresponding timestamp is added to the video frame and the voice data to ensure synchronization of the sent video frame and the voice data at the receiving end, which may be: when the voice data is collected. Add a timestamp to the voice data; add a timestamp to the voice data when the voice data is collected; and add a corresponding timestamp to the collected video frame when the video frame is collected. The timestamp may be a time at which voice data or a video frame is collected.

The parsing the at least one video frame sent by the at least one second electronic device includes:

Parsing voice and time stamp in the voice data; parsing the video frame and its corresponding time stamp;

The output voice data and the video frame are controlled according to the timestamp obtained by the parsing.

In the embodiment, the manner of controlling the output of the voice data and the video frame according to the timestamp obtained by the parsing may be simultaneous output of the voice data and the video frame having the same time stamp.

The first touch operation area may be a contact for detecting a touch operation set according to an actual situation, or a preset area for detecting a touch operation; wherein the contact may include at least: an abscissa of the contact and a vertical Coordinates; the preset area includes at least the horizontal and vertical coordinates of the lower left corner of the area and the horizontal and vertical coordinates of the upper right corner.

In addition, the presentation form of the first touch operation area may be a virtual button 31 displayed in the first electronic device as shown in FIG. 3. In this embodiment, the first touch can be clicked The function triggered by the operation area is set to the "one-eye" operation.

The operation duration may be a duration when the user clicks on the first touch operation area, that is, when the user clicks on the first touch operation area, the control turns on the camera, and at least one video frame is collected by the camera until the user is detected. Undo the operation of clicking the first touch operation area.

Further, sharing the video frame with the at least one second electronic device in the embodiment includes:

Transmitting a video frame to all of the at least one second electronic device;

or,

Selecting a target second electronic device from the at least one second electronic device, and transmitting the video frame to the target second electronic device.

That is, after the first communication state of the first electronic device and the plurality of electronic devices interacting with the voice data, switching to the second communication state may select to send the collected image to all the second electronic devices, or may be multiple A target device is selected in the electronic device to send an image to the selected target device.

Embodiment 4

In this embodiment, the signaling server and the data server are VOIP signaling servers and the VOIP data server as an example, and the operation of transmitting video frames is described:

As shown in Figure 4, a simple implementation flow chart of real-time video is given. The VOIP signaling server mainly controls and synchronizes the state during the call, and the data server is mainly If the video or voice data is forwarded (transparently transmitted), the following describes the process of performing real-time video between the two devices M1 and M2, where the device M1 is the first electronic device and the device M2 is the second electronic device:

1. M1 starts the camera device and collects video frames;

2. The video codec engine encodes the video frame collected by the M1;

3. The M1 video frame is encoded and then network packet is sent to the VOIP data server by UDP;

4. The VOIP data server transparently transmits the video frame to the device M2;

5. The device M2 receives and unpacks the video frame;

6. The video codec engine decodes the video frame;

7. The device M2 performs frame rendering and display on the decoded video frame.

Preferably, in the embodiment, voice data between two electronic devices and an interaction operation example of the video frames are respectively provided, and an example of interaction between the voice data and the video frames of the electronic device between the plurality of electronic devices is performed:

Example 1. A single-eye interaction between two people is a form of gameplay based on real-time video technology. The form here is to provide a button (not limited to a similar form), press and hold the button to start sending real-time video to the other party, and release the button to end The other party sends a live video. Take the visual interaction process between users A and B as an example, including 4 interactive situations, as shown in Figure 5. The aim between the two is based on real-time video technology. The form here is to provide a button when the user A presses When the first touch operation area provided in the display interface provided by the device is a virtual button, the “one-eye” operation is triggered, that is, the video frame that starts to send the real-time captured to the user B, that is, the second electronic device, is triggered. Operation; correspondingly, releasing the button ends the "one-eye" operation and stops sending real-time video to the other party. Specifically include:

Scenario 1: One-way sharing video

1)~2) User A presses the button to initiate a video sharing request to the VOIP signaling server. And starting the camera to start collecting video; wherein the button can be a button set according to actual conditions;

3) The VOIP signaling server notifies the user B that the user A starts the video sharing request;

4) ~ 5) User A sends the collected real-time video frame to the VOIP data server, and then the VOIP data server transparently transmits the video frame to the user B;

6) After receiving the real-time video frame from user A, user B will first check whether 3) notification has been received, and if it has been received, then render and display the video frame.

Scenario 2: Sharing videos in both directions

7)~8) On the basis of case 1, user B also starts to initiate a video sharing request to the VOIP signaling server, and starts the camera to start collecting video frames;

9) The VOIP signaling server notifies the user A that the user B has started to take a glance;

10) ~ 11) User B sends the collected real-time video frame to the VOIP data server, and then the VOIP data server transparently transmits the video frame to the user A;

12) After receiving the real-time video frame from user B, user A will first check whether 9) notification has been received, and if it has been received, then render and display the video frame. This completes the process of simultaneous real-time video sharing between users A and B.

Scenario 3: One party ends the video sharing

13)~14) User A releases the button, sends an end video sharing request to the VOIP signaling server, and stops the collection of the camera video frame, and at the same time, the VOIP data server no longer receives the video frame of the user A;

15) The VOIP signaling server notifies the user B that the user A has ended the video sharing;

16) User B no longer receives the video frame from User A, and switches to one-way video sharing on the basis of receiving the 15) notification.

Scenario 4: Both parties end video sharing

17) ~ 18) On the basis of Case 3, User B also releases the button to signal VOIP The server sends the end video sharing request, stops the collection of the camera video frame, and switches the interaction mode of the ordinary voice call. At the same time, the VOIP data server no longer receives the video frame of the user B;

19) The VOIP signaling server notifies the user A, and the user B ends the video sharing;

20) User A no longer receives the video frame from User B, and on the basis of receiving the 19) notification, switches the interactive form of the ordinary voice call.

A schematic diagram of interaction between voice data and video frames is performed on the premise that voice data is exchanged between two electronic devices provided in this embodiment, as shown in FIG. 6-7:

The interface 61 of FIG. 6 is a scenario in which the user A performs voice data interaction with the second electronic device of the user B by using the first electronic device in the first communication state, and long presses the “shooting” button 611 as the “initiating video sharing request”. Virtual button

Then, the scenario in which the video frame is sent by the unilateral party is opened, and the operation interface of the first electronic device of the user A enters the interface 62, and the currently acquired video frame is displayed in the interface 62, and the duration of the current call is also displayed and “is being shared in real time. a text prompt in which the virtual button 611 moves following the movement of the finger on the screen;

When the receiving user B receives the frame video of the user A, there is a simple sound effect reminder, and the user B's display interface 63 displays the video frame picture sent by the opener in full screen, and simultaneously displays the duration of the current call and the other party. Text prompts are being shared in real time. The recipient user B can use the preset button 631 as the touch operation area in this interface 63, and open the "video sharing" function of the second electronic device by clicking the button 631 to enter the two-way video sharing.

Video sharing between the two parties: After entering the two-way video sharing, the interfaces of the two parties are respectively shown in interface 64 and interface 65 in FIG. In the interface 64 and the interface 65, the camera framing content of the camera is displayed by the circular area 641 and the circular area 651, while the circular area moves following the movement of the finger on the screen, and the full-screen area displays the video frame picture of the other party. In this scenario, If one of the parties actively releases the button, the loosened one will return to interface 63, and the unreleased one will return to interface 62; if both parties are released, they will return to interface 61.

Example 2

This embodiment also provides an eye-catching interaction between multiple people (3 persons and above) as shown in FIG. 8 is also a game form based on real-time video technology, but unlike the form between two people, there can only be one at a time. If the multiple users start the video sharing request at the same time, the VOIP signaling server will arbitrate to return the video sharing request to the user who first arrives at the VOIP signaling server for the video sharing request, and the other person returns to initiate the video sharing request. Failure; if someone else wants to successfully start video sharing, they must wait for the user who is currently sharing the video to finish successfully. (This can be understood as the process of “grabbing” video sharing by multiple people). Take the video sharing interaction process between multiple users A1 to An as an example, including two interaction scenarios:

Case 1: Turn on video sharing

1) Several users in the user A1 to An simultaneously initiate a video sharing request to the multi-person VOIP signaling server;

2) The multi-person VOIP signaling server will arbitrate according to the order in which the video sharing request is received, and return the video sharing success to the user A1 who first received the request, and return the video sharing request to other users;

3) The multi-person VOIP signaling server notifies other users that the user A1 has started to take a glance;

4) User A1 starts the camera to start collecting video frames;

5-6) User A1 sends the collected real-time video frame to the multi-person VOIP data server, and then the VOIP data server transparently transmits the video frame to other users;

7) After receiving the real-time video frame from user A1, other users will first check whether 3) notification has been received, and if they have received, the video frame is rendered and displayed.

Scenario 2: End video sharing

8~9) On the basis of Case 1, the user A1 releases the button to send the end video sharing request to the multi-person VOIP signaling server, and the camera stops collecting the video frame and converts to the normal multi-person voice state; at the same time, the VOIP data The server also no longer receives the video frame of user A1;

10) The VOIP signaling server notifies other users that the user A1 has finished taking a glance;

11) Other users no longer receive the video frame from the user A1, and switch the interactive form of the ordinary multi-person voice call on the basis of receiving the 10) notification.

As shown in Figure 9, each effect diagram is described as follows:

The multi-person video sharing portal: the multi-person call interface 91, in the interface 91, long press the "photograph" 911 button as the operation of triggering the first touch operation area, that is, the video sharing function is turned on, and a simple sound effect can also be prompted. . By opening the video sharing function, the interface 92 is entered. After entering the interface 92, there is a "grab" process. If the "grab" is successful, the interface 92 is retained, and if the interface is returned to the interface 91 or the display interface 93 is received by other people. Take a look at the data.

In the interface 92, that is, after entering the video sharing function, the open side displays its own camera framing content in full screen, displays the duration of the current call and the text prompt "is sharing in real time", wherein the button moves following the movement of the finger on the screen. If the opener releases the button, it returns to interface 91.

The interface 93 indicates that when the video frame sharing is performed by multiple parties, the device that does not perform image sharing has a simple sound effect reminder when the video frame sent by the initiator sends a video frame, and then displays the video frame image sent by the opener in full screen, and simultaneously displays the current video frame. The length of the call and the text prompt "Something is sharing in real time."

It can be seen that, by adopting the foregoing solution, only voice data can be exchanged when the user is in the first communication state, and when the first operation is responded, the control electronic device switches to the second communication state, and both the voice data and the currently collected video frame are shared. In response to the second operation, the electronic device switches back to the first communication state. In this way, the user can interact with the voice communication. Flexible control of the interaction timing of video frames, thereby reducing the processing resources of the server occupied by video frames and voice data while communicating, and improving the user experience.

Embodiment 4

An embodiment of the present invention provides an electronic device, as shown in FIG. 10, including:

The first processing unit 1001 is configured to control the first electronic device to be in a first communication state, perform voice data interaction with the at least one second electronic device in the first communication state, and control the slave device from the second operation Switching the second communication state to the first communication state;

The second processing unit 1002 is configured to, in response to the first operation, control to switch from the first communication state to the second communication state, and maintain voice data interaction with the at least one second electronic device in the second communication state, and Sharing a video frame with at least one second electronic device.

Preferably, the electronic device can establish a communication connection with at least one second electronic device. For example, as shown in FIG. 2, the first electronic device 21 establishes a communication connection with the second electronic device 221 and the second electronic device 222.

It can be understood that, in this embodiment, the manner in which the electronic device establishes a communication connection with the second electronic device may establish a connection through the mobile cellular network, or may establish a connection through the IP network.

The electronic device further includes: an image acquisition unit 1003, configured to collect at least one image information;

Correspondingly, the second processing unit 1002 is specifically configured to control the first operation in response to the first operation. Switching to a second communication state, maintaining a voice data interaction with the at least one second electronic device in the second communication state; generating a first video frame by using the collected at least one image information, and transmitting The first video frame to the at least one second electronic device.

The acquiring at least one image information may be to turn on the camera, and the image information is periodically collected by the camera to obtain at least one image information. The period may be set according to the performance of the camera. For example, the image information may be set to be collected every 10 ms.

Alternatively, the acquiring at least one image information may be to open a camera for image capturing, and only one image is acquired, and the image is taken as a video frame.

Preferably, the electronic device further includes: a touch detection unit 1004, configured to determine that the first operation is detected if a touch operation for the first touch operation area and the operation duration is greater than the first threshold value is detected; It is detected that the touch operation for the first touch operation area is revoked, and it is determined that the second operation is detected.

The operation duration may be a duration when the user clicks on the first touch operation area, that is, when the user clicks on the first touch operation area, the control turns on the camera, and at least one image information is collected by the camera, and the at least One image information is used as a video frame until it is detected that the user cancels the operation of clicking the first touch operation area.

As shown in FIG. 4, a simple implementation flow chart of transmitting a video frame in this embodiment is given. The VOIP signaling server mainly controls and synchronizes the state during the call. The signaling server controls and synchronizes the call process by using TCP. The data server mainly forwards (transparently transmits) video or voice data, and forwards data according to the UDP protocol.

The following describes the process of real-time video between the first electronic device and the second electronic device of the two devices:

The first electronic device starts the camera device to collect image information; the video encoding and decoding engine encodes the image information; and the encoded video frame is network-encapsulated and sent to the data server through UDP; the data server may be VOIP data server;

The VOIP data server is transparently transmitted to the second electronic device;

The second electronic device receives and unpacks the video frame; the video codec engine decodes the video frame; and performs frame rendering and display on the decoded video frame.

It can be seen that, by adopting the foregoing solution, only voice data can be exchanged when the user is in the first communication state, and when the first operation is responded, the control electronic device switches to the second communication state, and both the voice data and the currently collected video frame are shared. In response to the second operation, the electronic device switches back to the first communication state. In this way, the user can flexibly control the interaction timing of the video frame during the interaction of the voice communication, thereby reducing the processing resources of the server occupied by the video frame and the voice data during communication, thereby improving the user experience.

Embodiment 5

An embodiment of the present invention provides an electronic device, as shown in FIG.

The first processing unit 1101 is configured to control the first electronic device to be in a first communication state, perform voice data interaction with the at least one second electronic device in the first communication state, and control the slave device from the second operation Switching the second communication state to the first communication state;

a second processing unit 1102, configured to switch from the first communication state to the second communication state in response to the first operation, to maintain voice data interaction with the at least one second electronic device in the second communication state, and Sharing a video frame with at least one second electronic device.

The embodiment is different from the first embodiment in that the electronic device further includes: a parsing unit 1103, configured to parse the second video frame to obtain at least one image information, and respectively display the at least one image information;

The second processing unit 1102 is further configured to receive a second video frame sent by the at least one second electronic device.

The parsing the second video frame may be that the first electronic device parses the video frame by the image parsing function to obtain one or more image information.

Embodiment 6

An embodiment of the present invention provides an electronic device, as shown in FIG. 12, including:

The first processing unit 1201 is configured to control the first electronic device to be in a first communication state, perform voice data interaction with the at least one second electronic device in the first communication state, and control the slave device from the second operation Switching the second communication state to the first communication state;

a second processing unit 1202, configured to switch from the first communication state to the second communication state in response to the first operation, to maintain voice data interaction with the at least one second electronic device in the second communication state, and Sharing a video frame with at least one second electronic device.

It can be understood that the manner of establishing a communication connection with the second electronic device in this embodiment may establish a connection through the mobile cellular network, or may establish a connection through the IP network.

The present embodiment provides a scenario in which the first electronic device can not only collect and transmit a video frame, but also receive and display a video frame. Specifically, the electronic device further includes: an image collecting unit 1203, configured to: Collecting at least one image information; correspondingly, the second processing unit 1202 is configured to, in response to the first operation, control to switch from the first communication state to the second communication state, and maintain the same in the second communication state The at least one second electronic device performs voice data interaction; generates the first video frame by using the collected at least one image information, and sends the first video frame to the at least one second electronic device.

In addition, the electronic device further includes: a parsing unit 1205, configured to parse the second video frame to obtain at least one image information, and respectively display the at least one image information;

The second processing unit 1202 is further configured to receive the at least one second electronic device Second video frame.

Preferably, the sending the video frame in this embodiment may include: the second processing unit 1202 sends a video frame sharing request to the signaling server; wherein the signaling server may be a VOIP signaling server, or may be a mobile a cellular network signaling server; sending the video frame sharing request to the at least one second electronic device by using the signaling server; turning on the camera, acquiring at least one image information, encoding the image information to obtain a video frame; and transmitting the video a frame to a data server; wherein the data server may be a VOIP data server, or may be a data server in a mobile cellular network;

The electronic device further includes: a touch detection unit 1204, configured to determine that the first operation is detected if a touch operation for the first touch operation area and the operation duration is greater than the first threshold is detected; For the touch operation of the first touch operation area, it is determined that the second operation is detected.

Further, the second processing unit 1202 is specifically configured to send a video frame to all second electronic devices in the at least one second electronic device;

or,

The second processing unit 1202 is specifically configured to select a target second electronic device from the at least one second electronic device, and send the video frame to the target second electronic device.

The electronic device further includes: a voice collecting unit 1206, configured to collect voice data, and add a time stamp in the voice data;

The image collection unit 1203 is specifically configured to add a corresponding timestamp to the collected video frame when the video frame is collected.

The first processing unit 1201 is configured to parse the voice and the timestamp in the voice data, and control the output voice data according to the parsed timestamp. The parsing unit 1205 is specifically configured to parse the video frame and its corresponding timestamp. The output video frame is controlled according to the timestamp obtained by the parsing.

It can be seen that, by adopting the foregoing solution, only voice data can be exchanged when the user is in the first communication state, and when the first operation is responded, the control electronic device switches to the second communication state, and both the voice data and the currently collected video frame are shared. In response to the second operation, the electronic device switches back to the first communication state. In this way, the user can flexibly control the interaction timing of the video frame when performing the interaction of the voice communication, thereby reducing the transmission of the video frame all the time during the communication. And the processing resources of the server occupied by the voice data to enhance the user experience.

In the several embodiments provided by the present application, it should be understood that the disclosed apparatus and method may be implemented in other manners. The device embodiments described above are merely illustrative. For example, the division of the unit is only a logical function division. In actual implementation, there may be another division manner, such as: multiple units or components may be combined, or Can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, or direct coupling, or communication connection of the components shown or discussed may be indirect coupling or communication connection through some interfaces, devices or units, and may be electrical, mechanical or other forms. of.

The units described above as separate components may or may not be physically separated, and the components displayed as the unit may or may not be physical units, that is, may be located in one place or distributed to multiple network units; Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, each functional unit in each embodiment of the present invention may be integrated into one processing module, or each unit may be separately used as one unit, or two or more units may be integrated into one unit; the above integration The unit can be implemented in the form of hardware or in the form of hardware plus software functional units.

A person skilled in the art can understand that all or part of the steps of implementing the above method embodiments may be completed by using hardware related to the program instructions. The foregoing program may be stored in a computer readable storage medium, and the program is executed when executed. The foregoing storage device includes the following steps: the foregoing storage medium includes: a mobile storage device, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk. A medium that can store program code.

The embodiment provides a specific hardware based on the foregoing device embodiment, and an electronic device includes at least: a processor, a memory, a microphone, and a camera;

The processor is configured to control the electronic device to be in a first communication state, where the first electronic device performs voice data interaction with at least one second electronic device in the first communication state; Controlling switching from the first communication state to the second communication state, maintaining voice data interaction with the at least one second electronic device in the second communication state, and sharing the video frame with the at least one second electronic device; In a second operation, the control is switched from the second communication state to the first communication state.

The above is only a specific embodiment of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily think of changes or substitutions within the technical scope of the present invention. It should be covered by the scope of the present invention. Therefore, the scope of the invention should be determined by the scope of the appended claims.

Claims

A communication processing method is applied to a first electronic device, the method comprising:

Controlling that the first electronic device is in a first communication state, and in the first communication state, the first electronic device performs voice data interaction with at least one second electronic device;

Controlling, in response to the first operation, switching from the first communication state to the second communication state, maintaining voice data interaction with the at least one second electronic device in the second communication state, and sharing with the at least one second electronic device Video frame

In response to the second operation, control switches from the second communication state to the first communication state.
The method of claim 1, wherein said controlling switching from a first communication state to a second communication state in response to the first operation, and maintaining with said at least one second electronic device in said second communication state The voice data interacts and shares the video frame with the at least one second electronic device, including:

Controlling, in response to the first operation, switching from the first communication state to the second communication state, maintaining voice data interaction with the at least one second electronic device in the second communication state;

And acquiring at least one video frame, and transmitting the at least one video frame to the at least one second electronic device.
The method of claim 2, wherein

Before the responding to the first operation, the method further includes: determining that the first operation is detected if a touch operation for the first touch operation area and the operation duration is greater than the first threshold is detected;

Before the responding to the second operation, the method further includes determining to detect the second operation if it is detected that the touch operation for the first touch operation area is revoked.
The method of claim 1 or 2, wherein the sharing the video frame with the at least one second electronic device comprises:

Receiving at least one video frame sent by the at least one second electronic device;

Parsing the at least one video frame sent by the at least one second electronic device.
The method of claim 1, wherein the sharing the video frame with the at least one second electronic device comprises:

Transmitting at least one video frame to all of the at least one second electronic device;

or,

Selecting a target second electronic device from the at least one second electronic device, and transmitting the at least one video frame to the target second electronic device.
The method according to claim 4, wherein the method further comprises: adding a time stamp to the voice data when the voice data is collected; and adding a corresponding time stamp to the collected video frame when the video frame is collected.
The method of claim 6, wherein the parsing the at least one video frame sent by the at least one second electronic device comprises:

Parsing voice and time stamp in the voice data; parsing the video frame and its corresponding time stamp;

The output voice data and the video frame are controlled according to the timestamp obtained by the parsing.
An electronic device comprising:

a first processing unit, configured to control the first electronic device to be in a first communication state, perform voice data interaction with the at least one second electronic device in the first communication state; and control the slave device in response to the second operation Switching the communication state to the first communication state;

a second processing unit configured to switch from the first communication state to the second communication state in response to the first operation, to maintain voice data interaction with the at least one second electronic device in the second communication state, and At least one second electronic device shares a video frame.
The electronic device of claim 8, wherein the electronic device further comprises:

An image acquisition unit configured to acquire at least one video frame;

Correspondingly, the second processing unit is configured to control the first communication in response to the first operation Switching the state to a second communication state, maintaining voice data interaction with the at least one second electronic device in the second communication state; transmitting the at least one video frame to the at least one second electronic device.
The electronic device of claim 9, wherein the electronic device further comprises: a touch detection unit configured to detect a touch operation for the first touch operation area and the operation duration is greater than the first threshold value, Determining that the first operation is detected; if detecting a revocation of the touch operation for the first touch operation area, determining to detect the second operation.
The electronic device according to claim 9 or 10, wherein the electronic device further comprises: a parsing unit configured to parse at least one video frame sent by the at least one second electronic device;

The second processing unit is configured to receive at least one video frame sent by the at least one second electronic device.
The electronic device according to claim 9, wherein

The second processing unit is configured to send at least one video frame to all second electronic devices in the at least one second electronic device;

or,

The second processing unit is configured to select a target second electronic device from the at least one second electronic device, and send the at least one video frame to the target second electronic device.
The electronic device according to claim 11, wherein the electronic device further comprises: a voice collecting unit configured to collect voice data, and add a time stamp in the voice data;

Correspondingly, the image collecting unit is configured to add a corresponding time stamp to the collected video frame when the video frame is collected.
The electronic device according to claim 13, wherein

The first processing unit is configured to parse the voice and the timestamp in the voice data, and control the output voice data according to the parsed timestamp;

Correspondingly, the parsing unit is configured to parse the video frame and its corresponding timestamp; The parsed timestamp controls the output video frame.
An electronic device comprising: a processor, a memory, a microphone, and a camera; wherein

The memory is configured to save a set of instructions that, when executed, cause at least one processor to perform an operation;

The processor is configured to control the electronic device to be in a first communication state, in the first communication state, the first electronic device performs voice data interaction with at least one second electronic device; in response to the first operation, controlling Switching from a first communication state to a second communication state, maintaining voice data interaction with the at least one second electronic device in the second communication state, and sharing a video frame with the at least one second electronic device; responding to the second Operating, controlling switching from the second communication state to the first communication state.