WO2022206651A1

WO2022206651A1 - Instant communication method and apparatus, electronic device, and storage medium

Info

Publication number: WO2022206651A1
Application number: PCT/CN2022/083247
Authority: WO
Inventors: 张驰
Original assignee: 维沃移动通信有限公司
Priority date: 2021-04-01
Filing date: 2022-03-28
Publication date: 2022-10-06
Also published as: CN113162844A; CN113162844B

Abstract

The present application relates to the field of communications, and discloses an instant communication method and apparatus, an electronic device, and a storage medium. The method comprises: receiving a first input to a target partner identifier of a group interface when a user instantly communicates with multiple chat partners at the same time by means of the group interface; and in response to the first input, using different modes to communicate with a peer device corresponding to a target chat partner and peer devices corresponding to the chat partners among the multiple chat partners other than the target chat partner, wherein the modes comprise a normal mode and a shielding mode.

Description

Instant messaging method, device, electronic device and storage medium

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the priority of the Chinese patent application with the application number 202110357580.X filed on April 1, 2021 and the invention title is "Instant Messaging Method, Device, Electronic Device and Storage Medium", which is fully incorporated by reference this application.

technical field

The present application belongs to the field of communication, and specifically relates to an instant communication method, device, electronic device and storage medium.

Background technique

At present, multiple clients can use social software for group video or group voice, which are widely used in work discussion or remote chat.

In group video or group voice, the video or voice sent by a client is forwarded by the server to all other clients participating in the group video or group voice. At the same time, the server forwards the video or voice sent by all other clients at the same time to the client. In the process of realizing this application, the inventor found that there are at least the following problems in the prior art:

In group video or group voice, if a client needs to send video or voice to other clients specified in group video or group voice, other chat tools are required, which is inconvenient to operate.

When the ambient sound of a client is too noisy, the client needs to be notified to turn off the microphone, which is inconvenient to operate.

SUMMARY OF THE INVENTION

The purpose of the embodiments of the present application is to provide an instant communication method, device, electronic device and storage medium, which can solve the problem of inconvenient directional communication in group video or group voice.

In order to solve the above technical problems, this application is implemented as follows:

In a first aspect, an embodiment of the present application provides an instant messaging method, which includes:

In the case where the user communicates with multiple chat objects simultaneously through the group interface, a first input of the target object identifier in the group interface is received; wherein the target object identifier is used to indicate the plurality of chat objects. The target chat object in the chat object;

In response to the first input, use different modes to perform a communication with the peer device corresponding to the target chat object and the peer device corresponding to other chat objects in the plurality of chat objects except the target chat object. communication;

Wherein, the modes include normal mode and shield mode;

The normal mode includes sending the actual audio and video data packets of the user, and receiving and playing at least one of the actual audio and video data packets sent by the peer device;

The shielding mode includes not sending the actual audio and video data packets of the user, and at least one of not receiving, receiving but not playing the actual audio and video data packets sent by the peer device.

In a second aspect, an embodiment of the present application provides an instant messaging device, including:

A receiving module, configured to receive the first input of the target object identifier in the group interface when the user conducts instant communication with multiple chat objects simultaneously through the group interface; wherein, the target object identifier is used for indicating a target chat object in the plurality of chat objects;

A response module, used for responding to the first input, using different modes to correspond to the peer device corresponding to the target chat object and other chat objects other than the target chat object in the plurality of chat objects to communicate with the peer device;

Wherein, the modes include normal mode and shield mode;

In a third aspect, embodiments of the present application provide an electronic device, the electronic device includes a processor, a memory, and a program or instruction stored on the memory and executable on the processor, the program or instruction being The processor implements the steps of the method according to the first aspect when executed.

In a fourth aspect, an embodiment of the present application provides a readable storage medium, where a program or an instruction is stored on the readable storage medium, and when the program or instruction is executed by a processor, the steps of the method according to the first aspect are implemented .

In a fifth aspect, an embodiment of the present application provides a chip, the chip includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is configured to run a program or an instruction, and implement the first aspect the method described.

In the embodiment of the present application, the local device uses different modes to communicate with the target chat and the peer device corresponding to other chat objects according to the user's input of the target object identifier in the group interface, so as to realize the use of shielding mode. Some chat objects in communication are directional shielded, and the flexibility of communication is improved through directional communication between the local device and some chat objects.

Description of drawings

Fig. 1 is one of the flow charts of the communication method provided by this application;

Fig. 2 is one of group interface schematic diagrams in the communication method provided by the present application;

3 is the second schematic diagram of the group interface in the communication method provided by the present application;

4 is the third schematic diagram of the group interface in the communication method provided by the present application;

5 is the fourth schematic diagram of the group interface in the communication method provided by the present application;

6 is the fifth schematic diagram of the group interface in the communication method provided by the present application;

7 is the sixth schematic diagram of the group interface in the communication method provided by the present application;

8 is the seventh schematic diagram of the group interface in the communication method provided by the present application;

9 is the eighth schematic diagram of the group interface in the communication method provided by the present application;

10 is the ninth schematic diagram of the group interface in the communication method provided by the present application;

11 is a tenth schematic diagram of a group interface in the communication method provided by the present application;

12 is an eleventh schematic diagram of a group interface in the communication method provided by the present application;

13 is the twelfth schematic diagram of the group interface in the communication method provided by the present application;

FIG. 14 is the thirteenth schematic diagram of the group interface in the communication method provided by the present application;

15 is a structural diagram of a communication device provided by the present application;

FIG. 16 is a schematic diagram of the hardware structure of the electronic device provided by the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

The terms "first", "second" and the like in the description and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It is to be understood that data so used may be interchanged under appropriate circumstances so that embodiments of the application can be practiced in sequences other than those illustrated or described herein. In addition, "and/or" in the description and claims indicates at least one of the connected objects, and the character "/" generally indicates that the associated objects are in an "or" relationship.

The instant messaging method, device, electronic device, and storage medium provided by the embodiments of the present application will be described in detail below with reference to the accompanying drawings through specific embodiments and application scenarios thereof.

As shown in FIG. 1 , the instant messaging method provided by the embodiment of the present application includes:

Step 101, in the case where the user conducts instant communication with multiple chat objects simultaneously through the group interface, receive a first input of the target object identifier in the group interface; wherein the target object identifier is used to indicate the Describe the target chat object among the plurality of chat objects;

The execution body of the instant messaging method provided by the embodiment of the present application is a terminal device. The local device may be a smart phone, a personal computer, a tablet computer, a wearable smart device, or the like.

The instant communication method provided by the embodiment of the present application is applied to the scenario of group video or group voice chat in social software.

The group interface is a chat interface for group video or group voice. In a group video or group voice scenario, the group communication interface includes at least two chat objects.

The target chat object is one or some chat objects determined by the user from the chat objects of group video or group voice according to actual needs.

For example, if the user only wants one or some of the chat objects participating in the group video or group voice to hear their own voice or see their own video, the chat object is used as the target chat object.

If the user does not want one or some of the chat objects participating in the group video or group voice to hear their own voice or see their own video, the chat object is used as the target chat object.

When the background sound of the video or voice of one or some chat objects played by the local device is noisy, or the user does not want to hear the speech of one or some chat objects, the chat object is used as the target chat object.

In the case that the user only wants to hear the speech of one or some chat objects, the chat object is taken as the target chat object. The user's first input includes a drag operation, a click operation, or a keyboard input, etc. This embodiment is not limited to the specific form of the first input.

The user first opens the social software on the local device and joins the group video or group voice chat. Then, the target object identifier of the target chat object is operated on the group interface of the group video or group voice chat, and the operation is taken as the user's first input to the target chat object.

The local device receives the user's first input to the target chat object.

Step 102, in response to the first input, use different modes to pair with the peer device corresponding to the target chat object and other chat objects other than the target chat object in the plurality of chat objects. The terminal device communicates; wherein, the mode includes a normal mode and a shielded mode;

Using different modes to communicate with the target chat object and peer devices corresponding to other chat objects includes:

Use the normal mode to communicate with the peer device corresponding to the target chat object, and use the shield mode to communicate with the peer device corresponding to other chat objects; or,

Use the shield mode to communicate with the peer device corresponding to the target chat object, and use the normal mode to communicate with the peer device corresponding to other chat objects.

The audio and video data packets are audio data packets or video data packets.

In the normal mode, the local device communicates normally with the target chat object or the peer device corresponding to other chat objects.

Optionally, in normal mode, the local device sends the actual audio and video data packets of the user on the local device to the peer device through the cloud server, or receives and plays the actual audio data packets sent by the peer device through the cloud server.

In shield mode, the local device does not communicate with the target chat object, or even if normal communication is performed, the local device receives but does not play the actual audio and video data packets sent by the peer device.

Optionally, when the local device does not communicate with the target chat object, the local device and the peer device send actual audio and video data packets to the cloud server, but the cloud server does not send the actual audio and video data packets of the local device. Forward to the peer device, or do not forward the actual audio and video data packets of the peer device to the local device.

In the embodiment of the present application, the local device uses different modes to communicate with the target chat and the peer device corresponding to other chat objects according to the user's input of the target object identifier in the group interface, so as to realize the communication using the shield mode. Some chat objects are directional shielded, and the flexibility of communication is improved through directional communication between the local device and some chat objects.

Optionally, the normal mode includes a normal sending mode and a normal receiving mode, and the masking mode includes a sending masking mode and a receiving masking mode;

The normal sending mode is to send the actual audio and video data packets of the user, the normal receiving mode is to receive and play the actual audio and video data packets sent by the peer device, and the transmission shielding mode is not to send the user The actual audio and video data packets, the reception mask mode is not receiving or receiving but not playing the actual audio and video data packets sent by the peer device;

The use of different modes to communicate with the target chat object and other chat objects in the plurality of chat objects includes:

Using the normal transmission mode and the transmission masking mode as a transmission mode, and using the normal reception mode and the reception masking mode as a reception mode;

Use the different sending modes or the receiving modes to conduct a communication with the peer device corresponding to the target chat object and the peer devices corresponding to other chat objects in the plurality of chat objects except the target chat object. communication.

Optionally, the normal sending mode is used to communicate with the peer device corresponding to the target chat object, and the transmission blocking mode is used to communicate with the peer device corresponding to other chat objects.

In this case, the local device first transmits the collected audio or video to the cloud server in the form of audio and video data packets, and then the cloud server only forwards the audio and video data packets sent by the local device to the target chat object. The peer device of the chat object will not be forwarded to the peer device corresponding to other chat objects.

Optionally, while sending the audio and video data packets to the cloud server, the local device sends the list of target chat objects to the cloud server. The cloud server updates the delivery object of the audio and video data packets of the local device to the target chat object in the list, so that only the audio and video data packets of the local device are forwarded to the peer device corresponding to the target chat object.

Optionally, communicate with the peer device corresponding to the target chat object by using the transmission blocking mode, and communicate with the peer device corresponding to other chat objects in the normal transmission mode.

In this case, the cloud server only forwards the audio and video data packets sent by the local device to the peer device corresponding to other chat objects, but not the peer device corresponding to the target chat object.

Optionally, while sending the audio and video data packets to the cloud server, the local device sends the list of other chat objects to the cloud server. The cloud server updates the delivery object of the audio and video data packets of the local device to other chat objects in the list, so that only the audio and video data packets of the local device are forwarded to the peer device corresponding to the other chat objects.

Optionally, use the reception blocking mode to communicate with the peer device corresponding to the target chat object, and use the normal reception mode to communicate with the peer device corresponding to other chat objects.

In this case, the target chat object sends audio and video data packets to the cloud server, and the cloud server does not forward the audio and video data packets sent by the target chat object to the local device. Alternatively, the cloud server forwards the audio and video data packets sent by the target chat object to the local device, but the local device does not perform playback operations after receiving the audio and video data packets from the target chat object, thereby achieving directional shielding.

Optionally, the local device sends the list of other chat objects to the cloud server. The cloud server updates the object of the voice video data package to the local device to other chat objects, thereby shielding the audio and video data packets sent by the target chat object to the local device.

Optionally, the normal reception mode is used to communicate with the peer device corresponding to the target chat object, and the reception mask mode is used to communicate with the peer device corresponding to the other chat objects.

In this case, other chat objects send audio and video data packets to the cloud server, and the cloud server does not forward the audio and video data packets sent by other chat objects to the local device. Alternatively, the cloud server forwards the audio and video data packets sent by other chat objects to the local device, but the local device does not perform playback operations after receiving the audio and video data packets of other chat objects, thereby achieving directional shielding.

Optionally, the local device sends the list of other chat objects to the cloud server. The cloud server updates the object of the voice video data package to the local device to the target chat object, thereby shielding the audio and video data packets sent by other chat objects to the local device.

In the embodiment of the present application, the local device forwards the audio and video data packets sent by the local device only to the target chat object or other chat objects according to the user's input to the target chat object, so as to realize directional chat, or not to the target chat object or other chat objects. The audio and video data sent by the chat object to the local device is forwarded to the local device. Even if it is forwarded to the local device, the local device will not play it, which realizes directional shielding. Through the communication between the local device and the target chat object or other chat objects Directed communication improves the flexibility of communication.

Optionally, the first input includes: an operation by the user to move a target control on the group interface group interface to the screen area where the group interface target object identifier is located; or,

Optionally, the target control includes one or more of the first control, the second control, the third control and the fourth control.

Optionally, the target control is a picture control or a button control. Target controls are displayed on the group interface.

In response to the user's first input on the first control operation, the normal sending mode is used to communicate with the peer device corresponding to the target chat object, and the transmission blocking mode is used to communicate with the peer device corresponding to the other chat objects.

In response to the user's first input of the second control operation, use the send shield mode to communicate with the counterpart device corresponding to the target chat object, and use the normal send mode to communicate with the counterpart device corresponding to other chat objects.

In response to the user's first input of the third control operation, the reception mask mode is used to communicate with the peer device corresponding to the target chat object, and the normal reception mode is used to communicate with the peer devices corresponding to other chat objects.

In response to the user's first input of the fourth control operation, the normal reception mode is used to communicate with the counterpart device corresponding to the target chat object, and the reception shield mode is used to communicate with the counterpart devices corresponding to other chat objects.

When the target control includes both the first control and the second control, in the case that the user only wants to chat with a few chat objects, the first control can be selected for operation; in the case that the user only wants to chat with a few chat objects without chatting , the second control can be operated, thereby reducing the number of operations.

When the target control includes both the third control and the fourth control, in the case that the user only wants to chat with a few chat objects, the fourth control can be selected for operation; in the case that the user only wants to chat with a few chat objects without chatting , the third control can be operated, thereby reducing the number of operations.

As shown in FIG. 2 , the smiling face image in the group interface is the object identification of the chatting object, and each area containing the smiling face image is the screen area where the object identification is located, and there are 9 chatting objects in total.

The target controls in the group interface include a first control and a third control.

The group interface also includes other controls, namely mute controls, hands-free controls, camera-on controls, and end controls.

Figure 2 is just an example of a group interface. This embodiment is not limited to the interface layout, the number of chat objects, the number of other controls, and the number of target controls in the group interface.

In the case where the user only needs to chat with a certain chat object in the group interface, the chat object is the target chat object. Drag and drop the first control onto the screen area where the target object identifier of the target chat object is located, as shown in FIG. 3 .

In the case of multiple target chat objects, the schematic diagram of the group interface after the user drags and drops the first control is shown in Fig. 4. At this point, the local device conducts a directed chat with multiple target chat objects.

Optionally, in response to the user's dragging operation on the first control, the local device performs communication with the peer device corresponding to the target chat object using the normal sending mode, and uses the sending shield mode to communicate with the peer device corresponding to other chat objects. devices to communicate.

In the case where the user only needs to directionally block a certain chat object in the group interface, the chat object is the target chat object. Drag and drop the third control onto the screen area where the target object identifier of the target chat object is located, as shown in FIG. 5 .

In the case where there are multiple target chat objects, a schematic diagram of the group interface after the user performs a drag operation on the third control is shown in FIG. 6 . At this time, the local device performs directional shielding on multiple target chat objects.

In response to the user's drag operation on the third control, the local device communicates with the peer device corresponding to the target chat object in the reception mask mode, and communicates with the peer device corresponding to other chat objects in the normal reception mode.

In addition, according to actual needs, the user can drag the first control to the screen area where one or more target chat objects are located in the group interface, and simultaneously drag the third control to one or more target chats in the group interface On the screen area where the object is located, as shown in Figure 7.

It should be noted that, the first control and the second control cannot perform the first input operation at the same time, and the third control and the fourth control cannot perform the first input operation at the same time. The first control and the fourth control can simultaneously perform the first input operation, and the first control and the third control can simultaneously perform the first input operation.

The picture area to which the first control and the third control are dragged may be the same or different. Likewise, the picture area to which the first control and the fourth control are dragged may be the same or different. When the mute control is selected in the group interface on the local device, the local device does not collect voice, so neither the target chat object nor other chat objects can receive the audio data packets of the local device.

The user clicks a target control displayed on the screen area where the target object identifier is located in the group interface, so that the target control is in a selected state.

As shown in FIG. 8 , the screen area where each chat object is located includes a first control and a third control. The first control and the third control are checkbox controls.

When the user needs to chat with one or some chat objects in the group interface, click the first control and/or the third control on the screen area where the corresponding chat object is located, as shown in FIG. 9 .

When the first control on the screen area where the chat object is located is selected, the local device uses the normal sending mode to communicate with the peer device corresponding to the target chat object, and uses the send shield mode to communicate with the peer device corresponding to other chat objects. communicate with end devices.

When the third control on the screen area where the chat object is located is selected, the local device communicates with the peer device corresponding to the target chat object in the reception blocking mode, and communicates with the peer device corresponding to other chat objects in the normal reception mode. communicate with end devices.

The first control and the third control on the screen area where each chat object identifier is located may both be in the selected state.

As shown in FIG. 10 , the first control and the third control are radio button controls.

When the user needs to chat with one or some chat objects in the group interface, click the first control or the third control on the screen area where the corresponding chat object ID is located, as shown in FIG. 11 .

When the first control on the screen area where the chat object identifier is located is selected, the local device uses the normal sending mode to communicate with the peer device corresponding to the target chat object, and uses the send shield mode to communicate with other chat objects corresponding to Communicate with peer devices.

When the third control on the screen area where the chat object ID is located is selected, the local device uses the reception blocking mode to communicate with the peer device corresponding to the target chat object, and uses the normal reception mode to communicate with other chat objects corresponding to other chat objects. Communicate with peer devices.

Due to the characteristics of radio buttons, the first control and the third control on the screen area where each chat object is located cannot both be in the selected state.

In this embodiment of the present application, the target control on the screen area where the target object identifier is located is set by dragging, or the selection state of the target control on the screen area where the target object identifier is located is set by clicking the mouse, which is convenient for operation and provides follow-up directed chat or directed shielding. Lay the foundation.

Optionally, the method further includes: in the case of communicating with the opposite terminal device using the sending shielding mode, sending the simulated image data packet or the avatar image data packet of the user to the other chat objects.

Optionally, in the case of using the sending shielding mode with the peer device corresponding to other chat objects, send the user's simulated screen data packet or avatar screen data packet to other chat objects, so that other chat objects correspond to The peer device displays the simulation picture or the avatar picture.

Optionally, in the case of a video call, the simulated picture data packet of the user on the local device is sent. The simulated picture data packets are different from the actual audio and video data packets.

Optionally, the simulated picture data packet is obtained by processing the video in the actual audio and video data packet sent by the local device.

The processing process of the video in the actual audio and video data packets sent by the local device is: identify the mouth in each video frame, and replace the mouth in each video frame with the mouth in the reference frame, so that other chat objects can see 's mouth does not move in the video.

The reference frame is any video frame in the video.

Optionally, the avatar picture data package is a user avatar preset by the user in the group video or group voice on the local device.

Optionally, the simulated image data packet or the avatar image data packet of the local device is sent to other chat objects through the cloud server.

Optionally, in the case of using the sending shielding mode with the peer device corresponding to the target chat object, send the simulated screen data packet or avatar screen of the user to the peer device corresponding to the target chat object. data package, so that the peer device corresponding to the target chat object displays the simulated picture or the avatar picture.

Optionally, the simulated image data packet or the avatar image data packet of the local device is sent to the target chat object through the cloud server.

In the embodiment of the present application, in the case of directional chatting with the target chat object or other chat objects, the simulated screen data packet or the avatar screen data packet of the local device is sent to other chat objects or target chat objects shielded in the group interface, In the case of implementing directional chat, other blocked chat objects are not easily noticed, and user experience is improved.

Optionally, the method further includes: using the reception masking mode to communicate with the opposite end device, and the reception masking mode is receiving but not playing the actual audio and video data sent by the opposite end device. Next, the audio data in the actual audio and video data packets sent by the peer device is parsed into corresponding text, and the text is displayed in a preset area in the group interface.

In the case where the local device receives but does not play the actual audio and video data sent by the peer device corresponding to the target chat object, the user may not want to hear or see the audio corresponding to the audio and video data packets sent by the target chat object. Video, but because the background sound of the target chat object's audio and video is noisy and inaudible, and may affect the listening of other chat objects' audio and video.

Therefore, in order to prevent the local device from losing the important information sent by the target chat object, after receiving the audio and video data packets sent by the target chat object, although the audio and video data packets corresponding to the audio and video data packets are not played, the audio and video data packets are not played. The audio data in is parsed into corresponding text and displayed in the group interface, as shown in Figure 12 to Figure 14.

Optionally, use the STT (Speech to Text, language to text) function of the local device to parse the audio data in the audio and video data packets sent by the target chat object into corresponding text.

In the case that the local device receives but does not play the actual audio and video data sent by the peer device corresponding to other chat objects, the user may not want to hear or see the audio and video data packets corresponding to the audio and video data packets sent by other chat objects. video, but because the background sound of the audio and video of other chat objects is noisy and inaudible, and may affect the listening of the audio and video of the target chat object.

Therefore, in order to prevent the local device from losing important information sent by other chat objects, after receiving the audio and video data packets sent by other chat objects, although the audio and video data packets corresponding to the audio and video data packets are not played, the audio and video data packets are not played. The audio data in is parsed into corresponding text and displayed in the group interface.

In the embodiment of the present application, in the case of directional shielding of the target chat object or other objects, the audio data in the audio and video data packets sent by the target chat object or other objects is parsed into corresponding text and displayed, and the target chat object can still be received. information sent by the object.

Optionally, the method further includes: receiving a second input from the user to the target chat object;

The second input is the operation of canceling the first input.

Under the condition that the user no longer needs to use the shielding mode to communicate with a certain target chat object or other chat objects, the user makes a second input to the corresponding target chat object in the group interface.

The local device receives the user's second input to the target chat object.

The form of the second input is determined according to the form of the first input.

In response to the second input, in the case of using the masked mode to communicate with the opposite device, switching to use the normal mode to communicate with the opposite device.

In response to the second input, the local device resends the user's actual audio and video data packets to the target chat object or other chat objects without sending the user's actual audio and video data packets to the target chat object or other chat objects.

In the case of not receiving or receiving but not playing the actual audio and video data packets sent by the target chat object or other chat objects, re-receive and play the actual audio and video data packets sent by the target chat object or other chat objects.

In the case of canceling the targeted chat and targeted blocking of a target chat object, the target chat object is regarded as a normal chat object. The chat mode before the first input is restored, that is, the default mode.

The embodiment of the present application cancels the directional chatting or directional shielding of the target chat object according to the second input of the user, which improves the flexibility of communication.

Optionally, the second input includes: an operation by the user to drag the target control in the group interface to leave the screen area where the target chat object is located in the group interface;

Since the second input is an operation of canceling the first input, the form of the second input is determined according to the form of the first input. Therefore, when the first input is the user dragging the target control to the screen area where the target chat object is located, the second input is The input is that the user drags the target control to leave the screen area where the target chat object is located.

The user can drag the target control to leave the screen area where some or all of the target chat objects are located.

When the user drags the target control away from the screen area where all the target chat objects are located, the group interface is restored to FIG. 2 .

Or the user clicks on the target control displayed on the screen area where the target chat object is located in the group interface, so that the target control is in an unselected state.

In the case where the first input is that the user clicks the target control displayed on the screen area where the target chat object is located, so that the target control is selected, the second input is that the user clicks the target control displayed on the screen area where the target chat object is located. target control to make the target control unselected.

The user clicks on the target controls displayed on the screen area where some or all of the target chat objects are located, so that the target controls are in an unselected state.

When the user clicks the target controls displayed on the screen area where all the target chat objects are located, the group interface returns to FIG. 8 or FIG. 10 .

In this embodiment of the present application, the target control on the screen area where the target chat object is located is canceled by dragging, or the target control on the screen area where the target chat object is located is unselected by clicking the mouse, which is convenient for operation and cancels the directional chat in the future. Or cancel directional masking to lay the groundwork.

The complete steps of the instant messaging method provided by the embodiments of the present application are as follows:

1. First, the user opens the social software on the local device and joins the group video or group voice chat.

2. The user operates on one or more target chat objects on the group interface of the video or group voice chat, and uses the operation as the user's first input to the target chat objects.

3. In response to the first input, the local device performs directional chatting or directional shielding on the target chat object.

4. The user makes a second input to the target chat object in the group interface.

5. In response to the second input, the local device cancels the directional chatting or directional shielding of the target chat object.

6. The target chat object restores the default chat mode.

Optionally, the method further includes: converting the actual audio and video data to be sent into corresponding text; and sending a data packet carrying the text to the peer device corresponding to each of the chat objects.

When the local device detects that its own network status is not good, there is a delay in sending audio and video data packets from the local device to the cloud server, which causes the chat object in the group interface to play the video or voice of the local device intermittently, causing lag and poor experience. it is good.

In order to solve this problem, the local device converts the actual audio and video data to be sent by the local device into corresponding text, and then forwards it to each chat object through the cloud server. The converted text is displayed on the client side of each chat partner.

When converting a video into corresponding text, first extract the audio from the video, and then convert the extracted audio into text.

Since the size of the text data packets is much smaller than that of the audio and video data packets, the transmission speed of the text data packets is faster than that of the audio and video data packets, which can avoid jamming.

The local device judges its own network status, and decides whether to convert the audio and video data to be sent into corresponding text according to the judgment result.

When the local device determines that the network status is poor, it converts the audio and video data to be sent into the corresponding text, but after monitoring that the network status is good, it stops converting the audio and video data to be sent into the corresponding text.

When the local device judges that the network status is good, it does not perform text conversion on the audio and video data to be sent.

The embodiment of the present application improves the transmission speed of data packets by converting the audio and video data to be sent by the local device into corresponding text and forwarding it to each chat object through the cloud server. When the network status of the local device is not good, Avoid stuttering.

Optionally, after sending the data packet carrying the text to the peer device corresponding to each chat object, the method further includes: sending the simulation of the user to the peer device corresponding to each chat object. Screen data packet or avatar screen data packet.

Optionally, the simulated picture data packet is obtained by processing the video in the audio and video data packet previously sent by the local device.

The processing process of the video in the audio and video data packets sent by the local device is as follows: identify the mouth in each video frame, and replace the mouth in each video frame with the mouth corresponding to the converted text, so that other chat objects can see it. The movement of the mouth in the received video is consistent with the converted text.

The mouth corresponding to the text sent to each chat partner refers to the mouth that matches the lip shape when the converted text is read.

Optionally, a voice with a tone similar to that of the user on the local device is generated according to the converted text, and the generated voice is sent to each chat object via the cloud server for playback.

In the implementation of the present application, the user experience is improved by sending the converted text to each chat object, and also sending the simulated screen data packet or the avatar screen data packet.

It should be noted that, in the instant messaging method provided by the embodiments of the present application, the execution subject may be an instant messaging device, or a control module in the instant messaging device for executing the loading instant messaging method. In the embodiment of the present application, the instant messaging method provided by the embodiment of the present application is described by taking the instant messaging device executing the loading instant messaging method as an example.

As shown in FIG. 15 , the instant messaging device provided by the embodiment of the present application includes a receiving module 1501 and a response module 1502, wherein:

The receiving module 1501 is configured to receive the first input of the target object identifier in the group interface when the user conducts instant communication with multiple chat objects simultaneously through the group interface; wherein, the target object identifier is used for indicating a target chat object in the plurality of chat objects;

The user first opens the social software on the local device and joins the group video or group voice chat. Then, the target chat object is operated on the group interface of the group video or group voice chat, and the operation is taken as the user's first input to the target chat object.

Response module 1502, configured to respond to the first input, use different modes to correspond to the peer device corresponding to the target chat object and other chat objects other than the target chat object among the plurality of chat objects. Corresponding peer device communicates; wherein, the modes include normal mode and shielding mode;

The response module is configured to: use the normal transmission mode and the transmission masking mode as transmission modes, and use the normal reception mode and the reception masking mode as reception modes; use different transmission modes or reception modes The mode communicates with the peer device corresponding to the target chat object and the peer devices corresponding to other chat objects in the plurality of chat objects except the target chat object.

Optionally, the first input includes: an operation by the user to move a target control on the group interface to a screen area where the target object identifier is located; or, the user clicks on the target object Identify the target control displayed on the screen area where it is located, so that the target control is in a selected state.

Optionally, the execution module is further configured to: in the case of using the transmission shield mode to communicate with the peer device, send the user's simulated screen data packet or avatar screen data to the peer device Bag.

Optionally, the response module is further configured to: use the reception masking mode to communicate with the peer device, and the reception mask mode is to receive but not play the actual audio and video data sent by the peer device. In the case of , the audio data in the actual audio and video data packets sent by the peer device is parsed into corresponding text, and the text is displayed in a preset area in the group interface.

Optionally, the response module is further configured to: receive a second input of the target object identifier from the user; in response to the second input, in the case of using the shielding mode to communicate with the opposite terminal device , and switch to use the normal mode to communicate with the peer device.

Optionally, the response module is further configured to: convert the actual audio and video data to be sent into corresponding text; and send a data packet carrying the text to the peer device corresponding to each chat object.

Optionally, after the response module sends the data packet carrying the text to the peer device corresponding to each chat object, it is also used to:

Send the simulated image data packet or the avatar image data packet of the user to the peer device corresponding to each chat object.

The instant messaging device in this embodiment of the present application may be a device, or may be a component, an integrated circuit, or a chip in a terminal. The apparatus may be a mobile electronic device or a non-mobile electronic device. Exemplarily, the mobile electronic device may be a mobile phone, a tablet computer, a notebook computer, a palmtop computer, an in-vehicle electronic device, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (personal digital assistant). assistant, PDA), etc., non-mobile electronic devices can be servers, network attached storage (Network Attached Storage, NAS), personal computer (personal computer, PC), television (television, TV), teller machine or self-service machine, etc., this application Examples are not specifically limited.

The instant messaging in this embodiment of the present application may be a device having an operating system. The operating system may be an Android (Android) operating system, an ios operating system, or other possible operating systems, which are not specifically limited in the embodiments of the present application.

The instant messaging provided in the embodiments of the present application can implement each process implemented by the instant messaging device in the method embodiments of FIG. 1 to FIG. 16 , and to avoid repetition, details are not described here.

FIG. 16 is a schematic diagram of the hardware structure of an electronic device provided by an embodiment of the application. As shown in FIG. 16 , the electronic device 1600 includes but is not limited to: a radio frequency unit 1601, a network module 1602, an audio output unit 1603, an input unit 1604, At least some of the components in the sensor 1605, the display unit 1606, the user input unit 1607, the interface unit 1608, the memory 1609, and the processor 1610 and the like.

Those skilled in the art can understand that the electronic device 1600 may also include a power source (such as a battery) for supplying power to various components, and the power source may be logically connected to the processor 1610 through a power management system, so as to manage charging, discharging, and power management through the power management system. consumption management and other functions. The structure of the electronic device shown in FIG. 16 does not constitute a limitation on the electronic device. The electronic device may include more or less components than the one shown, or combine some components, or arrange different components, which will not be repeated here. .

It should be understood that, in this embodiment of the present application, the input unit 1604 may include a graphics processor (Graphics Processing Unit, GPU) 16041 and a microphone 16042. Such as camera) to obtain still pictures or video image data for processing. The display unit 1606 may include a display panel 16061, which may be configured in the form of a liquid crystal display, an organic light emitting diode, or the like. The user input unit 1607 includes a touch panel 16071 and other input devices 16072 . Touch panel 16071, also called touch screen. The touch panel 16071 may include two parts, a touch detection device and a touch controller. Other input devices 16072 may include, but are not limited to, physical keyboards, function keys (such as volume control keys, switch keys, etc.), trackballs, mice, and joysticks, which are not described herein again.

In the embodiment of the present application, the radio frequency unit 1601 receives the downlink data from the network side device, and then processes it to the processor 1610; in addition, sends the uplink data to the network side device. Generally, the radio frequency unit 1601 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, and the like.

Memory 1609 may be used to store software programs or instructions as well as various data. The memory 1609 may mainly include a storage program or instruction area and a storage data area, wherein the stored program or instruction area may store an operating system, an application program or instruction required for at least one function (such as a sound playback function, an image playback function, etc.) and the like. In addition, the memory 1609 may include a high-speed random access memory, and may also include a non-volatile memory, wherein the non-volatile memory may be a read-only memory (Read-Only Memory, ROM), a programmable read-only memory (Programmable ROM) , PROM), erasable programmable read-only memory (Erasable PROM, EPROM), electrically erasable programmable read-only memory (Electrically EPROM, EEPROM) or flash memory. For example at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device.

The processor 1610 may include one or more processing units; optionally, the processor 1610 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, application programs or instructions, etc., Modem processors mainly deal with wireless communications, such as baseband processors. It can be understood that, the above-mentioned modulation and demodulation processor may not be integrated into the processor 1610.

The user input unit 1607 is configured to receive the first input of the target object identifier in the group interface when the user conducts instant communication with multiple chat objects through the group interface; wherein the target The object identifier is used to indicate a target chat object among the plurality of chat objects; the processor 1610 is configured to, in response to the first input, use different modes to correspond with the peer device and the target chat object corresponding to the target chat object. communicating with peer devices corresponding to other chat objects other than the target chat object among the plurality of chat objects;

Wherein, the modes include normal mode and shield mode;

The processor 1610 is configured to: use the normal transmission mode and the transmission masking mode as a transmission mode, and use the normal reception mode and the reception masking mode as a reception mode;

Optionally, the processor is further configured to: in the case of using the transmission mask mode to communicate with the opposite end device, send the user's simulated picture data packet or avatar picture data to the opposite end device Bag.

Optionally, the processor is further configured to: communicate with the peer device in the reception mask mode, and the reception mask mode is to receive but not play the actual audio and video data sent by the peer device. In the case of , the audio data in the actual audio and video data packets sent by the peer device is parsed into corresponding text, and the text is displayed in a preset area in the group interface.

Optionally, the processor is further configured to: receive a second input of the target object identifier from the user; in response to the second input, in the case of using the shielded mode to communicate with the opposite terminal device , and switch to use the normal mode to communicate with the peer device.

Optionally, the processor is further configured to: convert the audio and video data to be sent by the local device into corresponding text; send the audio and video data carrying the text to the peer device corresponding to each chat object. data pack.

Optionally, after the processor sends the data packet carrying the text to the peer device corresponding to each chat object, the processor is further configured to: send the message to the peer device corresponding to each chat object. User's simulation screen data packet or avatar screen data packet.

The electronic device embodiments in the embodiments of the present application are product embodiments corresponding to the foregoing method embodiments, and all implementations in the foregoing method embodiments are applicable to the electronic device embodiments, and can also achieve the same or similar technical effects. Therefore, it will not be repeated here.

Embodiments of the present application further provide a readable storage medium, where a program or an instruction is stored on the readable storage medium, and when the program or instruction is executed by a processor, each process of the foregoing object distance measurement method embodiment can be achieved, and can achieve The same technical effect, in order to avoid repetition, will not be repeated here.

Wherein, the processor is the processor in the electronic device described in the foregoing embodiments. The readable storage medium includes a computer-readable storage medium, such as a computer read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a magnetic disk or an optical disk, and the like.

An embodiment of the present application further provides a chip, where the chip includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is configured to run a program or an instruction to implement the above embodiment of the method for measuring distance between objects and can achieve the same technical effect, in order to avoid repetition, it will not be repeated here.

It should be understood that the chip mentioned in the embodiments of the present application may also be referred to as a system-on-chip, a system-on-chip, a system-on-a-chip, or a system-on-a-chip, or the like.

It should be noted that, herein, the terms "comprising", "comprising" or any other variation thereof are intended to encompass non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements, It also includes other elements not expressly listed or inherent to such a process, method, article or apparatus. Without further limitation, an element qualified by the phrase "comprising a..." does not preclude the presence of additional elements in the process, method, article or apparatus that includes the element. In addition, it should be noted that the scope of the methods and apparatus in the embodiments of the present application is not limited to performing the functions in the order shown or discussed, but may also include performing the functions in a substantially simultaneous manner or in the reverse order depending on the functions involved. performing, may also include performing the functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, eg, the described methods may be performed in a different order than described, and may also be added, omitted, or Combine various steps. Additionally, features described with reference to some examples may be combined in other examples.

From the description of the above embodiments, those skilled in the art can clearly understand that the method of the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is better implementation. Based on this understanding, the technical solution of the present application can be embodied in the form of a software product in essence or in a part that contributes to the prior art, and the computer software product is stored in a storage medium (such as ROM/RAM, magnetic disk, CD-ROM), including several instructions to make a terminal (which may be a mobile phone, a computer, a server, or a network device, etc.) execute the methods described in the various embodiments of this application.

The embodiments of the present application have been described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific embodiments, which are merely illustrative rather than restrictive. Under the inspiration of this application, without departing from the scope of protection of the purpose of this application and the claims, many forms can be made, which all fall within the protection of this application.

Claims

An instant communication method, comprising:

In the case where the user communicates with multiple chat objects simultaneously through the group interface, a first input of the target object identifier in the group interface is received; wherein the target object identifier is used to indicate the plurality of chat objects. The target chat object in the chat object;

In response to the first input, use different modes to perform a communication with the peer device corresponding to the target chat object and the peer device corresponding to other chat objects in the plurality of chat objects except the target chat object. communication;

Wherein, the modes include normal mode and shield mode;

The normal mode includes sending the actual audio and video data packets of the user, and receiving and playing at least one of the actual audio and video data packets sent by the peer device;

The shielding mode includes not sending the actual audio and video data packets of the user, and at least one of not receiving, receiving but not playing the actual audio and video data packets sent by the peer device.
The instant messaging method according to claim 1, wherein the normal mode includes a normal sending mode and a normal receiving mode, and the shielding mode includes a sending shielding mode and a receiving shielding mode;

The normal sending mode is to send the actual audio and video data packets of the user, the normal receiving mode is to receive and play the actual audio and video data packets sent by the peer device, and the transmission shielding mode is not to send the user The actual audio and video data packets, the reception mask mode is not receiving or receiving but not playing the actual audio and video data packets sent by the peer device;

The use of different modes to communicate with the target chat object and other chat objects in the plurality of chat objects includes:

Using the normal transmission mode and the transmission masking mode as a transmission mode, and using the normal reception mode and the reception masking mode as a reception mode;

Use the different sending modes or the receiving modes to conduct communication with the peer device corresponding to the target chat object and the peer devices corresponding to other chat objects in the plurality of chat objects except the target chat object. communication.
The instant messaging method according to claim 1 or 2, wherein the first input comprises:

An operation by the user to move the target control on the group interface to the screen area where the target object identifier is located; or,

The user clicks a target control displayed on the screen area where the target object identifier is located, so that the target control is in a selected state.
The instant messaging method according to claim 2, wherein the method further comprises:

In the case of using the transmission mask mode to communicate with the opposite end device, send the user's simulated picture data packet or the avatar picture data packet to the opposite end device.
The instant messaging method according to claim 2, wherein the method further comprises:

In the case that the reception masking mode is used to communicate with the peer device, and the reception masking mode is to receive but not play the actual audio and video data sent by the peer device, send the audio and video data sent by the peer device. The audio data in the actual audio and video data package is parsed into corresponding text, and the text is displayed in a preset area in the group interface.
The instant messaging method according to claim 1 or 2, wherein the method further comprises:

receiving a second input of the target object identifier from the user;

In response to the second input, in the case of using the masked mode to communicate with the opposite device, switching to use the normal mode to communicate with the opposite device.
The instant messaging method according to claim 2, wherein the method further comprises:

Convert the actual audio and video data to be sent into corresponding text;

Send a data packet carrying the text to the peer device corresponding to each of the chat objects.
The instant messaging method according to claim 7, wherein after sending the data packet carrying the text to the peer device corresponding to each of the chat objects, the method further comprises:

Send the simulated image data packet or the avatar image data packet of the user to the peer device corresponding to each of the chat objects.
An instant messaging device, comprising:

A receiving module, configured to receive the first input of the target object identifier in the group interface when the user conducts instant communication with multiple chat objects simultaneously through the group interface; wherein, the target object identifier is used for indicating a target chat object in the plurality of chat objects;

A response module for responding to the first input, using different modes to correspond to the peer device corresponding to the target chat object and other chat objects other than the target chat object in the plurality of chat objects to communicate with the peer device;

Wherein, the modes include normal mode and shield mode;

The normal mode includes sending the actual audio and video data packets of the user, and receiving and playing at least one of the actual audio and video data packets sent by the peer device;

The shielding mode includes not sending the actual audio and video data packets of the user, and at least one of not receiving, receiving but not playing the actual audio and video data packets sent by the peer device.
The instant messaging device according to claim 9, wherein the normal mode includes a normal sending mode and a normal receiving mode, and the shielding mode includes a sending shielding mode and a receiving shielding mode;

The normal sending mode is to send the actual audio and video data packets of the user, the normal receiving mode is to receive and play the actual audio and video data packets sent by the peer device, and the transmission shielding mode is not to send the user The actual audio and video data packets, the reception mask mode is not receiving or receiving but not playing the actual audio and video data packets sent by the peer device;

The response module is configured to: use the normal transmission mode and the transmission masking mode as transmission modes, and use the normal reception mode and the reception masking mode as reception modes; use different transmission modes or reception modes The mode communicates with the peer device corresponding to the target chat object and the peer devices corresponding to other chat objects in the plurality of chat objects except the target chat object.
The instant messaging device according to claim 9 or 10, wherein the first input comprises: an operation by the user to move a target control on the group interface to a screen area where the target object identifier is located ; or, the user clicks on the target control displayed on the screen area where the target object identifier is located, so that the target control is in a selected state.
The instant messaging apparatus according to claim 10, wherein the executing module is further configured to: in the case of using the transmission masking mode to communicate with the opposite end device, send the user to the opposite end device The simulation screen data packet or the avatar screen data packet.
The instant communication device according to claim 10, wherein the response module is further configured to: communicate with the peer device using the reception masking mode, and the reception masking mode is to receive but not play the In the case of actual audio and video data sent by the peer device, parse the audio data in the actual audio and video data packets sent by the peer device into corresponding text, and display all the audio and video data in the preset area in the group interface. description text.
The instant messaging device according to claim 9 or 10, wherein the response module is further configured to: receive a second input of the target object identifier from the user; in response to the second input, when using the shielding mode In the case of communicating with the opposite end device, switch to use the normal mode to communicate with the opposite end device.
The instant messaging device according to claim 10, wherein the response module is further configured to: convert the actual audio and video data to be sent into corresponding text; A packet with said text.
The instant messaging device according to claim 15, wherein after the response module sends the data packet carrying the text to the peer device corresponding to each chat object, the response module is further configured to:

Send the simulated image data packet or the avatar image data packet of the user to the peer device corresponding to each chat object.
An electronic device, comprising a processor, a memory, and a program or instruction stored on the memory and executable on the processor, the program or instruction being executed by the processor to achieve as claimed in claim 1 -8 any of the steps of the instant messaging method.
A readable storage medium, wherein a program or an instruction is stored on the readable storage medium, and when the program or instruction is executed by a processor, the steps of the instant messaging method according to any one of claims 1-8 are implemented.
A chip, wherein the chip includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is used for running programs or instructions to implement the method according to any one of claims 1-8 .