CN116800707A

CN116800707A - Communication message processing method and display device

Info

Publication number: CN116800707A
Application number: CN202210767185.3A
Authority: CN
Inventors: 范传恒; 李鑫; 赖园园; 薛梅
Original assignee: Juhaokan Technology Co Ltd
Current assignee: Juhaokan Technology Co Ltd
Priority date: 2022-06-30
Filing date: 2022-06-30
Publication date: 2023-09-22

Abstract

The present disclosure relates to a communication message processing method and a display device, including: monitoring communication information of the first equipment in the process that the first equipment throws a screen to the second equipment; the method comprises the steps that communication information is monitored, state information of a follow-up video stream displayed by a second device is obtained, the state information comprises a follow-up state and a pause state, and the communication information represents communication information sent by a third device to a first device; according to the state information of the follow-up video stream displayed by the second device, a control instruction is sent to the second device to control the second device to display notification content, notification content displayed by the second device corresponding to different control instructions is different, the situation that when a user follows the training video stream played by the second device, notification content received by the first device is directly sent to the second device is avoided, the training video stream played by the second device and the follow-up video stream are paused, the user in the follow-up state cannot respond timely, and the use experience of the user is reduced.

Description

Communication message processing method and display device

Technical Field

The disclosure relates to the technical field of display, and in particular relates to a communication message processing method and display equipment.

Background

With the development of terminal technology and internet technology, the multi-screen collaborative technology is widely applied. Multi-screen collaboration refers to the collaboration of multiple screens, which involves projecting the screen content of one device onto another device for display. For example, screen contents of the mobile phone are projected to the computer for display, so that a user can watch the contents displayed on the screen of the mobile phone on the computer, and the same screen display between the mobile phone and the computer is realized.

In the prior art, in the process that a first device throws a screen to a second device, the second device displays screen content of the first device. However, after the first device receives the communication message, the content displayed by the second device may pause displaying and access the communication message. However, in some usage scenarios, such as a fitness scenario, the user may exercise the training video played by the second device, the first device collects the heel exercise video of the user and throws a screen to the second device, the second device displays the training video and the heel exercise video, and after the first device receives the communication message, the first device interrupts the training video and the heel exercise video played by the second device, so that the user experience is reduced.

Disclosure of Invention

In order to solve the above technical problems or at least partially solve the above technical problems, the present disclosure provides a communication message processing method and a display device.

In a first aspect, an embodiment of the present disclosure provides a method for processing a communication message, including:

monitoring communication information of a first device in the process of screen projection of the first device to a second device;

acquiring state information of a follow-up video stream displayed by the second device when the communication message is monitored, wherein the state information comprises a follow-up state and a pause state, and the communication message represents a communication message sent by a third device to the first device;

and sending a control instruction to the second equipment according to the state information of the follow-up video stream displayed by the second equipment so as to control the second equipment to display notification content, wherein the notification content displayed by the second equipment corresponding to different control instructions is different.

As an embodiment, optionally, the notification content includes a first notification content and a second notification content, where the first notification content includes text and the second notification content includes text and audio;

and sending a control instruction to the second device according to the state information of the follow-up video stream displayed by the second device, including:

When the state information of the follow-up video stream displayed by the second equipment is the follow-up state, a first control instruction is sent to the second equipment so as to control the second equipment to display a text corresponding to the first notification content;

and when the state information of the follow-up video stream displayed by the second equipment is in a pause state, sending a second control instruction to the second equipment so as to control the second equipment to display a text corresponding to the second notification content and control a loudspeaker to play audio corresponding to the second notification content.

As an alternative embodiment, the method further comprises:

when the state information of the follow-up video stream displayed by the second equipment is the follow-up state, responding to the pause follow-up indication signal received by the second equipment, controlling a display to display a text corresponding to the notification content, and controlling a loudspeaker to play audio corresponding to the notification content.

As an alternative embodiment, the method further comprises:

when the state information of the follow-up video stream displayed by the second device is the follow-up state, responding to the trigger operation received by the second device, and sending a first indication instruction to the first device so as to indicate the first device to open an application corresponding to the notification content; or controlling the display to pause playing the training video stream and the follow-up training video stream, and controlling the loudspeaker to play the audio corresponding to the notification content.

As an optional implementation manner, after the sending the first instruction to the first device, the method further includes:

monitoring the communication state of the audio and video corresponding to the communication message of the first equipment;

when the communication state of the audio and video corresponding to the communication message of the first equipment is changed from the communication state to the communication ending state, the screen-throwing communication channels of the first equipment and the second equipment are established.

As an alternative embodiment, the method further comprises:

when the second equipment comprises an application corresponding to the notification content, opening the notification content through the application of the second equipment;

monitoring the communication state of the audio and video corresponding to the communication message;

and when the communication state of the audio and video corresponding to the communication message of the second equipment is changed from the communication state to the communication ending state, controlling the display to continuously play the training video stream and the follow-up video stream.

As an optional implementation manner, the acquiring the state information of the follow-up video stream displayed by the second device includes:

and when the prompt time length of the communication message received by the first equipment is monitored to be greater than or equal to a preset threshold value, acquiring the state information of the follow-up video stream displayed by the second equipment.

In a second aspect, embodiments of the present disclosure provide a display apparatus including:

a display;

a controller configured to:

and sending a control instruction to the second equipment according to the state information of the follow-up video stream displayed by the second equipment so as to control the second equipment to display notification content, wherein the notification content displayed by the display corresponding to different control instructions is different.

As an alternative embodiment, the controller is specifically configured to:

As an alternative, the device may, alternatively,

the controller is further configured to:

Compared with the prior art, the technical scheme provided by the embodiment of the disclosure has the following advantages:

according to the communication message processing method and the display device, in the screen casting process of the first device to the second device, the communication message of the first device is monitored, the communication message is monitored, whether the user follows the training video stream played by the second device is determined by acquiring the state information of the follow-up video stream displayed by the second device, then a control instruction is sent to the second device according to the acquired state information of the follow-up video stream displayed by the second device, so that the second device is controlled to display notification content, namely, whether the user follows the training video stream played by the second device is determined by acquiring the state information of the follow-up video stream displayed by the second device, further, the second device is controlled to display notification content according to whether the user follows the training video stream played by the second device, the notification content received by the first device is prevented from being directly sent to the second device when the user follows the training video stream played by the second device, the training video stream played by the second device and the follow-up video stream are paused, the user in the follow-up state cannot respond timely, and the use feeling of the user is reduced.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description, serve to explain the principles of the disclosure.

In order to more clearly illustrate the embodiments of the present disclosure or the solutions in the prior art, the drawings that are required for the description of the embodiments or the prior art will be briefly described below, and it will be obvious to those skilled in the art that other drawings can be obtained from these drawings without inventive effort.

Fig. 1A is a system network architecture diagram of a communication message processing method according to an embodiment of the present disclosure;

fig. 1B is a schematic structural diagram of a terminal device according to an embodiment of the present disclosure;

fig. 2A is a flow chart of a communication message processing method according to an embodiment of the disclosure;

FIG. 2B is a signal flow diagram of a communication message processing method according to an embodiment of the present disclosure;

FIG. 3A is a flow chart of another communication message processing method provided by an embodiment of the present disclosure;

FIG. 3B is a signal flow diagram of another communication message processing method according to an embodiment of the present disclosure;

FIG. 4A is a flow chart of yet another method for processing communication messages provided by an embodiment of the present disclosure;

fig. 4B is a signal flow diagram of yet another communication message processing method according to an embodiment of the present disclosure;

FIG. 5A is a flow chart of yet another method for processing communication messages provided by an embodiment of the present disclosure;

FIG. 5B is a signal flow diagram of yet another communication message processing method provided by an embodiment of the present disclosure;

FIG. 6A is a flow chart of yet another method for processing communication messages provided by an embodiment of the present disclosure;

FIG. 6B is a signal flow diagram of yet another communication message processing method provided by an embodiment of the present disclosure;

FIG. 7A is a flow chart of yet another method for processing communication messages provided by an embodiment of the present disclosure;

fig. 7B is a signal flow diagram of yet another communication message processing method according to an embodiment of the present disclosure;

FIG. 8A is a flow chart of yet another method for processing communication messages provided by an embodiment of the present disclosure;

fig. 8B is a signal flow diagram of yet another communication message processing method according to an embodiment of the present disclosure;

fig. 9 is a schematic structural diagram of an electronic device according to an embodiment of the disclosure.

Detailed Description

In order that the above objects, features and advantages of the present disclosure may be more clearly understood, a further description of aspects of the present disclosure will be provided below. It should be noted that, without conflict, the embodiments of the present disclosure and features in the embodiments may be combined with each other.

In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present disclosure, but the present disclosure may be practiced otherwise than as described herein; it will be apparent that the embodiments in the specification are only some, but not all, embodiments of the disclosure.

Fig. 1A schematically illustrates a network architecture diagram of a screen projection system of a screen projection method according to an embodiment of the present application. The screen projection system network architecture diagram comprises a first device 10, a second device 20, a router 30, a server 40 and the like. Wherein: the first device can be used as a transmitting end in a network architecture of the screen projection system and is used for transmitting the multimedia content and various instructions to the second device. In an exemplary embodiment, the first device includes, but is not limited to, a smart phone, a tablet, a gaming machine, a laptop, and the like. The second device can be used as a display end in the network architecture of the screen throwing system and is used for receiving the multimedia content and various instructions of the first device and synchronously playing the received multimedia content to a display panel of the self device or a display panel of the related device. The second device may be a related device with a larger screen, such as a virtual reality device, a set top box, a smart television, a vehicle terminal, a large open-air display panel, or the like, and of course, may also be a related device with a smaller screen, such as a smart phone, a tablet computer, a laptop computer, or the like.

The router can be used as a gateway in a network architecture of a screen throwing system for constructing a local area network and providing internet service. The router is capable of providing a local connection between the first device and the second device.

The server can be used as cloud equipment in a network architecture of the screen projection system and is used for providing cloud services, such as cloud account management services and the like. The server may be a rack server, a blade server, a tower server, or a rack server (including an independent server or a server cluster formed by a plurality of servers), etc.

The first device may discover the second device through the first network or the second network and establish a network connection with the second device based on the first network or the second network. Through the established network connection, the first device can put the multimedia content needing to be put on the screen into the second device, and the multimedia content is played through the second device. The first network may be a local area network, which may be a wired local area network or a wireless local area network. As shown in fig. 1A, the wireless local area network may be a Wi-Fi network built by router 8. The second network may be a wide area network, such as the internet. The wide area network may include physical links such as coaxial cable links, twisted pair cable links, fiber optic links, combinations thereof, and the like. The wide area network may also include wireless links, such as cellular links, satellite links, and the like.

In an exemplary embodiment, as shown in fig. 1B, the first and second apparatuses include at least one of a modem 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output interface 270, a memory, a power supply, and a user interface.

In some embodiments the controller includes a processor, a video processor, an audio processor, a graphics processor, RAM, ROM, a first interface for input/output to an nth interface.

The display 260 includes a display screen component for presenting a picture, and a driving component for driving an image display, a component for receiving an image signal from the controller output, displaying video content, image content, and a menu manipulation interface, and a user manipulation UI interface.

The display 260 may be a liquid crystal display, an OLED display, a projection device, or a projection screen.

The communicator 220 is a component for communicating with external devices or servers according to various communication protocol types. For example: the communicator may include at least one of a Wifi module, a bluetooth module, a wired ethernet module, or other network communication protocol chip or a near field communication protocol chip, and an infrared receiver. The display device 200 may establish transmission and reception of control signals and data signals with the external control device 100 or the server 400 through the communicator 220.

A user interface, which may be used to receive control signals from the control device 100 (e.g., an infrared remote control, etc.).

The detector 230 is used to collect signals of the external environment or interaction with the outside. For example, detector 230 includes a light receiver, a sensor for capturing the intensity of ambient light; alternatively, the detector 230 includes an image collector such as a camera, which may be used to collect external environmental scenes, user attributes, or user interaction gestures, or alternatively, the detector 230 includes a sound collector such as a microphone, or the like, which is used to receive external sounds.

The external device interface 240 may include, but is not limited to, the following: high Definition Multimedia Interface (HDMI), analog or data high definition component input interface (component), composite video input interface (CVBS), USB input interface (USB), RGB port, or the like. The input/output interface may be a composite input/output interface formed by a plurality of interfaces.

The modem 210 receives broadcast television signals through a wired or wireless reception manner, and demodulates audio and video signals, such as EPG data signals, from a plurality of wireless or wired broadcast television signals.

In some embodiments, the controller 250 and the modem 210 may be located in separate devices, i.e., the modem 210 may also be located in an external device to the main device in which the controller 250 is located, such as an external set-top box or the like.

The controller 250 controls the operation of the display device and responds to the user's operations through various software control programs stored on the memory. The controller 250 controls the overall operation of the display apparatus 200. For example: in response to receiving a user command to select a UI object to be displayed on the display 260, the controller 250 may perform an operation related to the object selected by the user command.

In some embodiments the controller includes at least one of a central processing unit (Central Processing Unit, CPU), video processor, audio processor, graphics processor (Graphics Processing Unit, GPU), RAM Random Access Memory, RAM), ROM (Read-Only Memory, ROM), first to nth interfaces for input/output, a communication Bus (Bus), and the like.

The user may input a user command through a Graphical User Interface (GUI) displayed on the display 260, and the user input interface receives the user input command through the Graphical User Interface (GUI). Alternatively, the user may input the user command by inputting a specific sound or gesture, and the user input interface recognizes the sound or gesture through the sensor to receive the user input command.

A "user interface" is a media interface for interaction and exchange of information between an application or operating system and a user, which enables conversion between an internal form of information and a user-acceptable form. A commonly used presentation form of the user interface is a graphical user interface (Graphic User Interface, GUI), which refers to a user interface related to computer operations that is displayed in a graphical manner. It may be an interface element such as an icon, a window, a control, etc. displayed in a display screen of the electronic device, where the control may include a visual interface element such as an icon, a button, a menu, a tab, a text box, a dialog box, a status bar, a navigation bar, a Widget, etc.

The respective functions of the first device and the second device are briefly described above.

The first device may perform a screen-casting operation to the second device.

However, in the prior art, during the process of the first device projecting the screen to the second device, the second device displays the screen content of the first device. After the first device receives the communication message, the content displayed by the second device can pause the display, and the communication message is accessed. However, in some usage scenarios, such as a fitness scenario, the user may exercise the training video played by the second device, the first device collects the heel exercise video of the user and throws a screen to the second device, the second device displays the training video and the heel exercise video, and after the first device receives the communication message, the first device interrupts the training video and the heel exercise video played by the second device, so that the user experience is reduced.

The embodiment of the disclosure provides a communication message processing method, which monitors a communication message of a first device in a process that the first device throws a screen to a second device; the communication message is monitored, state information of the follow-up video stream displayed by the second equipment is obtained, and the state information comprises a follow-up state and a pause state; according to the state information of the follow-up video stream displayed by the second device, a control instruction is sent to the second device to control the second device to display notification content, the notification content displayed by a display corresponding to different control instructions is different, and when the state information of the follow-up video stream displayed by the second device is in a follow-up state, the communication information sent by the third device received by the first device is ensured not to influence the follow-up video stream and the training video stream displayed by the second device, and the use experience of a user is improved.

Fig. 2A is a schematic flow chart of a communication message processing method provided by an embodiment of the present disclosure, and fig. 2B is a schematic flow chart of signals of the communication message processing method provided by the embodiment of the present disclosure, where, in combination with fig. 2A and fig. 2B, the communication message processing method includes:

s100, monitoring communication information of the first equipment in the process that the first equipment throws a screen to the second equipment.

In a specific embodiment, the first device includes an image acquisition module, which may be a camera, for example, and the image acquisition module based on the first device acquires user video data.

As a specific implementation mode, the second device plays a training video stream, such as a body building training video stream and a dance training video stream, a user plays the training video stream by training the second device, and the image acquisition module of the first device acquires the user training video stream and displays the training video stream on the second device through screen throwing.

The distance between the first device and the second device is set to be smaller than a preset distance threshold, and the angle between the orientation of the image acquisition module of the first device and the orientation of the display screen of the second device is set to be smaller than a preset angle threshold.

In the embodiment of the disclosure, when the user performs continuous limb movements, for example, dance movements, gymnastics movements, etc., through the training video stream played by the heel training second device, the image acquisition module of the first device is used to acquire the user heel training video stream. It can be appreciated that, in order to keep the position of the image acquisition module fixed, the first device can be arranged on the fixed support, that is, the distance between the first device and the second device is smaller than a preset distance threshold, and the angle between the orientation of the image acquisition module of the first device and the orientation of the display screen of the second device is smaller than a preset angle threshold, so that the user can practice the training video stream played by the second device, and the image acquisition module of the first device can acquire the user practice video stream.

As one example, during a screen cast of a first device to a second device, the second device may display a training video stream at a presentation window and display screen cast content (i.e., follow-up video stream) of the first device at the screen cast window. For example, the screen-drop window is used for displaying the screen content of the first device, i.e. the display content in the screen-drop window is the same as the content displayed by the screen of the first device.

It should be noted that, the screen projection generally includes a push screen projection and a mirror image screen projection. The method and the device are mainly applied to the mirror image screen projection scene.

The pushing of the screen throwing refers to throwing the multimedia resource of one device to other devices for playing, wherein the multimedia resource comprises audio and video resources. For example, the video on the mobile phone is put on the smart television for playing, the use of the mobile phone by the user is not influenced in the process, and the user can still operate the mobile phone. The mirror image screen projection refers to projecting the screen content of one device to other devices for display, namely, displaying the whole screen mirror image of one device on the other devices. For example, the screen content of the mobile phone is put on the projector to be displayed, and in the process, what content is displayed by the mobile phone, what content is displayed by the screen projection window of the projector.

In addition, during the process of the first device 10 projecting the screen onto the second device 20, the second device 20 may also perform reverse control on the first device 10 according to the projection window in the second device 20.

In a specific embodiment, in a process of screen projection from the first device to the second device, in order to avoid the influence of the communication message received by the first device on the training video stream and the follow-up video stream played by the second device, in a process of screen projection from the first device to the second device, the communication message of the first device is monitored first.

Communication refers to tools that can communicate, and the content of a message can be of the type voice, video, telephone, text, etc.

The communication message characterizes that the first device receives the communication message in the call request sent by the third device, wherein the call request comprises: cellular network call requests (requiring a SIM card, no application) or non-cellular network call requests (requiring no SIM card, no application), the non-cellular network call requests may include: the communication message types may be voice, video, telephone, text, etc. (the third device has an audio prompt message in the process of sending the communication message to the first device).

S200, acquiring state information of the follow-up video stream displayed by the second equipment when the communication message is monitored.

The state information includes a heel state and a pause state.

The second device displays a follow-up video stream and a training video stream, wherein the training video stream is a video stream played by the second device, the follow-up video stream is follow-up video data corresponding to the training video stream played by the first device and acquired by an image acquisition module of the first device, and then a screen is thrown to the second device for display.

When the server of the second device monitors that the first device receives the communication message, the controller of the second device acquires the state information of the follow-up video stream displayed by the screen-throwing window of the second device, namely, based on the state information of the follow-up video stream displayed by the screen-throwing window of the second device, whether the user follows the training video stream played by the second device can be determined, if the state information of the follow-up video stream displayed by the second device is the follow-up state, the training video stream played by the second device can be determined, and if the state information of the follow-up video stream displayed by the second device is the pause state, the training video stream which is not played by the second device can be determined.

As a specific embodiment, the state information of the follow-up video stream may be determined by continuously acquiring the image information of the follow-up video stream with a preset frame value, and comparing the image information corresponding to the adjacent frame values. For example, the image information corresponding to three continuous frames of the follow-up video stream is obtained, whether the image information corresponding to the first frame is consistent with the image information corresponding to the second frame, whether the image information corresponding to the second frame is consistent with the image information corresponding to the third frame or not are sequentially compared, and when the image information corresponding to the first frame is consistent with the image information corresponding to the second frame, the image information corresponding to the second frame is consistent with the image information corresponding to the third frame, then it can be determined that the follow-up video stream is in a pause state. When the image information corresponding to the first frame is inconsistent with the image information corresponding to the second frame and/or the image information corresponding to the second frame is inconsistent with the image information corresponding to the third frame, it can be determined that the follow-up video stream is in the follow-up state.

And S300, according to the state information of the follow-up video stream displayed by the second equipment, sending a control instruction to the second equipment so as to control the second equipment to display notification content.

The notification content displayed by the second device corresponding to the different control instructions is different.

When the first device drops the screen to the second device and the first device receives the communication message, the embodiment of the disclosure sends the control instruction to the second device by acquiring the state information of the follow-up video stream displayed by the second device so as to control the second device to display the notification content, so that the notification content corresponding to the communication message is prevented from being displayed on the screen of the second device directly after the first device drops the screen of the communication message received by the first device in the process of dropping the screen of the second device, the notification content corresponding to the communication message breaks the follow-up video stream and the training video stream displayed by the second device, and the use experience of a user is reduced.

According to the communication message processing method provided by the embodiment of the disclosure, in the screen casting process of the first device to the second device, the communication message of the first device is monitored, the communication message is monitored, whether the user follows the training video stream played by the second device is determined by acquiring the state information of the follow-up video stream displayed by the second device, then a control instruction is sent to the second device according to the acquired state information of the follow-up video stream displayed by the second device, so that the second device is controlled to display notification content, namely, whether the user follows the training video stream played by the second device is determined by acquiring the state information of the follow-up video stream displayed by the second device, further, the second device is controlled to display notification content according to whether the user follows the training video stream played by the second device, the notification content received by the first device is prevented from being directly sent to the second device when the user follows the training video stream played by the second device, the training video stream played by the second device and the follow-up video stream are suspended, so that the user in the follow-up state cannot respond timely, and the use feeling of the user is reduced.

Fig. 3A is a schematic flow chart of another communication message processing method provided by an embodiment of the disclosure, fig. 3B is a schematic signal flow chart of another communication message processing method provided by an embodiment of the disclosure, and in combination with fig. 3A and fig. 3B, an implementation of step S300 includes:

and S301, when the state information of the follow-up video stream displayed by the second device is the follow-up state, sending a first control instruction to the second device so as to control the second device to display a text corresponding to the first notification content.

In a specific embodiment, when the controller of the second device monitors that the first device receives the communication message and the acquired state information of the follow-up video stream displayed by the second device is the follow-up state, the controller of the second device sends a first control instruction to the second device to control the second device to display a text corresponding to the first notification content, that is, only the text corresponding to the notification content received by the first device is displayed, and the speaker does not play audio corresponding to the notification content.

If the first device opens the exercise application program, the image acquisition module of the exercise application program acquires the follow-up video stream of the user and throws the video stream to the second device for display, in the process that the first device throws the video stream to the second device, the exercise application program of the first device monitors that the third device sends a communication message to the first device, for example, the first device receives the incoming call of the user a, at this time, the first device acquires the caller information from the address book of the user according to the incoming call, invokes the first device middleware for sending the caller information to the second device, and after receiving the caller information sent by the middleware of the first device, the controller of the second device sends text data corresponding to the caller information to the second device for display (the caller information is notification content), namely, only the display screen of the second device displays the incoming call text of the user a, and does not play the incoming call audio, for example, the incoming call music of the first device set by the user is prevented from playing in the second device, and the follow-up video stream and the exercise video stream played by the second device corresponding to the notification content is interrupted.

And S302, when the state information of the follow-up video stream displayed by the second equipment is in a pause state, sending a second control instruction to the second equipment so as to control the second equipment to display a text corresponding to the second notification content and control the loudspeaker to play an audio corresponding to the second notification content.

In a specific embodiment, when the controller of the second device monitors that the first device receives the communication message and the acquired state information of the follow-up video stream displayed by the second device is in a pause state, the controller of the second device sends a second control instruction to the second device, the second device informs a text corresponding to the content, and controls the loudspeaker to play audio corresponding to the second informing content, namely, not only the text corresponding to the informing content received by the first device is displayed, but also the audio corresponding to the informing content is played.

If the first device opens the exercise application program, the image acquisition module of the exercise application program acquires the follow-up video stream of the user and throws the screen to the second device for display, in the process that the first device throws the screen to the second device, the exercise application program of the first device monitors the incoming call of the user A, at this time, the first device acquires the caller information from the address book of the user according to the incoming call, calls the first device middleware to send the caller information to the second device, and after receiving the caller information sent by the middleware of the first device, the server of the second device sends text data corresponding to the caller information to the second device for screen throwing display, and plays the caller music set by the first device, so that the user can display the text data corresponding to the communication message and play the audio data through the second device when pausing the follow-up state.

Fig. 4A is a schematic flow chart of another communication message processing method provided by an embodiment of the present disclosure, fig. 4B is a schematic signal flow chart of another communication message processing method provided by an embodiment of the present disclosure, where the embodiment of the present disclosure is based on the embodiment corresponding to fig. 3A, and as shown in fig. 4A and fig. 4B, the communication message processing method further includes:

and S303, when the state information of the follow-up video stream displayed by the second device is the follow-up state, responding to the pause follow-up indication signal received by the second device, controlling the display to display a text corresponding to the notification content, and controlling the loudspeaker to play an audio corresponding to the notification content.

In a specific embodiment, when the communication message is monitored and the state information of the follow-up training video stream displayed by the second device is the follow-up training state, the display is controlled to display the text corresponding to the notification content so as to prompt the user to have the communication message access, and on the basis that the follow-up training video stream of the user is not influenced, the user can be ensured to normally receive the communication message. The user can timely receive the prompt information of the communication message access based on the text corresponding to the display notification content, at this time, the user can control the second device to pause the training video stream through voice control or control equipment, for example, the user can control the second device to pause the training video stream through voice, the second device responds to the voice control signal output by the user and pauses the training video stream, at this time, the controller responds to the pause follow-up instruction information received by the second device, namely, after the controller receives the signal of the second device for pausing the playing of the training video stream, the controller controls the display to display the text corresponding to the notification content and controls the loudspeaker to play the audio corresponding to the notification content, namely, after the instruction signal of the user for pausing the training video stream played by the second device is received, the display is controlled to display the text corresponding to the notification content and the loudspeaker is controlled to play the audio corresponding to the notification content, and the user is ensured to realize long-distance receiving of the notification content.

Fig. 5A is a schematic flow chart of another communication message processing method provided by an embodiment of the present disclosure, fig. 5B is a schematic flow chart of a signal of another communication message processing party provided by an embodiment of the present disclosure, and the embodiment of the present disclosure is based on the embodiment corresponding to fig. 3A, as shown in fig. 5A and fig. 5B, where the communication message processing method further includes:

and S304, when the state information of the follow-up video stream displayed by the second device is the follow-up state, responding to the trigger operation received by the second device, and sending a first indication instruction to the first device so as to indicate the first device to open an application corresponding to the notification content.

Specifically, the triggering operation includes an opening operation, and the triggering operation may be performed based on the control device or may be controlled based on voice.

When the state information of the follow-up video stream displayed by the second device is the follow-up state, as an implementation manner, the user can control the notification content displayed by the second device through voice control or the control device, at this time, the controller responds to the triggering operation of the user on the second device, sends a first indication instruction to the first device, and opens an application/function corresponding to the notification content on the first device.

If the first device monitors that the application program a has a communication message, the communication message is that the user a receives a video call of the user B, when the state information of the follow-up video stream displayed by the second device is the follow-up state, the second device displays a text "video call of the user B" corresponding to the notification content on the second device in a message prompting manner, at this time, if the user triggers the text "video call of the user B" corresponding to the notification content displayed by the second device, the controller responds to the text corresponding to the notification content triggered by the user on the second device, and sends a first instruction to the first device to instruct the first device to open the application program a.

S305, controlling the display to pause playing the training video stream and the follow-up training video stream, and controlling the loudspeaker to play the audio corresponding to the notification content.

When the state information of the follow-up video stream displayed by the second device is the follow-up state, as another implementation manner, the user can control the text corresponding to the notification content displayed by the second device through voice control or the control device, at this time, the controller responds to the triggering operation of the user on the second device, controls the display to pause playing of the training video stream and the follow-up video stream, and controls the loudspeaker to play the audio corresponding to the notification content.

If the first device monitors that the application program a has a communication message, the communication message is that the user a receives a video call of the user B, when the state information of the follow-up video stream displayed by the second device is the follow-up state, the second device displays a text "the video call of the user B" corresponding to the notification content on the second device in a message prompting manner, at this time, if the user triggers the text "the video call of the user B" corresponding to the notification content displayed by the second device, the controller responds to the text corresponding to the notification content triggered by the user on the second device, controls the display to pause playing the training video stream and the follow-up video stream, and controls the speaker to play the audio corresponding to the notification content.

Fig. 6A is a schematic flow chart of another communication message processing method provided by an embodiment of the present disclosure, and fig. 6B is a schematic signal flow chart of another communication message processing method provided by an embodiment of the present disclosure, where, on the basis of the embodiment corresponding to fig. 5A, when the communication message processing method includes step S304, the method further includes:

s306, monitoring the communication state of the audio and video corresponding to the communication message of the first device.

S307, when the communication state of the audio and video corresponding to the communication message of the first device is changed from the communication state to the communication ending state, a screen-throwing communication channel of the first device and the second device is established.

When the state information of the follow-up video stream displayed by the second device is the follow-up state, the user can trigger a text corresponding to the notification content displayed by the second device, so that the controller sends a first indication instruction to the first device, the first device opens an application corresponding to the notification content, at this time, the user can answer a communication message on the application of the first device, and because the first device answers the communication message, the first device exits to throw a screen to the second device. When the controller monitors that the communication state of the audio and video corresponding to the communication message of the first device is changed from the communication state to the communication ending state, the first device can send a screen throwing request again, and the controller responds to the screen throwing request sent by the first device and sends the screen throwing request to the second device, so that the reestablishment of the screen throwing channels of the first device and the second device is realized.

Fig. 7A is a schematic flow chart of another communication message processing method provided by an embodiment of the present disclosure, fig. 7B is a schematic signal flow chart of another communication message processing method provided by an embodiment of the present disclosure, and the embodiment of the present disclosure is based on the embodiment corresponding to fig. 5A, when the communication message processing method includes step S305, the method further includes:

and S308, when the second device comprises an application corresponding to the notification content, opening the notification content through the application of the second device.

S309, monitoring the communication state of the audio and video corresponding to the communication message.

And S310, when the communication state of the audio and video corresponding to the communication message of the second device is changed from the communication state to the communication ending state, controlling the display to continuously play the training video stream and the follow-up video stream.

When the state information of the follow-up video stream displayed by the second device is the follow-up state, the user can trigger and open a text corresponding to the notification content displayed by the second device to further control the second device to pause playing of the training video stream and the follow-up video stream and play of the audio corresponding to the notification content, and at the moment, when the second device comprises an application corresponding to the notification content, the user can open the notification content through the second device to realize that the second device answers the communication message, and because the second device answers the communication message, the follow-up video stream and the training video stream played by the second device are in a pause state. When the controller monitors that the communication state of the audio and video corresponding to the communication message of the second device is changed from the communication state to the communication ending state, the controller controls the display to continuously play the training video stream and the follow-up video stream.

Fig. 8A is a flow chart of another communication message processing method provided by an embodiment of the present disclosure, and fig. 8B is a signal flow chart of another communication message processing method provided by an embodiment of the present disclosure, where, based on the embodiment corresponding to fig. 2A, obtaining state information of a follow-up video stream displayed by a second device includes:

s201, when the fact that the prompting time length of the communication message received by the first device is greater than or equal to a preset threshold value is monitored, state information of the follow-up video stream displayed by the second device is obtained.

And when the prompting time length of the communication message is smaller than the preset threshold value, ensuring that the second device normally displays the follow-up video stream of the first device screen throwing, and judging that the communication message received by the first device is useless communication message without accessing the communication message into the second device.

The embodiment of the disclosure provides a display device, the display device includes:

a display;

a controller configured to:

monitoring communication information of the first equipment in the process that the first equipment throws a screen to the second equipment;

The method comprises the steps that communication information is monitored, state information of a follow-up video stream displayed by a second device is obtained, the state information comprises a follow-up state and a pause state, and the communication information represents communication information sent by a third device to a first device;

In the display device provided by the embodiment of the disclosure, in the process that the first device throws a screen to the second device, monitoring a communication message of the first device; the communication message is monitored, state information of the follow-up video stream displayed by the second equipment is obtained, and the state information comprises a follow-up state and a pause state; according to the state information of the follow-up video stream displayed by the second device, a control instruction is sent to the second device to control the second device to display notification content, the notification content displayed by a display corresponding to different control instructions is different, and when the state information of the follow-up video stream displayed by the second device is in a follow-up state, the communication information sent by the third device received by the first device is ensured not to influence the follow-up video stream and the training video stream displayed by the second device, and the use experience of a user is improved.

As an alternative embodiment, the controller is specifically configured to:

when the state information of the follow-up video stream displayed by the second equipment is in a pause state, a second control instruction is sent to the second equipment so as to control the second equipment to display a text corresponding to the second notification content and control the loudspeaker to play audio corresponding to the second notification content.

As an alternative, the device may, alternatively,

the controller is further configured to:

when the state information of the follow-up video stream displayed by the second device is the follow-up state, responding to the pause follow-up indication signal received by the second device, controlling the display to display a text corresponding to the notification content, and controlling the loudspeaker to play audio corresponding to the notification content.

The present disclosure also provides an electronic device, including: and a processor for executing a computer program stored in a memory, which when executed by the processor implements the steps of the method embodiments described above.

Fig. 9 is a schematic structural diagram of an electronic device provided in the present disclosure, and fig. 9 shows a block diagram of an exemplary electronic device suitable for implementing an embodiment of the present invention. The electronic device shown in fig. 9 is merely an example, and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.

As shown in fig. 9, the electronic device 800 is embodied in the form of a general purpose computing device. Components of electronic device 800 may include, but are not limited to: one or more processors 810, a system memory 820, and a bus 830 that connects the different system components (including the system memory 820 and the processors).

Bus 830 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of a variety of bus architectures. By way of example, and not limitation, such architectures include Industry Standard Architecture (ISA) bus, micro channel architecture (MAC) bus, enhanced ISA bus, video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus.

Electronic device 800 typically includes a variety of computer system readable media. Such media can be any medium that is accessible by electronic device 800 and includes both volatile and non-volatile media, removable and non-removable media.

The system memory 820 may include computer system readable media in the form of volatile memory, such as Random Access Memory (RAM) 840 and/or cache memory 850. Electronic device 800 may further include other removable/non-removable, volatile/nonvolatile computer system storage media. By way of example only, the storage system 860 may be used to read from and write to non-removable, non-volatile magnetic media (commonly referred to as a "hard disk drive"). Disk drives for reading from and writing to removable nonvolatile magnetic disks (e.g., a "floppy disk"), and optical disk drives for reading from and writing to removable nonvolatile optical disks (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In such cases, each drive may be coupled to bus 830 through one or more data medium interfaces. The system memory 820 may include at least one program product having a set (e.g., at least one) of program modules configured to carry out the functions of embodiments of the invention.

A program/utility 880 having a set (at least one) of program modules 870 may be stored, for example, in system memory 820, such program modules 870 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each or some combination of which may include an implementation of a network environment. Program modules 870 generally perform the functions and/or methodologies of the embodiments described herein.

The processor 810 executes various functional applications and information processing, such as implementing method embodiments provided by embodiments of the present invention, by running at least one program of a plurality of programs stored in the system memory 820.

The present disclosure also provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the above-described method embodiments.

Any combination of one or more computer readable media may be employed. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

The computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, either in baseband or as part of a carrier wave. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

Computer program code for carrying out operations of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider).

The present disclosure also provides a computer program product which, when run on a computer, causes the computer to perform the steps of implementing the method embodiments described above.

It should be noted that in this document, relational terms such as "first" and "second" and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

The foregoing is merely a specific embodiment of the disclosure to enable one skilled in the art to understand or practice the disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the disclosure. Thus, the present disclosure is not intended to be limited to the embodiments shown and described herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method of processing a communication message, comprising:

2. The method of claim 1, wherein the notification content comprises a first notification content and a second notification content, the first notification content comprising text and the second notification content comprising text and audio;

3. The method according to claim 2, wherein the method further comprises:

4. The method according to claim 2, wherein the method further comprises:

5. The method of claim 4, wherein after the sending the first indication instruction to the first device, further comprises:

6. The method according to claim 4, wherein the method further comprises:

7. The method of claim 1, wherein the obtaining the state information of the follow-up video stream displayed by the second device comprises:

8. A display device, the display device comprising:

a display;

a controller configured to:

9. The display device of claim 8, wherein the display device further comprises a display device,

the controller is specifically configured to:

10. The display device of claim 8, wherein the display device further comprises a display device,

the controller is further configured to: