CN112133300A

CN112133300A - Multi-device interaction method, related device and system

Info

Publication number: CN112133300A
Application number: CN201910555097.5A
Authority: CN
Inventors: 孙铭银; 吕远方; 饶凌河; 李智; 张锦建
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2019-06-25
Filing date: 2019-06-25
Publication date: 2020-12-25

Abstract

The invention provides a multi-device interaction method, related devices and a system. The main control device obtains the multimedia data stream and the corresponding control information from the server, and controls the playing device to play the multimedia data between the switching time point corresponding to the playing device and the next switching time point in the multimedia data stream. The control information is used for designating a plurality of time points of the multimedia data stream as switching time points and designating at least one corresponding playing device for each switching time point. The playing device comprises a plurality of controlled devices or at least one controlled device and a main control device. In the scheme, the multimedia data to be played by the playing device is directly obtained from the main control device, and is not required to be downloaded from a server through the Internet, so that the influence of the Internet transmission rate on the interaction process can be effectively avoided.

Description

Multi-device interaction method, related device and system

Technical Field

The present invention relates to the field of automatic control technologies, and in particular, to a multi-device interaction method, a related device, and a system.

Background

The smart sound box is a novel audio playing device, which can acquire audio from the internet or other electronic devices and play the audio, and can respond to sound signals sent by users or other devices. The current application mode of an intelligent sound box is that a plurality of intelligent sound boxes respectively acquire specific voice data from a server through a network according to a preset time sequence and play the acquired voice data, so that the voice interaction effect among the intelligent sound boxes is realized.

For example, after the smart speaker a acquires the voice "hello" from the server and finishes playing, as a response, the smart speaker B acquires the voice "hello" from the server and plays, and so on, the smart speaker a and the smart speaker B alternately acquire the voice data of the server and play, thereby realizing voice interaction between the two smart speakers.

In the existing interaction method, each smart speaker needs to download voice data from a server through a network and then can play the voice data. And the speed of network transmission is not stable, so in the interaction method, the response of the intelligent sound box is delayed because data cannot be downloaded in time, so that the interaction process is paused, and the voice interaction effect is reduced.

Disclosure of Invention

Based on the defects of the prior art, the invention provides a multi-device interaction method, related devices and a system, which are used for solving the problem that in the prior art, interaction among a plurality of intelligent sound boxes is not smooth.

The invention discloses a multi-device interaction method, which is applied to a main control device, wherein the main control device is connected with a controlled device; the multi-device interaction method comprises the following steps:

receiving a multimedia data stream sent by a server and control information corresponding to the multimedia data stream; the control information is used for designating a plurality of time points of the multimedia data stream as switching time points and designating at least one corresponding playing device for each switching time point; the playing device comprises a plurality of controlled devices, or at least one controlled device and the main control device;

controlling the playing device to play the multimedia data between the switching time point corresponding to the playing device and the next switching time point in the multimedia data stream; and if the switching time point corresponding to the playing device is the last switching time point of the multimedia data stream, the next switching time point is the end time point of the multimedia data stream.

Optionally, the controlling the playing device to play the multimedia data in the multimedia data stream between the switching time point corresponding to the playing device and the next switching time point includes:

for each playing device, determining multimedia data to be played of the playing device in the multimedia data stream; the multimedia data to be played is multimedia data between a switching time point corresponding to the playing device and the next switching time point;

and aiming at each playing device, sending the multimedia data to be played to the playing device, so that the playing device plays the multimedia data to be played.

and sending the multimedia data stream and the control information to each controlled device, so that the controlled device plays the multimedia data between the switching time point corresponding to the controlled device and the next switching time point when the multimedia data stream is played to the switching time point corresponding to the controlled device.

Optionally, before controlling the playing device to play the multimedia data in the multimedia data stream between the switching time point corresponding to the playing device and the next switching time point, the method further includes:

modifying the playing equipment appointed by at least one switching time point in the control information to obtain modified control information;

wherein the controlling the playing device to play the multimedia data in the multimedia data stream between the switching time point corresponding to the playing device and the next switching time point includes:

and controlling the playing device to play the multimedia data between the switching time point corresponding to the playing device and the next switching time point in the multimedia data stream by using the modified control information.

sending a request to the server; wherein the request is for requesting the server to modify a playback device specified by a switching time point in the multimedia data stream;

receiving modified control information sent by the server, wherein in the modified control information, a playing device specified by at least one switching time point is modified;

according to the role setting of the playing device, adjusting the multimedia data of the multimedia data stream between the switching time point corresponding to the playing device and the next switching time point to obtain the adjusted multimedia data;

and controlling the playing device to play the adjusted multimedia data between the switching time point corresponding to the playing device and the next switching time point in the multimedia data stream.

The second aspect of the present invention discloses a multi-device interaction method, applied to a server, including:

sending a multimedia data stream and control information corresponding to the multimedia data stream to a main control device; the control information is used for designating a plurality of time points of the multimedia data stream as switching time points and designating at least one corresponding playing device for each switching time point; the playing device comprises a controlled device, or comprises the controlled device and the main control device; the controlled equipment is connected with the main control equipment.

Optionally, before sending the multimedia data stream and the control information corresponding to the multimedia data stream to the main control device, the method further includes:

receiving a playing request sent by the master control equipment or the controlled equipment;

the sending of the multimedia data stream and the control information corresponding to the multimedia data stream to the main control device includes:

and sending the multimedia data stream corresponding to the playing request and the control information corresponding to the multimedia data stream to the main control equipment.

Optionally, the multimedia data stream is a voice conversation;

the playing request carries a conversation theme, the number of the playing devices and an identifier;

and generating the multimedia data stream corresponding to the playing request by the server according to the conversation theme carried by the playing request, the number of the playing devices and the identification.

Optionally, the interaction method further includes:

receiving a request sent by the master control equipment; wherein the request is for requesting the server to modify a playback device specified by a switching time point in the multimedia data stream;

and sending the modified control information to the main control equipment.

The third aspect of the present invention discloses an interaction method for multiple devices, which is characterized in that the method is applied to a playing device, wherein the playing device comprises a controlled device, or comprises the controlled device and a main control device; the master control equipment is connected with the controlled equipment; the multi-device interaction method comprises the following steps:

the playing device responds to the control of the main control device and plays the multimedia data between the switching time point corresponding to the playing device and the next switching time point in the multimedia data stream;

if the switching time point corresponding to the playing device is the last switching time point of the multimedia data stream, the next switching time point is the end time point of the multimedia data stream; the multimedia data stream is received by the main control device and corresponds to control information, and the control information is used for designating a plurality of time points of the multimedia data stream as switching time points and designating at least one corresponding playing device for each switching time point.

A fourth aspect of the present invention discloses a master control device, which is connected to a plurality of controlled devices, and includes:

the receiving unit is used for receiving a multimedia data stream sent by a server and control information corresponding to the multimedia data stream; the control information is used for designating a plurality of time points of the multimedia data stream as switching time points and designating at least one corresponding playing device for each switching time point; the playing device comprises each controlled device, or each controlled device and the main control device;

a control unit, configured to control the playback device to play back multimedia data in the multimedia data stream between a switching time point corresponding to the playback device and a next switching time point; and if the switching time point corresponding to the playing device is the last switching time point of the multimedia data stream, the next switching time point is the end time point of the multimedia data stream.

Optionally, the control unit is specifically configured to:

Optionally, the control unit is further configured to: modifying the playing equipment appointed by at least one switching time point in the control information to obtain modified control information;

when the control unit executes control of the playing device to play the multimedia data in the multimedia data stream between the switching time point corresponding to the playing device and the next switching time point, the control unit is specifically configured to: and controlling the playing device to play the multimedia data between the switching time point corresponding to the playing device and the next switching time point in the multimedia data stream by using the modified control information.

Optionally, the master control device further includes: a sending unit, configured to send a request to the server; wherein the request is for requesting the server to modify a playback device specified by a switching time point in the multimedia data stream;

the receiving unit is further configured to: receiving modified control information sent by the server, wherein in the modified control information, a playing device specified by at least one switching time point is modified;

Optionally, the control unit is further configured to: according to the role setting of the playing device, adjusting the multimedia data of the multimedia data stream between the switching time point corresponding to the playing device and the next switching time point to obtain the adjusted multimedia data;

when the control unit executes control of the playing device to play the multimedia data in the multimedia data stream between the switching time point corresponding to the playing device and the next switching time point, the control unit is specifically configured to: and controlling the playing device to play the adjusted multimedia data between the switching time point corresponding to the playing device and the next switching time point in the multimedia data stream.

A fifth aspect of the present invention discloses a server, comprising:

the device comprises a sending unit, a receiving unit and a processing unit, wherein the sending unit is used for sending a multimedia data stream and control information corresponding to the multimedia data stream to a main control device; the control information is used for designating a plurality of time points of the multimedia data stream as switching time points and designating at least one corresponding playing device for each switching time point; the playing device comprises a controlled device, or comprises the controlled device and the main control device; the controlled equipment is connected with the main control equipment.

Optionally, the server further includes:

a receiving unit, configured to receive a play request sent by the master control device or the controlled device;

wherein the sending unit is specifically configured to:

Optionally, the multimedia data stream is a voice conversation;

wherein the server further comprises:

and the generating unit is used for generating a multimedia data stream corresponding to the playing request and control information corresponding to the multimedia data stream according to the conversation theme carried by the playing request, the number of the playing devices and the identification.

Optionally, the receiving unit is further configured to receive a request sent by the master device, where the request is used to request the server to modify a playback device specified by a switching time point in the multimedia data stream;

the generating unit is further configured to modify the control information according to the request to obtain modified control information, where in the modified control information, a playback device specified by at least one switching time point is modified;

the sending unit is further configured to send the modified control information to the master control device.

A sixth aspect of the present invention discloses a playback device, where the playback device includes a controlled device, or includes the controlled device and a main control device; the master control device is connected with the controlled device, and the controlled device comprises:

the playing unit is used for responding to the control of the main control equipment and playing the multimedia data between the switching time point corresponding to the playing equipment and the next switching time point in the multimedia data stream;

A seventh aspect of the present invention discloses an electronic apparatus, comprising: a processor and a memory; wherein:

the memory is to store computer instructions;

the processor is configured to execute the computer instructions stored in the memory, and is configured to execute the multi-device interaction method disclosed in any one of the first aspects of the present invention, or is configured to execute the multi-device interaction method disclosed in any one of the second aspects of the present invention; or for performing the multi-device interaction method disclosed in the third aspect of the invention.

An eighth aspect of the present invention discloses a storage medium for storing a program for executing the multi-device interactive method disclosed in any one of the first aspects of the present invention or for executing the multi-device interactive method disclosed in any one of the second aspects of the present invention when the program is executed; or for performing the multi-device interaction method disclosed in the third aspect of the invention.

The ninth aspect of the present invention discloses a multi-device interactive system, comprising:

the server is used for sending a multimedia data stream and control information corresponding to the multimedia data stream to the main control equipment; the control information is used for designating a plurality of time points of the multimedia data stream as switching time points and designating at least one corresponding playing device for each switching time point; the playing device comprises a controlled device, or comprises the controlled device and the main control device; the controlled equipment is connected with the main control equipment;

the main control device is used for controlling the playing device to play the multimedia data between the switching time point corresponding to the playing device and the next switching time point in the multimedia data stream; if the switching time point corresponding to the playing device is the last switching time point of the multimedia data stream, the next switching time point is the end time point of the multimedia data stream;

and the playing device is used for responding to the control of the main control device and playing the multimedia data between the switching time point corresponding to the playing device and the next switching time point in the multimedia data stream.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.

FIG. 1 is a schematic view of a voice interaction scenario among a plurality of smart speakers;

FIG. 2 is a schematic diagram illustrating the structure and operation of a conventional interactive system;

fig. 3 is a signaling flowchart of an interaction method of multiple devices according to an embodiment of the present invention;

FIG. 4 is a diagram illustrating multimedia data streams and switching time points according to an embodiment of the present invention;

fig. 5 is a signaling flowchart of a multi-device interaction method according to another embodiment of the present invention;

fig. 6 is a flowchart of a method for controlling a playback device of a main control device according to an embodiment of the present invention;

fig. 7 is a signaling flowchart of a multi-device interaction method according to another embodiment of the present invention;

fig. 8 is a flowchart of a method for generating a multimedia data stream and control information by a server according to an embodiment of the present invention;

FIG. 9 is a diagram illustrating a client interface for obtaining modification opinions of a user on a dialog text according to an embodiment of the present invention;

fig. 10 is a schematic structural diagram of a master device according to an embodiment of the present invention;

fig. 11 is a schematic structural diagram of a playing device according to an embodiment of the present invention;

fig. 12 is a schematic structural diagram of a server according to an embodiment of the present invention;

fig. 13 is a schematic structural diagram of an interactive system with multiple devices according to an embodiment of the present invention;

fig. 14 is a schematic structural diagram of a multi-device interactive system according to another embodiment of the present invention;

fig. 15 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

With the development of internet technology and artificial intelligence technology, smart speakers are currently being used by more and more users. On one hand, the intelligent sound box can acquire and play audio from the Internet and other electronic equipment in a wireless transmission and wired transmission mode; alternatively, a sound signal may be received and, in response thereto, audio corresponding to the received sound signal may be played. The sound signal may be a user voice or an audio played by other electronic devices.

Based on the above functions, the intelligent sound box can be used for realizing human-computer interaction on the one hand. For example, the user says for the smart speaker: "how do the weather now? After receiving the user voice, the intelligent sound box sends the user voice to the server through the network, acquires audio generated by the server aiming at the user voice, and plays the acquired audio as response information of the user voice. For the aforementioned user voice, the audio played by the smart speaker may be the audio including the current weather information, for example, the smart speaker plays: "the weather is clear at present, the outdoor temperature is 32 ℃, and people can pay attention to sun protection when going out". By receiving the voice of the user and playing the corresponding audio, the intelligent sound box equivalently realizes the man-machine interaction with the user.

On the other hand, the smart sound box can also be used for realizing voice interaction among a plurality of smart sound boxes, namely multi-machine voice interaction. Referring to the application scenario shown in fig. 1, an example of a multi-machine voice interaction may be:

smart speaker A plays first audio "good morning! After receiving the first audio, the smart speaker B plays a second audio, "do you get good, do not have breakfast? "as a response to the first audio, after the second audio is played, as a response to the second audio, smart speaker a plays the third audio" eat, you are? ". In the process, the intelligent sound box A and the intelligent sound box B display the process of voice interaction between the two intelligent sound boxes through alternately playing specific audio, which is equivalent to realizing multi-machine voice interaction. The audio played by the two intelligent sound boxes is obtained from the server through the internet.

However, the effect of multi-machine voice interaction realized by the existing interactive system has a high requirement on the real-time internet download rate, and if the current internet download rate is low, the effect of multi-machine voice interaction of the existing interactive system is remarkably reduced, which affects the user experience.

Referring to fig. 2 in combination with fig. 1, in the existing interactive system, the smart speaker a needs to download the first audio from the server through the internet, and only after the download is completed, the smart speaker B sends an audio acquisition request to the server through the internet after receiving the first audio played by the smart speaker a, and then downloads the second audio from the server, and then plays the downloaded second audio. That is, after the smart speaker a finishes playing the first audio, it takes a certain time to download the second audio before the smart speaker B starts playing the second audio. If the second audio is downloaded by the smart sound box B, the internet download rate is low, and after the first audio is played, the smart sound box B needs to wait for a long time, for example, to wait for 2 seconds before playing the second audio.

This may cause multiple pauses of a long time to occur in the multi-machine voice interaction realized by the existing interactive system, so that the whole interactive process is not smooth, and the user experience is seriously reduced.

Based on the above shortcomings in the prior art, embodiments of the present application provide a multi-device interaction method to avoid the unsmooth interaction process caused by a low internet download rate.

It should be noted that the multi-device interaction method provided in any embodiment of the present application can be used to control interaction among multiple devices, and the interaction described in any embodiment of the present application is not limited to multi-device voice interaction among multiple smart speakers, and may further include multi-device video interaction among multiple playback devices and a combination of multi-device video interaction and multi-device voice interaction, which are implemented by controlling multiple playback devices to play videos acquired in advance at a specific time point.

The playing device may be an intelligent sound box, or other electronic devices having an audio playing function and/or a video playing function, for example, an intelligent robot having audio playing and video playing functions.

The multi-device interaction method provided by any embodiment of the present application may be applicable to two types of systems, where one type of system is composed of a server and multiple playback devices, any one of the multiple playback devices serves as a master control device, and the other playback devices except the master control device serve as controlled devices, and the master control device is configured to control the multiple playback devices to interact with each other, where the master control device may control all the playback devices including itself according to requirements of an interaction process. Specifically, which playing device is used as the main control device may be preset by the provider of the system or may be set by the user during the use of the system.

Another type of system includes a server, a plurality of controlled devices and a master device. Each controlled device is a playing device, and the master control device is another electronic device different from the playing device, for example, if an intelligent robot with a playing function is used as the controlled device, the master control device may be an intelligent base for placing the intelligent robots.

In the second system, the main control device is configured to control the multiple playing devices to interact. Of course, when the system includes three or more playing devices, only a part of the playing devices in the system may be controlled, or all the playing devices in the system may be controlled, according to the requirement of the interactive process.

In the above two types of systems, each controlled device is connected to at least the master control device, and on this basis, any controlled device may also be connected to other controlled devices, specifically which controlled devices are connected to each other, and may be configured according to an actual use situation, which is not limited herein.

The connection between two devices (including the connection between the master device and the controlled device, and the connection between two controlled devices) described in any embodiment of the present application may be a wired connection, that is, the data transmission interface of one device is connected with the data transmission interface of another device through a data line, or the data transmission interface of one device is directly inserted into the data transmission interface of another device; or a wireless connection, for example, two devices connected via bluetooth.

On the other hand, in the above two systems, the server may perform data transmission at least with the master control device, and the data transmission manner includes, but is not limited to, data transmission through the internet, that is, the master control device may send data to the server through the internet, or may download data from the server through the internet. Of course, the server may also communicate with the controlled device.

For convenience of understanding, the present embodiment mainly introduces the multi-device interaction method provided in the present application based on a simple system including a server, two controlled devices and a master device, where the two controlled devices are respectively denoted as a first controlled device and a second controlled device. Of course, the multi-device interaction method described in this embodiment can be directly generalized to a complex system including three or more controlled devices.

Referring to fig. 3, the multi-device interaction method provided in this embodiment includes the following steps:

s301, the server sends the multimedia data stream and the control information to the main control device.

The control information is control information corresponding to a multimedia data stream.

The sending action in step S301 may be initiated by the server actively, or initiated by the server after responding to the play request of the master control device or the controlled device. That is, the server may monitor the main control device in real time, and send the multimedia data stream and the control information to the main control device when the monitored condition is satisfied, where the specific condition may be that the main control device accesses the server through the internet, or that the current time is a certain preset time.

The server may also send the multimedia data stream and the control information to the main control device after receiving the play request to respond to the play request, where the play request may be sent by the main control device, or sent by the controlled device that establishes a data transmission relationship with the server, or sent to the main control device by the controlled device, and then forwarded to the server by the main control device.

A multimedia data stream, which may be an audio data stream, i.e. a piece of audio; or a video data stream, i.e. a piece of video. Specifically, the multimedia data stream may be a song, a speech dialog, a video episode, or a video, wherein the first two belong to the audio data stream and the second two belong to the video data stream. A video data stream may comprise a corresponding audio data stream and a plurality of consecutive images, or may comprise only a plurality of consecutive images and no audio data stream.

The control information includes a plurality of switching time points in the multimedia data stream and the corresponding relationship between the switching time points and the playing device. In this embodiment, only two controlled devices participate in the playing, and the master control device does not participate in the playing, so in this embodiment, the correspondence is a correspondence between the switching time point and the controlled device.

Any time during the playing process of the multimedia data stream is a time point of the multimedia data stream. For example, the instant a song starts playing, i.e., the start time of the first second of the song, is a time point, and the end time of the 10 th second of the song is also a time point. In any embodiment of the present application, for a multimedia data stream, if a time point of the multimedia data stream is included in the corresponding control information, the time point is a switching time point of the multimedia data stream.

Each switching time point corresponds to at least one playing device, and the corresponding relation between the switching time point and the playing device is recorded in the control information. When the multimedia data stream is played to any switching time point, the playing device corresponding to the switching time point starts playing the multimedia data stream from the switching time point, and meanwhile, the playing devices except the playing device corresponding to the switching time point stop playing the multimedia data stream. This process is equivalent to switching to the playing device corresponding to the switching time point to continue playing when the multimedia data stream is played to any switching time point.

As regards the relation between the multimedia data stream and the switching point in time, it can be further understood with reference to fig. 4. The horizontal straight line in fig. 4 represents a segment of the multimedia data stream, and the corresponding point on the straight line represents a time point of the multimedia data stream. After the server records a plurality of time points in the control information corresponding to the multimedia data stream, these recorded time points are referred to as switching time points of the multimedia data stream (it can also be considered that the server designates the plurality of time points of the multimedia data stream as switching time points in the form of control information), and are represented by short vertical lines in fig. 4. The server specifies the switching time points and also records the playing device corresponding to each switching time point in the control information (which is equivalent to specifying the corresponding playing device for each switching time point). When the multimedia data stream is played, the playing is started from the first switching time point, and in the playing process, the multimedia data between one switching time point and the next switching time point is only played by the playing device corresponding to the switching time point.

Taking fig. 4 as an example, the multimedia data between the first switching time point and the second switching time point is played by the playing device corresponding to the first switching time point, and after the playing is finished, the multimedia data between the second switching time point and the third switching time point is played by the playing device corresponding to the second switching time point, and so on.

In practical applications, the switching time point is generally expressed as a few seconds of a multimedia data stream. For example, if a switching time point is denoted as 12 seconds of the multimedia data stream, the switching time point represents the time point just after the first 11 seconds of the multimedia data stream. If one switching time point is 1 st second, it indicates the moment when the multimedia data stream starts, i.e., the starting time of the first second.

For ease of understanding, the present embodiment introduces the subsequent steps based on the following assumptions:

the multimedia data stream is an audio data stream having a total duration of 45 seconds, and the control information corresponding to the audio data stream can be recorded in the form shown in table 1.

Switching time point	Corresponding controlled equipment
		Second 1	First controlled device
Second 15	Second controlled device
		30 th second	First receiverControl device

TABLE 1

Further, as an example, it can be assumed that the audio data stream is a speech dialog, and the speech dialog includes three sentences, wherein the first sentence corresponds to the audio data of the 1 st second to the 15 th second of the audio data stream, the second sentence corresponds to the audio data of the 15 th second to the 30 th second of the audio data stream, and the third sentence corresponds to the audio data of the 30 th second to the 45 th second.

It should be noted that, in any embodiment of the present application, for any switching time point (tth second), if the switching time point is not the last switching time point of the multimedia data stream, the multimedia data from the switching time point to the next switching time point (for example, tth second to tth + nth second) refers to the multimedia data in the period from the start time of tth second to the end time of tth + n-1 second (or, the start time of tth + nth second); if the switching time point is the last switching time point of the multimedia data stream, the multimedia data from the switching time point to the next switching time point is the multimedia data in the period from the starting time of the switching time point (the starting time of the t-th second) to the ending time of the ending time point of the multimedia data stream (the ending time of the t + n-th second).

Thus, in the above example, the first sentence corresponds to audio data between the start time of the 1 st second and the start time of the 15 th second, the second sentence corresponds to audio data between the start time of the 15 th second and the start time of the 30 th second, and the third sentence corresponds to audio data between the start time of the 30 th second and the end time of the 45 th second.

A sentence corresponds to the audio data in a period of time, which is equivalent to the sentence recorded in the audio data in the period of time, and the playing device plays the audio data in the period of time, which is equivalent to the playing device speaking the sentence.

The ending time point may be specified by the control information, for example, the 60 th second of the audio data stream with the total duration of 70 seconds is specified as the ending time point, in this embodiment, the subsequent audio data starting from the 61 th second is not played again; instead of specifying the end time of the audio data stream itself by the control information, the latter scheme may be adopted in this embodiment, and the audio data stream is played from the start time of 30 seconds to the end of the audio data stream.

The first column of table 1 records switching time points of the multimedia data stream, and the second column records a controlled device corresponding to each switching time point. In other embodiments of the present application, one switching time point may also correspond to two or more playback devices.

Optionally, the control information according to any embodiment of the present application may be configured by the server according to the multimedia data stream to be played and the number of the playing devices. For the same multimedia data stream, the number of playing devices is different, and the control information configured by the server may be different, that is, the switching time point and the corresponding relationship between the switching time point and the playing device may be different.

For example, for an audio data stream with a total duration of 100 seconds, if there are only two playing devices, the server may set the 1 st second as a switching time point corresponding to one playing device, and set the 50 th second as a second switching time point corresponding to another playing device, so that the two playing devices respectively play half of the audio data stream.

If the audio data stream needs to be played by 4 playing devices, the server may set the 1 st second, the 25 th second, the 50 th second and the 75 th second as switching time points, respectively, and each switching time point corresponds to one playing device, respectively; of course, the server may also set 5 or more switching time points, the time interval between each switching time may be adjusted according to the actual situation, and may only be 1 second or 40 seconds apart, and the number of the playing devices corresponding to each switching time point is not limited, and may be one or multiple.

S302, the main control device controls the first controlled device to play the first audio.

The first audio refers to audio formed by audio data between 1 st second and 14 th second of the audio data stream, so step S302 corresponds to the main control device controlling the first controlled device to play audio data between a corresponding switching time point (i.e. 1 st second) and a next switching time point (as shown in table 1, the next switching time point of the 1 st second is 15 th second).

As assumed in the foregoing, the multimedia data stream in the present embodiment is a voice conversation, and therefore, the step S302 exhibits an effect that the first controlled device plays the first sentence of the voice conversation for a time of 14 seconds.

It should be noted that, in any embodiment of the present application, multimedia data between one switching time point (referred to as a switching time point a) and the next switching time point can only be switched to the playback device corresponding to the switching time point a, and during the period from the switching time point a to the next switching time point, the playback devices other than the playback device corresponding to the switching time point a do not play any multimedia data.

Optionally, in the time from the switching time point a to the next switching time point, the other playing devices except the playing device corresponding to the switching time point a may continue to read the multimedia data stream, but set the volume of itself to zero, or directly stop reading the multimedia data stream.

Therefore, in step S302, when the first controlled device plays the first audio, the second controlled device remains silent.

And S303, the main control device controls the second controlled device to play the second audio.

Similar to step S302, step S303 corresponds to the master device controlling the second controlled device to play the audio data (i.e., the second audio) of the 15 th to 30 th seconds in the audio data stream, and corresponds to the second controlled device playing the second words of the voice dialog in the period from the beginning time of the 15 th second to the beginning time of the 30 th second.

It should be noted that, in step S302, the first controlled device plays the audio data with the total duration of 14 seconds between the start time of the 1 st second and the start time of the 15 th second, and it can be considered that step S301 is an action with the duration of 14 seconds. And the second controlled device starts playing from the 15 th second starting time in step S303, that is, immediately after the first controlled device stops playing, the second controlled device starts playing, that is, step S303 is performed immediately after the duration of step S302 is over, and step S303 may also be regarded as a continuous action, and the duration is 15 seconds from the 15 th second starting time to the 30 th second starting time.

In the duration of step S303, only the second switching time point in table 1, i.e. the playback device corresponding to the 15 th second, i.e. the second controlled device, plays, while the first controlled device remains silent.

And S304, the main control device controls the first controlled device to play the third audio.

Similar to the previous steps, step S304 corresponds to an action lasting from the start time of the 30 th second to the end time of the 45 th second, i.e. to the end time of the audio data stream, and the total duration is 16 seconds.

During the duration of step S304, the second controlled device remains silent, and the first controlled device plays the third audio, i.e., the audio data of the audio data stream from the 30 th second to the 45 th second, which is equivalent to the first controlled device playing the third speech in the voice conversation.

Step S304 is executed, similarly to step S303, starting immediately after step S303 ends.

Through four steps, the present embodiment is equivalent to achieving such an effect:

first, the first controlled device speaks (i.e., plays) the first sentence in the voice dialog acquired by the main control device, the time consumption is 14 seconds, the second controlled device immediately speaks the second sentence of the voice dialog after the first controlled device speaks the first sentence (i.e., after the duration of 14 seconds is over), the time consumption is 15 seconds, which is equivalent to a response to the first sentence spoken before the first controlled device, after the second controlled device speaks the second sentence, the first controlled device immediately starts speaking the third sentence in the voice dialog as a response to the second sentence spoken by the second controlled device, and then the whole interactive process is over.

That is to say, in the method provided in this embodiment, the main control device controls the controlled device to play the corresponding audio data from the switching time point corresponding to the controlled device itself to the next switching time point according to the audio data stream and the corresponding control information acquired from the server, so that an effect of multi-machine voice interaction is achieved among multiple controlled devices.

Meanwhile, referring to the connection mode between the master control device and the controlled device described above and the implementation process of this embodiment, it can be found that: the audio data to be played by each controlled device can be acquired from the main control device by means of wired transmission or wireless transmission, and the audio data streams including all the audio data to be played are completely downloaded by the main control device before the controlled device starts playing from the first switching time point of the audio data streams. Therefore, in the implementation process of the multi-device interaction method provided by this embodiment, each controlled device does not need to download the multimedia data to be played from the server through the internet, but obtains the multimedia data from the main control device through wired transmission and wireless transmission including bluetooth technology, so that the influence of the internet transmission rate on the interaction process is effectively avoided, and even under the condition that the internet transmission rate is low, the method provided by this embodiment can ensure that the interaction process among the multiple devices is performed smoothly.

Optionally, the main control device mentioned in this application controls the playback device to play the multimedia data at the corresponding switching time point and the next switching time point, and the specific control modes may include the following two modes.

The first control mode is that the main control device sends the multimedia data between the first switching time point and the next switching time point of the multimedia data stream to the playing device corresponding to the first switching time point for playing, then the main control device monitors the playing progress of the multimedia data stream, after the playing of the multimedia data between the first switching time point and the next switching time point is finished, the main control device sends the multimedia data between the second switching time point (i.e. the next switching time point of the first switching time point) and the next switching time point to the corresponding playing device of the second switching time point for playing, and so on until the multimedia data stream is played to the end time point.

In short, the first control manner is equivalent to that the main control device respectively sends the multimedia data between each switching time point and the next switching time point in the multimedia data stream to the corresponding playing device in time sequence, the playing device plays the multimedia data once after receiving the multimedia data corresponding to the playing device, and each playing device which is not the main control device only receives the multimedia data corresponding to the playing device, but does not receive the whole multimedia data stream.

In the second control mode, the entire multimedia data stream and the control information corresponding to the multimedia data stream are transmitted to each playing device participating in the interactive process, the playing device determines the switching time point corresponding to itself in the multimedia data stream according to the control information, and then when the multimedia data stream is played to the switching time point corresponding to itself, the switching time point corresponding to itself is played to the next switching time point.

In the second way, each playback device receives the entire multimedia data stream and the corresponding control information. Compared with the first control mode, in the first control mode, the playing devices except the main control device only need to receive and play the multimedia data, specifically which multimedia data are played by which playing devices, and the main control device determines according to the control information that the playing devices except the main control device do not need to consider. In the second control manner, each playing device needs to determine which multimedia data in the multimedia data stream needs to be played by itself according to the control information, and determine the start time and the end time of the playing.

Another embodiment of the present application will be described below with reference to the aforementioned first control mode. Please refer to fig. 5 for the following example:

it should be noted that, in this embodiment, attention is mainly paid to how the main control device controls the playback device to perform playback, and therefore, a part of the server is omitted in fig. 5, but the multimedia data stream and the control information in this embodiment are still sent by the server, and the connection relationship between the server and the main control device is consistent with the foregoing embodiment. The situation of the next embodiment of the present application is similar and will not be described again.

The present embodiment mainly relates to interaction among three playback devices, and one of the playback devices serves as a master device, and the other two playback devices serve as controlled devices. The three playing devices are not recorded as playing device a, playing device B, and playing device C, where playing device a serves as the main control device.

In this embodiment, the specific implementation process is still described by taking the audio data stream as an example, the total time duration of the audio data stream to be played is 80 seconds, and the corresponding control information is shown in table 2.

Switching time point	Corresponding playing device
		Second 10	Player B
Second 28	Player A, player C
		48 th second	Player A
65 th second	Player C

TABLE 2

On the one hand, as shown in table 2, in the present embodiment, the first switching time point of the audio stream is not the start time of the audio stream, i.e. the 1 st second. In general, in the interaction method provided in this embodiment, the multimedia data stream to be played may be generated by the server according to the actual scene, so that the multimedia data stream exactly matches the scene of the current interaction.

However, in some scenarios, the server may only need to select a pre-generated multimedia data stream from the history database and specify a portion to play via the control information. In this case, the control information shown in table 2 may appear, that is, the server does not process the original multimedia data stream in the history database, but only specifies a certain time point after the start time of the multimedia data stream as the first switching time point through the control information. Based on the control information of table 2, the playing device B will start playing from the 10 th second of the multimedia data stream, and the first 9 seconds of the multimedia data stream will be ignored.

On the other hand, the control information of the present embodiment includes an end time point set to 75 seconds in addition to the switching time point shown in table 2. Referring to the description of the ending point in time in the previous embodiment, the playing device C will play the multimedia data between the beginning time of the 65 th second and the ending time of the 75 th second of the multimedia data stream, and the multimedia data beginning from the 76 th second will not be played in this embodiment. A certain time point before the ending time of the multimedia data stream is designated by the control information as an ending time point, and a part of a certain multimedia data stream in the historical database is also designated as the multimedia data stream required by the interaction under a specific scene, which is not described herein again.

S501, the playing device A sends the first audio of the audio data stream to the playing device B.

The first audio refers to audio data between 10 seconds to 28 seconds of the audio data stream.

In this embodiment, the playing device a participates in the interaction process, and simultaneously takes over the role of the main control device, and is configured to transmit the audio data between two adjacent switching time points, or the audio data between the last switching time point and the ending time point to the corresponding playing device.

The specific implementation process of step S501 may be that the playing device a extracts audio data between the 10 th second and the 28 th second of the audio data stream as the audio to be played, and then sends the audio to be played to the playing device B. The subsequent transmission steps are similar.

S502, the playing device B plays the first audio received by the playing device B.

In any embodiment of the present application, as described in step S501, data transmission between the master device (i.e., the playback device a) and the slave device (i.e., the playback device B) may be considered to be completed instantaneously due to the high transmission rate. The process of playing the multimedia data by the playing device in step S502 is a continuous step, and the duration is the duration of the played multimedia data.

S503, the playing device A sends the second audio of the audio data stream to the playing device C.

The second audio refers to audio data between the 28 th second to the 48 th second of the audio data stream.

As shown in table 2, the 28 th second corresponds to two playing devices, but the playing device a itself stores the whole audio data stream, so that only the second audio determined from the audio data stream needs to be sent to the playing device C, and in the subsequent playing step, the playing device a directly plays the second audio.

It should be noted that step S503 should be performed immediately after the end of the playback in step S502. The specific implementation method may be that, after the playing device B finishes playing the first audio, an end signal is sent to the playing device a, and the playing device a executes step S503 after receiving the end signal. After the playing device B finishes playing, the playing device B directly suspends playing without sending a signal to the playing device a, and after the playing device a finishes step S501, the playing device a counts time, and when the 18 th second is finished, it is considered that the playing device has finished playing the first audio, and then step S503 is executed. The subsequent sending step is performed similarly.

It should be further noted that, in the embodiment implemented based on the first control manner, the main control device sends the multimedia data to the playing devices other than the main control device, and actually includes an action of determining whether the playing of the multimedia data stream is finished, if it is determined that the playing of the multimedia data stream is finished, the subsequent steps are not executed, and if it is determined that the playing of the multimedia data stream is not finished, the multimedia data to be played is determined and the corresponding playing device is controlled to play.

The process of determining whether the multimedia data stream has been played may be determining whether the multimedia data that has been played in the previous step is the multimedia data between the last switching time point of the multimedia data stream and the end time point of the multimedia data stream, or determining whether the multimedia data played in the previous playing step ends at the end time point of the multimedia data stream, if so, it indicates that the multimedia data stream has been played, and if not, it indicates that the multimedia data stream has not been played.

S504, the playing device A and the playing device C play the second audio at the same time.

After the transmission in step S503 is completed, the playing device a directly plays the second audio determined from the audio data stream, and the playing device C plays the received second audio. Since the data transmission rate between the playback devices is high, it can be considered that the playback device C has successfully received the data after the playback device a completes transmission, and the two playback devices play the second audio at the same time.

And S505, the playing device A plays the third audio.

The third audio refers to audio data between the 48 th second and the 65 th second of the audio data stream.

As shown in table 2, the 48 th second switching time point only corresponds to the playing device a, so that the playing device a can directly play the third audio determined from the audio data stream. Of course, step S505 is executed after the playing process of step S504 is finished, and the method for determining that step S504 is finished is the same as the previously described determining method, and is not described again.

S506, the playing device A sends the fourth audio of the audio data stream to the playing device C.

The fourth audio refers to audio data between 65 seconds and 75 seconds of the audio data stream, wherein the fourth audio is audio data between a start time of the 65 seconds and an end time of the 75 seconds since the 75 seconds are end time points of the audio data stream.

And S507, the playing device C plays the fourth audio.

S508, the playing device A determines that the playing of the audio data stream is finished.

Optionally, after determining that the audio data stream is played, the main control device may send an end instruction to the playing devices other than the main control device.

In the multi-device interaction method provided by this embodiment, which specific multimedia data in the multimedia data stream is played by which playing devices are determined by the main control device according to the control information, and the playing devices other than the main control device only need to receive and play the multimedia data.

In order to better understand the foregoing first control manner, the following describes, with reference to fig. 6, an implementation process of the multi-device interaction method provided by the present application based on the first control manner, from the perspective of a master control device.

S601, receiving the multimedia data stream and the corresponding control information.

And S602, controlling the playing device corresponding to the current switching time point to play the multimedia data to be played.

The multimedia data to be played refers to multimedia data between the current switching time point and the next switching time point of the multimedia data stream. Wherein, the next switching time point refers to the next switching time point of the current switching time point; if the current switching time point is the last switching time point of the multimedia data stream, the next switching time point is the end time point of the multimedia data stream.

If the main control device is a control device and the main control device corresponds to the current switching time point, the controlling in step S602 includes the main control device directly playing the multimedia data.

If the current switching time point corresponds to a playing device other than the main control device, step S602 includes sending the multimedia data to the corresponding playing device, so that the corresponding playing device plays the multimedia data.

In any implementation process of the multi-device interaction method provided in this embodiment, when step S602 is executed for the first time, the current switching time point is the first switching time point of the multimedia data stream.

S603, judging whether the multimedia data to be played is played to be finished.

If the playing is finished, step S604 is executed, and if the playing is not finished, the step is continuously executed.

The content of the previous embodiment is referred to for the determination method, and details are not repeated here.

S604, judging whether the multimedia data stream is played completely.

If the playing of the multimedia data stream is finished, the method is finished, and if the playing of the multimedia data stream is not finished, step S605 is executed.

The determination method in this step still refers to the corresponding description of the previous embodiment, and is not described herein again.

S605, determining the next switching time point as the current switching time point, and returning to execute step S602.

The next switching time point here is the next switching time point mentioned in step S602. For example, for the first loop of the method, the current switching time point of step S601 is the first switching time point of the multimedia data stream, and the next switching time point is the second switching time point of the multimedia data stream. Correspondingly, step S605 determines the second switching time point of the multimedia data stream as the current switching time point, and then returns to step S602 to enter the second loop.

In the following, a further embodiment of the present application is described with reference to the aforementioned second control mode, please refer to fig. 7.

The embodiment still relates to three playing devices, and further comprises a device different from the playing device as a main control device. The multimedia data stream and the control information related to the present embodiment are identical to those of the embodiment shown in fig. 4.

S701, the main control device sends the audio data stream and the control information to the playing device A, the playing device B and the playing device C.

It should be noted that, because all three playing devices involved in this embodiment participate in the interaction process, in step S701, the main control device may directly send the multimedia data stream and the control information to the playing device a, the playing device B, and the playing device C.

Generally, for a situation where the main control device is connected to multiple playing devices and at least one playing device does not participate in the current interaction process, when the main control device performs step S701, it may determine all playing devices participating in the interaction process, that is, determine all playing devices recorded in the control information, and then send the multimedia data stream and the control information to these playing devices participating in the interaction.

For example, three playing devices a, B, and C are recorded in the control information shown in table 2, and if the main control device is connected to a playing device D in addition to the three playing devices, the main control device can determine, through the control information shown in table 2, that, of the four playing devices connected to the main control device, the playing devices a, B, and C are playing devices participating in the current interaction process, and the playing device D does not participate in the current interaction process.

S702, the playing device B plays the first audio of the audio data stream.

The first audio is audio data between 10 seconds and 28 seconds of the audio data stream.

S703, the playing device A and the playing device C play the second audio of the audio data stream.

The first audio is audio data between the 28 th and 48 th seconds of the audio data stream.

S704, the playing device A plays the third audio of the audio data stream.

The first audio is audio data between 48 seconds to 65 seconds of the audio data stream.

S705, the playing device C plays the fourth audio of the audio data stream.

The first audio is audio data between 65 seconds and 75 seconds of the audio data stream.

After the playing device C finishes playing the fourth audio, the playing of the entire audio data stream is finished, which is equivalent to the current interaction process.

In order to implement the second control manner, the operating principle of each playing device may be:

after receiving the multimedia data stream and the control information, each playing device immediately judges whether the playing device itself corresponds to a first switching time point of the multimedia data stream, if so, the playing is immediately started from the first switching time point, the judgment process consumes a short time, and after the main control device finishes sending the multimedia data stream and the control information, the playing device corresponding to the first switching time point starts playing the corresponding multimedia data.

Then, each playing device determines which switching time point in the multimedia data stream corresponding to itself, and takes the time when the multimedia data stream and the control information are received as the starting time to time, and after the time is counted to any switching time point corresponding to itself, the multimedia data between the switching time point and the next switching time point of the switching time point is played, or if the switching time point is the last switching time point, the audio data between the switching time point and the ending time point is played. Of course, if the next switching time point is still the switching time point corresponding to itself, the playing step is continued.

In the embodiment shown in fig. 7, after the multimedia data stream and the control information are sent, the control device B immediately determines that the first switching time point is the corresponding switching time point, so step S702 is executed. Subsequently, the playing device a determines that the self corresponds to two switching time points, namely, the 28 th second and the 48 th second, and the playing device C determines that the self corresponds to two switching time points, namely, the playing device a and the playing device C start to time, and when the time is counted to the 28 th second, the playing device a and the playing device C execute step S703, when the time is played to the 48 th second, the playing device C stops playing and times, the playing device a executes step S704, continues playing to the 65 th second, then stops playing, and at the same time, the playing device C times to the 65 th second, executes step S705, and plays the audio data of the 65 th second to the 75 th second.

It should be noted that, during the execution of the playing step of the interaction method provided in any embodiment of the present application, other playing devices except the playing device indicated in the step remain silent.

Optionally, in the multi-device interaction method disclosed in this embodiment of the present application, before the main control device controls the playing device to play the multimedia data between the corresponding switching time point of the main control device and the next switching time point in the multimedia data stream, the main control device may further modify the control information sent by the server to obtain the modified control information, and then control the playing device to play the multimedia data in the multimedia data stream by using the modified control information.

The master control device can modify the control information sent by the server according to the number of the playing devices currently controlled by the master control device. Specifically, after receiving the control information sent by the server, the main control device determines whether the playback device specified by the control information matches with the playback device capable of working normally controlled by the main control device, and if the playback device specified by the control information is more than the playback device capable of working normally controlled by the main control device, or the playback device specified by the control information is less than the playback device capable of working normally controlled by the main control device, the main control device performs the modification.

In addition, the main control device may also modify the control information when the playback device specified by the control information matches a playback device that can normally operate and is controlled by the main control device.

For example, the multimedia data played by a certain playback device assigned by the control information does not match the role of the playback device, in which case the main control device may also modify the control information.

It should be further noted that the modification of the control information by the master control device may be performed in the following two ways.

One way is that the master device sends a request to the server requesting the server to modify the playback device specified in the control information for the at least one switching time point in the multimedia data stream. And after receiving the request, the server modifies the control information according to the request to obtain modified control information, and sends the modified control information to the main control equipment.

Optionally, in this manner, the main control device may carry in the request: the switching time point in the multimedia data stream and the modified corresponding relation of the playing device. Therefore, after the server receives the request, the control information can be modified according to the modified corresponding relation carried in the request. Of course, the main control device may also carry a modification requirement in the request, and after the server analyzes the requirement, the switching time point that needs to be modified in the multimedia data stream and the specified modified playing device are determined, and the control information is modified according to the switching time point, so as to meet the modification requirement carried in the request.

In another mode, the main control device modifies the playback device specified by at least one switching time point in the control information to obtain the modified control information.

Specifically, the main control device defines a switching time point of the playing device to be adjusted in the multimedia data flow, and defines a playing device specified after modification of the switching time point of the playing device to be adjusted; the master device modifies the control information in this way, so that the playback device specified by the switching time point in the control information is modified to a new playback device.

For example, the control information specifies three switching time points in the multimedia data stream, a first switching time point corresponds to the playing device a and the playing device B, a second switching time point corresponds to the playing device C, and a third switching time point corresponds to the playing device D; for this control information, the master control device may modify the control information by changing the playback device corresponding to the first switching time point into playback device B, changing the playback device corresponding to the second switching time point into playback device a, and changing the playback device corresponding to the third switching time point into playback device C and playback device D.

It should be further noted that, in the manner of modifying the control information by the main control device, in addition to modifying the playback device specified by at least one switching time point in the control information, the time point in the multimedia data stream specified by the control information as the switching time point may be added or deleted, and the position of the switching time point may be adjusted.

For example, the control information before modification specifies four switching time points in the multimedia data stream, which correspond to four playing devices, and are sequentially recorded as a playing device a, a playing device B, a playing device C, and a playing device D, where the four switching time points divide the multimedia data stream into four segments of multimedia data, and the main control device may add a switching time point between a first segment of multimedia data, that is, between the first switching time point and a second switching time point, so that the modified control information includes five switching time points. Certainly, after the master device adds the new switching time point, it is also necessary to designate a playing device for the new switching time point, and which designated playing devices can be determined by the master device, on one hand, on the basis that the original corresponding relationship is not changed, one or more of the original four playing devices can be designated to the new switching time point, for example, the designated new switching time point corresponds to the playing device C, so that in the modified control information, the playing device C corresponds to the second switching time point (i.e., the new switching time point) and the fourth switching time point (i.e., the third switching time point of the control information before modification), on the other hand, the new playing device can also be designated for the new switching time point, for example, the playing device E is designated to the new switching time point.

Of course, the master control device may also delete one of the original four switching time points, and reassign the playing device corresponding to the switching time point to another switching time point, for example, the master control device may delete the second switching time point, and then assign the playing device B to the first switching time point, which is equivalent to that the modified control information includes three switching time points, the first switching time point corresponds to the playing device a and the playing device B, and the second switching time point and the third switching time point correspond to the playing device C and the playing device D, respectively.

For example, the control information before modification includes three switching time points, where the second switching time point is the 20 th second of the multimedia data stream, and the master control device may change the second switching time point to the 30 th second of the multimedia data stream to obtain the modified control information.

It should be understood by those skilled in the art that although the foregoing embodiments are described by taking an audio data stream as an example and only two or three playback devices are involved, the above method can be extended to an interactive method based on a video data stream and involving four or more playback devices without any creative effort according to the foregoing embodiments, and thus these methods are also within the scope of the present application.

It should be noted that, in the multi-device interaction method provided in any embodiment of the present application, the multimedia data stream and the control information sent by the server may be sent after the play request is sent, the play request may be sent by the master control device or the controlled device, and meanwhile, the user may set the play request through the client and the master control device, so that the server may generate the multimedia data stream corresponding to the play request and the corresponding control information according to the related information carried in the play request.

Referring to fig. 8, the method for generating the multimedia data stream and the control information according to the play request by the server includes:

s801, the server receives the playing request.

The playing request carries the number and identification of the playing devices and the conversation theme set by the user. For example, the user may set the conversation topic to "music".

S802, the server determines a plurality of sentences related to the topic from a preset conversation database according to the conversation topic.

For example, for the aforementioned conversation topic "music", the statements related to that topic may be: "do you like to listen to a popular song", "I like singer xxx", "I feel like the song" yyyy "to hear very well", etc.

And S803, the server determines the playing equipment corresponding to each sentence according to the number and the identification of the playing equipment to obtain the dialog text.

For example, in the case where there are two playing devices, the two playing devices may be configured to alternately play the sentence determined in step S802.

S804, the server sends the conversation text to the client to obtain a modification request fed back by the client.

After the dialog text is sent to the client, the user refers to the dialog text at the client and proposes a modification suggestion. And then the client feeds back a modification request carrying the modification opinions of the user to the server. When the user does not modify the opinion, the modification request may carry confirmation information.

The client displays the dialog text to the user and obtains an interface for modifying the opinions, as can be seen in fig. 9.

S805, the server determines, for each sentence of the confirmed dialog text, a playing time length of the sentence.

And when the modification request fed back by the client carries the modification opinions, the confirmed dialog text is the dialog text modified according to the modification opinions. When the modification request carries confirmation information, the dialog text obtained in step S803 is directly used as the dialog text after confirmation.

For step S805, it can be set that each word is played in a specific playing time length, for example, each word is played for 0.5S, and then the playing time length of each sentence is the number of words of the sentence multiplied by the playing time length of each word.

On the other hand, a spacer with a certain duration may be inserted between two words in the sentence, so that when the sentence is played to the spacer, the sentence is paused for the corresponding duration, and then the sentence is continuously played. The play duration of the sentence is then added to the total duration of the spacer on the basis of the number of words of the previous aspect multiplied by the play duration of a single word.

S806, the server generates voice dialogue and control information according to the confirmed dialogue text and the playing duration of each sentence.

The voice dialog is a multimedia data stream generated by the server.

Optionally, the correspondence between the sentences and the playing devices may be determined according to the roles played by the playing devices. On the other hand, the sentence determined in step S802 may also be adjusted according to the role played by the corresponding playback device, so as to conform to the role characteristics of the corresponding playback device. For example, based on the sentence determined in step S802, some semantic words may be added according to the role characteristics of the corresponding playback device, and partial vocabulary of the original sentence may be replaced.

Furthermore, the determined sentences can be translated into other languages or different dialects according to the set roles, for example, the determined Chinese sentences can be translated into English, French and the like.

Optionally, the role played by each playing device may be specified by the user, or may be specified by the server after the dialog theme set by the user is acquired.

Optionally, in the above adjusting the sentences, the first aspect may be that after the sentences are determined in step S802, the server adjusts the sentences, and then combines the adjusted sentences into a voice conversation.

The second aspect may be that, after the main control device receives the voice conversation sent by the server, the main control device adjusts the statements in the voice conversation according to the role setting of the playing device controlled by the main control device, so as to obtain the adjusted voice conversation. The main control device adjusts the statement in the voice conversation according to the role setting of the playing device, and mainly adjusts the voice conversation between the switching time point corresponding to the playing device and the next switching time point in the voice conversation.

The third aspect may be that, after receiving the voice conversation sent by the server, the main control device sends a modification request to the server, so that the server adjusts the statements in the voice conversation according to the role of the playback device to obtain an adjusted voice conversation, and the server sends the adjusted voice conversation to the main control device.

Further, the main control device may also perform the foregoing adjustment in real time during the playing process, for example, when one playing device plays a first sentence of a voice conversation, the main control device may adjust a subsequent second sentence, a subsequent third sentence, and the like to obtain an adjusted second sentence or an adjusted third sentence, and then after the playing of the first sentence is finished, control the corresponding playing device to play the adjusted second sentence or control the corresponding playing device to play the adjusted third sentence.

The method provided by this embodiment is mainly used for acquiring a plurality of sentences related to a conversation topic from a preset conversation database according to the conversation topic set in a play request by a user, the number of playing devices, identifiers and other scene information, then determining the play duration of each sentence and the corresponding playing device, combining the sentences to obtain an audio data stream, determining control information according to the play duration of each sentence and the corresponding playing device, and then sending the audio data stream and the control information to the main control device.

Further, the server can send the generated audio data stream and the corresponding dialog text to the client, the user confirms whether to adjust the audio data stream and how to adjust the audio data stream, and then the server sends the adjusted audio data stream and the control information to the main control equipment.

Based on the above technical solution, in this embodiment, multiple types of voice dialogs that can serve as multimedia data streams can be generated according to the needs of the user, so that the main control device controls multiple playing devices to respectively play corresponding sentences according to the voice dialogs and corresponding control information, thereby enabling the user to experience a personalized interaction process of multiple devices.

In combination with the multi-device interaction method provided in any embodiment of the present application, an embodiment of the present application further provides a server, a master control device, and a playback device. With reference to fig. 10, the master device provided in the embodiment of the present application may include the following structure:

a receiving unit 1001 is configured to receive a multimedia data stream and control information corresponding to the multimedia data stream sent by a server.

Wherein the control information includes: the multimedia data stream switching method comprises a plurality of switching time points of the multimedia data stream and playing equipment corresponding to each switching time point, wherein each switching time point corresponds to at least one playing equipment.

Alternatively, the playback device may include each controlled device.

Optionally, the playback device may include each controlled device, and the master device.

If the main control equipment is also a playing equipment, the main control equipment is connected with at least one controlled equipment; and if the main control equipment is not one playing equipment, the main control equipment is connected with at least two controlled equipment.

The connection mode of the main control device and the controlled device can be wired connection or wireless connection.

A control unit 1002, configured to control each playing device to play multimedia data in the multimedia data stream between a switching time point corresponding to the playing device and a next switching time point.

And if the switching time point corresponding to one playing device is the last switching time point of the multimedia data stream, the next switching time point refers to the ending time point of the multimedia data stream.

With reference to the multi-device interaction method provided in the foregoing embodiment, the control unit 802 may implement control over the playback device in two ways, and according to different specific control ways, the control unit of the main control device provided in this embodiment may have the following two structures:

based on a control manner, the control unit 1002 may include:

the determining unit is configured to determine a first switching time point of a multimedia data stream to be currently played as a current time point, and further determine multimedia data between the current switching time point and a next switching time point as multimedia data to be played.

And the sending unit is used for sending the multimedia data to be played determined by the determining unit to the playing device corresponding to the current switching time point, so that the playing device corresponding to the current switching time point plays the multimedia data to be played.

The judging unit is used for judging whether the multimedia data to be played is played completely after the sending unit sends the multimedia data to be played, further judging whether the multimedia data stream to be played currently is played completely after the multimedia data to be played is judged to be played completely, if the multimedia data stream is judged to be played completely, the current interaction process is finished, if the multimedia data stream is judged not to be played completely, the determining unit is triggered to determine the next switching time point as the current switching time point, and the multimedia data to be played is determined based on the determined current switching time point.

Further optionally, when the main control device is a playing device, the control unit 802 may further include a playing unit, configured to play the multimedia data between the current switching time point and the next switching time point when the current switching time point determined by the determining unit corresponds to the main control device itself.

Based on another control manner, the control unit 1001 may include:

and the determining unit is used for reading the control information and determining the playing equipment recorded in the control information as the playing equipment participating in the interactive process.

And the sending unit is used for sending the multimedia data stream and the corresponding control information to all the playing devices participating in the current interactive process, so that the playing devices control the playing devices to play the multimedia data between the corresponding switching time point and the next switching time point of the corresponding switching time point according to the multimedia data stream and the control information.

Optionally, when the main control device itself is also a playing device participating in the current interaction process, the sending unit is configured to send the multimedia data stream and the corresponding control information to the playing devices participating in the current interaction process except the main control device itself.

Referring to fig. 11, a playing device provided in an embodiment of the present application may include the following structure:

the receiving unit 1101 is configured to receive multimedia data sent by a master control device, or receive a multimedia data stream and corresponding control information sent by the master control device.

The playing unit 1102 is configured to respond to control of the main control device, and play the multimedia data in the multimedia data stream between a switching time point corresponding to the playing device itself and a next switching time point.

The main control equipment establishes communication connection with the playing equipment in a wireless connection or wired connection mode

And if the switching time point corresponding to the playing device is the last switching time point of the multimedia data stream, the next switching time point is the end time point of the multimedia data stream.

The multimedia data stream is received by the main control device and corresponds to control information, and the control information comprises: the multimedia data stream switching method comprises a plurality of switching time points of the multimedia data stream and playing equipment corresponding to each switching time point, wherein each switching time point corresponds to at least one playing equipment.

Optionally, a playback device including the structure of the main control device described in the previous embodiment may be used as the main control device.

Optionally, corresponding to the structure of the main control device under the second control manner mentioned in the previous embodiment, the playback device may include the following units:

a determining unit 1103, configured to determine, from the control information, a switching time point in the multimedia data stream corresponding to the playback device itself.

The timing unit 1104 is configured to start timing from receiving the multimedia data stream and the corresponding control information, and trigger the playing unit to play the multimedia data between the switching time point and a next switching time point of the switching time point when the timing reaches a switching time point corresponding to any playing device.

Referring to fig. 12, a server provided in an embodiment of the present application may include the following structure:

a sending unit 1201, configured to send a multimedia data stream and control information corresponding to the multimedia data stream to a master control device.

Optionally, the server may further include:

a receiving unit 1202, configured to receive a play request sent by a master device or a controlled device.

The play request may carry the number and identification of the playing devices, and the topic of the conversation, which may be determined by the user.

A generating unit 1203, configured to generate a corresponding voice dialog and control information according to a play request carrying the number and the identifier of the playing devices and the dialog theme, where the voice dialog is sent as a multimedia data stream by the sending unit 1201.

Further optionally, the generating unit 1203 may generate a corresponding dialog text according to the play request, then trigger the sending unit 1201 to send the dialog text to the client, then receive a modification request of the client through the receiving unit 1202, thereby generating a confirmed dialog text according to the modification request and the dialog text, and finally generate a voice dialog and corresponding control information according to the confirmed dialog text.

For the main control device, the playing device, and the server provided in any embodiment of the present application, specific working principles thereof may refer to corresponding steps in the multi-device interaction method provided in any embodiment of the present application, and are not described herein again.

According to the main control device, the playing device and the server, after the main control device receives the multimedia data stream and the corresponding control information sent by the server by using the receiving unit 1001, the control unit 1002 controls the playing device according to the control information, and the playing unit 1102 of the playing device responds to the control of the main control device and plays the multimedia data between the switching time point corresponding to the playing device and the next switching time point in the multimedia data stream. Because the playing device can directly acquire the multimedia data to be played from the main control device without downloading the multimedia data from the server through the internet, the interaction process of multiple devices realized by playing the multimedia data is not influenced by the data transmission rate of the internet, even if the data transmission rate of the internet is low, the device can also realize the smooth interaction process of the multiple devices among the multiple playing devices, and the user experience is effectively improved.

With reference to fig. 13, the interactive system includes:

the system comprises a main control device, a server and at least two controlled devices, wherein each controlled device is a playing device.

The server is used for sending the multimedia data stream and corresponding control information to the main control equipment through the Internet, and the control information comprises: the multimedia data stream switching method comprises a plurality of switching time points of the multimedia data stream and playing equipment corresponding to each switching time point, wherein each switching time point corresponds to at least one playing equipment.

The master control device is connected with each controlled device in a wireless connection or a wired connection mode. The main control device is used for controlling the playing device to play the multimedia data between the switching time point corresponding to the playing device and the next switching time point in the multimedia data stream after receiving the multimedia data stream sent by the server.

The playing device is used for responding to the control of the main control device and playing the multimedia data between the switching time point corresponding to the playing device and the next switching time point in the multimedia data stream.

Another embodiment of the present application further provides a multi-device interactive system, please refer to fig. 14, where the system includes:

a plurality of playback devices and a server, any one of the plurality of playback devices serving as a master control device performs the functions of the master control device in the system provided in the previous embodiment.

The functions of the playing device and the server in this embodiment are the same as those of the playing device and the server in the previous embodiment, and are not described again.

The main control device in this embodiment may control all the playing devices in the system including the main control device to play the corresponding multimedia data.

It should be noted that, for convenience of understanding, fig. 13 and fig. 14 only show two simplest embodiments of the interactive system provided in the present application, and there are only two playing devices in the interactive system of these two embodiments. However, the interactive system obtained by increasing the number of playing devices on the basis of fig. 13 and fig. 14 is only another embodiment of the multi-device interactive system provided by the present application, and therefore is also within the protection scope of the present application.

The specific working principles of the server and each device in the multi-device interaction system provided in the embodiments of the present application may refer to corresponding steps of the multi-device interaction method provided in any embodiment of the present application, and are not described herein again.

In the multi-device interactive system provided in the embodiment of the present application, after receiving a multimedia data stream and corresponding control information sent by a server, a main control device controls a playback device to play multimedia data between a switching time point corresponding to the playback device and a next switching time point in the multimedia data stream according to the control information. Because the playing device can directly acquire the multimedia data to be played from the main control device without downloading the multimedia data from the server through the internet, the interaction process of multiple devices realized by playing the multimedia data is not influenced by the data transmission rate of the internet, even if the data transmission rate of the internet is low, the device can also realize the smooth interaction process of the multiple devices among the multiple playing devices, and the user experience is effectively improved.

Another embodiment of the present application further provides an electronic device, where the structure of the electronic device refers to fig. 15, and the electronic device includes: a memory 1501 and a processor 1502, wherein:

memory 1501 is used to store computer instructions;

the processor 1502 is configured to execute the computer instructions stored in the memory 1501, and if the electronic device is used as a master device, the computer instructions are executed to perform the relevant steps of the master device in the multi-device interaction method provided in any embodiment of the present application.

If the electronic device is used as a server, the computer instructions are executed to perform steps related to the server in the multi-device interaction method provided by any embodiment of the present application.

If the electronic device is used as a playing device, the computer instructions are executed to perform relevant steps of the playing device in the multi-device interaction method provided by any embodiment of the present application.

An eighth aspect of the present invention discloses a storage medium for storing a program.

If the storage medium is a storage medium of the main control device, the program is used to execute relevant steps of the main control device in the multi-device interaction method provided in any embodiment of the present application when executed.

If the storage medium is a storage medium of a server, the program is executed to perform the steps related to the server in the multi-device interaction method provided in any embodiment of the present application.

If the storage medium is a storage medium of a playing device, the program is used to execute relevant steps of the playing device in the multi-device interaction method provided by any embodiment of the present application when executed.

Those skilled in the art can make or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. The multi-device interaction method is applied to a main control device, wherein the main control device is connected with a controlled device; the multi-device interaction method comprises the following steps:

receiving a multimedia data stream sent by a server and control information corresponding to the multimedia data stream; wherein the control information is configured to designate a plurality of time points of the multimedia data stream as switching time points, and to designate at least one corresponding playback device for each of the switching time points: the playing device comprises a plurality of controlled devices, or at least one controlled device and the main control device;

2. The interactive method according to claim 1, wherein said controlling the playing device to play the multimedia data in the multimedia data stream between the switching time point corresponding to the playing device and the next switching time point comprises:

3. The interactive method according to claim 1, wherein said controlling the playing device to play the multimedia data in the multimedia data stream between the switching time point corresponding to the playing device and the next switching time point comprises:

4. The interactive method according to claim 1, wherein before controlling the playing device to play the multimedia data in the multimedia data stream between the switching time point corresponding to the playing device and the next switching time point, the method further comprises:

5. The interactive method according to claim 1, wherein before controlling the playing device to play the multimedia data in the multimedia data stream between the switching time point corresponding to the playing device and the next switching time point, the method further comprises:

6. The interactive method according to claim 1, wherein before controlling the playing device to play the multimedia data in the multimedia data stream between the switching time point corresponding to the playing device and the next switching time point, the method further comprises:

7. A master device, wherein the master device is connected to a plurality of controlled devices, the master device comprising:

8. A server, comprising:

the device comprises a sending unit, a receiving unit and a processing unit, wherein the sending unit is used for sending a multimedia data stream and control information corresponding to the multimedia data stream to a main control device; wherein the control information is configured to designate a plurality of time points of the multimedia data stream as switching time points, and to designate at least one corresponding playback device for each of the switching time points: the playing device comprises a plurality of the controlled devices, or comprises at least one of the controlled device and the main control device.

9. The server of claim 8, further comprising:

a generating unit, configured to generate, according to the play request, a multimedia data stream corresponding to the play request and control information corresponding to the multimedia data stream;

wherein the sending unit is specifically configured to: and sending the multimedia data stream corresponding to the playing request and the control information corresponding to the multimedia data stream to the main control equipment.

10. The server according to claim 9, wherein the playback request carries a conversation topic, the number of playback devices, and an identifier of the playback device;

the generating unit is specifically configured to: and generating a multimedia data stream corresponding to the playing request and control information corresponding to the multimedia data stream according to the conversation theme carried by the playing request, the number of the playing devices and the identification of the playing devices.

11. The server according to claim 8, wherein the receiving unit is further configured to receive a request sent by the master device, wherein the request is used to request the server to modify a playback device specified by a switching time point in the multimedia data stream;

12. A multi-device interactive system, comprising: