CN112133300B

CN112133300B - Multi-device interaction method, related device and system

Info

Publication number: CN112133300B
Application number: CN201910555097.5A
Authority: CN
Inventors: 孙铭银; 吕远方; 饶凌河; 李智; 张锦建
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2019-06-25
Filing date: 2019-06-25
Publication date: 2024-05-07
Anticipated expiration: 2039-06-25
Also published as: CN112133300A

Abstract

The invention provides a multi-device interaction method, related devices and a system. The master control equipment acquires the multimedia data stream and the corresponding control information from the server, and controls the playing equipment to play the multimedia data between the corresponding switching time point and the next switching time point of the playing equipment in the multimedia data stream. The control information is used for designating a plurality of time points of the multimedia data stream as switching time points, and designating at least one corresponding playing device for each switching time point. The playing device comprises a plurality of controlled devices or at least one controlled device and a master control device. In the scheme, the multimedia data to be played by the playing device are directly obtained from the main control device and are not required to be downloaded from the server through the Internet, so that the influence of the Internet transmission rate on the interaction process can be effectively avoided.

Description

Multi-device interaction method, related device and system

Technical Field

The present invention relates to the field of automatic control technologies, and in particular, to a multi-device interaction method, related devices, and a system.

Background

The intelligent sound box is a novel audio playing device, which can acquire and play audio from the internet or other electronic devices on one hand and can respond to sound signals sent by users or other devices on the other hand. The current application mode of the intelligent sound box is that a plurality of intelligent sound boxes acquire specific voice data from a server through a network respectively according to a preset time sequence and play the acquired voice data, so that the effect of voice interaction among the plurality of intelligent sound boxes is achieved.

For example, after the intelligent sound box a acquires the voice "hello" from the server and plays the voice, the intelligent sound box B acquires the voice "hello" from the server and plays the voice, and so on, the intelligent sound box a and the intelligent sound box B alternately acquire the voice data of the server and play the voice data, thereby realizing the voice interaction between the two intelligent sound boxes.

In the existing interaction method, each intelligent sound box needs to download voice data from a server through a network and then can play the voice data. In the interactive method, the response of the intelligent sound box may be delayed because the data cannot be downloaded in time, so that the interactive process is stopped, and the voice interactive effect is reduced.

Disclosure of Invention

Based on the defects of the prior art, the invention provides a multi-device interaction method, related devices and a system, which are used for solving the problem that interaction among a plurality of intelligent sound boxes is not smooth in the prior art.

The invention discloses an interaction method of multiple devices, which is applied to a master control device, wherein the master control device is connected with a controlled device; the multi-device interaction method comprises the following steps:

Receiving a multimedia data stream and control information corresponding to the multimedia data stream sent by a server; the control information is used for designating a plurality of time points of the multimedia data stream as switching time points, and designating at least one corresponding playing device for each switching time point; the playing device comprises a plurality of controlled devices or at least one controlled device and the master control device;

controlling the playing device to play the multimedia data in the multimedia data stream between the corresponding switching time point and the next switching time point of the playing device; the next switching time point is the next switching time point of the switching time point corresponding to the playing device, and if the switching time point corresponding to the playing device is the last switching time point of the multimedia data stream, the next switching time point is the ending time point of the multimedia data stream.

Optionally, the controlling the playing device to play the multimedia data in the multimedia data stream between the corresponding switching time point and the next switching time point of the playing device includes:

Determining multimedia data to be played of each playing device in the multimedia data stream; wherein, the multimedia data to be played is the multimedia data between the corresponding switching time point and the next switching time point of the playing device;

And sending the multimedia data to be played to each playing device, so that the playing device plays the multimedia data to be played.

And sending the multimedia data stream and the control information to each controlled device, so that the controlled device plays the multimedia data from the corresponding switching time point to the next switching time point when the multimedia data stream is played to the corresponding switching time point.

Optionally, before the controlling the playing device to play the multimedia data in the multimedia data stream between the corresponding switching time point and the next switching time point of the playing device, the method further includes:

Modifying the playing equipment appointed by at least one switching time point in the control information to obtain modified control information;

Wherein the controlling the playing device to play the multimedia data in the multimedia data stream between the corresponding switching time point and the next switching time point of the playing device includes:

And controlling the playing device to play the multimedia data between the corresponding switching time point and the next switching time point of the playing device in the multimedia data stream by utilizing the modified control information.

Sending a request to the server; wherein the request is for requesting the server to modify a playback device specified by a switching time point in the multimedia data stream;

Receiving modified control information sent by the server, wherein in the modified control information, at least one play device designated by a switching time point is modified;

According to the role setting of the playing equipment, adjusting the multimedia data of the multimedia data stream between the corresponding switching time point and the next switching time point of the playing equipment to obtain adjusted multimedia data;

And controlling the playing device to play the adjusted multimedia data in the multimedia data stream between the corresponding switching time point and the next switching time point of the playing device.

The second aspect of the present invention discloses a multi-device interaction method, applied to a server, comprising:

transmitting a multimedia data stream to a master control device and control information corresponding to the multimedia data stream; the control information is used for designating a plurality of time points of the multimedia data stream as switching time points, and designating at least one corresponding playing device for each switching time point; the playing device comprises a controlled device or comprises the controlled device and the master control device; the controlled device is connected with the master control device.

Optionally, before sending the multimedia data stream and the control information corresponding to the multimedia data stream to the master control device, the method further includes:

Receiving a play request sent by the master control equipment or the controlled equipment;

Wherein the sending the multimedia data stream and the control information corresponding to the multimedia data stream to the master control device includes:

And sending a multimedia data stream corresponding to the playing request and control information corresponding to the multimedia data stream to the master control equipment.

Optionally, the multimedia data stream is a voice conversation;

the playing request carries dialogue theme, and the number and the identification of the playing devices;

The server generates the multimedia data stream corresponding to the playing request according to the dialogue theme carried by the playing request, and the number and the identification of the playing equipment.

Optionally, the interaction method further includes:

receiving a request sent by the main control equipment; wherein the request is for requesting the server to modify a playback device specified by a switching time point in the multimedia data stream;

And sending the modified control information to the main control equipment.

The invention discloses a multi-device interaction method, which is characterized by being applied to playing devices, wherein the playing devices comprise controlled devices or controlled devices and a master control device; the main control equipment is connected with the controlled equipment; the multi-device interaction method comprises the following steps:

The playing device responds to the control of the main control device and plays the multimedia data in the multimedia data stream between the corresponding switching time point and the next switching time point of the playing device;

The next switching time point is the next switching time point of the switching time point corresponding to the playing device, and if the switching time point corresponding to the playing device is the last switching time point of the multimedia data stream, the next switching time point is the ending time point of the multimedia data stream; the multimedia data stream is received by the main control device and corresponds to control information, wherein the control information is used for designating a plurality of time points of the multimedia data stream as switching time points and designating at least one corresponding playing device for each switching time point.

A fourth aspect of the present invention discloses a master device connected to a plurality of controlled devices, the master device comprising:

A receiving unit, configured to receive a multimedia data stream sent by a server and control information corresponding to the multimedia data stream; the control information is used for designating a plurality of time points of the multimedia data stream as switching time points, and designating at least one corresponding playing device for each switching time point; the playing device comprises each controlled device or comprises each controlled device and the master control device;

The control unit is used for controlling the playing device to play the multimedia data in the multimedia data stream between the corresponding switching time point of the playing device and the next switching time point; the next switching time point is the next switching time point of the switching time point corresponding to the playing device, and if the switching time point corresponding to the playing device is the last switching time point of the multimedia data stream, the next switching time point is the ending time point of the multimedia data stream.

Optionally, the control unit is specifically configured to:

Optionally, the control unit is further configured to: modifying the playing equipment appointed by at least one switching time point in the control information to obtain modified control information;

the control unit is configured to control the playing device to play the multimedia data in the multimedia data stream between the corresponding switching time point and the next switching time point of the playing device, where the control unit is specifically configured to: and controlling the playing device to play the multimedia data between the corresponding switching time point and the next switching time point of the playing device in the multimedia data stream by utilizing the modified control information.

Optionally, the master control device further includes: a sending unit, configured to send a request to the server; wherein the request is for requesting the server to modify a playback device specified by a switching time point in the multimedia data stream;

The receiving unit is further configured to: receiving modified control information sent by the server, wherein in the modified control information, at least one play device designated by a switching time point is modified;

Optionally, the control unit is further configured to: according to the role setting of the playing equipment, adjusting the multimedia data of the multimedia data stream between the corresponding switching time point and the next switching time point of the playing equipment to obtain adjusted multimedia data;

The control unit is configured to control the playing device to play the multimedia data in the multimedia data stream between the corresponding switching time point and the next switching time point of the playing device, where the control unit is specifically configured to: and controlling the playing device to play the adjusted multimedia data in the multimedia data stream between the corresponding switching time point and the next switching time point of the playing device.

A fifth aspect of the present invention discloses a server, comprising:

A sending unit, configured to send a multimedia data stream and control information corresponding to the multimedia data stream to a master control device; the control information is used for designating a plurality of time points of the multimedia data stream as switching time points, and designating at least one corresponding playing device for each switching time point; the playing device comprises a controlled device or comprises the controlled device and the master control device; the controlled device is connected with the master control device.

Optionally, the server further includes:

The receiving unit is used for receiving a play request sent by the master control equipment or the controlled equipment;

The sending unit is specifically configured to:

Optionally, the multimedia data stream is a voice conversation;

wherein, the server further includes:

and the generating unit is used for generating the number and the identification of the playing equipment according to the dialogue theme carried by the playing request, the multimedia data stream corresponding to the playing request and the control information corresponding to the multimedia data stream.

Optionally, the receiving unit is further configured to receive a request sent by the master device, where the request is used to request the server to modify a playback device specified by a switching time point in the multimedia data stream;

The generating unit is further configured to modify the control information according to the request, to obtain modified control information, where in the modified control information, a playback device specified by at least one switching time point is modified;

The sending unit is further configured to send the modified control information to the master control device.

The sixth aspect of the present invention discloses a playing device, where the playing device includes a controlled device, or includes the controlled device and a master control device; the master control device is connected with the controlled device, and the controlled device comprises:

A playing unit, configured to respond to control of the master control device, and play multimedia data in the multimedia data stream between a switching time point corresponding to the playing device and a next switching time point;

A seventh aspect of the present invention discloses an electronic device, comprising: a processor and a memory; wherein:

The memory is used for storing computer instructions;

the processor is configured to execute the computer instructions stored in the memory, and is configured to execute the interaction method of the multiple devices disclosed in any one of the first aspects of the present invention, or is configured to execute the interaction method of the multiple devices disclosed in any one of the second aspects of the present invention; or for performing the multi-device interaction method disclosed in the third aspect of the invention.

An eighth aspect of the present invention discloses a storage medium for storing a program which, when executed, is used to perform the interaction method of the multi-device disclosed in any one of the first aspects of the present invention, or is used to perform the interaction method of the multi-device disclosed in any one of the second aspects of the present invention; or for performing the multi-device interaction method disclosed in the third aspect of the invention.

A ninth aspect of the present invention discloses a multi-device interactive system, comprising:

The server is used for sending the multimedia data stream and control information corresponding to the multimedia data stream to the main control equipment; the control information is used for designating a plurality of time points of the multimedia data stream as switching time points, and designating at least one corresponding playing device for each switching time point; the playing device comprises a controlled device or comprises the controlled device and the master control device; the controlled device is connected with the master control device;

The main control equipment is used for controlling the playing equipment to play the multimedia data in the multimedia data stream between the corresponding switching time point of the playing equipment and the next switching time point; the next switching time point is the next switching time point of the switching time point corresponding to the playing device, and if the switching time point corresponding to the playing device is the last switching time point of the multimedia data stream, the next switching time point is the ending time point of the multimedia data stream;

And the playing device is used for responding to the control of the master control device and playing the multimedia data in the multimedia data stream between the corresponding switching time point and the next switching time point of the playing device.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are required to be used in the embodiments or the description of the prior art will be briefly described below, and it is obvious that the drawings in the following description are only embodiments of the present invention, and that other drawings can be obtained according to the provided drawings without inventive effort for a person skilled in the art.

FIG. 1 is a schematic diagram of a voice interaction scenario among a plurality of intelligent speakers;

FIG. 2 is a schematic diagram of the structure and operation principle of an interactive system in the prior art;

fig. 3 is a signaling flow chart of an interaction method of multiple devices according to an embodiment of the present invention;

fig. 4 is a schematic diagram of a multimedia data stream and a switching time point according to an embodiment of the present invention;

fig. 5 is a signaling flow chart of a multi-device interaction method according to another embodiment of the present invention;

fig. 6 is a flowchart of a method for controlling a playing device by a master control device according to an embodiment of the present invention;

Fig. 7 is a signaling flow chart of a multi-device interaction method according to another embodiment of the present invention;

fig. 8 is a flowchart of a method for generating a multimedia data stream and control information by a server according to an embodiment of the present invention;

FIG. 9 is a schematic diagram of a client interface for obtaining user opinion regarding modification of dialog text according to an embodiment of the present invention;

fig. 10 is a schematic structural diagram of a master control device according to an embodiment of the present invention;

Fig. 11 is a schematic structural diagram of a playback device according to an embodiment of the present invention;

fig. 12 is a schematic structural diagram of a server according to an embodiment of the present invention;

FIG. 13 is a schematic structural diagram of an interactive system with multiple devices according to an embodiment of the present invention;

FIG. 14 is a schematic diagram of an interactive system with multiple devices according to another embodiment of the present invention;

Fig. 15 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.

Detailed Description

The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

With the development of internet technology and artificial intelligence technology, intelligent sound boxes are currently being used by more and more users. On one hand, the intelligent sound box can acquire and play audio from the Internet and other electronic equipment in a wireless transmission and wired transmission mode; alternatively, the sound signal may be received and in response thereto, audio corresponding to the received sound signal may be played. The sound signal may be a user voice or audio played by other electronic devices.

Based on the above-mentioned function, the intelligent audio amplifier on the one hand can be used for realizing human-computer interaction at present. For example, the user may say to a smart speaker: what is the weather now? After receiving the user voice, the intelligent sound box sends the user voice to the server through the network, acquires the audio generated by the server aiming at the user voice, and plays the acquired audio as response information of the user voice. For the foregoing user voices, the audio played by the smart speaker may be audio including current weather information, e.g., smart speaker play: the weather is clear at present, the outdoor air temperature is 32 ℃, and people can pay attention to sun protection when going out. By receiving the voice of the user and playing the corresponding audio, the intelligent sound box is equivalent to the human-computer interaction with the user.

On the other hand, the intelligent sound box can also be used for realizing voice interaction among a plurality of intelligent sound boxes, namely multi-machine voice interaction. Referring to the application scenario shown in fig. 1, an example of a multi-machine voice interaction may be:

The intelligent sound box A plays the first audio frequency' good morning-! After receiving the first audio, the intelligent sound box B plays a second audio, "hello, do breakfast? "after the second audio is played as a response to the first audio, the intelligent sound box a plays the third audio as a response to the second audio" eat, you do? ". In the above process, the intelligent sound box A and the intelligent sound box B show the process of performing voice interaction between the two intelligent sound boxes by alternately playing the specific audio, which is equivalent to realizing multi-machine voice interaction. The audio frequency played by the two intelligent sound boxes is obtained from the server through the Internet.

However, the effect of multi-machine voice interaction realized by the existing interaction system has high requirement on real-time internet download rate, and if the current internet download rate is low, the effect of multi-machine voice interaction of the existing interaction system is obviously reduced, and user experience is affected.

Referring to fig. 2, and referring to fig. 1, in the conventional interactive system, an intelligent sound box a needs to download a first audio from a server through the internet, and can play the first audio after the downloading is completed, and after receiving the first audio played by the intelligent sound box a, the intelligent sound box B sends an audio acquisition request to the server through the internet, downloads a second audio from the server, and plays the downloaded second audio. That is, after the intelligent sound box a finishes playing the first audio, it takes a certain time to download the second audio before the intelligent sound box B starts playing the second audio. If the internet download rate is low when the intelligent audio box B downloads the second audio, after the first audio is played, the intelligent audio box B needs to wait for a long time, for example, 2s, before the second audio can be played.

Therefore, the multi-machine voice interaction realized by the existing interaction system can be stopped for a plurality of times for a long time, so that the whole interaction process is not smooth, and the user experience is seriously reduced.

Based on the defects of the prior art, the embodiment of the application provides a multi-device interaction method to avoid the phenomenon that the interaction process is not smooth due to low internet downloading rate.

Firstly, it should be noted that the interaction method of multiple devices provided by any embodiment of the present application can be used to control interactions between multiple devices, and the interactions described in any embodiment of the present application are not limited to multiple-machine voice interactions between multiple intelligent speakers, but may also include multiple-machine video interactions between multiple playback devices implemented by controlling multiple playback devices to play pre-acquired videos at specific time points, and a combination of multiple-machine video interactions and multiple-machine voice interactions.

The playing device may be an intelligent sound box, or may be other electronic devices with audio playing function and/or video playing function, for example, may be an intelligent robot with audio playing and video playing functions.

The multi-device interaction method provided by any embodiment of the application can be applied to two types of systems, wherein one type of system consists of a server and a plurality of playing devices, any one of the playing devices is used as a master control device, other playing devices except the master control device are used as controlled devices, and the master control device is used for controlling the plurality of playing devices to interact, wherein the master control device can control all the playing devices including the master control device according to the requirement of an interaction process. The specific playing device is used as the master control device, which can be preset by the provider of the system, or can be set by a user in the use process of the system.

Another type of system includes a server, a plurality of controlled devices and a master device. Each controlled device is a playing device, and the master device is another electronic device different from the playing device, for example, if intelligent robots with playing functions are used as controlled devices, the master device may be an intelligent base for placing the intelligent robots.

In the second type of system, the master control device is used for controlling the plurality of playing devices to interact. Of course, when the system includes three or more playing devices, only some of the playing devices in the system may be controlled, or all of the playing devices in the system may be controlled, according to the needs of the interaction process.

In the above two types of systems, each controlled device is at least connected with the master control device, on the basis that any one controlled device can also be connected with other controlled devices, specifically, which controlled devices are connected with each other, the controlled devices can be configured according to actual use conditions, and the controlled devices are not limited herein.

The connection between two devices (including connection between a master device and a controlled device and connection between two controlled devices) in any embodiment of the present application may be a wired connection, that is, a data transmission interface of one device is connected with a data transmission interface of another device through a data line, or the data transmission interface of one device is directly inserted into the data transmission interface of the other device; or a wireless connection, for example, where the two devices are connected via bluetooth.

On the other hand, in the above two types of systems, the server at least can perform data transmission with the master control device, and the data transmission mode includes, but is not limited to, performing data transmission through the internet, that is, the master control device can send data to the server through the internet, and can also download data from the server through the internet. Of course, the server may also communicate with the controlled device.

For easy understanding, the present embodiment mainly describes the multi-device interaction method provided by the present application based on a simple system including a server, two controlled devices and a master device, where the two controlled devices are respectively denoted as a first controlled device and a second controlled device. Of course, the multi-device interaction method described in this embodiment may be directly generalized to a complex system including three or more controlled devices.

Referring to fig. 3, the multi-device interaction method provided in this embodiment includes the following steps:

s301, the server sends the multimedia data stream and the control information to the master control equipment.

The control information is control information corresponding to a multimedia data stream.

The sending behavior in step S301 may be actively initiated by the server, or may be initiated after the server responds to a play request of the master control device or the controlled device. That is, the server may monitor the master device in real time, and send the multimedia data stream and the control information to the master device when it is monitored that a specific condition is satisfied, where the specific condition may be that the master device accesses the server through the internet, or that the current time is a certain preset time.

The server may also send the multimedia data stream and the control information to the master control device after receiving the play request to respond to the play request, where the play request may be sent by the master control device, or may be sent by the controlled device establishing a data transmission relationship with the server, or may be sent by the controlled device to the master control device, and then forwarded to the server through the master control device.

The multimedia data stream may be an audio data stream, i.e. a piece of audio; but also a video data stream, i.e. a piece of video. In particular, the multimedia data stream may be a song, a voice conversation, a episode of a television show, or a video, wherein the former two belong to an audio data stream and the latter two belong to a video data stream. A video data stream may include a corresponding audio data stream and multiple successive frames of images, or may include only multiple successive frames of images and not the audio data stream.

The control information includes a plurality of switching time points in the multimedia data stream and a corresponding relationship between the switching time points and the playing device. In this embodiment, only two controlled devices participate in the playing, and the master device does not participate in the playing, so in this embodiment, the above-mentioned correspondence is a correspondence between the switching time point and the controlled device.

Any time during the playing of the multimedia data stream is a point in time of the multimedia data stream. For example, the moment when a song starts to play, i.e., the first second of the song, is a time point, and the 10 th second of the song is also a time point. In any embodiment of the present application, for a multimedia data stream, if a time point of the multimedia data stream is included in the corresponding control information, the time point is a switching time point of the multimedia data stream.

Each switching time point corresponds to at least one playing device, and the corresponding relation between the switching time points and the playing devices is recorded in the control information. When the multimedia data stream is played to any switching time point, the playing device corresponding to the switching time point starts playing the multimedia data stream from the switching time point, and meanwhile, the playing devices except the playing device corresponding to the switching time point stop playing the multimedia data stream. This process is equivalent to that when the multimedia data stream is played to any switching time point, the playing device corresponding to the switching time point is switched to continue playing.

With respect to the relation between the multimedia data stream and the switching point in time, it can be further understood with reference to fig. 4. The horizontal straight line in fig. 4 represents a segment of the multimedia data stream, and the corresponding point on the straight line represents a point in time of the multimedia data stream. After the server records a plurality of time points in the control information corresponding to the multimedia data stream, the recorded time points are called switching time points of the multimedia data stream (the server can also be considered as designating the time points of the multimedia data stream as the switching time points in the form of the control information), and the short vertical line is shown in fig. 4. The server specifies the switching time points, and also records the playback devices corresponding to each switching time point in the control information (corresponding to the specification of the playback devices for each switching time point). When the multimedia data stream is played, the playing starts from the first switching time point, and in the playing process, the multimedia data between one switching time point and the next switching time point is only played by the playing device corresponding to the switching time point.

Taking fig. 4 as an example, the multimedia data between the first switching time point and the second switching time point is played by the playing device corresponding to the first switching time point, after the playing is finished, the multimedia data between the second switching time point and the third switching time point is played by the playing device corresponding to the second switching time point, and so on.

In the practical application process, the switching time point is generally recorded as what second of a multimedia data stream. For example, if a switching time point is recorded as 12 th second of the multimedia data stream, the switching time point indicates a time point when the first 11 seconds of the multimedia data stream just ends. If one switching time point is marked as 1 st second, the instant when the multimedia data stream starts, i.e. the starting time of the first second, is indicated.

For ease of understanding, the present embodiment introduces the following steps based on the following assumptions:

the multimedia data stream is an audio data stream having a total duration of 45 seconds, and control information corresponding to the audio data stream may be recorded in the form shown in table 1.

Switching time point	Corresponding controlled device
		Second 1	First controlled device
15 Th second	Second controlled device
		Second 30 th	First controlled device

TABLE 1

Further, as an example, it may be assumed that the above-mentioned audio data stream is a voice conversation, and the voice conversation includes three sentences, wherein the first sentence corresponds to audio data from 1 st second to 15 th second of the audio data stream, the second sentence corresponds to audio data from 15 th second to 30 th second of the audio data stream, and the third sentence corresponds to audio data from 30 th second to 45 th second.

It should be noted that, in any embodiment of the present application, if the switching time point is not the last switching time point of the multimedia data stream for any switching time point (t second), the multimedia data from the switching time point to the next switching time point (for example, t second to t+n second) refers to the multimedia data from the beginning time of the t second to the ending time of the t+n-1 second (or the beginning time of the t+n second); if the switching time point is the last switching time point of the multimedia data stream, the multimedia data from the switching time point to the next switching time point refers to the multimedia data in the period from the starting time of the switching time point (the starting time of the t-th second) to the ending time of the ending time point of the multimedia data stream (the ending time of the t+n-th second).

Thus, in the above example, the first sentence corresponds to audio data between the start time of the 1 st second and the start time of the 15 th second, the second sentence corresponds to audio data between the start time of the 15 th second and the start time of the 30 th second, and the third sentence corresponds to audio data between the start time of the 30 th second and the end time of the 45 th second.

A sentence corresponds to the audio data in a period of time, which corresponds to the sentence recorded in the audio data in the period of time, and the playing device plays the audio data in the period of time, which corresponds to the playing device speaking the sentence.

The above-mentioned end time point may be specified by control information, for example, a 60 th second of the audio data stream having a total duration of 70 seconds is specified as the end time point, and then subsequent audio data from a 61 th second is not played any more in the present embodiment; instead of being specified by the control information, the end time of the audio data stream itself may be directly used as the end time point, and the present embodiment adopts the latter scheme, from the start time of the 30 th second to the end of the audio data stream.

The first column of table 1 records switching time points of the multimedia data stream, and the second column records controlled devices corresponding to each switching time point. In other embodiments of the present application, one switching time point may correspond to two or more playback devices.

Optionally, the control information related to any embodiment of the present application may be configured by the server according to the multimedia data stream to be played and the number of playing devices. The number of playback devices may be different for the same multimedia data stream, and the control information configured by the server may be different, that is, the switching time point and the correspondence between the switching time point and the playback device may be different.

For example, for an audio data stream having a total duration of 100 seconds, if there are only two playback devices, the server may set the 1 st second as a switching time point, corresponding to one playback device, and the 50 th second as a second switching time point, corresponding to the other playback device, so that the two playback devices respectively play half of the audio data stream.

If the audio data stream needs to be played by 4 playing devices, the server may set the 1 st second, the 25 th second, the 50 th second and the 75 th second as switching time points respectively, and each switching time point corresponds to one playing device respectively; of course, the server may also set 5 or more switching time points, and the time intervals between the switching time points may be adjusted according to practical situations, may be only 1 second or 40 seconds, and the number of the playing devices corresponding to each switching time point is not limited, and may be one or more.

S302, the master control device controls the first controlled device to play the first audio.

The first audio refers to audio composed of audio data between the 1 st second and the 14 th second of the audio data stream, so step S302 corresponds to the master device controlling the first controlled device to play audio data between one switching time point (i.e. 1 st second) and the next switching time point (as shown in table 1, the next switching time point of 1 st second is 15 th second) corresponding to the master device itself.

As described in the foregoing assumption, the multimedia data stream in this embodiment is a voice session, and thus, the step S302 shows an effect that the first controlled device plays the first sentence of the voice session for 14 seconds.

It should be noted that, in any embodiment of the present application, the multimedia data between one switching time point (denoted as a switching time point a) and the next switching time point can only be played by the playing device corresponding to the switching time point a, and in the period from the switching time point a to the next switching time point, the other playing devices except the playing device corresponding to the switching time point a do not play any multimedia data.

Optionally, in the time from the switching time point a to the next switching time point, other playing devices except the playing device corresponding to the switching time point a may continue to read the multimedia data stream, but the volume of the playing device is set to zero, and may also directly stop reading the multimedia data stream.

Therefore, in the above step S302, the second controlled device remains silent while the first controlled device plays the first audio.

S303, the master control device controls the second controlled device to play the second audio.

Step S303, similar to step S302, corresponds to the master device controlling the second controlled device to play the audio data (i.e. the second audio) from the 15 th second to the 30 th second in the audio data stream, and corresponds to the second controlled device playing the second sentence of the voice conversation in the period from the 15 th second starting time to the 30 th second starting time.

In step S302, the first controlled device plays audio data with a total duration of 14 seconds from the start time of the 1 st second to the start time of the 15 th second, and step S301 may be considered as an action with a duration of 14 seconds. In step S303, the second controlled device starts playing from the start time of the 15 th second, that is, immediately after the first controlled device pauses playing, that is, step S303 is performed immediately after the duration of step S302 is finished, and step S303 may also be considered as a continuous action, where the duration is 15 seconds from the start time of the 15 th second to the start time of the 30 th second.

In the duration of step S303, only the second switching time point in table 1, that is, the playing device corresponding to the 15 th second, that is, the second controlled device plays, and the first controlled device remains silent.

S304, the master control device controls the first controlled device to play the third audio.

Similar to the previous step, step S304 corresponds to an action lasting from the start time of the 30 th second to the end time of the 45 th second, i.e. to the end time of the audio data stream, for a total duration of 16 seconds.

The second controlled device remains silent during the duration of step S304, and the first controlled device plays the third audio, that is, the audio data of the audio data stream from 30 seconds to 45 seconds, which corresponds to the first controlled device playing the third sentence in the voice conversation.

Step S304 is started immediately after step S303 ends, similarly to step S303.

Through four steps, this embodiment corresponds to achieving the following effects:

Firstly, the first controlled device speaks (i.e. plays) the first sentence in the voice dialogue acquired by the master control device, the time is 14 seconds, the second controlled device speaks the second sentence of the voice dialogue immediately after the first controlled device speaks the first sentence (i.e. after the duration of 14 seconds is over), the time is 15 seconds, the second controlled device speaks the second sentence as the response to the first sentence which the first controlled device speaks before, the first controlled device immediately starts speaking the third sentence in the voice dialogue after the second controlled device speaks the second sentence as the response to the second sentence which the second controlled device speaks, and then the whole interaction process is ended.

That is, in the method provided in this embodiment, the master control device controls the controlled device to play the corresponding audio data from the corresponding switching time point of the controlled device to the next switching time point according to the audio data stream and the corresponding control information obtained from the server, so as to achieve the effect of multi-machine voice interaction between the multiple controlled devices.

Meanwhile, referring to the connection manner of the master control device and the controlled device described above, and the implementation procedure of this embodiment, it can be found that: the audio data to be played by each controlled device can be obtained from the master control device in a wired or wireless transmission mode, and the audio data streams including all the audio data to be played are downloaded completely by the master control device before the controlled device starts playing from the first switching time point of the audio data streams. Therefore, in the implementation process of the interaction method of multiple devices provided by the embodiment, each controlled device does not need to download multimedia data to be played from a server through the internet, but obtains the multimedia data from the master control device through wired transmission and wireless transmission including a bluetooth technology, so that the influence of the internet transmission rate on the interaction process is effectively avoided, and even under the condition of low internet transmission rate, the interaction process between multiple devices can be ensured to be smooth.

Optionally, the main control device in the present application controls the playing device to play the multimedia data at the corresponding switching time point and the next switching time point, and the specific control modes may be as follows.

The first control mode is that the main control device firstly sends the multimedia data between the first switching time point and the next switching time point of the multimedia data stream to the playing device corresponding to the first switching time point for playing, then the main control device monitors the playing progress of the multimedia data stream, after the playing of the multimedia data between the first switching time point and the next switching time point is finished, the main control device sends the multimedia data between the second switching time point (namely the next switching time point of the first switching time point) and the next switching time point to the corresponding playing device of the second switching time point for playing, and so on until the multimedia data stream is played to the finishing time point.

In short, the first control manner corresponds to that the master control device sends the multimedia data between each switching time point and the next switching time point in the multimedia data stream to the corresponding playing device according to the time sequence, and the playing device plays the multimedia data once after receiving the corresponding multimedia data, and each playing device which is not the master control device only receives the corresponding multimedia data and does not receive the whole multimedia data stream.

In the second control manner, the entire multimedia data stream and the control information corresponding to the multimedia data stream are transmitted to each playing device participating in the interaction process, and the playing device determines the corresponding switching time point in the multimedia data stream according to the control information, and then plays the multimedia data stream from the corresponding switching time point to the next switching time point when the multimedia data stream is played to the corresponding switching time point.

In the second way, each playback device receives the entire multimedia data stream and the corresponding control information. Compared with the first control mode, in the first control mode, the playing equipment except the main control equipment only needs to receive and play the multimedia data, and the specific multimedia data are played by the playing equipment, the main control equipment determines according to the control information, and the playing equipment except the main control equipment does not need to consider. In the second control manner, each playback device needs to determine, according to the control information, which multimedia data in the multimedia data stream needs to be played back by itself, and determine a start time and an end time of playback.

Another embodiment of the present application will be described below in conjunction with the aforementioned first control mode. Please read the following examples with reference to fig. 5:

It should be noted that, in this embodiment, the main control device mainly focuses on how to control the playing device to play, so in fig. 5, a part of the server is omitted, but the multimedia data stream and the control information in this embodiment are still sent by the server, and the connection relationship between the server and the main control device is consistent with the foregoing embodiment. The case of the next embodiment of the present application is similar and will not be described again.

The present embodiment mainly relates to interaction between three playback devices, and one playback device serves as a master control device and the other two playback devices serve as slave devices. The three playback devices may be referred to as playback device a, playback device B, and playback device C, where playback device a serves as a master device.

In this embodiment, the specific implementation process of the audio data stream is still described by taking the audio data stream as an example, the total duration of the audio data stream to be played is 80 seconds, and the corresponding control information is shown in table 2.

Switching time point	Corresponding playing device
		Second 10	Playing device B
Second of 28 th	Playback device A, playback device C
		48 Th second	Playing device A
65 Th second	Playing device C

TABLE 2

On the one hand, as shown in table 2, in the present embodiment, the first switching time point of the audio data stream is not the start time of the audio data stream, i.e., 1 st second. In general, in the interaction method provided in this embodiment, a multimedia data stream to be played may be generated by a server according to an actual taste scene, so that the multimedia data stream exactly matches the scene of the present interaction, and as described in the previous embodiment, the starting time of the multimedia data stream is also used as the first switching time point, and the ending time of the multimedia data stream is used as the ending time point.

However, in some scenarios, the server may only need to select a pre-generated multimedia data stream from the history database and play a portion of the multimedia data stream via the control information. In this case, control information as shown in table 2 may occur, that is, the server does not process the original multimedia data stream in the history database any more, but only designates a certain time point after the start time of the multimedia data stream as the first switching time point by the control information. Based on the control information of table 2, the playback device B will start playback from the 10 th second of the multimedia data stream, and the first 9 seconds of the multimedia data stream is ignored.

On the other hand, the control information of the present embodiment includes an end time point set to 75 seconds in addition to the switching time point shown in table 2. Referring to the description of the previous embodiment for the end point in time, the playback device C will play back the multimedia data between the start time of the 65 th second of the multimedia data stream and the end time of the 75 th second, and the multimedia data that starts from 76 seconds will not be played back in the present embodiment. The control information designates a certain time point before the end time of the multimedia data stream as an end time point, and is also used for designating a part of a certain multimedia data stream in the history database as a multimedia data stream required for the present interaction in a specific scene, which is not described herein.

S501, the playback device a sends the first audio of the audio data stream to the playback device B.

The first audio refers to audio data between 10 th and 28 th seconds of the audio data stream.

In this embodiment, the playing device a participates in the interaction process and also bears the task of the master control device, so as to transmit the audio data between two adjacent switching time points or the audio data between the last switching time point and the ending time point to the corresponding playing device.

The specific implementation process of step S501 may be that the playback device a extracts audio data between 10 th and 28 th seconds of the audio data stream as audio to be played, and then sends the audio to be played to the playback device B. Subsequent transmission steps are similar.

S502, the playing device B plays the received first audio.

In any embodiment of the present application, the data transmission between the master device (i.e. the playing device a) and the controlled device (i.e. the playing device B) as described in step S501 may be considered to be instantaneously completed due to the high transmission rate. The process of playing the multimedia data by the playing device in step S502 is a continuous step, and the duration is the duration of the played multimedia data.

S503, the playback device a sends the second audio of the audio data stream to the playback device C.

The second audio refers to audio data between 28 seconds and 48 seconds of the audio data stream.

As shown in table 2, the 28 th second corresponds to two playing devices, but the playing device a itself stores the entire audio data stream, so only the second audio determined from the audio data stream needs to be sent to the playing device C, and in the subsequent playing step, the playing device a directly plays the second audio.

It should be noted that step S503 should be performed immediately after the end of the playing in step S502. The specific implementation method may be that after the playing device B plays the first audio, an end signal is sent to the playing device a, and the playing device a executes step S503 after receiving the end signal. Alternatively, the playback device B may directly pause the playback after the playback device B has finished playing back without sending a signal to the playback device a, and after the playback device a has finished executing step S501, time counting is performed, and when the time counting is finished until the 18 th second is finished, the playback device and the playback of the first audio can be considered to be finished, so that step S503 is performed. The subsequent transmission steps are performed similarly.

It should be noted that, in the embodiment implemented based on the first control manner, the master control device sends the multimedia data to the other playing devices, and actually further includes an action of judging whether the playing of the multimedia data stream is finished, if it is judged that the playing of the multimedia data stream is finished, no subsequent step is executed, and if it is judged that the playing of the multimedia data stream is not finished, the multimedia data to be played is determined and the corresponding playing device is controlled to play.

The process of judging whether the multimedia data stream is played and ended may be to judge whether the multimedia data already played in the previous step is the multimedia data between the last switching time point of the multimedia data stream and the ending time point of the multimedia data stream, or to judge whether the multimedia data played in the previous playing step ends with the ending time point of the multimedia data stream, if so, it indicates that the multimedia data stream is played and ended, if not, it indicates that the multimedia data stream is not played and ended.

S504, the playback device a and the playback device C play back the second audio simultaneously.

The playing device a directly plays the second audio determined from the audio data stream after completing the sending in step S503, and the playing device C plays the received second audio. Because the data transmission rate between the playing devices is higher, it can be considered that the playing device C has also successfully received after the playing device a finishes transmitting, and the two playing devices are equivalent to playing the second audio at the same time.

S505, the playing device A plays the third audio.

The third audio refers to audio data between 48 seconds and 65 seconds of the audio data stream.

As shown in table 2, the 48 th second switching time point corresponds to only the playing device a, so that the playing device a determines the third audio from the audio data stream and then can directly play the third audio. Of course, step S505 is executed after the playing process of step S504 is finished, and the method for determining the ending of step S504 is identical to the previously described determining method, and will not be described again.

S506, the playback device a sends the fourth audio of the audio data stream to the playback device C.

The fourth audio refers to audio data between 65 th and 75 th seconds of the audio data stream, wherein the fourth audio is audio data between a start time of 65 th and an end time of 75 th seconds since 75 th second is an end time point of the audio data stream.

S507, playing the fourth audio by the playing device C.

S508, the playing device A determines that the playing of the audio data stream is finished.

Optionally, after determining that the audio data stream is played, the master control device may send an end instruction to a playing device other than the master control device.

According to the multi-device interaction method provided by the embodiment, specific multimedia data in the multimedia data stream are played by the playing devices, the master control device determines according to the control information, and the playing devices except the master control device only need to receive and play the multimedia data.

In order to better understand the foregoing first control manner, the implementation process of the multi-device interaction method provided by the present application based on the first control manner is described below with reference to fig. 6 in view of standing on the master control device.

S601, receiving a multimedia data stream and corresponding control information.

S602, controlling the playing equipment corresponding to the current switching time point to play the multimedia data to be played.

The multimedia data to be played refers to multimedia data between a current switching time point and a next switching time point of the multimedia data stream. Wherein, the next switching time point refers to the next switching time point of the current switching time point; if the current switching time point is the last switching time point of the multimedia data stream, the next switching time point is the ending time point of the multimedia data stream.

If the master control device is a control device and the master control device corresponds to the current switching time point, the control in step S602 includes the master control device directly playing the multimedia data.

If the current switching time point corresponds to other playing devices except the master control device, step S602 includes sending the multimedia data to the corresponding playing device, so that the corresponding playing device plays the multimedia data.

In the implementation process of the multi-device interaction method provided in this embodiment, when step S602 is performed for the first time, the current switching time point is the first switching time point of the multimedia data stream.

S603, judging whether the playing of the multimedia data to be played is finished.

If the playing is finished, step S604 is executed, and if the playing is not finished, the step is continued.

The judging method refers to the content of the previous embodiment, and is not described herein.

S604, judging whether the playing of the multimedia data stream is finished.

If the playing of the multimedia data stream is finished, the method is finished, and if the playing of the multimedia data stream is not finished, step S605 is executed.

The judging method in this step is still referred to the corresponding description of the previous embodiment, and will not be repeated here.

S605, determining the next switching time point as the current switching time point, and returning to the step S602.

The next switching time point here is the next switching time point mentioned in step S602. For example, for the first cycle of the method, the current switching time point of step S601 is the first switching time point of the multimedia data stream, and the next switching time point is the second switching time point of the multimedia data stream. Correspondingly, in step S605, the second switching time point of the multimedia data stream is determined as the current switching time point, and then step S602 is returned to enter the second cycle.

In the following, a further embodiment of the present application will be described with reference to fig. 7 in conjunction with the aforementioned second control mode.

The present embodiment still involves three playback devices, and further includes a device different from the playback device as a master device. The multimedia data stream and the control information related to the present embodiment are identical to those of the embodiment shown in fig. 4.

S701, the master control device sends the audio data stream and the control information to the playing device A, the playing device B and the playing device C.

It should be noted that, since all three playback devices involved in the present embodiment participate in the interaction process, the master control device in step S701 may directly send the multimedia data stream and the control information to the playback device a, the playback device B, and the playback device C.

In general, for the case that the master control device is connected to a plurality of playback devices, and there is a case where at least one playback device does not participate in the present interaction process, when the master control device performs step S701, it may first determine all playback devices that participate in the interaction process, that is, determine all playback devices recorded in the control information, and then send the multimedia data stream and the control information to those playback devices that participate in the interaction.

For example, the control information shown in table 2 records the above three playback devices a, B, and C, and if the master device is further connected to the playback device D in addition to the three playback devices, the master device may determine, through the control information shown in table 2, that among the four playback devices connected to the master device, playback devices a, B, and C are playback devices that participate in the present interaction process, and that playback device D does not participate in the present interaction process.

S702, playing equipment B plays first audio of the audio data stream.

The first audio is audio data between 10 th and 28 th seconds of the audio data stream.

S703, the playback device a and the playback device C play back the second audio of the audio data stream.

The first audio is audio data between 28 seconds and 48 seconds of the audio data stream.

S704, the playing device a plays the third audio of the audio data stream.

The first audio is audio data between 48 seconds and 65 seconds of the audio data stream.

S705, the playing device C plays the fourth audio of the audio data stream.

The first audio is audio data between 65 th and 75 th seconds of the audio data stream.

After the playing device C plays the fourth audio, the playing of the whole audio data stream is ended, which is equivalent to the interactive process.

In order to implement the aforementioned second control manner, the working principle of each playing device may be:

After each playing device receives the multimedia data stream and the control information, immediately judging whether the playing device corresponds to the first switching time point of the multimedia data stream, if so, immediately starting playing from the first switching time point, wherein the judging process takes a short time, and the playing device corresponding to the first switching time point starts playing the corresponding multimedia data after the master control device sends the multimedia data stream and the control information.

Then, each playing device judges which switching time points in the multimedia data stream correspond to itself, and counts the time when the multimedia data stream and the control information are received as the starting time, and plays the multimedia data from the switching time point to the next switching time point of the switching time point after counting to any one of the switching time points corresponding to itself, or plays the audio data from the switching time point to the ending time point if the switching time point is the last switching time point. Of course, if the next switching time point is still the switching time point corresponding to itself, the above playing step is continued.

In the embodiment shown in fig. 7, after the transmission of the multimedia data stream and the control information is completed, the control device B immediately determines that the first switching time point is the switching time point corresponding to itself, so step S702 is performed. Then, the playback device a determines that there are two switching time points of the own correspondence of 28 th second and 48 th second, and the playback device C determines that there are two switching time points of the own correspondence of 28 th second and 65 th second, so that the playback device a and the playback device C start timing, and when the timing reaches 28 th second, the playback device a and the playback device C execute step S703, and when the playback reaches 48 th second, the playback device C pauses the playback and timing, and the playback device a executes step S704, continues the playback until 65 th second, and then pauses the playback, while the playback device C clocks until 65 th second, and executes step S705, and audio data of 65 th to 75 th seconds is played.

It should be noted that, in the execution of the playing step of the interaction method provided in any embodiment of the present application, other playing devices except the playing device indicated by the step remain silent.

Optionally, in the multi-device interaction method disclosed in the embodiment of the present application, before controlling the playing device to play the multimedia data between the corresponding switching time point and the next switching time point in the multimedia data stream, the master control device may further modify the control information sent by the server to obtain modified control information, and then control the playing device to play the multimedia data in the multimedia data stream by using the modified control information.

The master control device can modify control information sent by the server according to the number of playing devices currently controlled by the master control device. Specifically, after receiving the control information sent by the server, the master control device determines whether the playing device specified by the control information is matched with the playing device capable of working normally controlled by the master control device, and if the playing device specified by the control information is more than the playing device capable of working normally controlled by the master control device or the playing device specified by the control information is less than the playing device capable of working normally controlled by the master control device, the master control device carries out the modification.

In addition, the master control device may also modify the control information when the playback device specified by the control information matches with the playback device that can operate normally and is controlled by the master control device.

For example, the control information is assigned to the multimedia data played by a certain playing device, and does not match the role of the playing device, in which case the master control device may also modify the control information.

It should be noted that, the modification of the control information by the master control device may be performed in the following two ways.

In one way, the master device sends a request to the server requesting the server to modify the playback device specified in the control information for at least one switching point in time in the multimedia data stream. After receiving the request, the server modifies the control information according to the request to obtain modified control information, and sends the modified control information to the main control equipment.

Optionally, in this manner, the master device may carry in the request: and the corresponding relation between the switching time point in the multimedia data stream and the modified playing equipment. Thus, after receiving the request, the server can modify the control information according to the modified corresponding relationship carried in the request. Of course, the master control device may also carry the modification requirement in the request, and after the server analyzes the requirement, the server determines the switching time point to be modified in the multimedia data stream and the designated modified playing device, and modifies the control information according to the switching time point to meet the modification requirement carried in the request.

In another way, the master control device modifies the playback device specified by at least one switching time point in the control information, and obtains modified control information.

Specifically, the master control device definitely determines a switching time point of the playing device to be adjusted in the multimedia data flow, and definitely determines the designated playing device after modification of the switching time point of the playing device to be adjusted; the master control device modifies the control information in this way so that the playback device specified by the switching point in time in the control information is modified to a new playback device.

For example, the control information specifies three switching time points in the multimedia data stream, the first switching time point corresponds to the playing device a and the playing device B, the second switching time point corresponds to the playing device C, and the third switching time point corresponds to the playing device D; for this control information, the modification of the master control device may be that the playing device corresponding to the first switching time point is changed to the playing device B, the playing device corresponding to the second switching time point is changed to the playing device a, and the playing device corresponding to the third switching time point is changed to the playing device C and the playing device D.

It should be noted that, in addition to modifying the playback device specified by at least one switching time point in the control information, the manner in which the master control device modifies the control information may also add or delete a time point in the multimedia data stream specified by the control information as the switching time point, and may also adjust the position of the switching time point.

For example, the control information before modification designates four switching time points in the multimedia data stream, which correspond to four playing devices respectively, and are sequentially designated as a playing device a, a playing device B, a playing device C and a playing device D, where the four switching time points divide the multimedia data stream into four pieces of multimedia data, and the master control device may add a switching time point between the first piece of multimedia data, that is, between the first switching time point and the second switching time point, so that the modified control information includes five switching time points. Of course, after the new switching time point is added, the master control device needs to specify the playing device for the new switching time point, which playing devices can be determined by the master control device, on one hand, one or more of the four original playing devices can be specified to the new time point on the basis of the original corresponding relation, for example, the new time point is specified to correspond to the playing device C, so that in the modified control information, the playing device C corresponds to the second switching time point (i.e., the new switching time point) and the fourth switching time point (i.e., the third switching time point of the control information before modification), and on the other hand, the new playing device can be specified for the new switching time point, for example, the playing device E is specified to the new switching time point.

Of course, the master control device may delete one of the four original switching time points and reassign the playback device corresponding to the switching time point to the other switching time point, for example, the master control device may delete the second switching time point and then assign the playback device B to the first switching time point, which is equivalent to that the modified control information includes three switching time points, where the first switching time point corresponds to the playback device a and the playback device B, and the second switching time point and the third switching time point correspond to the playback device C and the playback device D, respectively.

The position of the switching time point is adjusted, for example, the control information before modification includes three switching time points, wherein the second switching time point is the 20 th second of the multimedia data stream, and the master control device can change the second switching time point to the 30 th second of the multimedia data stream to obtain the modified control information.

It will be appreciated by those skilled in the art that although the foregoing embodiments are described in terms of audio data streams and only involving two or three playback devices, the foregoing methods may be extended without inventive effort to interactive methods based on video data streams involving four or more playback devices, and are therefore within the scope of the present application.

In the method for interaction between multiple devices provided in any embodiment of the present application, the multimedia data stream and the control information sent by the server may be sent after the play request, where the play request may be sent by the master control device or the controlled device, and meanwhile, the user may set the play request through the client and the master control device, so that the server may generate the multimedia data stream and the corresponding control information corresponding to the play request according to the related information carried in the play request.

Referring to fig. 8, the method for generating the multimedia data stream and the control information by the server according to the play request includes:

S801, the server receives a play request.

The playing request carries the number and the identification of the playing devices and the dialogue theme set by the user. For example, the user may set the dialog topic to "music".

S802, the server determines a plurality of sentences related to the topic from a preset dialogue database according to the dialogue topic.

For example, for the aforementioned dialog topic "music", the sentence associated with that topic may be: "do you like to listen to popular songs", "i like singer xxx", "i feel that the song" yyy "is very good, etc.

S803, the server determines the playing device corresponding to each sentence according to the number and the identification of the playing devices, and a dialogue text is obtained.

For example, where there are two playback devices, two playback devices may be provided to alternately play the sentence determined in step S802.

S804, the server sends the dialogue text to the client to acquire a modification request fed back by the client.

After the dialogue text is sent to the client, the user refers to the dialogue text at the client and proposes a modification opinion. The client then feeds back to the server a modification request carrying the user's modification opinion. When the user does not modify the opinion, the modification request may carry confirmation information.

The client displays the dialog text to the user and obtains an interface for modifying opinions, see fig. 9.

S805, the server determines the playing time length of each sentence of the confirmed dialogue text.

When the modification request fed back by the client carries the modification opinion, the confirmed dialogue text is the dialogue text modified according to the modification opinion. When the modification request carries the confirmation information, the dialog text obtained in step S803 is directly used as the dialog text after confirmation.

For step S805, on the one hand, each word may be set to play with a specific play duration, for example, each word is played for 0.5S, and then the play duration of each sentence is the number of words of the sentence multiplied by the play duration of each word.

On the other hand, a spacer with a certain time length can be inserted between certain two words in the sentence, so that when the sentence is played to the spacer, the corresponding time length is stopped, and then the sentence is played continuously. Then the play duration of the sentence is added to the total duration of the spacer on the basis of the number of words of the previous aspect multiplied by the play duration of the individual words.

S806, the server generates a voice dialogue and control information according to the confirmed dialogue text and the playing time length of each sentence.

The voice conversation is a multimedia data stream generated by a server.

Alternatively, the correspondence between the sentence and the playback device may be determined according to the roles played by the respective playback devices. On the other hand, the sentence determined in step S802 may also be adjusted according to the role played by the corresponding playing device, so as to conform to the role characteristics of the corresponding playing device. For example, based on the sentence determined in step S802, some of the mood words may be added according to the character characteristics of the corresponding playing device, and a part of the vocabulary of the original sentence may be replaced.

Furthermore, the determined sentence can be translated into other languages or different dialects according to the set roles, for example, the determined Chinese sentence can be translated into English, french and the like.

Alternatively, the role played by each playback device may be specified by the user, or may be specified by the server after the dialog topic set by the user is acquired.

Alternatively, the above-mentioned sentence adjustment may be performed in the first aspect, where after the sentences are determined in step S802, the server adjusts the sentences, and then combines the adjusted sentences into a voice dialogue.

In the second aspect, after receiving the voice dialogue sent by the server, the master control device adjusts sentences in the voice dialogue according to the role setting of the play device controlled by the master control device, so as to obtain the adjusted voice dialogue. The main control device adjusts sentences in the voice dialogue according to the role setting of the playing device, and mainly adjusts the voice dialogue between the corresponding switching time point and the next switching time point of the playing device in the voice dialogue.

The third aspect may be that, after receiving the voice dialogue sent by the server, the master control device sends a modification request to the server, so that the server adjusts the sentences in the voice dialogue according to the role of the playing device, and obtains an adjusted voice dialogue, and the server sends the adjusted voice dialogue to the master control device.

Further, the master control device may also perform the foregoing adjustment in real time during the playing process, for example, when one playing device plays the first sentence of the voice dialogue, the master control device may adjust the subsequent second sentence, the third sentence, and so on, to obtain the adjusted second sentence, or the adjusted third sentence, and then after the playing of the first sentence is finished, control the corresponding playing device to play the adjusted second sentence, or control the corresponding playing device to play the adjusted third sentence.

The method provided by the embodiment is mainly used for acquiring a plurality of sentences related to the dialogue theme from a preset dialogue database according to the dialogue theme, the number of playing devices, the identification and other scene information set in a playing request by a user, then determining the playing time length of each sentence and the corresponding playing device, combining the sentences to obtain an audio data stream, determining control information according to the playing time length of each sentence and the corresponding playing device, and then sending the audio data stream and the control information to a main control device.

Further, the server may send the generated audio data stream and the corresponding dialog text to the client, and the user confirms whether to adjust the audio data stream and how to adjust the audio data stream and the control information to the master control device.

Based on the above technical scheme, the embodiment can generate various types of voice dialogs capable of being used as multimedia data streams according to the needs of users, so that the master control device controls the plurality of playing devices to respectively play corresponding sentences according to the voice dialogs and corresponding control information, and the users can experience personalized interaction processes of the plurality of devices.

In combination with the multi-device interaction method provided by any embodiment of the present application, the embodiment of the present application further provides a server, a main control device and a playing device. With reference to fig. 10, the master control device provided in the embodiment of the present application may include the following structures:

a receiving unit 1001 is configured to receive a multimedia data stream and control information corresponding to the multimedia data stream sent by a server.

Wherein the control information includes: the system comprises a plurality of switching time points of the multimedia data stream and playing devices corresponding to each switching time point, wherein each switching time point corresponds to at least one playing device.

Alternatively, the playback device may include each controlled device.

Alternatively, the playback device may include each of the controlled devices, and the master device described above.

If the master control equipment is also a playing equipment, the master control equipment is connected with at least one controlled equipment; and if the master control equipment is not a playing equipment, the master control equipment is connected with at least two controlled equipment.

The connection mode of the master control device and the controlled device can be wired connection or wireless connection.

A control unit 1002, configured to control each playback device to play back the multimedia data in the multimedia data stream between the corresponding switching time point and the next switching time point of the playback device.

The next switching time point is the next switching time point of the switching time point corresponding to the playing device, and if the switching time point corresponding to one playing device is the last switching time point of the multimedia data stream, the next switching time point refers to the ending time point of the multimedia data stream.

In combination with the multi-device interaction method provided in the foregoing embodiment, the control unit 802 may implement control on the playing device in two ways, and according to different specific control manners, the control unit of the master control device provided in this embodiment may have the following two structures:

based on one control scheme, the control unit 1002 may include:

And the determining unit is used for determining the first switching time point of the multimedia data stream to be played currently as the current time point, and further determining the multimedia data between the current switching time point and the next switching time point as the multimedia data to be played.

And the sending unit is used for sending the multimedia data to be played determined by the determining unit to the playing equipment corresponding to the current switching time point, so that the playing equipment corresponding to the current switching time point plays the multimedia data to be played.

The judging unit is used for judging whether the playing of the multimedia data to be played is finished after the sending unit sends the multimedia data to be played, further judging whether the playing of the current multimedia data stream to be played is finished after judging that the playing of the multimedia data to be played is finished, finishing the interaction process if judging that the playing of the multimedia data stream is finished, triggering the determining unit to determine the next switching time point as the current switching time point if judging that the playing of the multimedia data stream is not finished, and determining the multimedia data to be played based on the determined current switching time point.

Further optionally, when the master device is a playing device, the control unit 802 may further include a playing unit, configured to play the multimedia data between the current switching time point and the next switching time point when the current switching time point determined by the determining unit corresponds to the master device itself.

Based on another control manner, the control unit 1002 may include:

And the determining unit is used for reading the control information and determining the playing equipment recorded in the control information as the playing equipment participating in the interaction process.

And the sending unit is used for sending the multimedia data stream and the corresponding control information to all playing devices participating in the interactive process, so that the playing devices control the multimedia data between the corresponding switching time point and the next switching time point of the corresponding switching time point according to the multimedia data stream and the control information.

Optionally, when the master control device is also a playing device participating in the present interaction process, the sending unit is configured to send the multimedia data stream and the corresponding control information to playing devices participating in the present interaction process other than the master control device.

Referring to fig. 11, a playback apparatus provided in an embodiment of the present application may include the following structure:

The receiving unit 1101 is configured to receive multimedia data sent by the master device, or receive a multimedia data stream and corresponding control information sent by the master device.

And the playing unit 1102 is configured to respond to control of the master device, and play multimedia data in the multimedia data stream between a switching time point corresponding to the playing device and a next switching time point.

The main control equipment establishes communication connection with the playing equipment in a wireless connection or wired connection mode

The next switching time point is the next switching time point of the switching time point corresponding to the playing device, and if the switching time point corresponding to the playing device is the last switching time point of the multimedia data stream, the next switching time point is the ending time point of the multimedia data stream.

The multimedia data stream is received by the main control equipment and corresponds to control information, and the control information comprises: the system comprises a plurality of switching time points of the multimedia data stream and playing devices corresponding to each switching time point, wherein each switching time point corresponds to at least one playing device.

Alternatively, a playback device including the configuration of the master device described in the previous embodiment may be used as the master device.

Alternatively, corresponding to the structure of the master control device in the second control manner mentioned in the previous embodiment, the playback device may include the following units:

A determining unit 1103 is configured to determine a switching time point in the multimedia data stream corresponding to the playing device from the control information.

The timing unit 1104 is configured to trigger the playing unit to play the multimedia data between the switching time point and the next switching time point of the switching time point when the timing unit starts timing from receiving the multimedia data stream and the corresponding control information and timing to the switching time point corresponding to any one playing device.

Referring to fig. 12, a server provided by an embodiment of the present application may include the following structures:

A sending unit 1201, configured to send a multimedia data stream and control information corresponding to the multimedia data stream to a master control device.

Optionally, the server may further include:

a receiving unit 1202, configured to receive a play request sent by a master device or a controlled device.

The play request may carry the number and identity of the play devices, as well as a dialog theme, which may be determined by the user.

A generating unit 1203, configured to generate, according to the number and the identifier of the playback devices and the playback request of the session theme, a corresponding voice session and control information, where the voice session is sent by the sending unit 1201 as a multimedia data stream.

Further alternatively, the generating unit 1203 may generate the corresponding dialogue text according to the above-mentioned play request, then trigger the sending unit 1201 to send the dialogue text to the client, and then receive the modification request of the client through the receiving unit 1202, thereby generating the confirmed dialogue text according to the modification request and the dialogue text, and finally generate the voice dialogue and the corresponding control information according to the confirmed dialogue text.

The specific working principles of the master control device, the playing device and the server provided by any embodiment of the present application may refer to corresponding steps in the interaction method of multiple devices provided by any embodiment of the present application, which are not described herein.

After the master control device receives the multimedia data stream and the corresponding control information sent by the server, the master control device uses the receiving unit 1001 to control the playing device according to the control information, and the playing unit 1102 of the playing device responds to the control of the master control device and plays the multimedia data between the corresponding switching time point and the next switching time point of the playing device in the multimedia data stream. The playing device can directly acquire the multimedia data to be played from the main control device without downloading the multimedia data from the server through the Internet, so that the interaction process of the multimedia devices realized by playing the multimedia data is not influenced by the data transmission rate of the Internet, and even if the data transmission rate of the Internet is lower, the device can realize the smooth interaction process of the multimedia devices among a plurality of playing devices, and effectively improve the user experience.

In combination with the multi-device interaction method, the master control device, the playing device and the server provided in the embodiment of the present application, referring to fig. 13, the embodiment of the present application provides a multi-device interaction system, which includes:

A master control device, a server and at least two controlled devices, each controlled device being a playback device.

The server is used for sending the multimedia data stream and the corresponding control information to the main control equipment through the Internet, and the control information comprises: the system comprises a plurality of switching time points of the multimedia data stream and playing devices corresponding to each switching time point, wherein each switching time point corresponds to at least one playing device.

The master control device is connected with each controlled device in a wireless or wired manner. After receiving the multimedia data stream sent by the server, the master control device is used for controlling the playing device to play the multimedia data between the corresponding switching time point and the next switching time point of the playing device in the multimedia data stream.

The playing device is used for responding to the control of the master control device and playing the multimedia data between the corresponding switching time point and the next switching time point of the playing device in the multimedia data stream.

Another embodiment of the present application further provides a multi-device interaction system, referring to fig. 14, the system includes:

a plurality of playback devices and a server, any one of the plurality of playback devices serving as a master device performs the functions of the master device in the system provided in the previous embodiment.

The functions of the playing device and the server in this embodiment are identical to those of the playing device and the server in the previous embodiment, and will not be described again.

The master control device in this embodiment may control all the playing devices in the system including itself to play the corresponding multimedia data.

It should be noted that, for convenience of understanding, fig. 13 and fig. 14 only show two simplest embodiments of the interactive system provided by the present application, and only two play devices in the interactive system of these two embodiments are shown. However, the interactive system obtained by increasing the number of playing devices based on fig. 13 and 14 is just another embodiment of the multi-device interactive system provided by the present application, and therefore, is also within the scope of the present application.

The specific working principles of the server and each device in the multi-device interaction system provided by the embodiment of the present application may refer to corresponding steps of the multi-device interaction method provided by any embodiment of the present application, and will not be described herein.

After receiving the multimedia data stream and the corresponding control information sent by the server, the master control device controls the playing device to play the multimedia data between the corresponding switching time point and the next switching time point of the playing device according to the control information. The playing device can directly acquire the multimedia data to be played from the main control device without downloading the multimedia data from the server through the Internet, so that the interaction process of the multimedia devices realized by playing the multimedia data is not influenced by the data transmission rate of the Internet, and even if the data transmission rate of the Internet is lower, the device can realize the smooth interaction process of the multimedia devices among a plurality of playing devices, and effectively improve the user experience.

Another embodiment of the present application further provides an electronic device, with reference to fig. 15, including: a memory 1501 and a processor 1502, wherein:

memory 1501 is used to store computer instructions;

The processor 1502 is configured to execute the computer instructions stored in the memory 1501, and if the electronic device is used as a master device, the computer instructions are configured to execute the steps related to the master device in the multi-device interaction method according to any of the embodiments of the present application when executed.

If the electronic device is used as a server, the computer instructions are used to execute relevant steps of the server in the multi-device interaction method according to any embodiment of the present application when executed.

If the electronic device is used as a playing device, the computer instructions are used for executing relevant steps of the playing device in the multi-device interaction method provided by any embodiment of the present application when executed.

An eighth aspect of the present invention discloses a storage medium storing a program.

If the storage medium is a storage medium of a master device, the program is used to execute the relevant steps of the master device in the multi-device interaction method provided by any embodiment of the present application when the program is executed.

If the storage medium is a storage medium of a server, the program is used to execute the steps related to the server in the multi-device interaction method according to any of the embodiments of the present application when the program is executed.

If the storage medium is a storage medium of a playback device, the program is configured to execute the steps related to the playback device in the multi-device interaction method according to any one of the embodiments of the present application when the program is executed.

Those skilled in the art will be able to make or use the application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. The multi-device interaction method is characterized by being applied to a master control device, wherein the master control device is connected with a controlled device; the multi-device interaction method comprises the following steps:

Receiving a multimedia data stream and control information corresponding to the multimedia data stream sent by a server; the control information is used for designating a plurality of time points of the multimedia data stream as switching time points, and designating at least one corresponding playing device for each switching time point; the playing device comprises a plurality of controlled devices or at least one controlled device and the master control device; before the controlled device starts playing from the first switching time point of the multimedia data stream, the multimedia data stream is downloaded by the main control device;

2. The interactive method according to claim 1, wherein said controlling the playback device to play back the multimedia data in the multimedia data stream between the corresponding switching time point and the next switching time point of the playback device comprises:

3. The interactive method according to claim 1, wherein said controlling the playback device to play back the multimedia data in the multimedia data stream between the corresponding switching time point and the next switching time point of the playback device comprises:

4. The interactive method according to claim 1, wherein said controlling the playback device to play back the multimedia data in the multimedia data stream between the corresponding switching time point and the next switching time point of the playback device further comprises:

5. The interactive method according to claim 1, wherein said controlling the playback device to play back the multimedia data in the multimedia data stream between the corresponding switching time point and the next switching time point of the playback device further comprises:

6. The interactive method according to claim 1, wherein said controlling the playback device to play back the multimedia data in the multimedia data stream between the corresponding switching time point and the next switching time point of the playback device further comprises:

7. A master device, wherein the master device is connected to a plurality of controlled devices, the master device comprising:

A receiving unit, configured to receive a multimedia data stream sent by a server and control information corresponding to the multimedia data stream; the control information is used for designating a plurality of time points of the multimedia data stream as switching time points, and designating at least one corresponding playing device for each switching time point; the playing device comprises each controlled device or comprises each controlled device and the master control device; before the controlled device starts playing from the first switching time point of the multimedia data stream, the multimedia data stream is downloaded by the main control device;

8. The master control apparatus according to claim 7, wherein the control unit comprises:

A determining unit, configured to determine a first switching time point of a multimedia data stream to be played currently as a current time point, and further determine multimedia data between the current switching time point and a next switching time point as multimedia data to be played;

The sending unit is used for sending the multimedia data to be played determined by the determining unit to the playing device corresponding to the current switching time point, so that the playing device corresponding to the current switching time point plays the multimedia data to be played;

9. The master control apparatus of claim 7, wherein the control unit further comprises:

And the playing unit is used for playing the multimedia data between the current switching time point and the next switching time point when the current switching time point determined by the determining unit corresponds to the master control equipment.

10. The master control apparatus according to claim 7, wherein the control unit comprises:

The determining unit is used for reading the control information and determining the playing equipment recorded in the control information as the playing equipment participating in the interaction process;

11. A server, comprising:

A transmitting unit for transmitting a multimedia data stream and control information corresponding to the multimedia data stream to the master control device according to any one of claims 7 to 10; the control information is used for designating a plurality of time points of the multimedia data stream as switching time points, and designating at least one corresponding playing device for each switching time point: the playing device comprises a plurality of controlled devices or at least one controlled device and the master control device.

12. The server of claim 11, further comprising:

a generating unit, configured to generate a multimedia data stream corresponding to the play request and control information corresponding to the multimedia data stream according to the play request;

The sending unit is specifically configured to: and sending a multimedia data stream corresponding to the playing request and control information corresponding to the multimedia data stream to the master control equipment.

13. The server of claim 12, wherein the play request carries a dialog theme, a number of the play devices and an identification of the play devices;

the generating unit is specifically configured to: and generating a multimedia data stream corresponding to the playing request and control information corresponding to the multimedia data stream according to the dialogue theme carried by the playing request and the number of the playing devices and the identification of the playing devices.

14. The server according to claim 12, wherein the receiving unit is further configured to receive a request sent by the master device, where the request is for requesting the server to modify a playback device specified by a switching time point in the multimedia data stream;

15. A multi-device interactive system, comprising:

The server is used for sending the multimedia data stream and control information corresponding to the multimedia data stream to the main control equipment; the control information is used for designating a plurality of time points of the multimedia data stream as switching time points, and designating at least one corresponding playing device for each switching time point; the playing device comprises a controlled device or comprises the controlled device and the master control device; the controlled device is connected with the master control device; before the controlled device starts playing from the first switching time point of the multimedia data stream, the multimedia data stream is downloaded by the main control device;

16. A device for user interface playback, the device comprising a processor and a memory:

The memory is used for storing program codes and transmitting the program codes to the processor;

The processor is configured to perform the method of any of claims 1-6 according to instructions in the program code.

17. A computer readable storage medium, characterized in that the computer readable storage medium is for storing a program code for performing the method of any one of claims 1-6.