WO2017107309A1

WO2017107309A1 - Control method, control device, terminal, and synchronous audio playback system

Info

Publication number: WO2017107309A1
Application number: PCT/CN2016/075665
Authority: WO
Inventors: 罗炜; 贾鑫
Original assignee: 中兴通讯股份有限公司
Priority date: 2015-12-24
Filing date: 2016-03-04
Publication date: 2017-06-29
Also published as: CN106921882A

Abstract

A control method, control device, terminal, and synchronous audio playback system. The method comprises: first, identifying a multimedia object being played by a first terminal to obtain corresponding audio information and playback state information (S10); and then, controlling a second terminal to play, according to the audio information and the playback state information, a multimedia file corresponding to the audio information, so as to synchronously play the multimedia object with the first terminal (S20). By employing an audio identification technique to identify the multimedia object being played by the first terminal, and automatically controlling the second terminal to play a corresponding multimedia file, the control method, control device, terminal, and synchronous audio playback system can synchronize a start of multimedia playback, and achieve synchronous control during the playback process, thus realizing synchronous multimedia playback.

Description

Control method, control device, terminal and audio synchronous playback system

Technical field

The audio synchronous playback technology of the present invention particularly relates to a control method, a control device, a terminal and an audio synchronous playback system.

Background technique

With the development of smart terminals and the Internet, users are increasingly demanding multimedia services. For example, a user may wish to use multiple smart terminals to simultaneously play the same music for a surround sound effect. In addition, users who like music may also want to play the same music simultaneously through their respective smart terminals to achieve interactive and social purposes.

However, the existing smart terminal also needs to realize synchronous play of audio through manual operation, so that the user needs to operate accurately, for example, start playing music at the same time, so that the music starts playing at the same time. However, even in this case, it may be because there is a difference in the running speed between the various smart terminals, or an individual smart terminal encounters an event such as a crash restart, so that even if the music starts playing at the same time, the playing process cannot be strictly synchronized, thus affecting the audiovisual. effect.

Summary of the invention

Embodiments of the embodiments of the present invention aim to at least solve one of the technical problems existing in the prior art. To this end, embodiments of the present invention are required to provide a control method, a control device, a terminal, and an audio synchronous playback system.

The control method of the embodiment of the present invention includes the following steps:

The identifying step of identifying the multimedia played by the first terminal to obtain corresponding audio information and playing status information;

And a controlling step of controlling the second terminal to play the multimedia file corresponding to the audio information according to the audio information and the playing state information to play the multimedia synchronously with the first terminal.

In some embodiments, the first terminal or the second terminal comprises a mobile phone, a tablet or a wearable device.

In some embodiments, the identifying step comprises:

Extracting a sub-step, extracting feature values of the multimedia;

A matching sub-step performs feature matching according to the feature value and the feature library to identify the multimedia.

In some embodiments, the feature value comprises an acoustic feature or a linguistic feature,

The acoustic features include melody features or scale features.

The multimedia is a song and the linguistic features include lyrics.

In some embodiments, the feature library is stored at the second terminal or remote server.

In some embodiments, the identifying step comprises:

Taking the sub-step, the multimedia played by the first terminal is recorded as a recording segment;

The extracting substep extracts the feature value by analyzing the recorded segment.

In some embodiments, the identifying step comprises:

The sub-step is determined to determine whether the multimedia is successfully identified, and if not, the recording sub-step is returned.

In some embodiments, the multimedia is a song, and the audio information includes a song title, a lyrics, an author, a singer, or a genre.

In some embodiments, the play status information includes a current play position, whether to pause, whether to fast forward, or whether to rewind.

In some embodiments, the controlling step comprises:

Obtaining a sub-step of acquiring the multimedia file according to the audio information;

a first control sub-step, the second terminal is configured to configure a play parameter according to the audio information, and start playing the multimedia file according to the play parameter and the play status information;

The second control sub-step performs synchronous play control during playback.

In some embodiments, the multimedia file is stored at the second terminal or stored at a remote server.

In some embodiments, the second control sub-step includes:

Intermittently closing the speaker of the second terminal;

Identifying multimedia played by the first terminal; and

Determining whether the multimedia played by the first terminal and the second terminal is synchronized, and if not, returning to the identifying step.

In some embodiments, the second control sub-step comprises the following grandchild steps:

Receiving multimedia played by the first terminal and the second terminal;

Identifying multimedia played by the first terminal after filtering the multimedia played by the second terminal; and

Receiving multimedia played by the first terminal and the second terminal;

Comparing the relevance of the multimedia played by the first terminal and the second terminal; and

Determining, according to the correlation, whether the multimedia played by the first terminal and the second terminal is synchronized, and if not, returning to the identifying step.

In some embodiments, the correlation includes the linearity of the speech spectrum or the number of identical syllables within a predetermined time.

In some embodiments, the multimedia is a song, the play status information includes a play position of the lyrics, and the second control sub-step includes the following grand steps:

Receiving multimedia played by the first terminal and the second terminal;

Identifying multimedia played by the first terminal and the second terminal to obtain a corresponding playing position of the lyrics;

Determining, according to the playing position of the lyrics, whether the multimedia played by the first terminal and the second terminal is synchronized, and if not, returning to the identifying step.

In some embodiments, the controlling method further includes:

It is judged whether a request for synchronous play is received, and if so, the identification step is entered.

The control device of the embodiment of the present invention includes:

An identification module, configured to identify multimedia played by the first terminal to obtain corresponding audio information and play status information;

a control module, the control module is configured to control the second terminal to play the multimedia file corresponding to the audio information according to the audio information and the play status information to play the multimedia synchronously with the first terminal.

In some embodiments, the identification module comprises:

An extraction module, configured to extract feature values of the multimedia;

a matching module, the matching module being configured to perform feature matching according to the feature value and the feature library to identify the multimedia.

The acoustic features include melody features or scale features.

The multimedia is a song and the linguistic features include lyrics.

In some embodiments, the identification module comprises:

An admission module, the admission module is configured to record the multimedia played by the first terminal as a recording segment;

The extraction module is configured to extract the feature value by analyzing the recorded segment.

In some embodiments, the identification module comprises:

a judging module, the judging module being configured to determine whether the multimedia is successfully identified and notifying the admission module when the multimedia is not successfully identified.

In some embodiments, the control module comprises:

Obtaining a module, the extraction module being configured to acquire the multimedia file according to the audio information;

a first sub-control module, the first sub-control module is configured to control the second terminal to configure a play parameter according to the audio information, and start playing the multimedia file according to the play parameter and the play status information;

a second sub-control module, the second sub-control module being configured to perform synchronous play control during playback.

In some embodiments, the second sub-control module is configured to intermittently turn off the speaker of the second terminal, identify the multimedia played by the first terminal, and determine that the first terminal and the second terminal play Whether the multimedia is synchronized and notifies the identification module when it is not synchronized.

In some embodiments, the second sub-control module is configured to: capture the multimedia played by the first terminal and the second terminal, filter the multimedia played by the second terminal, and identify the first terminal to play And determining whether the multimedia played by the first terminal and the second terminal is synchronized and notifying the identification module when not synchronized.

In some embodiments, the second sub-control module is configured to record the multimedia played by the first terminal and the second terminal, and compare the relevance of the multimedia played by the first terminal and the second terminal. And determining, according to the correlation, whether the multimedia played by the first terminal and the second terminal is synchronized, and notifying the identification module when not synchronized.

In some embodiments, the multimedia is a song, the play status information includes a play position of the lyrics, and the second sub-control module is configured to: capture the multimedia played by the first terminal and the second terminal, and identify The multimedia played by the first terminal and the second terminal obtains a play position of the lyrics and determines, according to the play position of the lyrics, whether the multimedia played by the first terminal and the second terminal is synchronized and not The identification module is notified when synchronizing.

In some embodiments, the control device comprises:

The request receiving module is configured to determine whether a request for synchronous play is received and notify the identification module when the request is received.

A terminal of an embodiment of the present invention includes the control device.

The terminal of the audio synchronization playing system of the embodiment of the present invention.

In the embodiment of the present invention, a computer storage medium is further provided, and the computer storage medium may store an execution instruction for executing the playback control method in the foregoing embodiment.

The control method, the control device, the terminal, and the audio synchronous playback system of the embodiment of the present invention use the voice recognition technology to identify the multimedia played by the first terminal, and automatically control the second terminal to play the corresponding multimedia file, so that The synchronization of the multimedia start playback can be realized, and the synchronization control can also be realized during the playback process, so that the multimedia synchronous playback can be realized.

In this way, the second terminal can be associated with the first terminal to form a network, and can change the situation that the past terminals are independently controlled or can only transmit files through the local area network to realize synchronous control of multimedia playback, thereby achieving better performance. The sound effects, such as the surround sound effect in the family, and the second, can achieve interactive, social purposes.

The additional aspects and advantages of the embodiments of the present invention will be set forth in part in the description which follows.

DRAWINGS

The above and/or additional aspects and advantages of the embodiments of the present invention will become apparent and readily understood from the following description

FIG. 1 is a schematic flow chart of a control method according to an embodiment of the present invention.

2 is a schematic diagram of functional modules of an audio synchronous playback system according to an embodiment of the present invention.

3 is a functional block diagram of an audio synchronous playback system in accordance with some embodiments of the present invention.

4 is a flow chart of a control method of some embodiments of the present invention.

FIG. 5 is a schematic diagram of functional modules of an audio synchronous playback system according to some embodiments of the present invention.

6 is a functional block diagram of an audio synchronous playback system in accordance with some embodiments of the present invention.

7 is a flow chart of a control method of some embodiments of the present invention.

FIG. 8 is a schematic diagram of functional modules of an audio synchronous playback system according to some embodiments of the present invention.

9 is a flow chart of a control method of some embodiments of the present invention.

FIG. 10 is a schematic diagram of functional modules of an audio synchronous playback system according to some embodiments of the present invention.

11-15 are schematic flow diagrams of a control method of some embodiments of the present invention.

16 is a functional block diagram of an audio synchronous playback system in accordance with some embodiments of the present invention.

detailed description

The embodiments of the present invention are described in detail below, and the examples of the embodiments are illustrated in the drawings, wherein the same or similar reference numerals are used to refer to the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the drawings are intended to be illustrative of the embodiments of the invention and are not to be construed as limiting.

Referring to FIG. 1, a control method of an embodiment of the present invention includes the following steps:

S10: Identify the multimedia played by the first terminal to obtain corresponding audio information and play status information; and

S20: Control the second terminal to play the multimedia file corresponding to the audio information according to the audio information and the playing state information to play the multimedia synchronously with the first terminal.

Referring to FIG. 2 , the control device 100 of the embodiment of the present invention includes an identification module 10 and a control module 20 . As an example, the control method of the embodiment of the present invention can be implemented by the control device 100 of the embodiment of the present invention, and can be applied to the audio synchronous playback system 1000. The audio synchronous playback system 1000 can include a control device 100, a first terminal 200, and a second terminal 300.

The step S10 of the control method of the embodiment of the present invention may be implemented by the identification module 10, and the step S20 may be implemented by the control module 20. That is to say, the identification module 10 is configured to recognize the multimedia played by the first terminal 200 to obtain audio information and play status information. The control module 20 is configured to control the second terminal 300 to play the multimedia file corresponding to the audio information according to the audio information and the playing state information to play the multimedia synchronously with the first terminal 200.

The control method, the control device 100 and the audio synchronization playing system 1000 of the embodiment of the present invention use the voice recognition technology to identify the multimedia played by the first terminal 200, and automatically control the second terminal 300 to play the corresponding multimedia file, so that one can The synchronization of the multimedia start playback is realized, and the synchronization control can also be realized during the playback process, so that the multimedia synchronous playback can be realized.

In this way, the second terminal 300 can be associated with the first terminal 200 to form a network, and can change the situation that the previous terminals are independently controlled or can only transmit files through the local area network to realize the synchronous control of the multimedia play, thereby achieving better sound effects. For example, in a home setting, a three-dimensional sound effect can be achieved; secondly, an interactive and social purpose can be achieved.

In some embodiments, the first terminal 200 or the second terminal 300 may be a communication terminal, an entertainment terminal, or a smart terminal having a multimedia playing function, such as a mobile phone, a tablet computer, a wearable device, a computer, or the like. The wearable device can be a smart bracelet, a smart watch or smart glasses.

As such, the first terminal 200 or the second terminal 300 is a product commonly carried by people in modern society and has a real demand for multimedia synchronous playback functions.

In this embodiment, the number of the second terminals 300 is one.

Referring to FIG. 3, in the audio synchronous playback system 1000a of other embodiments of the present invention, the number of the second terminals 300 may be multiple, and is not limited to the above embodiments. In these embodiments, each of the second terminals 300 may apply the control device 100 of the embodiment of the present invention.

It should be noted that the types of the first terminal 200 and the second terminal 300 may be the same or different. For example, in the example of FIG. 2, the first terminal 200 may be a mobile phone, and the second terminal 300 is also a mobile phone, that is, the two mobile phones are associated to form an audio synchronous playback system 1000. In the example of FIG. 3, the first terminal 200 can be a computer, and the second terminal 300 has two, which can be a mobile phone and a tablet computer, respectively.

It can be understood that the control device 100 can be integrated in whole or in part in the second terminal 300. Of course, the control device 100 can also be set independently of the second terminal 300.

In an embodiment in which the control device 100 is all integrated in the second terminal 300, the second terminal 300 can be considered to include the control device 100. In these embodiments, the control device 100 may also share some or all of the components with the slave device 300. For example, the control device 100 is an application installed in the second terminal 300 and implements a corresponding function when the second terminal 300 operates.

In some embodiments, the first terminal 200 can also be applied with the control device 100, that is, the first terminal 200 can also be used as the second terminal 300 to follow other first terminals 200 for multimedia synchronous playback, or first. The role between the terminal 200 or the second terminal 300 can be switched as needed, and is not limited to any of the embodiments of the present invention.

Referring to FIG. 4, in some embodiments, step S10 includes sub-steps:

S12: extracting feature values of the multimedia; and

S14: Perform feature matching according to the feature value and the feature library to identify the multimedia.

Please refer to FIG. 5 , and in some embodiments, the identification module 10 includes an extraction module 12 and a matching module 14 . Step S12 can be implemented by the extraction module 12, and step S14 can be implemented by the matching module 14. Alternatively, the extraction module 12 is configured to extract feature values of the multimedia, and the matching module 14 is configured to perform feature matching based on the feature values and the feature library to identify the multimedia.

In some embodiments, the feature value comprises an acoustic feature or a linguistic feature. For example, the multimedia can be pure music or songs, and the like. For pure music, since there is no language information, it can be identified mainly by acoustic features, that is, the feature values are acoustic features, such as melody features or scale features.

For songs, since language information (such as lyrics) is included, it can be identified by linguistic features, such as by lyrics, that is, linguistic features include lyrics.

In other embodiments, if the multimedia is a song, the audio feature and the linguistic feature can be simultaneously used to simultaneously identify the multimedia.

It can be understood that the feature library should be established in advance, and various multimedia can be analyzed and then features can be established. The creation of a signature library can also be a machine learning process that builds and constantly updates the signature library by continuously training the machine.

Referring to FIG. 6, in some embodiments, the feature library is stored in the second terminal 300 or stored in the remote server 400.

It can be understood that if the feature library is stored in the second terminal 300, the speed of voice recognition can be improved. And if the feature inventory is stored in the remote server 400, the resources occupying the second terminal 300 can be reduced.

In an embodiment in which the feature store is stored in the server 400, the audio sync play system 1000b includes a server 400.

Referring to FIG. 7, in some embodiments, step S10 includes sub-steps:

S11: The multimedia played by the first terminal 200 is recorded as a recording segment, and in step S12, the feature value is extracted by analyzing the recorded segment.

In still other embodiments, step S10 further includes the substeps:

S15: It is judged whether the multimedia is successfully recognized, and if not, the process returns to step S11.

Referring to FIG. 8 , in some embodiments, the identification module 10 includes an admission module 11 and a determination module 15 . Step S11 can be implemented by the admission module 11, and step S15 can be implemented by the determination module 15. In other words, the admission module 11 is configured to record the multimedia played by the first terminal 200 as a recording segment, and the determining module 15 is configured to determine whether the multimedia is successfully recognized.

For example, the admission module 11 can set a predetermined sampling rate to record the multimedia played by the first terminal 200. The extraction module 12 uses a specific extraction algorithm to extract feature values of the recorded segments, such as melody features, scale features, or lyrics. The matching module 14 searches for audio information matching the feature value by performing a search comparison in the feature library after obtaining the feature value. If the audio information matching the feature value is found, the judging module 15 judges that the multimedia recognition is successful; otherwise, the judging module 15 judges that the multimedia recognition has failed.

In some cases, such as a large ambient noise, the recorded segment includes more noise, and the extraction module 12 may not be able to accurately extract the feature values, thus causing the matching module to fail to match. In these embodiments, the decision module 15 should notify the admission module 11 to re-admit the multimedia to obtain different audio segments, and the extraction module 12 re-extracts the feature values. This cycle directly identifies multimedia successfully.

In the embodiment of the present invention, a computer storage medium is further provided, and the computer storage medium may store an execution instruction for executing the method in the foregoing embodiment.

In some embodiments, the multimedia is a song and the audio information includes a song title, lyrics, author, singer, or genre.

The play status information includes the current play position, whether it is paused, whether it is fast forward, or whether it is fast backward.

For example, if the first terminal 200 plays the song "The Volunteer March", the audio information may include:

Song name: The March of the Volunteers;

Lyrics: Get up! People who don't want to be slaves! Make our flesh and blood into our new Great Wall! When the Chinese nation reached its most dangerous time, everyone was forced to make the final snoring. stand up! stand up! ! stand up! ! ! We are all in one heart, braving the enemy's artillery, and marching forward! Take the enemy's gunfire and move on! go ahead! go ahead! ! Into! ! !

Author: Tian Han lyricist, composer Nie Er;

Singer: Chorus; and

Genre: Revolutionary songs.

The play status information may include a play position, for example, the play position is 01 minutes 22 seconds (represented as 01:22).

Referring to FIG. 9, in some embodiments, step S20 includes the following sub-steps:

S22: Acquire a multimedia file according to the audio information;

S24: Control the second terminal to configure a play parameter according to the audio information, and start playing the multimedia file according to the play status information; and

S26: Synchronous playback control is performed during playback.

Referring to FIG. 10, in some embodiments, the control module of the embodiment of the present invention may include an acquisition module 22, a first sub-control module 24, and a second sub-control module 26.

Step S21 can be implemented by the acquisition module 22, step S24 can be implemented by the first sub-control module 24, and step S26 can be implemented by the second sub-control module 26. Alternatively, the acquisition module 22 is configured to retrieve the multimedia file based on the audio information. The first sub-control module 24 is configured to control the second terminal to configure the play parameters according to the audio information and start playing the multimedia file according to the play status information. The second sub-control module 26 is arranged to perform synchronized play control during playback.

In some embodiments, in step S22, in the case of obtaining audio information, the obtaining module 22 may control the second terminal 300 to search for and acquire the corresponding multimedia file, for example, in the case of obtaining the above audio information, the second control may be performed. The terminal 300 searches for and acquires a file of the song "The March of the Volunteers".

In some embodiments, the multimedia file can be stored in the second terminal 300 or stored with the remote server 400.

Therefore, the acquisition module 22 can control the second terminal 300 to perform a local search first. If the multimedia file is local, there is no need to download it, which can speed up. In the case where the multimedia file is not searched locally, the acquisition module 22 may control the second terminal 300 to search for the remote server 400. After the remote server 400 searches for the multimedia file, it can download to the second terminal 300, and can also play online. The multimedia file can reduce the resources of the second terminal 300 at the remote server 400.

In some embodiments, in step S24, setting the play parameters may be setting an equalizer, for example, the equalizer includes different play modes: classical music, rock music, heavy metal music, jazz music, romantic music, or country music. For the song "The Volunteer March" of the present embodiment, the playing mode of the equalizer can be set to classical music.

In addition, playing the multimedia file according to the playing status information may be based on the playing position. For example, in the above embodiment, the playing position is 01:22, and the second terminal 300 should be controlled to start playing from 01:22 when playing the multimedia file.

For some second terminals 300 that can simultaneously play lyrics, playing the multimedia file according to the playing state information should also play the lyrics at the same time.

In order to prevent the first terminal 200 from being stopped, fast forwarded or rewinded due to a device difference between the second terminal 300 and the first terminal 200, or due to a user operation or a malfunction or the like. Therefore, the control device 20 needs to perform synchronous control during the entire multimedia playback process to achieve true synchronous playback.

Thus, referring to Figure 11, in some embodiments, sub-step S26 includes a grandchild step:

S262: Turn off the speaker of the second terminal 300;

S264: voice recognition of the multimedia played by the first terminal 200; and

S266: Determine whether the multimedia played by the first terminal 200 and the second terminal 300 is synchronized, and if no, return to step S10.

The step S262 can be implemented by the second sub-control module 26 to control the second terminal 300. The grand step S264 can be implemented by the second sub-control module 26 to control the identification module 10, and the grand step S266 can also be implemented by the second sub-control module 26. .

It can be understood that the above steps S262-S266 can be continuously performed to ensure the synchronization control of the multimedia playback. However, this may require more resources, and may affect the multimedia playback of the second terminal 300. Therefore, in this embodiment, Steps S262-266 are continued at predetermined time intervals, for example every 10 seconds.

In the step S266, the determination may be based on the comparison of the playback position, for example, the playback position of the recognized multimedia is 01:56, and the first sub-control module 24 may obtain the playback position of the multimedia played by the second terminal 300, if 01:56, you can judge the synchronous play, and if it is 02:01, you can judge that it is not synchronized.

When it is determined that the synchronous play is performed, the second terminal 300 can continue to play the multimedia synchronously with the first terminal 200, and in the case of determining that the synchronous play is not synchronized, it should return to the control step of re-synchronizing the playback.

Of course, the implementation of the synchronization control is not limited to the above embodiment.

Referring to Figure 12, in some embodiments, sub-step S26 includes the following grand steps:

S262a: accepting multimedia played by the first terminal 200 and the second terminal 300;

S264a: After filtering the multimedia played by the second terminal 300, identifying the multimedia played by the first terminal 200; and

S266a: It is determined whether the multimedia played by the first terminal 200 and the second terminal 300 is synchronized. If not, the process returns to step S10.

The grand step S262a can be implemented by the second sub-control module 26 to control the second terminal 300, the grand step S264a can be controlled by the second sub-control module 26 to control the identification module 10, and the grandchild step S266a can also be implemented by the second sub-control module 26. .

The grandchild steps S262a-S266a are similar to steps S262-S266a, but the recorded segments in step S262a include the multimedia played by the first terminal 200 and the multimedia played by the second terminal 300. In step S264a, the multimedia played by the second terminal 300 can be obtained according to the first sub-control module 24, so as to be filtered out in the recorded segment, and the multimedia played by the first terminal 200 is obtained. Step S266a is substantially the same as step S266.

Referring to Figure 13, in some embodiments, sub-step S26 includes the following grand steps:

S262b: the multimedia played by the first terminal 200 and the second terminal 300;

S264b: comparing the relevance of the multimedia played by the first terminal 200 and the second terminal 300; and

S266b: Determine whether the multimedia played by the first terminal 200 and the second terminal 300 is synchronized according to the correlation degree, and if not, return to the control step S10.

The grand step S262b can be implemented by the second sub-control module 26 to control the second terminal 300, and the grand step S264b and the grandchild step S266b can be implemented by the second sub-control module 26.

In the grand step S264b, it can be judged that the sub-step realizes the judgment by comparing the linearity of the speech spectrum or the number of identical syllables within a preset time.

A syllable is the most natural unit of speech that can be felt by hearing. One or several phonemes are combined according to certain rules. A Chinese character in Chinese is a syllable. Each syllable consists of three parts: initial, final and tonal. A vowel phoneme in English can form a syllable, and a vowel phoneme and one or several consonant phonemes can also form a syllable. In the present embodiment, the determination rule of the syllable can be defined according to actual needs before the judgment.

In some embodiments, if the same syllable reaches a certain number within a preset time, it is considered that the multimedia played by the first terminal 200 is synchronized with the multimedia played by the second terminal 300.

Referring to FIG. 14, in some embodiments, sub-step S26 includes:

S262c: the multimedia played by the first terminal 200 and the second terminal 300;

S264c: identify the multimedia played by the first terminal 200 and the second terminal 300;

S266c: Determine, according to the playing state information, whether the multimedia played by the first terminal and the second terminal is synchronized, and if not, return to the identifying step.

The grand step S262c can be implemented by the second sub-control module 26 to control the second terminal 300, the grand step S264c can be controlled by the second sub-control module 26 to control the identification module 10, and the grandchild step S266c can also be implemented by the second sub-control module 26. .

In some embodiments, the playback position of the playback status information can be utilized for comparison. Additionally, in some embodiments, the playback position of the lyrics can also be utilized for comparison.

Referring to FIG. 15, in some embodiments, the control method further includes:

S00: It is determined whether a request for synchronous play is received, and if yes, the process proceeds to step S10.

Referring to FIG. 16, the audio synchronous playback system 1000d of some embodiments is substantially the same as the audio synchronous playback system 1000, except that the control device 100d of the audio synchronous playback system 1000d is different from the control device 100, and the control device further includes a request receiving module 30.

Step S00 can be implemented by the request receiving module 30, that is, the request receiving module 30 is set to determine whether a request for synchronous play is received. If yes, go to step S10, if not, return to continue judgment.

The

control device

100 or 100d of the embodiment of the present invention and the other portions of the audio synchronization playback system 1000-100d that are not deployed may be referred to the corresponding portions of the control method of the above embodiment, and will not be developed in detail herein.

In the description of the embodiments of the present invention, it is to be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "previous" "," "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", etc. The orientation or positional relationship is based on the orientation or positional relationship shown in the drawings, and is merely for the convenience of describing the embodiments and the simplified description of the present invention, and does not indicate or imply that the device or component referred to has a specific orientation, The orientation configuration and operation are therefore not to be construed as limiting the embodiments of the invention. Moreover, the terms "first" and "second" are used for descriptive purposes only and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated. Thus, features defining "first" or "second" may include one or more of the described features either explicitly or implicitly. In the description of the embodiments of the present invention, the meaning of "a plurality" is two or more unless specifically defined otherwise.

In the description of the embodiments of the present invention, it should be noted that the terms "installation", "connected", and "connected" should be understood broadly, and may be a fixed connection, for example, or They are detachable or integrally connected; they can be mechanically connected, they can be electrically connected or can communicate with each other; they can be connected directly or indirectly through an intermediate medium, which can be internal or two components of two components. Interaction relationship. For those skilled in the art, the specific meanings of the above terms in the embodiments of the present invention can be understood on a case-by-case basis.

In the embodiments of the present invention, the "on" or "below" of the second feature may include direct contact of the first and second features, and may also include the first sum, unless otherwise specifically defined and defined. The second feature is not in direct contact but through additional features between them. Moreover, the first feature "above", "above" and "above" the second feature includes the first feature directly above and above the second feature, or merely indicating that the first feature level is higher than the second feature. The first feature "below", "below" and "below" the second feature includes the first feature directly above and above the second feature, or merely the first feature level being less than the second feature.

The following disclosure provides many different embodiments or examples for implementing different structures of embodiments of the present invention. In order to simplify the disclosure of embodiments of the present invention, the components and arrangements of specific examples are described below. Of course, they are merely examples and are not intended to limit the invention. In addition, the embodiments of the present invention may repeat reference numerals and/or reference letters in different examples, which are for the purpose of simplicity and clarity, and do not in themselves indicate the relationship between the various embodiments and/or arrangements discussed. . Moreover, embodiments of the present invention provide examples of various specific processes and materials, but one of ordinary skill in the art will recognize the use of other processes and/or the use of other materials.

In the description of the present specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiment", "example", "specific example" or "some examples", etc. Particular features, structures, materials or features described in the manner or examples are included in at least one embodiment or example of the invention. In the present specification, the schematic representation of the above terms does not necessarily mean the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in a suitable manner in any one or more embodiments or examples.

Any process or method description in the flowcharts or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a particular logical function or process. And The scope of the preferred embodiments of the present invention includes additional implementations in which the functions may be performed in a substantially simultaneous manner or in an opposite order depending on the functions involved, in the order shown or discussed. Embodiments of the invention will be understood by those skilled in the art.

The logic and/or steps represented in the flowchart or otherwise described herein, for example, may be considered as an ordered list of executable instructions for implementing logical functions, and may be embodied in any computer readable medium, Used in conjunction with, or in conjunction with, an instruction execution system, apparatus, or device (eg, a computer-based system, a system including a processing module, or other system that can fetch instructions and execute instructions from an instruction execution system, apparatus, or device) Or use with equipment. For the purposes of this specification, a "computer-readable medium" can be any apparatus that can contain, store, communicate, propagate, or transport a program for use in an instruction execution system, apparatus, or device, or in conjunction with the instruction execution system, apparatus, or device. More specific examples (non-exhaustive list) of computer readable media include the following: electrical connections (electronic devices) having one or more wires, portable computer disk cartridges (magnetic devices), random access memory (RAM), Read only memory (ROM), erasable editable read only memory (EPROM or flash memory), fiber optic devices, and portable compact disk read only memory (CDROM). In addition, the computer readable medium may even be a paper or other suitable medium on which the program can be printed, as it may be optically scanned, for example by paper or other medium, followed by editing, interpretation or, if appropriate, other suitable The method is processed to obtain the program electronically and then stored in computer memory.

It should be understood that portions of the embodiments of the invention may be implemented in hardware, software, firmware or a combination thereof. In the above-described embodiments, multiple steps or methods may be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented by any one or combination of the following techniques well known in the art: having logic gates for implementing logic functions on data signals. Discrete logic circuits, application specific integrated circuits with suitable combinational logic gates, programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), etc.

One of ordinary skill in the art can understand that all or part of the steps carried by the method of implementing the above embodiments can be completed by a program to instruct related hardware, and the program can be stored in a computer readable storage medium. When executed, one or a combination of the steps of the method embodiments is included.

In addition, each functional unit in each embodiment of the present invention may be integrated into one processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above integrated modules can be implemented in the form of hardware or in the form of software functional modules. The integrated modules, if implemented in the form of software functional modules and sold or used as stand-alone products, may also be stored in a computer readable storage medium.

The above mentioned storage medium may be a read only memory, a magnetic disk or an optical disk or the like.

Although the embodiments of the present invention have been shown and described, it is understood that the above-described embodiments are illustrative and are not to be construed as limiting the scope of the invention. The embodiments are subject to variations, modifications, substitutions and variations.

Industrial applicability

According to the foregoing technical solution provided by the embodiment of the present invention, the voice recognition technology is used to identify the multimedia played by the first terminal, and the second terminal is automatically controlled to play the corresponding multimedia file, so that the multimedia can be started to play. Synchronization, and synchronization control can also be realized during playback, so that multimedia synchronization can be realized.

Claims

A control method comprising:

The identifying step of identifying the multimedia played by the first terminal to obtain corresponding audio information and playing status information;

And a controlling step of controlling the second terminal to play the multimedia file corresponding to the audio information according to the audio information and the playing state information to play the multimedia synchronously with the first terminal.
The control method according to claim 1, wherein the first terminal or the second terminal comprises a mobile phone, a tablet or a wearable device.
The control method according to claim 1, wherein said identifying step comprises:

Extracting a sub-step, extracting feature values of the multimedia;

A matching sub-step performs feature matching according to the feature value and the feature library to identify the multimedia.
The control method according to claim 3, wherein said feature value comprises an acoustic feature or a linguistic feature, said acoustic feature comprises a melody feature or a scale feature, said multimedia is a song, and said linguistic feature comprises lyrics.
The control method according to claim 3, wherein said feature library is stored in said second terminal or a remote server.
The control method according to claim 3, wherein said identifying step comprises:

Taking the sub-step, the multimedia played by the first terminal is recorded as a recording segment;

The extracting substep extracts the feature value by analyzing the recorded segment.
The control method according to claim 6, wherein said identifying step comprises:

The sub-step is determined to determine whether the multimedia is successfully identified, and if not, the recording sub-step is returned.
The control method according to claim 1, wherein said multimedia is a song, and said audio information includes a song name, a lyrics, an author, a singer, or a genre.
The control method according to claim 1, wherein the play status information includes a current play position, whether to pause, whether to fast forward, or whether to rewind.
The control method according to claim 1, wherein said controlling step comprises:

Obtaining a sub-step of acquiring the multimedia file according to the audio information;

a first control sub-step, the second terminal is configured to configure a play parameter according to the audio information, and start playing the multimedia file according to the play parameter and the play status information;

The second control sub-step performs synchronous play control during playback.
The control method according to claim 10, wherein said multimedia file is stored in said second terminal or stored in a remote server.
The control method according to claim 10, wherein said second control sub-step comprises:

Intermittently closing the speaker of the second terminal;

Identifying multimedia played by the first terminal; and

Determining whether the multimedia played by the first terminal and the second terminal is synchronized, and if not, returning to the identifying step.
The control method according to claim 10, wherein said second control substep comprises the following grandchild steps:

Receiving multimedia played by the first terminal and the second terminal;

Identifying multimedia played by the first terminal after filtering the multimedia played by the second terminal; and

Determining whether the multimedia played by the first terminal and the second terminal is synchronized, and if not, returning to the identifying step.
The control method according to claim 10, wherein said second control substep comprises the following grandchild steps:

Receiving multimedia played by the first terminal and the second terminal;

Comparing the relevance of the multimedia played by the first terminal and the second terminal; and

Determining, according to the correlation, whether the multimedia played by the first terminal and the second terminal is synchronized, and if not, returning to the identifying step.
The control method according to claim 14, wherein the correlation includes a linearity of the speech spectrum or the number of identical syllables within a preset time.
The control method according to claim 10, wherein said multimedia is a song, said play status information includes a play position of the lyrics, and said second control sub-step includes the following grand steps:

Receiving multimedia played by the first terminal and the second terminal;

Identifying multimedia played by the first terminal and the second terminal to obtain a corresponding playing position of the lyrics;

Determining, according to the playing position of the lyrics, whether the multimedia played by the first terminal and the second terminal is synchronized, and if not, returning to the identifying step.
The control method according to claim 1, wherein the control method further comprises:

It is judged whether a request for synchronous play is received, and if so, the identification step is entered.
A control device comprising:

An identification module, configured to identify multimedia played by the first terminal to obtain corresponding audio information and play status information;

a control module, the control module is configured to control the second terminal to play the multimedia file corresponding to the audio information according to the audio information and the play status information to play the multimedia synchronously with the first terminal.
The control device according to claim 18, wherein said first terminal or said second terminal comprises a mobile phone, a tablet or a wearable device.
The control device according to claim 18, wherein said identification module comprises:

An extraction module, configured to extract feature values of the multimedia;

a matching module, the matching module being configured to perform feature matching according to the feature value and the feature library to identify the multimedia.
The control device according to claim 20, wherein said feature value comprises an acoustic feature or a linguistic feature,

The acoustic features include melody features or scale features.

The multimedia is a song and the linguistic features include lyrics.
The control device according to claim 20, wherein said feature library is stored in said second terminal or remote server.
The control device of claim 20, wherein the identification module comprises:

An admission module, the admission module is configured to record the multimedia played by the first terminal as a recording segment;

The extraction module is configured to extract the feature value by analyzing the recorded segment.
The control device according to claim 23, wherein said identification module comprises:

a judging module, the judging module being configured to determine whether the multimedia is successfully identified and notifying the admission module when the multimedia is not successfully identified.
The control device according to claim 18, wherein said multimedia is a song, and said audio information includes a song name, a lyrics, an author, a singer or a genre.
The control device according to claim 18, wherein said play status information includes a current play position, whether to pause, whether to fast forward, or whether to rewind.
The control device of claim 18, wherein the control module comprises:

Obtaining a module, the extraction module being configured to acquire the multimedia file according to the audio information;

a first sub-control module, the first sub-control module is configured to control the second terminal to configure a play parameter according to the audio information, and start playing the multimedia file according to the play parameter and the play status information;

a second sub-control module, the second sub-control module being configured to perform synchronous play control during playback.
The control device according to claim 27, wherein said multimedia file is stored in said second terminal or stored in a remote server.
The control device according to claim 27, wherein said second sub-control module is arranged to intermittently turn off the speaker of said second terminal, identify multimedia played by said first terminal, and determine said first terminal and said Whether the multimedia played by the second terminal is synchronized and notifies the identification module when not synchronized.
The control device according to claim 27, wherein the second sub-control module is configured to: capture the multimedia played by the first terminal and the second terminal, filter the multimedia played by the second terminal, and identify the The multimedia played by the first terminal determines whether the multimedia played by the first terminal and the second terminal is synchronized and notifies the identification module when not synchronized.
The control device according to claim 27, wherein the second sub-control module is configured to listen to the multimedia played by the first terminal and the second terminal, compare the first terminal and the second terminal to play And determining, according to the correlation degree, whether the multimedia played by the first terminal and the second terminal is synchronized according to the correlation degree, and notifying the identification module when not synchronized.
The control device according to claim 31, wherein said correlation includes a linearity of the speech spectrum or the number of identical syllables within a preset time.
The control device according to claim 27, wherein said multimedia is a song, said play status information comprises a play position of the lyrics, and said second sub-control module is configured to take said first terminal and said second terminal Playing the multimedia, identifying the multimedia played by the first terminal and the second terminal to obtain a play position of the lyrics, and determining, according to the play position of the lyrics, the multimedia played by the first terminal and the second terminal Whether to synchronize and notify the identification module when it is not synchronized.
The control device according to claim 18, wherein said control device comprises:

The request receiving module is configured to determine whether a request for synchronous play is received and notify the identification module when the request is received.
A terminal comprising the control device according to any one of claims 18-34.
An audio synchronization playback system comprising the terminal of claim 35.
A computer storage medium storing execution instructions for performing the method of any one of claims 1 to 17.