CN106992971B - Interactive terminal switching method and device and interactive recording and broadcasting system - Google Patents
Interactive terminal switching method and device and interactive recording and broadcasting system Download PDFInfo
- Publication number
- CN106992971B CN106992971B CN201710135861.4A CN201710135861A CN106992971B CN 106992971 B CN106992971 B CN 106992971B CN 201710135861 A CN201710135861 A CN 201710135861A CN 106992971 B CN106992971 B CN 106992971B
- Authority
- CN
- China
- Prior art keywords
- meeting place
- interactive
- branch
- instruction
- main
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/40—Support for services or applications
- H04L65/403—Arrangements for multi-party communication, e.g. for conferences
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/3331—Query processing
- G06F16/334—Query execution
- G06F16/3344—Query execution using natural language analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/279—Recognition of textual entities
- G06F40/284—Lexical analysis, e.g. tokenisation or collocates
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/30—Semantic analysis
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
- G10L2015/223—Execution procedure of a spoken command
Abstract
The embodiment of the invention discloses an interactive terminal switching method, which is used for solving the problems that the switching mode of the existing interactive terminal is easy to make mistakes, the switching efficiency is low, and the interactive rhythm of the whole system is easy to be disturbed. The method provided by the embodiment of the invention comprises the following steps: detecting voice information of a main meeting place and/or branch meeting places which are interacted currently; carrying out voice recognition on the voice information obtained by detection to obtain a recognition result; extracting keywords from the recognition result to obtain instruction keywords; generating a terminal switching instruction according to the instruction keyword; and carrying out interactive terminal switching processing on the currently interactive main meeting place and/or branch meeting place according to the terminal switching instruction. The embodiment of the invention also provides an interactive terminal switching device and an interactive recording and broadcasting system.
Description
Technical Field
The invention relates to the technical field of video processing, in particular to an interactive terminal switching method and device and an interactive recording and playing system.
Background
In the field of recording and broadcasting, with the need of informatization and globalization development, a gradually mature audio and video interaction technology becomes a key for solving the problem of interactive recording and broadcasting. The interactive recording and broadcasting system can cross the space geographic position to realize multi-party communication and share high-quality resources.
As shown in fig. 1, the existing interactive recording and playing system needs a Multipoint Control Unit (MCU) to manage the interactive terminals in multiple branch conference rooms, only the video terminal with the MCU can be used as the main conference room, and other interactive terminals can only be used as branch conference rooms to access the main conference room. After the branch meeting places access the main meeting place, only one main meeting place and one branch meeting place interact in the interaction process, when another branch meeting place needs to interact with the main meeting place, a worker needs to manually input a switching signal in a background of the interactive recording and broadcasting system, and the branch meeting place currently participating in the interaction is switched into the other branch meeting place according to the switching signal.
However, the manual input of the switching signal is not only prone to error and low in switching efficiency, but also prone to disturb the interaction rhythm of the whole system, and greatly affects the use experience of the interactive recording and broadcasting system.
Disclosure of Invention
The embodiment of the invention provides an interactive terminal switching method and device and an interactive recording and broadcasting system, which can improve the switching efficiency of an interactive terminal, reduce the occurrence of error switching and avoid the influence of the switching process on the interactive rhythm of the interactive recording and broadcasting system.
The interactive terminal switching method provided by the embodiment of the invention is applied to an interactive recording and broadcasting system, wherein the interactive recording and broadcasting system comprises a plurality of interactive terminals, one interactive terminal in the plurality of interactive terminals is used as a main meeting place, and the other interactive terminals are used as branch meeting places;
the interactive terminal switching method comprises the following steps:
detecting voice information of a main meeting place and/or branch meeting places which are interacted currently;
carrying out voice recognition on the voice information obtained by detection to obtain a recognition result;
extracting keywords from the recognition result to obtain instruction keywords;
generating a terminal switching instruction according to the instruction keyword;
and carrying out interactive terminal switching processing on the currently interactive main meeting place and/or branch meeting place according to the terminal switching instruction.
Optionally, the detecting the voice information of the currently interactive main meeting place and/or branch meeting place includes:
acquiring first voice information in audio and video signals of a main meeting place and/or a branch meeting place which are interacted at present;
performing audio analysis on the obtained first voice information, and extracting second voice information of a designated user in the first voice information;
and determining the extracted second voice information as the detected voice information.
Optionally, the generating a terminal switching instruction according to the instruction keyword includes:
performing semantic recognition on the instruction keywords to obtain a semantic recognition result;
generating a terminal switching instruction according to the semantic recognition result;
or
Matching the instruction keywords with a preset keyword template;
and if the instruction keyword is successfully matched with the preset keyword template, generating a terminal switching instruction corresponding to the successfully matched keyword template.
Optionally, the performing, according to the terminal switching instruction, the switching processing of the interactive terminal on the currently interactive main meeting place and/or branch meeting place includes:
if the extracted instruction keyword comes from the currently interactive main meeting place and the instruction keyword comprises a unique mark of a non-interactive branch meeting place, switching the branch meeting place corresponding to the unique mark into the currently interactive branch meeting place, or switching the branch meeting place corresponding to the unique mark into the currently interactive main meeting place;
if the instruction keywords comprise a first instruction keyword and a second instruction keyword, the first instruction keyword is from the currently interactive branch meeting place, the second instruction keyword is from the currently interactive main meeting place, the first instruction keyword comprises a unique mark of a non-interactive branch meeting place, and the second instruction keyword comprises a keyword representing a determined meaning, the branch meeting place corresponding to the unique mark is switched to the currently interactive branch meeting place, or the branch meeting place corresponding to the unique mark is switched to the currently interactive main meeting place;
if the extracted instruction keywords come from the currently interactive main meeting place and the instruction keywords comprise the unique mark of the currently interactive branch meeting place, interchanging the currently interactive main meeting place and the currently interactive branch meeting place;
the non-interactive branch and meeting place refers to other branch and meeting places except the current interactive branch and meeting place.
Optionally, the unique identifier of the meeting place is determined by the following steps:
acquiring access time points of each branch meeting place accessed to the main meeting place;
determining the access sequence of each branch meeting place according to the access time point;
and setting the unique mark of each branch meeting place according to the access sequence.
The interactive terminal switching device provided by the embodiment of the invention is applied to an interactive recording and broadcasting system, the interactive recording and broadcasting system comprises a plurality of interactive terminals, one interactive terminal in the plurality of interactive terminals is used as a main meeting place, and other interactive terminals are used as branch meeting places;
the interactive terminal switching device comprises:
the voice detection module is used for detecting voice information of the current interactive main meeting place and/or branch meeting place;
the voice recognition module is used for carrying out voice recognition on the voice information obtained by detection to obtain a recognition result;
the keyword extraction module is used for extracting keywords from the identification result to obtain instruction keywords;
the switching instruction generating module is used for generating a terminal switching instruction according to the instruction keyword;
and the switching processing module is used for switching the interactive terminal to the currently interactive main meeting place and/or branch meeting place according to the terminal switching instruction.
Optionally, the voice detection module includes:
the first voice information acquisition unit is used for acquiring first voice information in audio and video signals of a main meeting place and/or a branch meeting place which are interacted currently;
the second voice information extraction unit is used for carrying out audio analysis on the obtained first voice information and extracting second voice information of a specified user in the first voice information;
and the detected voice information determining unit is used for determining the extracted second voice information as the detected voice information.
Optionally, the switching instruction generating module includes:
the semantic recognition unit is used for performing semantic recognition on the instruction keywords to obtain a semantic recognition result;
the first instruction generating unit is used for generating a terminal switching instruction according to the semantic recognition result;
or
The keyword matching unit is used for matching the instruction keywords with a preset keyword template;
and the second instruction generating unit is used for generating a terminal switching instruction corresponding to the keyword template which is successfully matched if the instruction keyword is successfully matched with the preset keyword template.
Optionally, the handover processing module includes:
the first switching unit is used for switching the branch meeting place corresponding to the unique mark to the current interactive branch meeting place or switching the branch meeting place corresponding to the unique mark to the current interactive main meeting place if the extracted instruction keyword comes from the current interactive main meeting place and the instruction keyword comprises a unique mark of a non-interactive branch meeting place;
a second switching unit, configured to switch the meeting place corresponding to the unique identifier to the currently interactive meeting place or switch the meeting place corresponding to the unique identifier to the currently interactive main meeting place if the instruction keyword includes a first instruction keyword and a second instruction keyword, where the first instruction keyword is from the currently interactive meeting place, the second instruction keyword is from the currently interactive main meeting place, the first instruction keyword includes a unique identifier of a non-interactive meeting place, and the second instruction keyword includes a keyword representing a determined meaning;
the third switching unit is used for interchanging the currently interactive main meeting place and the currently interactive branch meeting place if the extracted instruction keyword comes from the currently interactive main meeting place and the instruction keyword comprises the unique mark of the currently interactive branch meeting place;
the non-interactive branch and meeting place refers to other branch and meeting places except the current interactive branch and meeting place.
The interactive recording and broadcasting system provided by the embodiment of the invention comprises a plurality of interactive terminals, wherein one interactive terminal in the plurality of interactive terminals is used as a main meeting place, and other interactive terminals are used as branch meeting places;
the interactive recording and broadcasting system further comprises the interactive terminal switching device.
According to the technical scheme, the embodiment of the invention has the following advantages:
in the embodiment of the invention, firstly, the voice information of the main meeting place and/or the branch meeting place which are interacted at present is detected; then, carrying out voice recognition on the voice information obtained by detection to obtain a recognition result; then, extracting keywords from the recognition result to obtain instruction keywords; generating a terminal switching instruction according to the instruction keyword; and finally, carrying out interactive terminal switching processing on the currently interactive main meeting place and/or branch meeting place according to the terminal switching instruction. In the embodiment of the invention, the terminal switching instruction can be automatically generated according to the voice information of the current interactive main meeting place and/or branch meeting place, the switching of the interactive terminals can be realized without manually inputting the switching information, the switching efficiency of the interactive terminals is improved, the occurrence of wrong switching conditions is reduced, and the interactive rhythm of the interactive recording and broadcasting system cannot be influenced because the terminal switching instruction is originated from the current interactive main meeting place and/or branch meeting place, so that the use experience of the interactive recording and broadcasting system is greatly improved.
Drawings
Fig. 1 is a schematic diagram of a conventional interactive recording and playing system;
fig. 2 is a schematic diagram of an interactive recording and playing system according to an embodiment of the present invention;
fig. 3 is a flowchart of an embodiment of a method for switching an interactive terminal according to the embodiment of the present invention;
fig. 4 is a flowchart illustrating a specific step 301 of an interactive terminal switching method according to an embodiment of the present invention;
fig. 5a is a schematic diagram of an interactive picture layout of a conventional interactive recording and playing system in an application scene;
FIG. 5b is a schematic diagram of a screen layout for switching to a main call screen when the interactive recording and playing system of FIG. 5a requires interaction;
fig. 6 is a schematic diagram of an interactive picture layout of an interactive recording and playing system according to an embodiment of the present invention;
fig. 7 is a flowchart illustrating a step of determining a unique identifier of a branch venue according to a method for switching an interactive terminal in an embodiment of the present invention;
fig. 8 is a structural diagram of an embodiment of an interactive terminal switching device according to an embodiment of the present invention.
Detailed Description
The embodiment of the invention provides an interactive terminal switching method and device and an interactive recording and broadcasting system, which are used for solving the problems that the switching mode of the existing interactive terminal is easy to make mistakes, the switching efficiency is low, and the interactive rhythm of the whole system is easy to be disturbed.
In order to make the objects, features and advantages of the present invention more obvious and understandable, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is obvious that the embodiments described below are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Example one
The interactive terminal switching method of the embodiment can be applied to an interactive recording and broadcasting system, and the interactive recording and broadcasting system comprises a plurality of interactive terminals. As shown in fig. 2, any one of the interactive terminals on the interactive recording and playing system can be used as a main meeting place or a branch meeting place, and after the interactive recording and playing system is started, one of the interactive terminals can be determined as the main meeting place in a calling manner, and the other interactive terminals are used as the branch meeting places. It should be noted that at least one interactive terminal needs to be erected or installed in each conference place, and the interactive recording and broadcasting system collects and processes audio and video signals of the conference places through the interactive terminals. For convenience of description, the following description refers to switching the conference place, that is, switching the interactive terminal corresponding to the conference place accordingly.
Particularly, in the interactive recording and playing system, the interactive terminal can call any one of other interactive terminals. The calling refers to that the interactive terminal carries out primary network communication through the network address of the other interactive terminal, and after the other interactive terminal answers, the two interactive terminals are communicated, namely the two interactive terminals can carry out network communication. At this time, although the two interactive terminals have established the communication link, the two interactive terminals do not interact with each other, that is, there is no data interaction such as audio and video. By the method, after the interactive recording and broadcasting system is started, the interactive terminal serving as the main meeting place can actively call the interactive terminals of other branch meeting places, and communication links are established with the interactive terminals of the branch meeting places one by one. In addition, each branch meeting place can also actively call the main meeting place to request access, and a communication link is established with the main meeting place.
When each branch meeting place calls the main meeting place, the main meeting place can select the mode of accessing the branch meeting place. The access of the main conference place to the conference place can be selected to refuse the access or the video signal (data) of the conference place is switched into the interactive picture layout. If the main meeting place does not execute rejection operation or cut-in operation, the branch meeting place can only receive audio and video signals (data) of the main meeting place and cannot interactively communicate with the main meeting place; if the main meeting place executes cut-in operation, the branch meeting place can not only receive the audio and video signals of the main meeting place, but also can carry out interactive communication with the main meeting place, and the video picture of the branch meeting place can also appear in the interactive picture layout. Regarding the interactive screen layout, it will be described in the following.
Furthermore, communication links between every two branch meeting places on the interactive recording and broadcasting system can be pre-established, so that when any branch meeting place is switched to the main meeting place, the interactive terminal switched to the main meeting place can be in real time audio and video data connection or data interaction with all the branch meeting places.
Referring to fig. 3, a method for switching an interactive terminal in this embodiment includes:
301. detecting voice information of a main meeting place and/or branch meeting places which are interacted currently;
in this embodiment, in the interactive recording and playing system, only one main meeting place and one branch meeting place which are currently interacted with each other can perform audio and video interaction, and other branch meeting places generally can only receive audio and video signals processed by the main meeting place but cannot interact with the main meeting place. For the main meeting place, the main meeting place can receive the audio and video signals of all the accessed branch meeting places, but the main meeting place can only interact with one branch meeting place at the same time.
In the actual use process of the interactive recording and broadcasting system, the main meeting place generally interacts with more than one branch meeting place, so that the branch meeting places need to be switched quickly and accurately. In this embodiment, the voice information of the currently interactive main meeting place and/or branch meeting place is first detected, that is, only the voice information of the currently interactive main meeting place or only the voice information of the currently interactive branch meeting place may be detected as needed, or both the voice information of the currently interactive main meeting place and the voice information of the currently interactive branch meeting place may be detected.
The voice information of the main meeting place and/or the branch meeting place can be obtained by separating and extracting the audio and video signals interacted between the main meeting place and the branch meeting place. Further, as shown in fig. 4, the step 301 may include:
401. acquiring first voice information in audio and video signals of a main meeting place and/or a branch meeting place which are interacted at present;
402. performing audio analysis on the obtained first voice information, and extracting second voice information of a designated user in the first voice information;
403. and determining the extracted second voice information as the detected voice information.
For the step 401, first voice information in the audio/video signals of the main meeting place and/or the branch meeting place is obtained. Generally speaking, the speech information herein may include all speech signals of the main meeting place and/or the branch meeting place, and if more than two users are speaking at the same time in the meeting place, the speech signal currently speaking by the more than two users is included in the first speech information.
For the above step 402, when confronted with the mixed first voice information, in order to extract the desired voice information from the first voice information, that is, the voice information of the specified user, it is necessary to perform audio analysis on the first voice information, and extract the voice information conforming to the specific frequency from the first voice information as the second voice information.
It is understood that in the main meeting place and/or the branch meeting place, not all the users speaking have the authority to switch the interactive terminal. For example, in a lecture main conference place, only a main teacher generally has the authority to switch between an interactive terminal and a branch conference place, but students who attend a lecture do not have the authority, and even if the students yell "switch to the branch conference place 2", the system should not convert the lecture main teacher into a corresponding switching instruction for execution. Therefore, in step 402, the designated user is a preset user with the authority to switch the interactive terminal, and the second voice information belonging to the user is extracted from the first voice in an audio analysis manner.
For the above step 403, after the second voice information of the authorized user is extracted, the second voice information may be determined as the detected voice information, that is, the required voice information.
302. Carrying out voice recognition on the voice information obtained by detection to obtain a recognition result;
in this embodiment, after the voice information is detected, voice recognition is performed on the detected voice information to obtain a recognition result. For example, in an application scenario, the recognition result may be "switch to conference room 1", or "switch the main conference room to conference room 2", and so on.
303. Extracting keywords from the recognition result to obtain instruction keywords;
after the recognition result is obtained, keyword extraction may be performed on the recognition result to obtain an instruction keyword.
Before the interactive recording and broadcasting system is started, it can be preset which keywords need to be extracted, such as "switch", "branch meeting place", "main meeting place", and arabic numerals. In practical applications, the keywords to be extracted may be set according to practical situations, and are not described herein again.
304. Generating a terminal switching instruction according to the instruction keyword;
after the instruction keyword is obtained, a terminal switching instruction may be generated according to the instruction keyword. It can be understood that there are several instructions for switching the interactive terminal, for example, an instruction for switching the currently interactive branch meeting place to another branch meeting place; or, switching the current main meeting place to another branch meeting place; or, instructions for interchanging the main meeting place and the branch meeting place of the current interaction, and the like. For these instructions, the key conditions for these instructions may be preset. For example, if the command keyword includes "switch", "session", and an arabic number (assumed to be 2), a terminal switch command "switch the currently interactive session to session 2" is generated. Note: the above-mentioned "meeting place 2" is the name of a certain meeting place in an application scenario.
In this embodiment, a plurality of ways for generating a terminal switching instruction according to an instruction keyword are provided, and the following two ways are mainly described below:
first, instructions are generated by semantic recognition. The step 304 may specifically include: firstly, performing semantic recognition on the instruction keywords to obtain a semantic recognition result; and then, generating a terminal switching instruction according to the semantic recognition result. It can be understood that, through the semantic recognition technology, the system can be enabled to "understand" the whole meaning of the extracted instruction keyword, so that the terminal switching instruction meeting the user requirement is generated according to the meaning of the semantic recognition. In addition, in the embodiment, the keyword extraction is performed on the result of the voice recognition, and then the semantic meaning of the extracted instruction keyword is recognized by adopting the semantic recognition technology, so that the workload of the semantic recognition process can be greatly reduced, and the recognition efficiency of the semantic recognition is improved.
Second, the instructions are generated by way of keyword template matching. Then, the step 304 may specifically include: firstly, matching the instruction keywords with a preset keyword template; and if the instruction keyword is successfully matched with the preset keyword template, generating a terminal switching instruction corresponding to the successfully matched keyword template. Examples are as follows: assume that the extracted instruction keys include "switch", "branch meeting place", "2", and "main meeting place"; two keyword templates are preset, wherein the first keyword template comprises switching, a meeting place and any Arabic numeral; the second keyword template includes "interchange", "main meeting place", and "branch meeting place". After matching, it can be known that the extracted instruction keywords include all keywords of the first keyword template, and thus the extracted instruction keywords are successfully matched with the first keyword template. And then generating a terminal switching instruction corresponding to the first keyword template, wherein the terminal switching instruction can be an instruction of switching the currently interactive branch meeting place into a branch meeting place 2. It can be known that the corresponding relationship between the keyword templates and the terminal switching instruction is preset, and after the keyword templates are successfully matched, the corresponding relationship between the keyword templates and the terminal switching instruction is searched to generate the terminal switching instruction corresponding to the keyword templates.
305. And carrying out interactive terminal switching processing on the currently interactive main meeting place and/or branch meeting place according to the terminal switching instruction.
In this embodiment, different from the prior art, the interactive recording and playing system sets a specific interactive picture layout for the main meeting place and the branch meeting places. The interactive picture layout in the prior art is shown in fig. 5a and 5b, and after receiving the video streams of all the branch sites, the interactive terminal in the main site integrates the pictures of the main site and all the branch sites into one interactive picture, as shown in fig. 5 a. When the interaction is needed, the meeting place of the other party of the interaction is switched to the whole large screen, as shown in fig. 5 b.
In this embodiment, the interactive recording and playing system adopts a split-screen picture layout, as shown in fig. 6, the whole picture is divided into a main display area and three small display areas, which are respectively used for displaying a main speaking picture, a main meeting place picture 1, a main meeting place picture 2 and a picture of the currently interactive split meeting place. For example, in a teaching main meeting place, a main meeting place picture is a computer picture of the main meeting place, a main meeting place picture 1 and a main meeting place picture 2 are shot pictures of two different viewing angles of the main meeting place respectively, and a branch meeting place picture is a meeting place picture of a currently interactive branch meeting place. If the interactive recording and broadcasting system is started in the initial stage and the switching operation of the meeting places does not occur, the earliest accessed meeting place is defaulted as the current interactive meeting place.
In the process of interactive recording and playing, the interactive terminal of the main meeting place processes the locally received audio/video signals (audio/video signals of the main meeting place) and the audio/video signals of the currently interactive branch meeting places to form an interactive picture layout as shown in fig. 6, and transmits the processed audio/video signals to all branch meeting places for playing.
In the interactive recording and broadcasting system, regardless of the layout of the interactive picture or the switching of the interactive terminal/meeting place, each branch meeting place needs to be distinguished, so that each meeting place can be operated and managed in the interactive recording and broadcasting system. In this embodiment, the unique identifier of the branch venue can be determined through the following steps, please refer to fig. 7:
701. acquiring access time points of each branch meeting place accessed to the main meeting place;
702. determining the access sequence of each branch meeting place according to the access time point;
703. and setting the unique mark of each branch meeting place according to the access sequence.
For the above steps 701 to 703, for example, the meeting place which is accessed to the main meeting place at the earliest time may be named as branch meeting place 1, then branch meeting place 2, branch meeting place 3, … …, and so on, and each branch meeting place may be named according to the time when each branch meeting place accesses to the main meeting place, that is, each branch meeting place is marked with a unique mark.
For the above step 305, different terminal switching instructions, the switching processing for the currently interactive main meeting place and/or branch meeting place, are different.
For the handover process aspect, there are generally several cases: a. switching the current interactive main meeting place into another branch meeting place, and if the current interactive main meeting place is switched into the branch meeting place 2, the branch meeting place 2 becomes the main meeting place after switching; b. switching the currently interactive branch meeting place into another meeting place, and if the currently interactive branch meeting place is switched into a branch meeting place 3, the switched branch meeting place 3 becomes the currently interactive branch meeting place with the main meeting place; c. and exchanging the currently interactive main meeting place with the branch meeting places, wherein if the meeting place 1 is the main meeting place, the meeting place 2 is the currently interactive branch meeting place, and after the exchange, the meeting place 2 is the main meeting place, and the meeting place 1 is the currently interactive branch meeting place.
In the case of the three types of switching processing, the step 305 may specifically implement the following three processing modes by using different terminal switching commands:
the first processing mode is as follows: and if the extracted instruction keyword comes from the currently interactive main meeting place and the instruction keyword comprises a unique mark of a non-interactive branch meeting place, switching the branch meeting place corresponding to the unique mark into the currently interactive branch meeting place, or switching the branch meeting place corresponding to the unique mark into the currently interactive main meeting place. For the first processing mode, it can be understood that, in a general scenario, the main meeting place has the authority to switch the interactive terminal. For example, in a lecture meeting place for teaching, a meeting place where a teacher is located is a main meeting place, and at this time, if an instruction keyword comes from the teacher in the main meeting place and the instruction keyword includes a name (unique identifier) of another branch meeting place, the system may consider that the branch meeting place needs to be switched to the currently interactive branch meeting place or the main meeting place. Specifically, the branch meeting place corresponding to the unique identifier is switched to the currently interactive branch meeting place or the main meeting place, which depends on the terminal switching instruction. If the terminal switching instruction is an instruction aiming at the currently interactive branch and meeting place, switching to the branch and meeting place; otherwise, the conference is switched to the main conference place.
The second treatment method comprises the following steps: if the instruction keywords comprise a first instruction keyword and a second instruction keyword, the first instruction keyword is from the currently interactive branch meeting place, the second instruction keyword is from the currently interactive main meeting place, the first instruction keyword comprises a unique mark of a non-interactive branch meeting place, and the second instruction keyword comprises a keyword representing a determined meaning, the branch meeting place corresponding to the unique mark is switched to the currently interactive branch meeting place, or the branch meeting place corresponding to the unique mark is switched to the currently interactive main meeting place. For the second processing mode, because the main meeting place only has the authority to switch the interactive terminal in a general scene, if the first instruction keyword is from the currently interactive branch meeting place and includes a name (unique identifier) of another branch meeting place, it can be considered that a user in the currently interactive branch meeting place requests to switch to another branch meeting place, at this time, if the main meeting place agrees to switch, that is, the second instruction keyword includes a keyword indicating a determined meaning, the switching request is established, the branch meeting place corresponding to the unique identifier is switched to the currently interactive branch meeting place according to the terminal switching instruction, or the branch meeting place corresponding to the unique identifier is switched to the currently interactive main meeting place. And the specific switching to the currently interactive branch meeting place or the main meeting place depends on the terminal switching instruction. If the terminal switching instruction is an instruction aiming at the currently interactive branch and meeting place, switching to the branch and meeting place; otherwise, the conference is switched to the main conference place.
The third treatment method comprises the following steps: and if the extracted instruction keywords come from the currently interactive main meeting place and comprise the unique mark of the currently interactive branch meeting place, interchanging the currently interactive main meeting place and the currently interactive branch meeting place. For the third processing mode, it can be understood that, if the instruction keyword from the main meeting place includes the name of the currently interactive branch meeting place, at this time, it may be considered that the main meeting place wants to exchange with the branch meeting place.
In the above three processing modes, the non-interactive branch and meeting place refers to other branch and meeting places except the currently interactive branch and meeting place.
For the sake of understanding, the three processing modes will be illustrated below in three different application scenarios:
the first application scenario: the meeting place A is a main meeting place, and the meeting place B is a branch meeting place 1 which is interacted with the main meeting place currently. A teacher in the main meeting place says 'switching to the branch meeting place 2', wherein the branch meeting place 2 is a meeting place C, the branch meeting place 2 is switched to a branch meeting place which is interacted with the main meeting place at present, and the meeting place C is interacted with the meeting place A at the moment; if the teacher in the main meeting place says "switch the main meeting place to meeting place 2", the main meeting place is switched to meeting place C, and at this time, the meeting place C interacts with the meeting place B.
Second application scenario: the meeting place A is a main meeting place, and the meeting place B is a branch meeting place 1 which is interacted with the main meeting place currently. The students in the branch meeting place 1 say 'switch to the branch meeting place 2', the teacher in the main meeting place replies to say 'ok', then the branch meeting place 2 (namely meeting place C) is switched to the branch meeting place interacted with the main meeting place at present, and at the moment, the meeting place C interacts with the meeting place A; if the students in the branch meeting place 1 say that the main meeting place is switched to the branch meeting place 2, and the teachers in the main meeting place reply to say that the students can, the main meeting place is switched to a meeting place C, and at the moment, the meeting place C interacts with the meeting place B.
The third application scenario: the meeting place A is a main meeting place, and the meeting place B is a branch meeting place 1 which is interacted with the main meeting place currently. The teacher in the main meeting place says 'switching to the branch meeting place 1', the meeting place B is switched to the main meeting place, and the meeting place A is switched to the branch meeting place 1, so that the interchange of the main meeting place and the branch meeting place is realized.
As can be seen from the above, the interactive terminal switching method of the embodiment of the present invention has the following advantages:
1. the method comprises the steps that a terminal switching instruction is automatically generated according to voice information of a current interactive main meeting place and/or branch meeting place, switching of interactive terminals can be achieved without manually inputting switching information, switching efficiency of the interactive terminals is improved, and the occurrence of wrong switching conditions is reduced;
2. because the terminal switching instruction is from the current interactive main meeting place and/or branch meeting place, the interactive rhythm of the interactive recording and broadcasting system is not influenced, and the use experience of the interactive recording and broadcasting system is greatly improved.
Example two
The above mainly describes an interactive terminal switching method, and a detailed description will be given below of an interactive terminal switching device.
Fig. 8 is a structural diagram illustrating an embodiment of an interactive terminal switching device according to an embodiment of the present invention.
In this embodiment, an interactive terminal switching device is applied to an interactive recording and broadcasting system, where the interactive recording and broadcasting system includes a plurality of interactive terminals, one of the interactive terminals serves as a main meeting place, and the other interactive terminals serve as branch meeting places;
the interactive terminal switching device may include:
a voice detection module 801, configured to detect voice information of a current interactive main meeting place and/or branch meeting place;
a voice recognition module 802, configured to perform voice recognition on the detected voice information to obtain a recognition result;
a keyword extraction module 803, configured to perform keyword extraction on the recognition result to obtain an instruction keyword;
a switching instruction generating module 804, configured to generate a terminal switching instruction according to the instruction keyword;
and a switching processing module 805, configured to perform, according to the terminal switching instruction, interactive terminal switching processing on the currently interactive main meeting place and/or branch meeting place.
Further, the voice detection module may include:
the first voice information acquisition unit is used for acquiring first voice information in audio and video signals of a main meeting place and/or a branch meeting place which are interacted currently;
the second voice information extraction unit is used for carrying out audio analysis on the obtained first voice information and extracting second voice information of a specified user in the first voice information;
and the detected voice information determining unit is used for determining the extracted second voice information as the detected voice information.
Further, the switching instruction generating module may include:
the semantic recognition unit is used for performing semantic recognition on the instruction keywords to obtain a semantic recognition result;
the first instruction generating unit is used for generating a terminal switching instruction according to the semantic recognition result;
or
The keyword matching unit is used for matching the instruction keywords with a preset keyword template;
and the second instruction generating unit is used for generating a terminal switching instruction corresponding to the keyword template which is successfully matched if the instruction keyword is successfully matched with the preset keyword template.
Further, the handover processing module may include:
the first switching unit is used for switching the branch meeting place corresponding to the unique mark to the current interactive branch meeting place or switching the branch meeting place corresponding to the unique mark to the current interactive main meeting place if the extracted instruction keyword comes from the current interactive main meeting place and the instruction keyword comprises a unique mark of a non-interactive branch meeting place;
a second switching unit, configured to switch the meeting place corresponding to the unique identifier to the currently interactive meeting place or switch the meeting place corresponding to the unique identifier to the currently interactive main meeting place if the instruction keyword includes a first instruction keyword and a second instruction keyword, where the first instruction keyword is from the currently interactive meeting place, the second instruction keyword is from the currently interactive main meeting place, the first instruction keyword includes a unique identifier of a non-interactive meeting place, and the second instruction keyword includes a keyword representing a determined meaning;
the third switching unit is used for interchanging the currently interactive main meeting place and the currently interactive branch meeting place if the extracted instruction keyword comes from the currently interactive main meeting place and the instruction keyword comprises the unique mark of the currently interactive branch meeting place;
the non-interactive branch and meeting place refers to other branch and meeting places except the current interactive branch and meeting place.
Further, the unique identifier of the branch venue can be determined by the following modules:
an access time point obtaining module, configured to obtain an access time point at which each of the branch meeting places accesses the main meeting place;
an access sequence determining module, configured to determine an access sequence of each branch meeting place according to the access time point;
and the meeting place mark setting module is used for setting the unique mark of each branch meeting place according to the access sequence.
EXAMPLE III
The embodiment provides an interactive recording and broadcasting system, which comprises a plurality of interactive terminals, wherein one interactive terminal in the plurality of interactive terminals is used as a main meeting place, and the other interactive terminals are used as branch meeting places;
the interactive recording and playing system further comprises any one of the interactive terminal switching devices described in the second embodiment.
In addition, for convenience and simplicity of description, the interactive recording and playing system provided by the third embodiment may further include the technical features related to the interactive recording and playing system described in the first embodiment.
Therefore, the interactive recording and broadcasting system provided by the embodiment of the invention has the following advantages:
1. in the interactive recording and broadcasting system, any party of the interactive terminals can serve as a central controller to initiate multi-party audio and video interaction, the utilization rate of the interactive terminals is improved, and the rest branch meeting places are accessed into the interaction at any time through the IP of the main meeting place to receive audio and video signals of the main meeting place for interaction.
2. The audio and video signals of all the branch meeting place interaction terminals can be accessed into the main meeting place, but only the earliest branch meeting place of the access time or the branch meeting place of voice recognition in actual interaction can be switched in to form a one-to-one interaction layout, so that the interaction subjects of the main meeting place and the branch meeting place can be determined, and the interaction efficiency can be improved.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
The above-mentioned embodiments are only used for illustrating the technical solutions of the present invention, and not for limiting the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (1)
1. An interactive terminal switching method is characterized by being applied to an interactive recording and broadcasting system, wherein the interactive recording and broadcasting system comprises a plurality of interactive terminals, one interactive terminal in the plurality of interactive terminals is used as a main meeting place, and other interactive terminals are used as branch meeting places;
the interactive terminal switching method comprises the following steps: detecting voice information of a main meeting place and/or branch meeting places which are interacted currently; carrying out voice recognition on the voice information obtained by detection to obtain a recognition result;
extracting keywords from the recognition result to obtain instruction keywords; generating a terminal switching instruction according to the instruction keyword;
performing interactive terminal switching processing on the currently interactive main meeting place and/or branch meeting place according to the terminal switching instruction;
the switching processing of the interactive terminal to the currently interactive main meeting place and/or branch meeting place according to the terminal switching instruction comprises the following steps: if the extracted instruction keyword comes from the currently interactive main meeting place and the instruction keyword comprises a unique mark of a non-interactive branch meeting place, switching the branch meeting place corresponding to the unique mark into the currently interactive branch meeting place, or switching the branch meeting place corresponding to the unique mark into the currently interactive main meeting place;
if the instruction keywords comprise a first instruction keyword and a second instruction keyword, the first instruction keyword is from the currently interactive branch meeting place, the second instruction keyword is from the currently interactive main meeting place, the first instruction keyword comprises a unique mark of a non-interactive branch meeting place, and the second instruction keyword comprises a keyword representing a determined meaning, the branch meeting place corresponding to the unique mark is switched to the currently interactive branch meeting place, or the branch meeting place corresponding to the unique mark is switched to the currently interactive main meeting place;
if the extracted instruction keywords come from the currently interactive main meeting place and the instruction keywords comprise the unique mark of the currently interactive branch meeting place, interchanging the currently interactive main meeting place and the currently interactive branch meeting place; the non-interactive branch and meeting place refers to other branch and meeting places except the current interactive branch and meeting place;
the unique mark of the branch hall is determined by the following steps: acquiring access time points of each branch meeting place accessed to the main meeting place; determining the access sequence of each branch meeting place according to the access time point; setting unique marks of all the branch places according to the access sequence;
the detecting the voice information of the main meeting place and/or the branch meeting place of the current interaction comprises the following steps: acquiring first voice information in audio and video signals of a main meeting place and/or a branch meeting place which are interacted at present; performing audio analysis on the obtained first voice information, and extracting second voice information of a designated user in the first voice information; determining the extracted second voice information as detected voice information;
the generating of the terminal switching instruction according to the instruction keyword comprises: performing semantic recognition on the instruction keywords to obtain a semantic recognition result; generating a terminal switching instruction according to the semantic recognition result; or matching the instruction keywords with a preset keyword template; if the instruction keyword is successfully matched with a preset keyword template, generating a terminal switching instruction corresponding to the successfully matched keyword template;
the interactive terminal can call any one of other interactive terminals, the interactive terminal carries out primary network communication through the network address of the other interactive terminal, after the other interactive terminal answers, the two interactive terminals are communicated, the interactive terminal serving as the main meeting place can actively call the interactive terminals of other meeting places, and communication links are established with the interactive terminals of the meeting places one by one; in addition, each branch meeting place can also actively call the main meeting place to request access, and a communication link is established with the main meeting place;
when each branch meeting place calls the main meeting place, the access of the main meeting place to the branch meeting places can select to refuse the access or cut the video signal of the meeting place into the interactive picture layout, if the main meeting place does not execute the refuse operation or the cut-in operation, the branch meeting place can only receive the audio and video signal of the main meeting place and cannot interactively communicate with the main meeting place; if the main meeting place executes cut-in operation, the branch meeting place can not only receive the audio and video signals of the main meeting place, but also carry out interactive communication with the main meeting place, and the video picture of the branch meeting place can also appear in the interactive picture layout;
and after the interactive terminal of the main meeting place receives the video streams of all the branch meeting places, integrating the pictures of the main meeting place and all the branch meeting places into one interactive picture.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710135861.4A CN106992971B (en) | 2017-03-09 | 2017-03-09 | Interactive terminal switching method and device and interactive recording and broadcasting system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710135861.4A CN106992971B (en) | 2017-03-09 | 2017-03-09 | Interactive terminal switching method and device and interactive recording and broadcasting system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106992971A CN106992971A (en) | 2017-07-28 |
| CN106992971B true CN106992971B (en) | 2021-10-26 |
Family
ID=59411545
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710135861.4A Active CN106992971B (en) | 2017-03-09 | 2017-03-09 | Interactive terminal switching method and device and interactive recording and broadcasting system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106992971B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109698927A (en) * | 2017-10-23 | 2019-04-30 | 中兴通讯股份有限公司 | Conference management method, device and storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101252670A (en) * | 2008-03-17 | 2008-08-27 | 深圳华为通信技术有限公司 | Apparatus and method for processing conference television |
| CN101867768A (en) * | 2010-05-31 | 2010-10-20 | 杭州华三通信技术有限公司 | Picture control method and device for video conference place |
| CN102131071A (en) * | 2010-01-18 | 2011-07-20 | 华为终端有限公司 | Method and device for video screen switching |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20020138842A1 (en) * | 1999-12-17 | 2002-09-26 | Chong James I. | Interactive multimedia video distribution system |
| EP2700244B1 (en) * | 2011-04-21 | 2016-06-22 | Shah Talukder | Flow-control based switched group video chat and real-time interactive broadcast |
-
2017
- 2017-03-09 CN CN201710135861.4A patent/CN106992971B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101252670A (en) * | 2008-03-17 | 2008-08-27 | 深圳华为通信技术有限公司 | Apparatus and method for processing conference television |
| CN102131071A (en) * | 2010-01-18 | 2011-07-20 | 华为终端有限公司 | Method and device for video screen switching |
| CN101867768A (en) * | 2010-05-31 | 2010-10-20 | 杭州华三通信技术有限公司 | Picture control method and device for video conference place |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106992971A (en) | 2017-07-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10630738B1 (en) | Method and system for sharing annotated conferencing content among conference participants | |
| KR102085383B1 (en) | Termial using group chatting service and operating method thereof | |
| EP3131257B1 (en) | Program, information processing apparatus, and information processing system for use in an electronic conference system | |
| CN112653902B (en) | Speaker recognition method and device and electronic equipment | |
| CN109361527B (en) | Voice conference recording method and system | |
| US20160065895A1 (en) | Method, apparatus, and system for presenting communication information in video communication | |
| US20120259924A1 (en) | Method and apparatus for providing summary information in a live media session | |
| CN114827094A (en) | Cloud desktop-based authority control method and device, computer equipment and medium | |
| CN106992971B (en) | Interactive terminal switching method and device and interactive recording and broadcasting system | |
| CN114227702A (en) | Intelligent conference guiding method and device based on robot and robot | |
| CN111767898B (en) | Service data processing method, device, equipment and storage medium | |
| CN113747247B (en) | Live broadcast method, live broadcast device, computer equipment and storage medium | |
| CN113596381A (en) | Audio data acquisition method and device | |
| CN112333050A (en) | Conference performance testing method, device, equipment and storage medium based on simulation | |
| CN115311920B (en) | VR practical training system, method, device, medium and equipment | |
| US11729489B2 (en) | Video chat with plural users using same camera | |
| US20220303319A1 (en) | Method and device for conference control and conference participation, server, terminal, and storage medium | |
| CN110163777A (en) | A kind of public good assistance to improve schooling in backward areas management method and device | |
| CN115396626A (en) | Video conference method, device, equipment and storage medium | |
| CN113345281A (en) | Intelligent teaching system | |
| CN113225521B (en) | Video conference control method and device and electronic equipment | |
| CN110808960A (en) | Method, equipment and system for establishing data connection | |
| KR20150145303A (en) | Multiview telepresence service providing method and apparaus thereof | |
| CN115396404B (en) | Synchronous screen throwing method and related device for explanation positions of main speakers in cloud conference scene | |
| CN113724547B (en) | Distributed teaching system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |