Online live television teleconference intelligent management system based on cloud computing and artificial intelligence
Technical Field
The invention belongs to the technical field of conference management, and relates to an intelligent management system for an online live television teleconference based on cloud computing and artificial intelligence.
Background
Under the promotion of big data development strategy, people can acquire information more conveniently, and the national government actively promotes the information disclosure of a plurality of public fields all the time, so that an online live television conference is generated, real-time information of the conference can be shared in a live broadcast mode, but later adjustment cannot be performed, and the online live television conference management is particularly important.
In traditional online live telecommand conference management mode, manage network and the equipment operation state that corresponds in this meeting room, therefore traditional online live telecommand conference management mode has still existed many drawbacks of violence, on the one hand can't trace each speech person's speech time, can't accomplish timely warning to each speech person, on the one hand can't adjust the speech time of next speech person according to current speech person's speech time, and then can't improve the effect of this live telecommand conference, on the other hand lacks the management to the meeting flow, can't the managerial efficiency of effectual guarantee online live telecommand conference.
Disclosure of Invention
In view of this, in order to solve the problems existing in the background art, an intelligent management system for an online live tv teleconference based on cloud computing and artificial intelligence is proposed, so as to realize intelligent management of the online live tv teleconference.
The purpose of the invention can be realized by the following technical scheme:
an online live television teleconference intelligent management system based on cloud computing and artificial intelligence comprises a participant information acquisition module, a conference flow importing module, a conference flow processing module, a speech tracking module, an information verification module, a time correction module, a warning module and a database;
the speech tracking module is respectively connected with the conference flow importing module, the information verification module, the database, the time judging module and the warning module, the conference flow importing module is respectively connected with the participant information acquisition module and the conference flow processing module, and the warning module is connected with the information verification module and the speech tracking module;
the conference personnel information acquisition module is used for acquiring basic information of the conference personnel, counting the number of the conference personnel, numbering the counted conference personnel according to a preset sequence, sequentially marking the number as 1,2, aSetting the corresponding seat position of each participant, and acquiring the corresponding seat number of each participant, wherein the basic information of each participant comprises the name of each participant, the face image of each participant and the seat number of each participant, thereby constructing a participant information set Fw(Fw1,Fw2,...,Fwj,...Fwm),Fwj represents the w-th basic information corresponding to the jth participant, w represents the basic information, and w is a1, a2, a3, a1, a2 and a3 which are respectively represented as the participant name, the participant image and the participant seat number;
the conference flow importing module is used for importing flow information of the online live television teleconference, the flow information of the online live television teleconference comprises conference starting time and conference ending time, speaking subjects corresponding to all speaking persons, speaking time points and speaking durations corresponding to all speaking persons, and the imported conference starting time points and the imported conference ending time points, the speaking subjects corresponding to all speaking persons, the speaking time points corresponding to all speaking persons and the speaking durations are sent to the conference flow processing module;
the conference flow processing module is used for receiving the conference starting time point and the conference ending time point sent by the conference flow importing module, the speaking subjects corresponding to the speakers, the speaking time points and the speaking durations corresponding to the speakers, sequencing the received flow information of the online live video teleconference according to the speaking time points of the speakers, further acquiring the conference flow corresponding to the conference, and sending the conference flow to the speaking tracking module;
the speech tracking module is used for receiving the conference flow sent by the conference flow processing module and tracking the time of the current speech person according to the flow, and comprises a primary speech tracking module and a secondary speech tracking module;
the first-stage speaking tracking module is used for recording the initial speaking time point of the current speaking person when the current speaking person starts speaking, timing the current speaking person at the initial speaking time point, tracking the current speaking person in real time, counting the timing speaking duration of the current speaking person, acquiring the time point corresponding to the timing speaking duration according to the corresponding speaking time point and the timing speaking duration of the current speaking person in the conference flow, recording the time point as the speaking process time point, comparing the speaking process time point corresponding to the current speaking person with the speaking reminding time point corresponding to the current speaking person, if the speaking process time point corresponding to the current speaking person is at the speaking reminding time point corresponding to the current speaking person, extracting the number of the current speaking person to be reminded, sending the number of the speaking person to the warning module, and according to the speaking time point and the speaking duration of the current speaking person corresponding to the conference flow, acquiring a predicted speech stop time point of a current speaker, comparing the predicted speech stop time point of the current speaker with a speech reminding time point corresponding to the current speaker, further acquiring a difference value between the predicted speech stop time point of the current speaker and the speech reminding time point corresponding to the current speaker, recording the difference value as the predicted speech stop time of the current speaker, further extracting a name corresponding to a next speaker from a conference flow, extracting a face image and a seat number of the next speaker from a participant information set according to the name of the speaker, recording the face image as an original face image, and sending the extracted original face image, the seat number and the predicted speech stop time of the current speaker to an information verification module;
the information verification module is used for verifying the information of the next speaking person, wherein the specific verification process of the information verification module comprises the following steps:
b1, receiving an original face image, a seat number and an estimated stop duration of the current speaker of the next speaker sent by the primary speaker tracking module, and recording the next speaker as a to-be-uttered speaker;
b2, according to the seat number corresponding to the person to be announced, calling the camera of the seat to collect the face image of the person with the seat number, and recording the collected face image as an actual face image;
b3, extracting the features of the actual face image corresponding to the seat number to obtain the actual face feature image of the person waiting to speak, and extracting the features of the original face image corresponding to the seat number to obtain the original face feature image of the person waiting to speak;
b4, comparing the actual face feature image of the person to be pronounced with the original face feature image of the person to be pronounced, and further obtaining the matching degree of the actual face feature image of the person to be pronounced and the original face feature image of the person to be pronounced;
b5, if the image matching degree difference is smaller than zero, judging that the face image corresponding to the seat number does not accord with the face image input by the person to be spoken, and further starting a camera of the seat where each participant is located to acquire the face image of each participant;
b6, further carrying out face feature extraction on the collected face images of the participants to obtain face feature images corresponding to the participants;
b7, comparing the face characteristic image corresponding to each participant with the original face characteristic image of the person to be spoken, further screening out the face characteristic image with the highest matching degree with the face image input by the person to be spoken, further extracting the seat number corresponding to the face characteristic image, recording the seat number as the actual seat number of the person to be spoken, and further sending the actual seat number of the person to be spoken and the estimated stop time of the current speaker to the warning module;
the warning module comprises a primary warning module and a secondary warning module and is used for reminding the current person and the person to be called;
the first-stage warning module is used for receiving the number of the current speaker to be reminded, which is sent by the conference processing module, and further reminding the current speaker;
the second-level warning module is used for receiving the actual seat number of the person to be uttered and the estimated stop time of the current speaker sent by the information verification module, calling the position of the person to be uttered according to the actual seat number of the person to be uttered, calling a display corresponding to the position of the speaker, and displaying the estimated stop time and the estimated signal to be uttered corresponding to the current speaker through the display;
the secondary speaking tracking module is used for acquiring the actual speaking duration of the current speaking person when the current speaking person stops speaking, and sending the actual speaking duration of the current speaking person to the time judging module;
the time judging module is used for receiving the actual speaking duration of the current speaking person sent by the secondary speaking tracking module, and compares the actual speaking duration of the current speaking person with the speaking duration of the current speaking person corresponding to the conference flow, further obtaining the difference value between the actual speaking time length of the current speaking person and the speaking time length of the current speaking person corresponding to the conference flow, if the difference value is larger than zero, judging the speaking timeout of the speaking person, counting the timeout time of the current speaking person, if the difference is less than zero, judging that the speaker finishes speaking in advance, counting the advance time of the current speaker, if the difference between the actual speaking time of the current speaker and the speaking time of the current speaker corresponding to the conference flow is equal to zero, judging that the speaking person stops speaking according to the speaking duration corresponding to the conference flow, and sending the overtime time of the current speaking person and the advance time of the current speaking person to a time correction module;
the time correction module is used for correcting the speaking time point of the next speaking person when the speaking time of the current speaking person is not equal to the speaking time of the current speaking person corresponding to the conference flow, wherein the time correction module comprises an overtime correction module and a forward correction module;
the overtime correction module is used for receiving the overtime of the current speaker sent by the time judgment module, delaying the starting time corresponding to the next speaker backwards according to the overtime of the current speaker and the speaker time point corresponding to the next speaker corresponding to the conference flow, counting the speaker time point corresponding to the next speaker after correction, recording the speaker time point as an overtime correction speaker time point, and sending the overtime correction speaker time point corresponding to the next speaker to the speaker tracking module;
the advance correction module is used for carrying out time advance on the starting time corresponding to the next speaker according to the advance time of the current speaker and the speaker time point corresponding to the next speaker corresponding to the conference flow, counting the speaker time point corresponding to the next speaker after correction, recording the corrected speaker time point as an advance corrected speaker time point, and sending the advance corrected speaker time point corresponding to the next speaker to the speaker tracking module;
the database is used for storing speaking reminding time points corresponding to all speaking persons.
Furthermore, the speaking time tracking module tracks speaking through a timing device and a sound detection device, wherein the timing device is a timer and is used for timing speaking time when a speaking person starts speaking, and the sound detection device is a sound sensor and is used for detecting the sound of the speaking person so as to judge whether the speaking person is in the speaking process.
Furthermore, the primary warning module reminds through a warning device, the warning device is a wearable warning terminal, when the speaking process time point corresponding to the current speaking person is located at the speaking reminding time point corresponding to the current speaking person, the current speaking person number needing to be reminded is extracted, and then the warning terminal corresponding to the number is called for vibration reminding.
Furthermore, the secondary level warning module reminds the current speaking person of estimating the stop duration and the signal to be spoken in a text reminding mode, wherein the text reminding mode is the increase of text fonts and the deepening of text colors, and therefore reminding of the person to be spoken is achieved.
Further, the system also comprises a device for installing the corresponding seats of the participants, wherein the device comprises a plurality of cameras and a plurality of displays, the seats of the participants correspond to one camera and one display respectively, the cameras are used for collecting images of the participants at the seats, and the displays are used for displaying video images corresponding to the current speakers and conference flows corresponding to the participants to be spoken.
Furthermore, the camera is high-definition and can automatically focus, and is used for collecting high-definition face images of each participant.
The invention has the beneficial effects that:
(1) according to the intelligent management system for the online live television teleconference based on the cloud computing and the artificial intelligence, the current speakers are tracked in real time through the speech tracking module and the warning module, the initial speech time point and the speech duration of each speaker are obtained, and then each speaker is reminded in time, so that the intelligent management of the online live television teleconference is realized, the stability of the conference flow is guaranteed, meanwhile, the next speaker can be adjusted according to the speech time of the current speaker, the effect of the live television teleconference is effectively improved, and the management efficiency of the live television teleconference is improved.
(2) According to the invention, the speaking time of the next speaking person is rapidly adjusted by correcting the speaking time point of the next speaking person in the time correction module, so that the progress of the live video teleconference is effectively promoted, and the order of the live video teleconference is effectively maintained.
(3) According to the invention, the warning module reminds the current speaker and the speakers to be sent, so that the current speaker and the speakers to be sent are reminded in time, and the smoothness of the progress of the television/telephone conference is effectively guaranteed.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a schematic diagram of the system module connections of the present invention;
fig. 2 is a schematic diagram of the connection of the utterance tracking module in the present invention;
FIG. 3 is a schematic diagram of an alarm module according to the present invention.
Detailed Description
While the foregoing is directed to embodiments of the present invention, other and further embodiments of the invention may be devised without departing from the basic scope thereof, and the scope thereof is determined by the claims that follow.
Referring to fig. 1, an online live tv conference intelligent management system based on cloud computing and artificial intelligence includes a participant information acquisition module, a conference flow import module, a conference flow processing module, a speech tracking module, an information verification module, a time correction module, a warning module, and a database;
the speech tracking module is respectively connected with the conference flow importing module, the information verification module, the database, the time judging module and the warning module, the conference flow importing module is respectively connected with the participant information acquisition module and the conference flow processing module, and the warning module is connected with the information verification module and the speech tracking module;
the participant information acquisition module is used for acquiring basic information of participants, counting the number of the participants, numbering the counted participants according to a preset sequence, sequentially marking the counted participants as 1,2, a, j, a, m, counting the number of seats in the conference room, numbering the seats in the conference room according to the preset sequence, sequentially marking the seats as 1,2, a, i, a, n, and acquiring the seat numbers corresponding to the participants according to the preset seat positions corresponding to the participants, wherein the basic information of the participants comprises names of the participants, face images of the participants and the seat numbers of the participants, and further establishing a participant information set Fw(Fw1,Fw2,...,Fwj,...Fwm),Fwj represents the jth participant correspondenceThe w-th basic information of the present invention, w represents basic information, w is a1, a2, a3, a1, a2 and a3 respectively represent names of participants, images of the participants and seat numbers of the participants, wherein each seat corresponds to a camera and a display, the camera is used for collecting images of the participants at the position, and the display is used for displaying video images corresponding to the current speaker and conference flows corresponding to the participants to be spoken;
the conference flow importing module is used for importing flow information of the online live television teleconference, the flow information of the online live television teleconference comprises conference starting time and conference ending time, speaking subjects corresponding to all speaking persons, speaking time points and speaking durations corresponding to all speaking persons, and the imported conference starting time points and the imported conference ending time points, the speaking subjects corresponding to all speaking persons, the speaking time points corresponding to all speaking persons and the speaking durations are sent to the conference flow processing module;
the conference flow processing module is used for receiving the conference starting time point and the conference ending time point sent by the conference flow importing module, the speaking subjects corresponding to the speaking persons, the speaking time points and the speaking durations corresponding to the speaking persons, and sequencing the received flow information of the online live television conference according to the speaking time points of the speaking persons, so that the conference flow corresponding to the conference is obtained, and the conference flow is sent to the speaking tracking module.
Referring to fig. 2, the utterance tracking module is configured to receive a conference flow sent by the conference flow processing module, where the utterance tracking module includes a primary utterance tracking module and a secondary utterance tracking module;
the embodiment of the invention tracks the current speakers in real time through the speaker tracking module, acquires the initial speaking time point and the speaking duration of each speaker, and further timely reminds each speaker, thereby realizing intelligent management of the on-line live television conference and ensuring the stability of the conference flow;
the first-stage speaking tracking module tracks the time of the current speaking person according to the flow through a timing device and a sound detection device, wherein the timing device is a timer and is used for timing the speaking time when the speaking person starts speaking, the sound detection device is a sound sensor and is used for detecting the sound of the speaking person so as to judge whether the current speaking person is in the speaking process, when the sound sensor detects the sound, the initial speaking time point of the current speaking person is recorded and timed, the speaking time of the current speaking person is counted, the initial speaking time point of the current speaking person is recorded, the current speaking person is tracked in real time, the timed speaking time of the current speaking person is counted, and the time point corresponding to the timed speaking time is obtained according to the speaking time point and the timed speaking time corresponding to the current speaking person in the conference flow, recording the current speech process time point as a speech process time point, comparing the speech process time point corresponding to the current speaker with a speech reminding time point corresponding to the current speaker, if the speech process time point corresponding to the current speaker is at the speech reminding time point corresponding to the current speaker, extracting the number of the current speaker to be reminded, sending the number of the speaker to a warning module, acquiring the estimated speech stop time point of the current speaker according to the speech time point and the speech duration of the current speaker corresponding to the conference flow, comparing the estimated speech stop time point of the current speaker with the speech reminding time point corresponding to the current speaker, further acquiring the difference value between the estimated speech stop time point of the current speaker and the speech reminding time point corresponding to the current speaker, and recording the difference value as the estimated speech stop time of the current speaker, further extracting a name corresponding to the next speaker from the conference flow, extracting a face image and a seat number of the next speaker from the participant information set according to the name of the speaker, recording the face image as an original face image, and sending the extracted original face image, the seat number and the estimated stop time of the current speaker to the information verification module;
the information verification module verifies information of the next speaking person through a camera, the camera is high-definition and can automatically focus, and the specific verification process of the information verification module for collecting high-definition face images of all participants comprises the following steps:
b1, receiving an original face image, a seat number and an estimated stop duration of the current speaker of the next speaker sent by the primary speaker tracking module, and recording the next speaker as a to-be-uttered speaker;
b2, according to the seat number corresponding to the person to be announced, calling the camera of the seat to collect the face image of the person with the seat number, and recording the collected face image as an actual face image;
b3, extracting the features of the actual face image corresponding to the seat number to obtain the actual face feature image of the person waiting to speak, and extracting the features of the original face image corresponding to the seat number to obtain the original face feature image of the person waiting to speak;
b4, comparing the actual face feature image of the person to be pronounced with the original face feature image of the person to be pronounced, and further obtaining the matching degree of the actual face feature image of the person to be pronounced and the original face feature image of the person to be pronounced;
b5, if the image matching degree difference is smaller than zero, judging that the face image corresponding to the seat number does not accord with the face image input by the person to be spoken, and further starting a camera of the seat where each participant is located to acquire the face image of each participant;
b6, further carrying out face feature extraction on the collected face images of the participants to obtain face feature images corresponding to the participants;
b7, comparing the face characteristic image corresponding to each participant with the original face characteristic image of the person to be spoken, further screening out the face characteristic image with the highest matching degree with the face image input by the person to be spoken, further extracting the seat number corresponding to the face characteristic image, recording the seat number as the actual seat number of the person to be spoken, and further sending the actual seat number of the person to be spoken and the estimated stop time of the current speaker to the warning module;
according to the embodiment of the invention, the information verification module verifies the information of the next speaker through the camera, so that the warning accuracy of the warning module is effectively guaranteed, and the stability of the live video teleconference is effectively maintained;
referring to fig. 3, the warning module includes a primary warning module and a secondary warning module, and is configured to remind a current person and a person to be called;
according to the embodiment of the invention, the warning module reminds the current speaker and the speakers to be announced, so that the current speaker and the speakers to be announced can be reminded in time, and the smoothness of the progress of the television teleconference can be effectively guaranteed.
The first-stage warning module is used for receiving a current speaker number which needs to be reminded and is sent by the first-stage speech tracking module, the first-stage warning module reminds through a warning device, the warning device is a wearable warning terminal, and when a speech process time point corresponding to a current speaker is located at a speech reminding time point corresponding to the current speaker, the current speaker number which needs to be reminded is extracted, and then the reminding terminal corresponding to the number is called for vibration reminding;
the second-level warning module is used for receiving the actual seat number of the person to be uttered and the estimated stop time of the current person to be uttered, which are sent by the information verification module, calling the position of the person to be uttered according to the actual seat number of the person to be uttered, calling a display corresponding to the position of the person to be uttered, and displaying and reminding the estimated stop time of the current person to be uttered and a signal to be uttered in a text reminding mode through the display, wherein the text reminding mode is that the font of the text is enlarged and the color of the text is deepened, so that the reminding of the person to be uttered is realized;
the secondary speaking tracking module is used for acquiring the actual speaking duration of the current speaking person when the current speaking person stops speaking, and sending the actual speaking duration of the current speaking person to the time judging module;
the time judging module is used for receiving the actual speaking time of the current speaking person sent by the second-stage speaking tracking module, comparing the actual speaking time of the current speaking person with the speaking time of the current speaking person corresponding to the conference flow, further acquiring a difference value between the actual speaking time of the current speaking person and the speaking time of the current speaking person corresponding to the conference flow, if the difference value is greater than zero, judging that the speaking of the speaking person is overtime, counting the overtime time of the current speaking person, if the difference value is less than zero, judging that the speaking person finishes speaking in advance, counting the advance time of the current speaking person, if the difference value between the actual speaking time of the current speaking person and the speaking time of the current speaking person corresponding to the conference flow is equal to zero, judging that the speaking person stops speaking according to the speaking time corresponding to the conference flow, and sending the overtime of the current speaking person and the advance time of the current speaking person to the time correcting module, according to the embodiment of the invention, the actual speaking duration of the current speaking person is compared with the speaking duration of the current speaking person corresponding to the conference flow through the time judgment module, so that the actual speaking duration of the current speaking person is quickly judged, and a data basis is provided for the processing of the time of the next speaking person;
the time correction module is used for correcting the speaking time point of the next speaking person when the speaking time of the current speaking person is not equal to the speaking time of the current speaking person corresponding to the conference flow, wherein the time correction module comprises an overtime correction module and an advance correction module;
according to the embodiment of the invention, through the time correction module, when the speaking duration of the current speaking person is not equal to the speaking duration of the current speaking person corresponding to the conference flow, the speaking time point of the next speaking person is corrected, and through the adjustment of the speaking time point of the next speaking person, the effect of the live telecommand is effectively improved, and the management efficiency of the live telecommand is improved;
the overtime correction module is used for receiving the overtime of the current speaker sent by the time judgment module, delaying the starting time corresponding to the next speaker backwards according to the overtime of the current speaker and the speaker time point corresponding to the next speaker corresponding to the conference flow, counting the speaker time point corresponding to the next speaker after correction, recording the speaker time point as an overtime correction speaker time point, and sending the overtime correction speaker time point corresponding to the next speaker to the speaker tracking module;
the advance correction module is used for carrying out time advance on the starting time corresponding to the next speaker according to the advance time of the current speaker and the speaker time point corresponding to the next speaker corresponding to the conference flow, counting the speaker time point corresponding to the next speaker after correction, recording the corrected speaker time point as an advance corrected speaker time point, and sending the advance corrected speaker time point corresponding to the next speaker to the speaker tracking module;
the database is used for storing speaking reminding time points corresponding to all speaking persons.
The foregoing is merely exemplary and illustrative of the principles of the present invention and various modifications, additions and substitutions of the specific embodiments described herein may be made by those skilled in the art without departing from the principles of the present invention or exceeding the scope of the claims set forth herein.