WO2019019406A1

WO2019019406A1 - Teaching recording data updating device

Info

Publication number: WO2019019406A1
Application number: PCT/CN2017/105553
Authority: WO
Inventors: 陈滢西; 赵鹏祥; 滕凯
Original assignee: 深圳市鹰硕技术有限公司
Priority date: 2017-07-28
Filing date: 2017-10-10
Publication date: 2019-01-31
Also published as: CN109324811A; CN109324811B

Abstract

The present invention provides a teaching recording data updating device, comprising a voice data updating unit and a courseware data updating unit. Replacement is performed using written content generated by voice data recognition, and old courseware data, particularly old PPT courseware data, is replaced by new courseware data, such that after a teaching process is recorded, updating is performed using the resulting teaching recording data. By using the device of the present invention, after a teaching process is recorded, a voice-related error, such as a mistaken, omitted, or inaccurate expression that occurs during teaching can be corrected by updating voice data. In addition, old courseware can be replaced by new courseware, such that new courseware is displayed to a user learning from the teaching recording data. Therefore, after a teaching process is completed, that is, after teaching recording data is formed, an error that occurs in the teaching process can be corrected by means of updating.

Description

Device for updating teaching recording data

Technical field

The invention belongs to the field of network teaching recording and broadcasting technology, and can be used for recording and playing of a teaching activity or a conference process based on network teaching or online conference, in particular, a voice data capable of recording and broadcasting data in a completed recording teaching. A device for updating courseware data.

Background technique

In recent years, as traditional teaching models have become less and less able to meet the needs of users for new teaching methods such as multimedia, informationization, and easy playback, with the rapid development and popularization of Internet technologies, especially mobile Internet technologies, various online teaching The recording and broadcasting system is booming. In the network teaching, the teaching process is recorded through the classroom recording, and the teaching resources can be shared on the Internet. The user can use the terminal to access these teaching resources online, which can meet the needs of the user for remote learning and review.

Early technologies in teaching recording and broadcasting, such as CN101141271A (publication date March 12, 2008), disclosed a recording and broadcasting system for network teaching, including: recorder, processor, first network, second network, server, database And three clients. Among them, the recorder mainly includes a camera and a wireless digital microphone to record video information and voice data of the courseware. The first network transmits the courseware information to the server. The server is used on the one hand to further process the courseware information, to generate courseware data, and on the other hand to search and call the courseware data in the database, and then convert the courseware data back to the courseware information. The database is used to store the courseware data. The second network is used to connect the client to the server. The client is used to facilitate the user to query courseware information and invoke courseware information. The patent application discloses a technique for recording a typical streaming media format recording course. Now, the main disadvantage is that the file formed after recording is relatively large, the uploading and downloading speed is slow, and the required storage space is large.

Recent technologies in teaching and recording, such as CN105306861A (publication date February 3, 2016), disclose an effective classroom teaching and recording method and system. In the process of online teaching or online meeting, multimedia whiteboard can be realized for users. The functional voice, speech/speech voice, communication with other users, and/or coaching, etc., are recorded to form different data streams, and a unified time stamp for various data streams is generated by the network teaching recording system. Marking, instead of recording the entire event in a streaming media format, the network user can conveniently download various data streams to be played from the cloud server or the LAN server through the network anytime and anywhere, and obtain the data stream after the customer of the user terminal. The end obtains the data stream according to the time stamp reproduction, and the organic combination plays out to display to the user, thereby completing the on-demand browsing. The patent application discloses a separate storage in three data stream formats according to time stamps. A classroom recording method for storing and recording classroom teaching data.

With the increasing pursuit of the quality of recorded courses, more and more teaching and recording systems use speech recognition technology, usually need to convert the speech into text, display it as a subtitle on the screen or save it as a text format. In the prior art, there are a few patent applications for speech recognition, especially converting speech to text or converting text to speech, such as:

CN101354748A (Publication Date January 28, 2009) discloses a character recognition device including an image pickup device, a character recognition device, a voice conversion device, and a voice output device for taking in text information and taking a photo The entered text information is sent to the character recognition device in the form of a picture; the character recognition device is configured to identify the text information in the picture and send the text information to the voice conversion device; the voice conversion device is configured to The text information is converted into voice data and sent to the voice output device; and the voice output device is configured to play the voice data. The patent application discloses a technique for collecting and recognizing text symbols in image information and then converting the text symbols into speech.

CN102956231A (Publication Date March 6, 2013) discloses a semi-automatic correction based speech key information recording apparatus and method in the field of speech recognition technology, the apparatus comprising: a key information extraction unit and an information correction unit connected thereto, The key information extracting unit obtains the uncorrected text data and extracts the key information, and outputs the key information to the information correcting unit, and the information correcting unit outputs the text data confirmed by the user feedback. The invention reduces the workload of manual correction by using a semi-automatic information correction unit; uses a database to correct special nouns such as place names and professional tool names, thereby reducing the influence caused by the operator's knowledge limit in manual correction; extracting voice data Key information in the message, thereby increasing the amount of information available for the recorded information. The patent application aims to solve the problem of semi-automatic correction of text data after speech conversion into text.

CN105159870A (Publication Date December 16, 2015) discloses a processing system for accurately completing continuous natural speech text, the processing system including a cloud speech recognition engine and a speech recognition update platform, and the speech recognition and update platform and The cloud speech recognition engine is connected, and the speech recognition post update platform comprises a display unit, an update operation unit, a control unit and a three-dimensional integrated generation unit, and the update operation unit comprises a voice update, a keyboard update, a mouse update, and a keyboard plus a mouse. The operation mode is updated, in which it is disclosed that the voice file to be identified can be finely segmented to achieve accurate recognition.

CN105808197A (Publication Date July 27, 2016) discloses an information processing method applied to an electronic device having a speech recognition module, the method comprising: receiving input speech data; After inputting voice data for recognition to obtain a recognition result, when When the first information in the recognition result is content that needs to be updated, the first information is at least one character in the recognition result, and the first information in the recognition result is updated by using an operation body input manner, The method of inputting the operation body updates the first information in the recognition result, and only needs to update the part of the target update, without the user inputting the voice data again to obtain the target result, the operation process is simple, and the information input is improved. The overall speed. The patent application discloses that it is only necessary to update the content that needs to be updated at the first place after voice recognition, thereby improving the speed of the update, but such update is only for the recognized text data, wherein the process of voice recognition In the middle, the method of comparing the information to be identified with the standard voice data is used, thereby improving the recognition accuracy.

CN106328145A (Publication Date, January 11, 2017) discloses a voice update method and apparatus, comprising: acquiring voice data input by a user; and identifying the voice data to obtain text content corresponding to the voice data; When the text content includes the first preset keyword, the text content is divided into original text and edit text according to the first preset keyword, wherein the edit text is used to perform the original text. Updating; extracting text to be updated from the original text according to the edited text; updating the original text according to the edited text and the to-be-updated text to obtain updated text. The patent application discloses that the text to be edited in the original text, that is, the edited text, can be obtained by means of keyword recognition, and the text content formed by the voice recognition is updated in a targeted manner.

CN102215233A (Publication Date, October 12, 2011) discloses an information system client installed in a user's terminal device, which can be applied to a microblog, a blog, a forum or a personal space, etc., including: a user interaction module and a connection office. The voice module of the user interaction module, preferably, further includes a feedback module, a conversion module, the voice module includes a voice collection unit, a voice recognition unit, and a voice synthesis unit, and the voice collection unit is configured to collect voices of the user; the voice recognition unit The voice recognition unit collects the voice recognition as text output to the user interaction module; the voice synthesis unit converts the text obtained by the user interaction module from the information system server into voice output to the user; the feedback module, the connection center a voice recognition unit, configured to confirm whether the voice recognition is correct, if the correct, the feedback module outputs the text to the user interaction module, and if not, the feedback module enables the voice collection unit Reacquiring the voice of the user or the voice recognition unit updates the text straight To confirm that it is correct. The patent application discloses a technology for converting between voice and text respectively, and aims to convert information of one format into information of another format, and if the outputted text information is incorrect, the feedback module re- Collect user voices or directly update the output text information.

CN106486113A (March 8, 2017) discloses a method for recording a conference, comprising: acquiring a voice signal; converting the voice signal into corresponding text information by a voice conversion software, and in the document The text information includes the correct text information and the incorrect text information; the wrong text information in the document is marked, and the marked error text information is associated with the voice signal corresponding to the incorrect text information. Linking; when clicking the error text information, using the voice conversion software to perform secondary recognition on the voice signal associated with the erroneous text information, and performing editable display on the second recognized text information in the document; The correction text information is corrected and edited in the editable display to obtain corrected text information, and the error text information is replaced with the corrected text information.

In summary, in the prior art, neither the teaching recording and broadcasting technology field nor the speech recognition conversion field involves the concept of updating the already formed classroom recording data, especially through updating the recognized text content. To update the voice itself and replace the old courseware with a new courseware, the old course can use the idea of a new courseware, etc. The prior art is concerned with the problem of the accuracy of speech recognition conversion, especially voice conversion into text. However, in any teaching or conference process, for any speaker, there may be misrepresentation, omission or non-standard pronunciation, and even non-standard expression. For these problems, it is usually used in speech recognition. That is, when converting to text (such as in subtitles), plus text annotations (such as explained in parentheses).

In particular, for the teaching and recording system, since the taught course is recorded and reproduced to the user through the network, the influence of the problems such as wrong saying, missing, and non-standard expression becomes prominent and affected because the language data is compressed. Very large, on the one hand, because users often find it difficult to identify these errors, and even if they are identified by subtitles, on the other hand, because of the use of the environment, the user may not be able to read the subtitles, and can only listen in the form of voice, voice expression Unclear, further affecting the effect of user learning.

For courseware data, especially the PPT class, the content of the courseware can be divided by the page. Once an error occurs, such as a text error or an expression error, after the course recording is completed, it can no longer be corrected, or deleted, or by adding a note description. Remedy, obviously these problems need to be solved through new ways.

In view of the problems existing in the prior art, the present invention aims to provide an apparatus for updating teaching recording and broadcasting data, which realizes by replacing the text content formed by the voice data and the replacement operation of the new and old courseware data, especially the PPT courseware data. After the completion of the recording of the teaching process, the purpose of updating the teaching recording data is further updated.

Summary of the invention

The present invention is directed to an apparatus for updating teaching recording data, which includes separately stored data streams such as voice data, video data, motion data, and the like, and specifically includes using a recording device to be taught in the network or The voice signal during the online conference is converted into the original voice data with time stamp, saved in the voice data stream + time stamp format, and the operation action data obtained by using the motion data recording software, such as the courseware, especially the PPT file, The courseware action action data stream + timestamp format is saved.

According to the apparatus of the present invention, the original speech data is converted into original text data using a speech recognition model, the original text data is collated, and the old text content to be updated is replaced with the new text content, thereby realizing the original text data. The update forms the updated text data, uses the timestamp for positioning, and replaces the standard voice data of the new text content with the corresponding voice data segment of the old text content to form updated voice data.

According to the device of the present invention, the new courseware is updated on the basis of the old courseware, and the relationship between the new and old coursewares is established, including the corresponding relationship between the page and the updated content, and the time of the old courseware data in the recorded data is obtained. The stamp information is used to replace the old courseware with the new courseware according to the correspondence relationship and the timestamp information, so that the new courseware content is presented to the user in the manner of displaying the old courseware formed during the recording when the recorded data is played back on demand.

It should be understood that although the embodiments of the present invention have been described primarily in the context of the recording of web-based teaching or web conferencing, it will be appreciated that the apparatus of the present invention may also be used for recording of other online online communication processes, Play and update. That is to say, the present invention relates to a method, system and computer program product for teaching and recording a network teaching, online training, emergency command (map annotation and voice recording), a financial system or an online conference boarding system, or recording and playing in a conference process. In the process of network teaching, online training, emergency command (map annotation and voice recording), financial system (marketing explanation) or online conference, as long as voice data and PPT files are involved, the voice data is identified and converted. The text data is updated, and the standard voice data of the updated text content is replaced with the corresponding recorded voice data, so that the update of the voice data and the replacement operation of the new and old courseware can be realized, and the courseware data is updated.

The present invention provides an apparatus for updating teaching recording data, during recording and on-demand review of a multimedia classroom (or online classroom) or the like, particularly when recording a multimedia classroom, including voice data, The action data (electronic whiteboard book) on the multimedia whiteboard, the operation action data on the screen of the user terminal, the video data recorded by the recording device, etc. are added in time stamps after the data stream format is added, and the teaching recording data is formed, and the user logs in to the network teaching record. After broadcasting the system, use wired or The wireless local area or the wide area network obtains the teaching and recording data, realizes the reproduction process on the user terminal by using the time stamp or simulates the teaching process of the recurring classroom, thereby realizing the review playback or the on-demand playback of the recorded classroom.

The device for updating teaching recording data includes: a file identification generating unit, a voice data collecting unit, a voice data updating unit, a courseware data collecting unit, a courseware data updating unit, other data collecting units, a recording data playing unit, and Error information feedback unit, etc., wherein

a file identification generating unit, configured to generate a file identification ID when starting the recording teaching process;

a voice data collecting unit, configured to convert a voice signal into original voice data by using an audio collecting device, and save the voice signal in a voice data stream format; preferably, collecting at least one voice data from at least one voice source, and adding a time stamp to the voice Data stream format is saved;

a voice data update unit, configured to update the voice data that needs to be updated by the original voice data, to form updated voice data, where the update of the voice data is implemented by a replacement operation of the text content formed by the voice data identification;

The courseware data collecting unit is configured to acquire the courseware data for the courseware file, in particular the operation action data of the PPT file during the teaching process;

a courseware data updating unit, configured to update courseware data in the teaching record data, wherein the update of the courseware data is implemented by replacing the old courseware with a new courseware;

The other data collection unit is configured to collect at least one of the following data: action data on the multimedia whiteboard, operation data on the screen of the user terminal, video data of the video recording device, and adding the timestamp to each data collected, Separately saved in a data stream format, and together with the voice data stream and the courseware data, form recording data that can be played on-demand;

Recording a data playing unit, the user uses the terminal to acquire the recorded data through a network, combines different data streams according to the time stamp, thereby playing the recorded data on the terminal, reproduces and/or simulates a recurring teaching process, and realizes Learning and/or reviewing the teaching process;

The error information feedback unit may: when the user plays the recorded data by using the terminal, may select and submit the error text content in the found updated text data, and the feedback content is updated by the administrator, and the update is updated. Text data, and repeating the voice data replacement unit, updating the updated voice data.

The voice data update unit further includes

a voice data identification unit, configured to convert the original voice data identification into original text data according to the voice recognition model, and determine a time coordinate of each text content in the text data according to the time stamp;

a text data updating unit, configured to proofread the original text data, and need to update The old text content is replaced with an updated new text content to form updated text data;

a voice data replacing unit, configured to replace, by using standard voice data of the new text content, a voice data segment of the old text content in the original voice data, to form updated voice data, to implement data recording and playing on the teaching Update of voice data.

The courseware data updating unit further includes

The courseware content update unit is used to update the content on the basis of the old courseware, replace the old content with the new content, form a new courseware, and record the correspondence data between the old courseware and the new courseware, including between the pages before and after the update. The relationship between the relationship and the content before and after each page update;

a courseware data updating unit, configured to acquire a timestamp identifier of the courseware data in the teaching recording and broadcasting data, and process the new courseware according to the corresponding relationship data, and add the timestamp identifier to the new courseware to form a new Courseware data;

The courseware replacement updating unit replaces the old courseware data in the teaching recording data with the new courseware data, and updates the courseware data of the teaching recording data.

The courseware is preferably a PPT file, and the standard voice data is obtained by searching from a standard voice database, and the voice data and the courseware data are separately saved. Preferably, the voice data is “voice data stream+” The format of the timestamp is saved, and the courseware data is saved in the format of "courseware operation data stream + timestamp".

The text data updating unit is further configured to: after the courseware data update is completed, determine, according to the timestamp identifier, the display time of each page of the PPT file, and update according to each page after the update Content correspondence, searching for old content in the text data during the presentation time, replacing the old content with the new content, forming updated text data, implementing further updating of the text data, and making the updated voice data and the updated courseware The data is adapted. Thereby, the interaction and association of the courseware update and the voice update are realized, so that when the content involved in the voice data has been updated and replaced with the new PPT file, the voice data is replaced according to the replacement relationship between the PPT and the new content. The related content and voice in the speech realize the adaptive synchronization modification of the voice explanation and the PPT display content, and further improve the completeness and achievability of the recorded data after updating.

The further updating of the text data reveals the correspondence between the old content and the new content in the text data to the administrator before the replacement is completed, and the administrator confirms whether to replace according to the context.

The updated text data is displayed on the screen in a subtitle manner according to a timestamp identifier, preferably displayed on a screen area in which the video data is played, and more preferably, the text data is editable. The selected mode is displayed in a specific area of the screen.

The device further includes an error information feedback unit,

The error information feedback unit may: when the user uses the terminal to play the teaching and recording data, select and submit the error text content in the found updated text data, and the feedback content is updated again after being confirmed by the administrator. The text data is updated, and the updated voice data is updated again by the voice data replacement unit.

When the text data and the voice data are updated, an update history is formed, which may include update time, update content, update operator, issue finder, and the like.

The voice data replacing unit is configured to calculate a smoothing coefficient according to the pronunciation time of the replaced old text content in the original voice data and the pronunciation time of the standard voice data of the new text content, and according to the smoothing coefficient, The pronunciation time of the new text content is adjusted, thereby smoothing and synchronizing the voice data before and after the replacement.

Through the method of the invention, the level of classroom recording is improved, and the storage manners of various data streams are respectively saved based on the time stamp, and the voice data is updated and updated, and the voice data is updated according to the updated text content, and the voice data is updated according to the updated text content. The content that needs to be updated in the original voice data overcomes the problems caused by “less talk, wrong talk and miss talk” in the classroom, and can obtain double updated voice data and text data (subtitle information) through the old courseware. On the basis of the update, a new courseware is formed, and a corresponding relationship is established between the new and old coursewares, and the timestamp information in the recorded data is combined to realize the replacement of the courseware.

By using the device of the invention, after the completion of the recording process of the teaching process, the voice problem is solved by updating the voice data, and the voice problem of non-standard expression in the teaching process is solved, and the new and old courseware can be replaced by the new and old courseware replacement operation. The courseware is displayed to the users who study and record the data, so that after the teaching process is completed, that is, after the teaching and recording data is formed, the problems existing in the teaching process can still be remedied and improved through the update operation.

The above and further objects and features of the present invention will become more apparent from the following detailed description.

DRAWINGS

1 is a block diagram of a recording and broadcasting system according to the present invention;

2 is a flow chart showing the steps of updating the recording data according to the present invention;

Figure 3 is a flow chart showing the steps of updating courseware according to the present invention; and

4 is a flow chart of a voice update step in accordance with the present invention.

Detailed ways

Hereinafter, specific embodiments of the present invention will be described in further detail with reference to the accompanying drawings.

The network teaching in the invention is not limited to the classroom teaching form of students and teachers, and may include online network teaching, remote network teaching, local network teaching, and employees of enterprises and institutions, with teachers and students, or trainers as participants. Participate in online web conferencing, remote web conferencing, local web conferencing, and other forms of communication/interaction that use the web for online communication and/or presentation of file content, such as remote collaborative work.

As shown in FIG. 1, the teacher 1 and the student 2 respectively connect to the teaching server 3 via the Internet using a terminal device installed with a client of the network teaching recording system, and the teaching server 3 is also connected with the multimedia device 4 including an intelligent electronic whiteboard and a camera. At least one of a microphone, a high-stakes instrument, and thus a network lecture/listening/recording/on-demand/review of the multimedia classroom.

The terminal device includes: a processor, a network module, a control module, a display module, and a smart operating system, and can be a smart phone, a PAD, a notebook computer, a desktop computer, or the like. The terminal may be provided with a plurality of data interfaces for connecting various extension devices and accessories through a data bus. The intelligent operating system includes Windows, Android and its improvements, iOS, on which application software can be installed and run, and functions of various application software, services, and application stores/platforms under the intelligent operating system are realized.

Terminal devices can be connected to the Internet via RJ45/Wi-Fi/Bluetooth/2G/3G/4G/G.hn/Zigbee/Z-ware/RFID connections and connected to other terminals or other computers and devices via the Internet. Through 1394/USB/Serial/SATA/SCSI/PCI-E/Thunderbolt/data card interface and other data interfaces or bus methods, through HDMI/YpbPr/SPDIF/AV/DVI/VGA/TRS/SCART/Displayport, etc. Connection methods such as audio and video interfaces to connect various expansion devices and accessories to form a conference/teaching device interactive system. The sound capture control module and the motion capture control module with software form, or the sound capture control module and the motion capture control module in the form of data bus onboard hardware, realize voice control and shape control function; connect display/projection through audio and video interface Modules, microphones, audio equipment and other audio and video equipment for display, projection, sound access, audio and video playback, and digital or analog audio and video input and output functions; connected to the camera, microphone, electronic whiteboard, RFID through the data interface The reading device realizes image access, sound access, use control and screen recording of the electronic whiteboard, RFID reading function, and can access and control mobile storage devices, digital devices and other devices through corresponding interfaces; through DLNA/ IGRS technology and internet technology are used to implement functions such as manipulation, interaction and screen switching between multi-screen devices.

In the present invention, a processor is defined to include, but is not limited to, an instruction execution system such as a computer/processor based system, an application specific integrated circuit (ASIC), a computing device, or a non-transitory or non-transitory computer. A hardware and/or software system that reads a storage medium to acquire or acquire logic and execute instructions contained in a non-transitory storage medium or a non-transitory computer readable storage medium. The processor may also include any controller, state machine, microprocessor, internetwork-based entity, service or feature, or any other analog, digital, and/or mechanical implementation thereof.

In the present invention, the Internet may include a local area network and a wide area Internet, and may be a wired Internet or a wireless Internet, or any combination of these networks.

As shown in FIG. 2, the main updating steps of the network teaching recording data according to the present invention are as follows:

S100: Starting the recording and broadcasting system: after the user (such as a teacher user) logs in using the terminal, various multimedia devices 4 such as an intelligent electronic whiteboard, a teacher terminal screen operation motion capturing program, a camera, a microphone, and the like are put into a working state, the camera There may be more than one, the microphone includes at least one, respectively for capturing the teacher's voice and for capturing the student's voice, and forming a voice data stream format together with the digital time stamp for saving; the screen operation motion capture program may be used to capture the teacher On the terminal, for the operation action of the courseware file, especially the PPT file, the page information, the action information and the timestamp of the operation PPT file are obtained, thereby forming the courseware operation action flow data, and the teaching server of the recording and broadcasting system can be used to generate the digital timestamp. .

S200: Start online teaching: the teacher starts classroom teaching, and the document identification generating unit generates a teaching document ID. In the teaching process, for example, the teacher uses the intelligent electronic whiteboard to display (as a teaching board or explain the problem board), and uses real-time voice to explain, Use real-time interactive voice to communicate, use electronic documents such as PPT documents on the teacher terminal for presentation and explanation, and then conduct multimedia teaching and interactive Q&A with students, and collect and form corresponding data streams through the devices as described in S100.

S300: Recording data saving: During the recording process, the actions on the intelligent whiteboard are transmitted and saved in the form of “action data stream + time stamp”, and the voice during the teaching and interaction process is “voice data stream” through the voice data collecting unit. + timestamp" transmission and saving, the operation actions of electronic documents such as PPT documents involved in the teacher terminal are transmitted and saved by the courseware data collection unit in the form of "electronic document operation data stream + time stamp" for other data streams It can be collected by other data acquisition units, such as video data transmitted and saved as "video data stream + time stamp". All of these data streams throughout the course of the course are tied to the teaching document ID to achieve the identity of the recorded course. These data can be added or deleted as needed. A typical case is that the recorded data includes voice data, video data, and PPT document presentation data. In the prior art, the classified recording and split screen display is relatively mature. Technology. The various data recorded can be saved to a local database or a terminal database, and then uploaded to the remote teaching server through the network, or directly saved to the remote teaching server.

In one example, for the collection of voice data by the voice data collection unit, a voice acquisition device such as various available microphones may be used to collect the voice signal, and the voice signal is converted into voice data and saved in a data stream format. In the case of a single voice source, the gender of the voice source can be marked so that when a subsequent voice update (replacement) operation is performed, the standard voice of the corresponding gender can be selected. For the case of multiple voice sources, the gender of the voice source can be separately identified, and the multiple voice sources can be identified and clustered to form a voice file marked with a voice source, and the time stamp is added and saved separately. The method for separately identifying the voice source can use the prior art, and details are not described herein again.

In an example, the electronic document is preferably a PPT document. Optionally, any other electronic document that can be divided into a course format and can be divided into a page format, such as a WORD document, the electronic document operation data stream + time stamp can be Specifically, the page information + action information + timestamp information of the PPT file.

S400: voice data recognition: for the recorded original voice data, firstly through the voice data recognition unit, the voice model is used for recognition conversion to form original text data, and then the original text data is proofreaded and updated by the text data update unit. When the original text data is formed, the time stamp of the original voice data is added to the text data so that the text content in the text data can be time-located. The text content may be at least one word, word, sentence or paragraph in the text data. The clock data of the time dimension of the audio data can be obtained by the time positioning, that is, the clock parameter of the time point at which a certain data segment in one audio data can be relatively located.

When the recognition conversion is performed, the original speech data identification can be converted into the original text data using various available speech models, and when the speech data recognition conversion is performed, the gender of the speech source is first recognized, and the gender information is added to the text data. in. Proofreading updates for text data include manual proofing, semi-automatic proofreading, and voice proofreading.

S500: Voice data update: The original text data is replaced by a voice data replacement unit using a voice update command (CN106406807A), but the present invention is not limited thereto. The voice data replacement operation includes: accepting a voice update instruction, identifying, in the text data to be updated, all the characters that are the same as the voice update instruction sound, and a time stamp of the text content, and determining the text to be updated among all the recognized texts, Displaying an alternative text list corresponding to the text to be updated, accepting an alternative text selection instruction, performing a replacement operation, forming an updated text data, thereby completing the text update, and after completing the text update, performing the corresponding relationship information according to the text update. The following voice data segment replacement operation.

In the process of completing the above text update, the standard pronunciation information of the updated text is retrieved from the standard voice database, and the corresponding voice data segment in the original voice data is replaced with the standard pronunciation information according to the time stamp of the updated text, and the voice data is updated. , forming new voice data. The standard voice data is obtained by searching from a standard voice database. The standard speech database may include a girls standard speech database, a boys standard speech database, and/or a personalized standard speech database. The personalized standard voice database is a voice model of a specific speaker formed by a standard voice database formed by recording a specific speaker, or by corpus training, and can be used for voice recognition, and can also be used to generate personalized standard voice. database.

When the standard pronunciation information is retrieved from the standard voice data, the corresponding standard voice is selected according to the voice source gender information of the original text data, or other personalized information. Alternatively, the old text content may be empty content, that is, the new text content replacing the empty content is missing, and the added text content is now required. The new text content may be empty content, that is, the old text content that is replaced is redundant, and the deleted text content is now required.

After the step S400 is started, the step S400 may be consistent with the step S500. The step sequence may be adjusted as needed, and is not specifically limited. However, after the S400 is completed, the S401 is performed first. : Courseware data update steps.

S401: Courseware data updating step: for updating the courseware data in the teaching and recording data by means of the courseware data updating unit and using the replacement operation of the new and old courseware.

As shown in FIG. 3, the step of updating the courseware data by the courseware data updating unit further includes the following steps.

The S4011 courseware content update step, through the courseware content update unit, updates the content on the basis of the old courseware, replaces the old content with the new content, forms a new courseware, and records the correspondence data between the old courseware and the new courseware, including the update. The relationship between the front and back pages and the content correspondence before and after each page update. This step can be performed at an appropriate time as needed, but the courseware data update operation can be performed after the courseware content update operation is completed.

S4012 courseware data updating step, obtaining a timestamp identifier of the courseware data in the teaching recording and broadcasting data through the courseware data updating unit, and processing the new courseware according to the corresponding relationship data, adding the timestamp identifier to the new In the courseware, new courseware data is formed.

The S4013 courseware replacement operation step replaces the old courseware data in the teaching recording data with the new courseware data by using the courseware replacement update unit, and implements updating the courseware data of the teaching recording and broadcasting data.

As a preferred example, after the update operation of the courseware data is completed, the text data formed for the step S400: voice data identification may be further used by the text data update unit, after the courseware data update is completed, according to the timestamp Determining the display time of each page of the PPT file, so that the positioning may be performed, and searching for the text data formed by the voice recognition in the display time according to the content correspondence relationship before and after each page update The old content, the old content is replaced with the new content, the updated text data is formed, and the text data formed by the voice recognition is further updated. After the text data is updated, according to the voice replacement method of step S500, the standard voice library is used. The corresponding speech segments in the original speech data are replaced with standard speech data segments to form further updated speech data. In this way, the content of the PPT file involved in the voice data can be synchronously updated according to the modification of the PPT file, so that the updated voice data is adapted to the updated courseware data, and the recording data is further completed. Update.

As an example, for further updating of the text data formed by the voice recognition, before the completion of the replacement of the text content, the correspondence between the old content and the new content in the text data is presented to the administrator, and the administrator confirms whether according to the context. Replace it.

That is, after completing the update of the courseware data, the corresponding relationship data updated according to the courseware data, and the timestamp information, for example, obtaining the start and end time period of an updated PPT page display, and within the start and end time period, Searching for textual data formed by voice conversion, whether there is the same replacement content between the old and new courseware, for example, the text “Ma Sauce” (which may be the reason for the description) in the page of the new courseware is modified to “hot sauce”. At the time of explanation, there is also "hemp sauce" in the content of the page mentioned. That is to say, this "hemp sauce" is inaccurate and should be replaced with "hot sauce", but for reasons of semantic complexity, the present invention is preferred, The system finds such problems, generates prompts or reports to the administrator, and the administrator confirms whether to perform the replacement.

As shown in FIG. 4, in one example, the specific steps of the voice update are as follows:

S110: receiving an instruction

When the identified text data finds a problem, if the text to be updated is “Hu Jian”, the voice update instruction is received, for example, the user can issue a voice instruction of “check Hu Jian” through the unit, and initiate an update of the problem text “Hu Jian”. instruction.

S120: Find text

All texts identical to the specified pronunciation of the voice update instruction are identified in the original text data.

S130: Determine the text

Determine all the text to be updated in the recognized text data.

Wherein, when a plurality of words corresponding to the specified pronunciation of the voice update instruction appear in the text data, the user can clarify which text needs to be updated by further voice instructions. For example, in the text data to be updated, the characters that recognize the pronunciation as "hujian" from the destination are: "Hu Jian", "mutual see", "shoulder shoulder", etc., the user currently wants to recognize the first When a text is updated, a "first" voice can be issued to determine the first recognized text as the text to be updated.

S140: Alternative list

Displaying an alternative text list corresponding to the text to be updated; the candidate text is homophone with the text to be updated.

Wherein, after the text to be updated is selected, a list of alternate characters of the homophone is displayed in the vicinity of the text, so that the user can select the alternative text later. For example, if the first word “Hu Jian” in the text data is “hujian” is determined as the text to be updated, the first word in the text data in this step is “hujian”. "A list of alternative texts is displayed nearby: 1, Fujian; 2, accessories; 3, shoulder pads; 4, mutual see,...

S150: Selected instruction

Receive an alternate text selection instruction.

Among them, the user can speak the position of the alternative text in the alternative text list by voice, and complete the work of selecting the alternative text. For example, use Fujian to replace Hu Jian.

S160: Update text

Updating the to-be-updated text to an alternate text specified by the alternate text selection instruction. In the process of performing the update replacement, the time position information of the text to be updated is marked with a time stamp, so as to accurately locate the time position information of the voice data corresponding to the updated text. Preferably, in the process of updating the text data and the voice data stream, an update history is formed, the update history including update time, update content, update operator, and the like.

S170: Voice clip

From the standard speech library, the standard speech data is searched according to the alternative text, and if a plurality of words or sentences are combined, a new piece of speech data is combined. Preferably, the text data includes gender information of the voice source, and when the search is performed, the girl's pronunciation or the boy's pronunciation, or various voice data such as various trebles and basses may be obtained according to the gender information.

S180: Voice replacement

The new voice data segment is replaced with the corresponding voice data segment in the original voice data according to the previously described time position information to form new voice data. Preferably, due to the standard voice The sound time and the pronunciation time of the replaced voice, even if the text content is exactly the same, the pronunciation time is not necessarily the same. For smooth seamless replacement, the smoothing coefficient can be calculated according to the pronunciation time of the two voice segments, according to the smoothing The coefficient speeds up or slows down the standard pronunciation time so that the pronunciation duration of the same text content is the same after the replacement and before the replacement.

The user uses the terminal to log in to the recording and broadcasting system through the Internet, and can realize the review playback or on-demand playback of the recorded classroom. Of course, for some users, such as online online conference users, these recording classrooms may be process record files of online online conferences, and the recording and playback system will send the teaching file IDs requested by the user for review or on-demand to the teaching server through the Socket encrypted channel, through teaching. The file ID obtains the time-stamped action data stream, the voice data stream, the electronic document such as the PPT file operation data stream, the video data stream, and the text data of the course and is sent to the user terminal requesting the corresponding teaching file ID, and the user terminal is locally Restore (reproduce or simulate reproduce) the entire classroom teaching process based on timestamps. These data streams can be displayed or switched display in each functional area of the user terminal. For video, it can generally be reproduced on the user terminal, but for the operation of the electronic whiteboard, simulation reproduction can be realized by the simulation program of the electronic whiteboard. For the operation of the PPT file, the replay display can be performed on the local terminal.

Of course, the user can choose to play only at least one of these data streams, for example, can only listen to the voice. For text data, it can be displayed in a specific area of the user terminal in the form of subtitles, such as a video display area.

In one example, text data (formed by voice recognition) for performing caption display may be displayed in a specific editable area so that the user can perform a selected operation or the like, such that for the found non-standard voice data or text information, Just select the appropriate text message to get feedback. After receiving the feedback from the user, the administrator of the recording and broadcasting system verifies. If it is found that there is an error, the previous text data and the updating process of the voice data are repeated, so that the text data and the voice data can be continuously improved and improved.

In the above embodiment, the terminal and the server are configured to be connected to a communication network including the Internet. Therefore, the medium may be a program that carries the program code in a streaming manner via the communication network. In the case where the program code is downloaded from the communication network as described above, the program for downloading may be stored in the main device or may be installed from another recording medium.

Furthermore, the present invention can be realized by the above-described program code in the form of a computer data signal embodied in an electronic transmission embodied in a carrier wave.

The preferred embodiments of the present invention have been described above, and are intended to be illustrative and not to limit the scope of the present invention. Modifications and improvements are intended to be included within the scope of the appended claims.

Industrial applicability

Claims

An apparatus for updating teaching recording data, comprising a voice data updating unit and a courseware data updating unit, wherein

The voice data updating unit is configured to update voice data in the teaching and recording data,

The courseware data updating unit is configured to update courseware data in the teaching and recording data,

The updating of the voice data is implemented by a replacement operation of the text content formed by the voice data identification,

The update of the courseware data is achieved by replacing the old courseware with a new courseware.
The device according to claim 1 wherein

The voice data update unit further includes

a voice data identification unit, configured to convert the original voice data identification into original text data according to the voice recognition model;

a text data updating unit, configured to perform proofreading on the original text data, and replace the old text content that needs to be updated into an updated new text content to form updated text data;

a voice data replacing unit, configured to replace, by using standard voice data of the new text content, a voice data segment of the old text content in the original voice data, to form updated voice data, to implement data recording and playing on the teaching Update of voice data.
The device according to claim 2, characterized in that

The courseware data updating unit further includes

The courseware content update unit is used to update the content on the basis of the old courseware, replace the old content with the new content, form a new courseware, and record the correspondence data between the old courseware and the new courseware, including between the pages before and after the update. The relationship between the relationship and the content before and after each page update;

a courseware data updating unit, configured to acquire a timestamp identifier of the courseware data in the teaching recording and broadcasting data, and process the new courseware according to the corresponding relationship data, and add the timestamp identifier to the new courseware to form a new Courseware data;

The courseware replacement updating unit replaces the old courseware data in the teaching recording data with the new courseware data, and updates the courseware data of the teaching recording data.
The device according to claim 3, characterized in that

The courseware is preferably a PPT file, and the standard voice data is obtained by searching from a standard voice database, and the voice data and the courseware data are separately saved. Preferably, the voice data is “voice data stream+” "Timestamp" format is saved, the courseware data is "courseware operation data stream" +Timestamp" format is saved.
The device according to claim 4, characterized in that

The text data updating unit is further configured to: after the courseware data update is completed, determine a presentation time of each page of the PPT file according to the timestamp identifier, and search for the content according to the content correspondence relationship before and after each page update. The old content in the text data in the presentation time is replaced with the new content, and the updated text data is formed to further update the text data, so that the updated voice data is adapted to the updated courseware data.
The device according to claim 5, characterized in that

The further updating of the text data reveals the correspondence between the old content and the new content in the text data to the administrator before the replacement is completed, and the administrator confirms whether to replace according to the context.
The device according to claim 6 wherein

The updated text data is displayed on the screen in a subtitle manner according to the timestamp identifier, preferably displayed on a screen area in which the video data is played, and more preferably, the text data is editable in an optional manner. The way it is displayed is displayed in a specific area of the screen.
The device according to claim 7, wherein

The device further includes an error information feedback unit,

The error information feedback unit may: when the user uses the terminal to play the teaching and recording data, select and submit the error text content in the found updated text data, and the feedback content is updated again after being confirmed by the administrator. The text data is updated, and the updated voice data is updated again by the voice data replacement unit.
The device according to claim 8 wherein

When the text data and the voice data are updated, an update history is formed, which may include update time, update content, update operator, issue finder, and the like.
The device according to claim 4, characterized in that

The voice data replacing unit is configured to calculate a smoothing coefficient according to the pronunciation time of the replaced old text content in the original voice data and the pronunciation time of the standard voice data of the new text content, and according to the smoothing coefficient, The pronunciation time of the new text content is adjusted, thereby smoothing and synchronizing the voice data before and after the replacement.