CN112241462B - Knowledge point mark generation system and method thereof - Google Patents

Abstract

A knowledge point mark generation system and method thereof, wherein at least one first key word emphasized in a classroom by text, at least one second key word emphasized in a classroom by sound, at least one first candidate word repeatedly appearing in a classroom by text, and at least one second candidate word repeatedly appearing in a classroom by sound are analyzed according to their corresponding weights to obtain label words, and corresponding to the time segment in which the label words appear, knowledge point marks are set on the time axis of the audio and video files shot in the classroom to form an audio and video file with knowledge point marks. Therefore, learners can understand the knowledge points of the class and the segments in which they exist without browsing the audio and video files of the entire class, which is convenient for learners to focus on learning or reviewing.

Description

Knowledge point tag generation system and method

Technical Field

The present invention relates to a tag generation system and method thereof, in particular to a knowledge point tag generation system and method thereof.

Background technique

With the advancement of technology and the development of the Internet, learners can study or review through the audio and video files recorded during the class after the class.

Currently, when learners want to study, they can only search by the title of the video file. However, the title can only provide limited information. Learners may need to browse the entire video to know whether it meets their learning needs, which is time-consuming.

In addition, when learners want to review, they usually do not know the playback time of the key points (i.e., knowledge points) of the class they want to review in the audio and video files, so they have to continuously drag the playback progress pointer on the playback timeline or fast-forward the video to search for the clips they want to watch, which obviously causes inconvenience to learners.

In summary, it can be seen that the prior art has the problem that learners need to browse the entire class audio and video files to know whether they meet their learning needs and need to search for the segments where the knowledge points are located by dragging the playback progress pointer or fast-forwarding the video, which causes inconvenience to learners. Therefore, it is necessary to propose improved technical means to solve this problem.

Summary of the invention

The invention discloses a knowledge point mark generation system and a method thereof.

First, the present invention discloses a knowledge point mark generation system, which includes: a capture device, a speech recognition device, a processing device and an integration device. The capture device is used to continuously capture and analyze computer screen images, projection images and/or blackboard images in the classroom to continuously obtain text, and capture at least one first key word in the text based on the font and/or font color of the computer screen image, projection image and/or blackboard image and the clicked text. The speech recognition device is used to continuously receive sound signals in the classroom, and continuously convert the sound signals into text strings through speech-to-text, and determine the identity of the sound signal through voiceprint recognition or sound source recognition, and capture at least one second key word in the text string based on the identity of the sound signal and/or multiple preset words. The processing device is used to analyze the text continuously obtained by the capture device in a statistical manner after the class is over to obtain at least one first candidate word; analyze the text string continuously obtained by the voice recognition device in a statistical manner after the class is over to obtain at least one second candidate word; and analyze at least one first key word, at least one second key word, at least one first candidate word and at least one second candidate word according to their corresponding weights to obtain a label word. The integration device is used to analyze the time segment of each sentence with the label word in the text string continuously obtained by the voice recognition device after the class is over, and when the time difference between the adjacent time segments is less than a specific time length, merge the adjacent time segments into a time interval, and then set a plurality of knowledge point marks corresponding to the unmerged time segments and time intervals on the time axis of the audio and video file shot in the class to form an audio and video file with the knowledge point marks.

In addition, the present invention discloses a knowledge point mark generation method, the steps of which include: providing a knowledge point mark generation system, which includes a capture device, a speech recognition device, a processing device and an integration device; the capture device continuously captures and analyzes computer screen images, projection images and/or blackboard images in the classroom to continuously obtain text; the capture device captures at least one first key word in the text based on the font and/or font color in the computer screen image, projection image and/or blackboard image and the clicked text; the speech recognition device continuously receives sound signals in the classroom, and continuously converts the sound signals into text strings through speech-to-text conversion; the speech recognition device determines the identity of the sound signal through voiceprint recognition or sound source recognition; the speech recognition device captures at least one second key word in the text string based on the identity of the sound signal and/or multiple preset words; the processing device analyzes the text through statistical methods after the class ends The text continuously obtained by the capture device is used to obtain at least one first candidate word; the processing device analyzes the text string continuously obtained by the voice recognition device in a statistical manner after the class is over to obtain at least one second candidate word; the processing device performs an analysis procedure on at least one first key word, at least one second key word, at least one first candidate word and at least one second candidate word according to their corresponding weights to obtain a label word; and the integration device analyzes the time segment of each sentence with the label word appearing in the text string continuously obtained by the voice recognition device after the class is over, and when the time difference between adjacent time segments is less than a specific time length, the adjacent time segments are merged into a time interval, and then, a plurality of knowledge point marks corresponding to the unmerged time segments and time intervals are set on the time axis of the audio and video file shot in the class to form an audio and video file with the knowledge point marks.

The system and method disclosed in the present invention are as described above. The difference from the prior art is that the present invention obtains at least one label word by analyzing at least one first key word emphasized in the classroom by text, at least one second key word emphasized in the classroom by sound, at least one first candidate word repeatedly appearing in the classroom by text, and at least one second candidate word repeatedly appearing in the classroom by sound according to their corresponding weights, and sets knowledge point marks on the time axis of the audio and video files shot in the classroom corresponding to the time segment and time interval when the label words appear, so as to form an audio and video file with knowledge point marks.

Through the above-mentioned technical means, the present invention allows learners to understand the knowledge points and existing fragments of the class without browsing the audio and video files of the entire class, which is convenient for learners to focus on learning or reviewing.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a system block diagram of an embodiment of a knowledge point tag generation system of the present invention.

2A and 2B are method flow charts of an embodiment of a knowledge point tag generation method executed by the knowledge point tag generation system of FIG. 1 .

[List of Reference Numerals]

50 Live Streaming Module

60 Marking Module

70 Transmission Module

100 Knowledge Point Marking Generation System

110 Capture Device

112 Photography Module

114 Parsing module

120 Voice recognition device

122 Microphone Module

124 Conversion Module

126 Voiceprint Recognition Module

130 Processing device

140 Integrated Device

150 Client

160 Behavior Detection Device

162 Photography Module

164 Parsing module

Step 210 provides a knowledge point tag generation system, which includes: a capture device, a speech recognition device, a processing device, and an integration device.

Step 220: The capture device continuously captures and analyzes computer screen images, projection images and/or blackboard images in the classroom to continuously obtain text

Step 230: The capture device captures at least one first keyword in the text based on the font and/or font color in the computer screen image, the projection image and/or the blackboard image and the clicked text.

Step 240: The voice recognition device continuously receives voice signals in the classroom and continuously converts the voice signals into text strings through voice-to-text conversion.

Step 250: The voice recognition device determines the identity of the person who sent the voice signal by voiceprint recognition or sound source recognition.

Step 260: The voice recognition device extracts at least one second key word from the text string based on the identity of the person emitting the voice signal and/or a plurality of preset words.

Step 270: After the class is over, the processing device analyzes the text continuously obtained by the capture device in a statistical manner to obtain at least one first candidate vocabulary.

Step 280: After the class is over, the processing device analyzes the text strings continuously obtained by the speech recognition device in a statistical manner to obtain at least one second candidate vocabulary.

Step 290 The processing device performs an analysis process on at least one first keyword, at least one second keyword, at least one first candidate word, and at least one second candidate word according to their corresponding weights to obtain a tag word.

Step 300 After the class is over, the integration device combines the adjacent time segments into a time interval based on the time segments of each sentence with the label vocabulary appearing in the text string continuously obtained by the voice recognition device, and when the time difference between the adjacent time segments is less than a specific time length, then sets a plurality of knowledge point marks corresponding to the uncombined time segments and time intervals on the time axis of the audio and video file shot in the class to form an audio and video file with the knowledge point marks

Detailed ways

The following will describe the implementation modes of the present invention in detail with reference to the accompanying drawings and embodiments, so that the implementation process of how the present invention applies technical means to solve technical problems and achieve technical effects can be fully understood and implemented accordingly.

Before explaining the knowledge point marking generation system and method disclosed in the present invention, the terms of the present invention are first explained. The knowledge point described in the present invention refers to the basic unit of information transmission in the course. Therefore, it can be known that knowledge points play an important role in the learning navigation of the course. The present invention can analyze the behaviors and events that occur in the classroom according to their corresponding weights to obtain the knowledge points of the classroom, so that learners can understand the knowledge points of the classroom and the existing fragments thereof when learning or reviewing through the audio and video files recorded during teaching, without browsing the audio and video files of the entire classroom. In addition, the capture device, voice recognition device and behavior detection device described in the present invention can be started and operated synchronously at the beginning of each class, and stopped synchronously after each class ends.

Please refer to Figures 1, 2A and 2B. Figure 1 is a system block diagram of an embodiment of the knowledge point tag generation system of the present invention, and Figures 2A and 2B are method flow charts of an embodiment of the knowledge point tag generation system of Figure 1 executing the knowledge point tag generation method. In this embodiment, the knowledge point tag generation system 100 includes: a capture device 110, a speech recognition device 120, a processing device 130 and an integration device 140 (step 210). Among them, the capture device 110 is connected to the processing device 130, the speech recognition device 120 is connected to the processing device 130, and the processing device 130 is connected to the integration device 140.

The capture device 110, the speech recognition device 120, the processing device 130 and the integration device 140 can be implemented in various ways, including software, hardware, firmware or any combination thereof. The technology proposed in the embodiments uses software or firmware that can be stored on a machine-readable storage medium, such as a read-only memory (ROM), a random access memory (RAM), a magnetic disk storage medium, an optical storage medium, a flash memory device, etc., and can be executed by one or more general or special programmable microprocessors. The capture device 110 and the processing device 130, the speech recognition device 120 and the processing device 130, and the processing device 130 and the integration device 140 can be connected to each other wirelessly or wiredly to transmit signals and data.

The capture device 110 continuously captures and analyzes computer screen images, projection images and/or blackboard images in the classroom to continuously obtain text (step 220). In more detail, the capture device 110 may include a photographing module 112 and an analysis module 114, and the photographing module 112 is connected to the analysis module 114. Among them, the photographing module 112 can be used to continuously shoot images on the podium in each class, and the content of the image on the podium may include: projection images and/or classroom blackboards or whiteboards, so as to capture the projection images and/or blackboard images, but this embodiment is not used to limit the present invention and can be adjusted according to actual needs. For example, when taking a computer course, the photographing module 112 can be used to continuously shoot the computer screen operated by the teacher to capture the computer screen image. It should be noted that the content continuously shot by the photographing module 112 must include auxiliary teaching materials with text provided by the teacher during teaching, such as: handouts, slides, blackboards or whiteboards, etc. The analysis module 114 can continuously receive and analyze the computer screen images, projection images and/or blackboard images captured by the photography module 112 to obtain the text in each computer screen image, each projection image and/or each blackboard image to generate corresponding text. The analysis module 114 uses optical character recognition (OCR) technology to extract the text in each computer screen image, each projection image and/or each blackboard image to form text (i.e., image to text conversion).

The capture device 110 captures at least one first key word in the text based on the font and/or font color in the computer screen image, the projection image and/or the blackboard image and the selected text (step 230). More specifically, since the text in the auxiliary teaching materials provided by the teacher during the classroom teaching may have different fonts and/or font colors to strengthen the transmission of certain knowledge points (i.e., key points), so that the learner can understand the knowledge points (i.e., key points) that the teacher wants to convey through the text with special fonts and/or font colors, the capture device 110 may capture at least one first key word (i.e., possible knowledge point) in the text based on the font and/or font color in the computer screen image, the projection image and/or the blackboard image, wherein the font may include but is not limited to font size, font weight, font type, whether the font is italic, whether the font has an underline, and whether the font has a text effect, and each first key word is a word composed of adjacent words with special fonts and/or font colors. In addition, in this embodiment, the text in the computer screen image, projection image and/or blackboard image that the teacher clicks (including pointing or selecting with the hand, laser pen or computer cursor) when teaching in the classroom can also be regarded as the knowledge point (i.e., key point) that the teacher wants to emphasize and convey. Therefore, the capture device 110 can also capture at least one first key word (i.e., possible knowledge point) in the text based on the text clicked in the computer screen image, projection image and/or blackboard image, wherein each first key word is a word composed of the clicked text. It should be noted that each first key word captured by different methods (for example: special fonts, special font colors and clicked text) may have the same or different corresponding weights in the subsequent processing device 130 for the analysis procedure, and can be adjusted according to actual needs.

The speech recognition device 120 continuously receives sound signals in the classroom, and continuously converts the sound signals into text strings through speech-to-text conversion (step 240). In more detail, the speech recognition device 120 may include a microphone module 122 and a conversion module 124. The microphone module 122 may be used to continuously receive the sounds (i.e., sound signals) emitted by teachers and learners in the classroom, and the conversion module 124 may convert the sound signals continuously received by the microphone module 122 into text strings through speech-to-text conversion. The microphone module 122 may include a plurality of microphone units (not shown) for configuration in various places in the classroom to fully receive the sounds (i.e., sound signals) emitted by teachers and learners in the classroom. The number and configuration positions of the microphone units may be adjusted according to actual needs.

The speech recognition device 120 determines the identity of the person who sends the sound signal by voiceprint recognition or sound source recognition (step 250). In more detail, since the speech recognition device 120 may also include a voiceprint recognition module 126, it is used to identify that the sound signal received by the microphone module 122 is sent by the teacher or the learner, and then determine that the text string converted by the conversion module 124 is the words spoken by the teacher or the learner. In addition, in this embodiment, since the teacher's position is usually near the podium (i.e., the front position of the classroom), and the learner's position is usually in the middle or back of the classroom compared to the teacher's position, the microphone module 122 can also be used to determine the sound source position and then determine the identity of the sound signal. In more detail, since the microphone module 122 may include multiple microphone units arranged in various places in the classroom, the microphone module 122 can determine the position of the sound signal based on the time difference when the microphone units receive the same sound signal and the relative configuration positions of the microphone units, and then determine whether the sound signal is emitted by the teacher or the learner based on the position of the sound signal, and then determine that the text string converted by the conversion module 124 is the words spoken by the teacher or the learner.

The speech recognition device 120 extracts at least one second key word from the text string based on the identity of the person who sends the sound signal and/or multiple preset words (step 260). In more detail, since the text string corresponding to the sound signal sent by the teacher and/or contains preset words (for example: special, key, key point, must memorize, test point, etc.) and the corresponding text string may contain the knowledge points of the class, the probability is very high. Therefore, the speech recognition device 120 can extract at least one second key word (i.e., possible knowledge point) from the text string corresponding to the sound signal sent by the teacher and/or contains preset words (for example: special, key, key point, must memorize, test point, etc.). The extraction of the second key word can be obtained by semantic analysis, but this embodiment is not used to limit the present invention. In addition, in another embodiment, the text string corresponding to the sound signal with a higher volume sent by the teacher during the teaching process can also be used as one of the parameters for extracting the second key word.

It should be noted that the text string corresponding to the sound signal emitted by the teacher and/or each second key word extracted from the text string corresponding to the preset vocabulary (for example: special, key, key point, must-memorize, test point, etc.) may have the same or different corresponding weights in the subsequent analysis process performed by the processing device 130, and can be adjusted according to actual needs.

After the class is over, the processing device 130 analyzes the text continuously obtained by the capture device 110 in a statistical manner to obtain at least one first candidate vocabulary (step 270). In more detail, the processing device 130 first counts the vocabulary in the texts obtained by the capture device 110, and then defines the first few vocabulary with higher frequency of occurrence as the first candidate vocabulary (i.e., possible knowledge points). It should be noted that since the frequency of occurrence of any word is too high, the word may be the main axis of the class and is not suitable to be the label vocabulary described in the subsequent steps. Therefore, when the processing device 130 analyzes the text continuously obtained by the capture device 110 in a statistical manner after the class is over, if it is determined that the frequency of occurrence of any word exceeds a preset value, the word is excluded from becoming the first candidate vocabulary, wherein the size of the preset value can be adjusted according to actual needs.

After the class is over, the processing device 130 analyzes the text strings continuously obtained by the speech recognition device 120 in a statistical manner to obtain at least one second candidate vocabulary (step 280). In more detail, the processing device 130 first counts the vocabulary in the text strings obtained by the speech recognition device 120, and then defines the first few words with higher frequency of occurrence as second candidate vocabulary (i.e., possible knowledge points). It should be noted that since the frequency of occurrence of any word is too high, the word may be the main axis of the class and is not suitable to be the label vocabulary described in the subsequent steps. Therefore, when the processing device 130 analyzes the text strings continuously obtained by the speech recognition device 120 in a statistical manner after the class is over, if it is determined that the frequency of occurrence of any word exceeds a preset value, the word is excluded from becoming the second candidate vocabulary, wherein the size of the preset value can be adjusted according to actual needs.

The processing device 130 performs an analysis procedure on at least one first key word, at least one second key word, at least one first candidate word, and at least one second candidate word according to their corresponding weights to obtain a label word (step 290). In more detail, since the probability of the first key word, the second key word, the first candidate word, and the second candidate word becoming a knowledge point is different, therefore, in the analysis procedure for determining the knowledge point of the class, the weights corresponding to the first key word, the second key word, the first candidate word, and the second candidate word are different, and can be adjusted according to actual needs. Among them, the analysis procedure determines the knowledge point (i.e., label word) of the class through the first key word, the second key word, the first candidate word, and the second candidate word and their corresponding weights, and the number of knowledge points (i.e., label words) can be adjusted according to actual needs.

When the number of knowledge points (i.e., label words) is one, the integration device 140 merges the adjacent time segments into a time interval according to the time segment of each sentence with the label word appearing in the text string continuously obtained by the speech recognition device 120 after the class ends, and when the time difference between the adjacent time segments is less than the specific time length, then sets a plurality of knowledge point marks corresponding to the unmerged time segments and time intervals on the time axis of the audio and video file shot in the class to form an audio and video file with the knowledge point marks (step 300). In more detail, the knowledge point mark generation system 100 may also include a camera device (not shown) for shooting the audio and video files to be placed on the platform or website for learners to learn or review, and for shooting the streaming audio and video required for live broadcasting the class (i.e., the streaming audio and video of the class can be simultaneously broadcast and stored to generate the audio and video file of the class after the class ends), wherein the camera device, the capture device 110 and the speech recognition device 120 can be started and run synchronously at the beginning of each class, and stopped synchronously after each class ends. Through the above step 290, a knowledge point (i.e., a label word) of the class can be obtained. Therefore, the integration device 140 can search the time segments in which each sentence with the knowledge point (i.e., the label word) appears in the text strings obtained by the speech recognition device 120, and when the time difference (i.e., the time interval) between the adjacent time segments is less than the specific time length, the adjacent time segments are merged into a time interval, wherein the size of the specific time length can be adjusted according to actual needs. Then, after the class ends, the integration device 140 can set multiple knowledge point marks on the time axis of the audio and video file captured by the camera device in the classroom according to the above-mentioned unmerged time segments and time intervals, so as to form an audio and video file with these knowledge point marks.

Among them, when the number of label words is a plurality, the integration device 140 can find the unmerged time segment and time interval corresponding to each label word according to the above process, and then distinguish the knowledge point marks corresponding to different label words according to different colors, so as to facilitate learners to distinguish the knowledge point marks corresponding to different label words. For example, when the label words are "Fourier transform" and "Laplace transform", the knowledge point mark corresponding to "Fourier transform" set on the time axis of the audio and video file can be but not limited to yellow, and the knowledge point mark corresponding to "Fourier transform" can be but not limited to green, but this example is not used to limit the present invention.

In this embodiment, in addition to determining the label vocabulary of the class by the first key vocabulary, the second key vocabulary, the first candidate vocabulary and the second candidate vocabulary and their corresponding weights, the behavior of each learner in the class, such as looking up at the blackboard, looking down to write notes, etc., can also be added as one of the parameters for determining the label vocabulary of the class, as described in detail below. In this embodiment, the knowledge point mark generation system 100 can also include a behavior detection device 160, and the knowledge point mark generation method can also include: the behavior detection device 160 continuously receives and analyzes the learner classroom image in the class to obtain the behavior recognition signal of each learner; when the behavior detection device 160 obtains the behavior recognition signal of any learner as looking up or writing notes, the processing device 130 generates a behavior string according to the text string obtained by the voice recognition device 120 within an expected time interval before and after; the processing device 130 analyzes the behavior strings by statistical methods, the head-up rate of the whole class and/or the proportion of the whole class writing notes to obtain at least one fourth candidate vocabulary; and the processing device 130 also adds the at least one fourth candidate vocabulary to the analysis program according to its corresponding weight to obtain the label vocabulary.

In more detail, the behavior detection device 160 may include a camera module 162 and an analysis module 164, wherein the camera module 162 is connected to the analysis module 164. The camera module 162 may be used to continuously capture images of each learner's location in the classroom (i.e., images of each learner in the classroom, i.e., learner classroom images) in each class, and the behavior recognition signal of each learner (i.e., the dynamic behavior of each learner) may be obtained by analyzing the images continuously captured by the camera module 162. When a learner looks up at a projection image, blackboard and/or whiteboard or lowers his head to write notes, it means that the content taught by the teacher at that time may be the key point (i.e., knowledge point). Therefore, when the behavior detection device 160 obtains a behavior recognition signal of any learner looking up at a projection image, blackboard and/or whiteboard or lowering his head to write notes, the processing device 130 can generate a behavior string based on the text string obtained by the speech recognition device 120 within an expected time interval before and after the time point when the learner looks up at the projection image, blackboard and/or whiteboard or lowers his head to write notes, wherein the size of the expected time interval can be adjusted according to actual needs. The processing device 130 can first count the words in the behavior strings it generates, and then define the first few words with higher frequency of occurrence as the fourth candidate words (i.e., possible knowledge points).

In addition, when the number of learners who look up at the projection image, blackboard and/or whiteboard or look down to take notes at the same time is greater, it means that the text string obtained by the speech recognition device 120 before and after the time point is more likely to be the knowledge point of the class. Therefore, in the process of obtaining the fourth candidate vocabulary, the processing device 130 needs to add the head-up rate of the whole class and/or the proportion of the whole class taking notes as a reference factor to obtain at least one fourth candidate vocabulary. Then, the processing device 130 can also add the at least one fourth candidate vocabulary to the analysis program according to its corresponding weight to obtain the label vocabulary, wherein the weight corresponding to the at least one fourth candidate vocabulary can be adjusted according to actual needs.

Furthermore, in this embodiment, in addition to determining the label vocabulary of the class through the first key vocabulary, the second key vocabulary, the first candidate vocabulary, the second candidate vocabulary, the fourth candidate vocabulary and their corresponding weights, the behavior of each learner who studies through live broadcasting can also be added to one of the parameters that determine the label vocabulary of the class, for example: at least one tag information is set during the live streaming video, and the detailed description is as follows. In this embodiment, the knowledge point tag generation system 100 can also include at least one user terminal 150, wherein each learner can learn through live broadcasting through the user terminal 150 he owns.

Each client 150 includes a live broadcast module 50, a marking module 60 and a transmission module 70. The knowledge point marking generation method may also include: the live broadcast module 50 of each client 150 continuously broadcasts the streaming video in the class; the marking module 60 of each client 150 allows at least one marking information to be set during the live streaming video; the transmission module 70 of each client 150 transmits the time point of the at least one marking information to the processing device 130; the processing device 130 generates a marking string based on the text string obtained by the speech recognition device 120 within a predetermined time interval before and after the time point of the at least one marking information set by each client 150 after the class ends; the processing device 130 analyzes the marking strings in a statistical manner to obtain at least one third candidate word; and the processing device 130 also adds the at least one third candidate word to the analysis program according to its corresponding weight to obtain a label word. Among them, the number of client 150 can be adjusted according to actual needs. In order to avoid over-complexity in the diagram of FIG. 1 , only two user terminals 150 are drawn here, and the actual number of user terminals 150 can be adjusted according to actual needs.

In other words, when each learner learns through live broadcast through the user terminal 150 owned by him (i.e., during the live streaming video process), he can set marking information for the part taught by the teacher in the current period at any time (similar to the concept of writing notes with his head down as mentioned above). Since when the learner sets the marking information, it means that the content taught by the teacher at that time may be the focus (i.e., the knowledge point), therefore, when any learner sets the marking information through the user terminal 150 owned by him, the processing device 130 can generate a marking string based on the text string obtained by the speech recognition device 120 within a predetermined time interval before and after the time point when the learner sets the marking information, wherein the size of the predetermined time interval can be adjusted according to actual needs. The processing device 130 can first count the words in the marked strings it has obtained, and then define the first few words with higher frequency of occurrence as third candidate words (i.e., possible knowledge points). Then, the processing device 130 can also add the at least one third candidate word to the analysis program according to its corresponding weight to obtain a label word, wherein the weight corresponding to the at least one third candidate word can be adjusted according to actual needs.

It should be noted that, except for the explanation of the causal relationship, the knowledge point tag generation method of this embodiment can perform the above steps in any order.

In summary, it can be seen that the difference between the present invention and the prior art lies in that at least one first key word emphasized in the classroom by text, at least one second key word emphasized in the classroom by sound, at least one first candidate word repeatedly appearing in the classroom by text, and at least one second candidate word repeatedly appearing in the classroom by sound are analyzed according to their corresponding weights to obtain label words, and corresponding to the time segment and time interval of the appearance of the label words, knowledge point marks are set on the time axis of the audio and video files shot in the classroom to form an audio and video file with knowledge point marks. This technical means can solve the problems existing in the prior art, so that learners can understand the knowledge points of the class and the existing fragments therein without browsing the audio and video files of the entire class, which is convenient for learners to focus on learning or reviewing.

Although the present invention is disclosed as above with the aforementioned embodiments, it is not intended to limit the present invention. Any person skilled in the art may make some changes and modifications without departing from the spirit and scope of the present invention. Therefore, the patent protection scope of the present invention shall be subject to that defined in the claims attached to this specification.

Claims (10)

1. A knowledge point tag generation system, comprising:

The capturing device is used for continuously capturing and analyzing the computer picture image, the projection image and/or the board image in a classroom so as to continuously obtain texts, and capturing at least one first key word in the texts based on the character types and/or the character colors in the computer picture image, the projection image and/or the board image and the clicked characters;

The voice recognition device is used for continuously receiving a voice signal in the classroom, continuously converting the voice signal into a character string in a voice-to-character mode, judging the identity of the voice signal in a voiceprint recognition or sound source recognition mode, and capturing at least one second key word in the character string based on the identity of the voice signal and/or a plurality of preset words;

The processing device is used for analyzing the text continuously acquired by the acquisition device in a statistical mode after the class is finished so as to acquire at least one first candidate vocabulary; after the class is finished, the character strings continuously obtained by the voice recognition device are analyzed in a statistical mode so as to obtain at least one second candidate vocabulary; the at least one first key word, the at least one second key word, the at least one first candidate word and the at least one second candidate word are subjected to an analysis program according to the corresponding weights so as to obtain a tag word; and

The integrating device is used for generating a time section of each sentence with the tag word in the word string continuously acquired by the voice recognition device after the class is finished, combining the adjacent time sections into a time section when the time difference between the adjacent time sections is smaller than a specific time length, and setting a plurality of knowledge point marks corresponding to the time sections which are not combined and the time section on a time axis of the video file shot in the class so as to form the video file with the knowledge point marks.

2. The knowledge point tag generation system of claim 1, wherein the knowledge point tag generation system further comprises:

At least one user terminal, each user terminal includes:

The live broadcast module is used for continuously broadcasting streaming video and audio in the class;

The marking module is used for allowing the setting of the at least one marking information in the process of live broadcasting the streaming video and audio; and

The transmission module is used for transmitting the set time point of the at least one piece of marking information to the processing device;

After the class is finished, the processing device generates a marked word string according to the word string acquired by the voice recognition device in a preset time interval before and after the time point of setting the at least one marked information by each user side; analyzing the marked word strings in a statistical mode to obtain at least one third candidate word; and adding the at least one third candidate vocabulary into the analysis program according to the corresponding weight to obtain the tag vocabulary.

3. The knowledge point marking generation system of claim 1 or 2, wherein the knowledge point marking generation system further comprises:

The behavior detection device is used for continuously receiving and analyzing the learner classroom images in the classroom so as to acquire behavior identification signals of each learner;

When the behavior detection device obtains that any behavior identification signal of the learner is head-up or writing, the processing device generates a behavior string according to the character string obtained by the voice recognition device in the expected time interval; analyzing the behavior word strings through a statistical mode, a whole-class head-up rate and/or a whole-class written note proportion to obtain at least one fourth candidate vocabulary; and adding the at least one fourth candidate vocabulary into the analysis program according to the corresponding weight to obtain the tag vocabulary.

4. The knowledge point tag generation system of claim 1, wherein when the processing device performs the analysis procedure on the at least one first keyword, the at least one second keyword, the at least one first candidate vocabulary, and the at least one second candidate vocabulary according to their corresponding weights to obtain a plurality of tag vocabularies, the integrating device distinguishes the knowledge point tags corresponding to different tag vocabularies according to different colors.

5. The knowledge point mark generation system according to claim 1, wherein when the processing device statistically analyzes the text obtained continuously by the capturing device or the text string obtained continuously by the voice recognition device after the end of the class, if it is determined that the frequency of occurrence of any vocabulary exceeds a predetermined value, the vocabulary is excluded as the first candidate vocabulary or the second candidate vocabulary.

6. The knowledge point mark generation method is characterized by comprising the following steps:

providing a knowledge point mark generation system, which comprises a capturing device, a voice recognition device, a processing device and an integration device;

The capturing device continuously captures and analyzes the computer picture image, the projection image and/or the board book image in the class so as to continuously obtain the text;

The capturing device captures at least one first keyword in the text based on the computer picture image, the projection image and/or the font color in the blackboard-writing image and the clicked text;

the voice recognition device continuously receives a voice signal in the class and continuously converts the voice signal into a character string in a voice-to-character mode;

The voice recognition device judges the identity of the sound signal through voiceprint recognition or sound source recognition;

the voice recognition device captures at least one second key word in the word string based on the identity of the sent sound signal and/or a plurality of preset words;

The processing device analyzes the text continuously obtained by the capturing device in a statistical mode after the class is finished so as to obtain at least one first candidate vocabulary;

the processing device analyzes the character strings continuously obtained by the voice recognition device in a statistical mode after the class is finished so as to obtain at least one second candidate vocabulary;

The processing device carries out an analysis procedure on the at least one first key word, the at least one second key word, the at least one first candidate word and the at least one second candidate word according to the corresponding weights so as to obtain a tag word; and

The integration device is used for generating a time zone of each sentence with the tag word in the word string continuously acquired by the voice recognition device after the class is finished, combining the adjacent time zones into a time zone when the time difference between the adjacent time zones is smaller than a specific time length, and setting a plurality of knowledge point marks corresponding to the time zone which is not combined and the time zone on a time axis of the video file shot in the class so as to form the video file with the knowledge point marks.

7. The knowledge point tag generation method of claim 6, wherein the knowledge point tag generation system further comprises at least one user side, each of the user sides comprises a live broadcast module, a tag module, and a transmission module, the knowledge point tag generation method further comprising:

the live broadcast module of each user terminal continuously broadcasts streaming video and audio in the class;

The marking module of each user terminal allows setting the at least one marking information in the process of directly broadcasting the streaming video;

the transmission module of each user side transmits the set time point of the at least one piece of marking information to the processing device;

The processing device generates a marked word string according to the word string acquired by the voice recognition device in a preset time interval before and after the user terminal sets the time point of the at least one marked information after the class is finished;

The processing device analyzes the marked word strings in a statistical mode to obtain at least one third candidate vocabulary; and

The processing device also adds the at least one third candidate vocabulary into the analysis program according to the corresponding weight to obtain the tag vocabulary.

8. The knowledge point mark generation method according to claim 6 or 7, wherein the knowledge point mark generation system further comprises a behavior detection device, the knowledge point mark generation method further comprising:

the behavior detection device continuously receives and analyzes learner classroom images in the classroom so as to acquire behavior identification signals of each learner;

When the behavior detection device obtains any behavior identification signal of the learner as a head-up or writing mark, the processing device generates a behavior string according to the character strings obtained by the voice recognition device in the expected time interval;

The processing device analyzes the behavior word strings through a statistical mode, a full-shift head-up rate and/or a full-shift writing ratio so as to obtain at least one fourth candidate vocabulary; and

The processing device also adds the at least one fourth candidate vocabulary into the analysis program according to the corresponding weight to obtain the tag vocabulary.

9. The method of claim 6, wherein when the processing device performs the analysis procedure on the at least one first keyword, the at least one second keyword, the at least one first candidate vocabulary, and the at least one second candidate vocabulary according to weights corresponding to the at least one first candidate vocabulary to obtain a plurality of tag vocabularies, the integrating device distinguishes the knowledge point tags corresponding to different tag vocabularies according to different colors.

10. The knowledge point tag generation method according to claim 6, wherein when the processing device statistically analyzes the text obtained continuously by the capturing device or the text string obtained continuously by the voice recognition device after the end of the class, if it is determined that the frequency of occurrence of any vocabulary exceeds a predetermined value, the vocabulary is excluded as the first candidate vocabulary or the second candidate vocabulary.