TW201520996A - Audiovisual automatic scoring and training system for presentation skill - Google Patents

Abstract

This invention provides an audiovisual automatic scoring and training system for presentation skill, which is characterized by having an image capturing module, a sound capturing module, a processor, a database, and a display. Image and sound characteristics of a target object are captured by the image capturing module and the sound capturing module and then transmitted to the processor for analysis. The processor further matches and compares the image and the tone quality of the target object with a sample image and a sample sound source in the database and then generates a score based on a predetermined parameter provided by the database. The score is stored in the database and then shown on the display.

Description

Audio and video automatic scoring and training system for presentation skills

The invention relates to an automatic audio and video scoring and training system for a speech skill which can automatically score points for speeches and speeches.

In the school's liberal arts department, the easiest way to understand the results of student learning is to use lectures. It is the most efficient way to use lectures to check students' language ability, manners and deep understanding of the subject. .

At present, the lectures of the class are usually time-consuming and laborious. It takes about 6-8 classes to listen to the lectures. The teacher is easy to receive the previous speech after listening to the lectures. Impressions affect the scores that are unfair.

Moreover, the general school teacher will not only bring a class, but also listen to a large number of speeches, which is also a burden for the teacher, which is easy to cause mistakes.

In view of this, how to provide an audio-visual automatic scoring and training system capable of automatically scoring and quickly scoring a speech without spending too much time becomes a subject to be improved by the present invention.

The object of the present invention is to provide an audio-visual automatic scoring and training system for a presentation skill that can automatically score points for speeches and speeches.

In order to solve the above problems and achieve the object of the present invention, the technical means of the present invention is realized in this way, and is an audio-visual automatic scoring and training system for speaking skills, which is characterized by The image capturing module (1), a sound capturing module (2), a processor (3), a database (4), and a display (5); an image capturing module (1) ), comprising a photographing device (11) for detecting an object image in a specific area, and transmitting the feature (12) captured to the object image to the processor (3) for analysis; The sound capturing module (2) is configured to detect a sound source (21) in a specific area, and capture the spectrum size and the speech passage of the sound source (21), and transmit the same to the processor (3) Performing an analysis; the processor (3) is connected to the image capturing module (1), the sound capturing module (2), the data library (4), and the display (5), and has logical operations, integration, and input. And output and other functions, in order to provide target image and tone matching, and according to a preset parameter provided by the database (4), give a score value, and establish the score value in the database (4), The score is displayed on a display (5); the database (4) provides storage and reading, and includes a sample audio, a speaker audio, and the same speaker image and performance. The speaker image, the database (4) can be used by the presenter to input the same preset parameters of the audio and the same speaker image, and the stored sound capturing module and the image capturing module are received. The speaker's audio and video, and the speaker's audio and video are fed back to the processor for cross-comparison; thereby, through this creation can be used to evaluate whether the speaker's performance on the stage is stable, and then develop into a training course guiding principle.

More preferably, the feature (12) of the target image is one of the following, a facial feature, a torso feature, and a gesture feature.

More preferably, when the feature (12) is a facial feature, the processor (3) determines the facial feature by one or a combination of the following: whether the expression is stiff and unnatural, whether or not the face is worn. Smile, whether your eyes are flickering, whether your eyes are escaping from the audience, and whether your eyes are taking care of all the audience.

More preferably, the feature (12) is a method for determining the torso feature of the processor (3) when it is a torso feature, and is one of the following or a combination thereof: whether the standing posture is tall and straight, whether the chest is strong or not. Is it straight, whether the upper body is relaxed but straight, and the lower body is stable and stable? Whether it is free to shake and wear is formal and decent.

More preferably, when the feature (12) is a gesture feature, the processor (3) determines the gesture feature by one or a combination of the following: whether the gesture is smooth, whether the hand is in contact with the facial feature. Whether the hand is in contact with the torso features.

More preferably, the sound source (21) is composed of the following features, a tonal feature, a timbre feature, and a linguistic feature.

More preferably, the processor (3), when determining the sound source (21), determines the feature subdivided into the following ways: whether the tone is smooth, whether the bite is clear, whether the volume is sufficient, and whether the tone speed is maintained. Between 110wpm and 130wpm.

More preferably, the database (4) is more capable of storing a large number of speech recordings, and is set by the rating person as a five-level rating, and the present invention generates a set of scoring units, so that the rating person can quickly rate the speech.

More preferably, the database (4) is capable of recording features (12) that are often present in the target image and inputting them into the database (4) for use as a basis for the next rating.

More preferably, the database (4) is capable of recording the characteristics of the target sound source (21) and inputting it to the database (4) for use as a basis for the next rating.

More preferably, the processor (3) and the database (4) are more connectable to the Internet for remote operation; the database (4) is more compatible with the Internet connection. The remotely located camera device (11) and the sound capture module (2) pass the camera device (11) and the sound capture module (2) at the distal end to capture the features of the image (12) With the sound source (21), the processor (3) is fed back for cross-comparison and the score is displayed on the remotely located display (5).

More preferably, the display (5) not only can display the score, but also can synchronously play the image recorded by the image capturing module (1) on the display (5), so that the speaker can correct his own state.

Compared with the prior art, the functions and effects of the present invention are as follows:

The first point: in the present invention, the image capturing module (1) cooperates with the sound capturing module (2) to capture the image and sound of the speaker, and then compare the data by the processor (3). Library (4) content, which scores the speaker's speech, and then displays the score to the reviewer or speaker through the display (5), which can automatically and quickly score the lecture without spending too much time and reducing the burden of the review. It is also possible to avoid the problem of unfairness in scoring.

The second point: In the present invention, the processor (3) works in conjunction with the database (4), and can not only compare the image and sound of the speaker, but also edit the recorded image and sound of the present speaker to The feature (12) and the sound source (21) set the scoring standard and give points according to the scoring standard.

1‧‧‧Image capture module

11‧‧‧Photographing device

12‧‧‧Characteristics

2‧‧‧Sound capture module

21‧‧‧ source

3‧‧‧ Processor

4‧‧‧Database

5‧‧‧ display

Fig. 1 is a schematic view showing a first embodiment of the present invention.

Figure 2 is a diagram showing the system architecture of the first embodiment of the present invention.

Figure 3 is a flow chart showing a first embodiment of the present invention.

Figure 4 is a schematic view showing the implementation of the first embodiment of the present invention.

The following is a detailed description of the following embodiments according to the drawings: as shown in FIG. 1 to FIG. 4, the figure discloses a video and audio automatic scoring and training system for speaking skills, which is characterized in that: an image capturing mode is included. a group (1), a sound capturing module (2), a processor (3), a database (4), and a display (5); the image capturing module (1) comprising a photographing device (11), which is used for detecting a target image in a specific area, and transmitting the captured feature (12) to the processor (3) for analysis; the sound capturing module (2) for detecting a sound source (21) in a specific area, and taking the spectrum size of the sound source (21) and the speech paragraph, and transmitting it to the processor (3) for analysis; (3), with images撷 The module (1), the sound capture module (2), the database (4) and the display (5) are connected, and have functions of logic operation, integration, input and output, etc., to provide a matching ratio of the target image and the sound color. Afterwards, according to a preset parameter provided by the database (4), a score value is given, and the score value is established in the database (4), and the score is displayed on a display (5); Library (4), providing storage and reading, including a sample audio, a speaker audio, the same speaker image and a speaker image, the database (4) can be used for the speaker to pre-input the same audio and Pre-set parameters of a sample of the presenter image, together with storing the sound capture module and the image capture module, the received speaker audio and image, and feeding the speaker's audio and image back to the processor for crossover ratio Yes, through this creation can be used to assess whether the speaker's performance on the stage is stable, and then develop into a guiding principle of the training course.

Through the image capture module (1) and the sound capture module (2), the image and sound can be recorded by themselves, and then the processor (3) can set the score and the data of the score according to the image and sound, and then The data is stored in the database (4) for the next rating.

Secondly, the image capture module (1) and the sound capture module (2) can be connected to the processor (3) and the database (4) through the Bluetooth device, so that the image capture module can be replaced by the mobile phone (1) And the sound capture module (2) facilitates carrying and mounting of the present invention.

In the above, the feature (12) is one of the following, a facial feature, a torso feature, and a gesture feature.

Among them, people have a variety of performances when they are nervous and excited. The most easily judged features (12) are subdivided into faces, torso, and gestures to judge whether the speaker is emotionally stable and to rate the speaker.

In the above, when the feature (12) is a facial feature, the processor (3) determines the facial feature by one or a combination of the following: whether the expression is stiff and unnatural, whether it is smiling, or not, Whether the eyes are flickering, whether the eyes are escaping from the audience, whether the eyes take care of all Audience.

Among them, judging by the distance between the pupil of the eye and the pupil, the frequency of the pupil disappearing on the screen can easily grasp whether the eyes are flickering, whether the eyes are facing the audience, and whether the eyes take care of the situation of all the audience, and then Match the expression of the mouth to judge the overall expression of the speaker.

In the above, the feature (12) is a method for determining the torso feature when the processor (3) is a torso feature, and is one of the following or a combination thereof: whether the standing posture is straight, whether the chest is straight and the shoulder is in line. Whether the upper body is relaxed but straight, whether the lower body is stable and stable, whether it is swaying at random, and whether it is officially decent.

Among them, the body part of the human body will be shaken and shaken unconsciously because of the tension, so that the nervous emotions are generated, so the trunk characteristics are judged by the processor (3), and the speaker is scored.

In the above, the feature (12) is a method for determining the gesture feature when the processor (3) is a gesture feature, and is one of the following or a combination thereof: whether the gesture is smooth, whether the hand is in contact with the facial feature, the hand Whether it is in contact with the torso features.

Among them, the distance between the hand and the hand is detected as the basis for the judgment, and the frequency of the gesture waving can be given according to the fluctuation of the sound.

In the above, the sound source (21) is composed of the following features, a tonal feature, a timbre feature, and a linguistic feature.

Among them, according to the sound, tone and biting of the sound is clear, as the key feature of the identification sound source (21).

In the above, the processor (3), when determining the sound source (21), the determined features are subdivided into the following ways: whether the tone is smooth, whether the biting is clear, whether the volume is sufficient, and whether the tone speed is maintained. 110wpm to 130wpm. [Note: wpm words per minute Clock, English: Words per minute, abbreviation wpm].

The processor (3) determines the characteristics of the sound source (21) by comparing the speech tone, the tone sample, and the text sense module in the database (4), and performs scoring or establishing a scoring parameter according to the above features.

In the above, the database (4) can store a large number of speech recordings, and the rating person sets a five-level rating, and the present invention generates a set of scoring units, so that the rating person can quickly score the speech.

The video capture module (1) and the sound capture module (2) are directly stored in the database (4), and then edited by the processor (3) to digitize various features and sound sources, and The scoring is set so that the presenter can set the sample speaker and sample audio through the invention.

In the above, the processor (3) and the database (4) are more connectable to the Internet for remote operation; the database (4) is more suitable for the Internet connection. The photographic device (11) and the sound capturing module (2) pass the photographic device (11) and the sound capturing module (2) at the distal end, and the features (12) of the image are captured. The sound source (21) is fed back to the processor (3) for cross-comparison and the score is displayed on the remotely located display (5).

Among them, directly connecting the processor (3) and the database (4) through the Internet can achieve the purpose of long-distance teaching, and also enables students to conduct remote speaking exercises through the Internet.

Secondly, the present invention is more convenient to apply through a network connection, and even the form of a mobile application (Mobile App, or App) can be implemented by using a mobile phone.

In the above, the display (5) not only can display the score, but also can synchronously play the image recorded by the image capturing module (1) on the display (5), so that the speaker can correct his own state.

The image capturing module (1) is used to display the image at the same time as the recording On the display (5), the speaker can instantly correct his or her own state through the image of the display (5).

The embodiments, features, and effects of the present invention are described in detail above with reference to the embodiments shown in the drawings. The above description is only a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Modification changes shall fall within the scope of the invention as long as they are within the same scope.

1‧‧‧Image capture module

11‧‧‧Photographing device

12‧‧‧Characteristics

2‧‧‧Sound capture module

21‧‧‧ source

3‧‧‧ Processor

4‧‧‧Database

5‧‧‧ display

Claims (12)

An audio and video automatic scoring and training system for speech skills, comprising: an image capturing module (1), a sound capturing module (2), a processor (3), a database (4), and a display (5); the image capture module (1) includes a photographing device (11) for detecting an image of a target in a specific area and extracting features of the target image ( 12), sent to the processor (3) for analysis; the sound capture module (2) is configured to detect a sound source (21) in a specific area, and capture the spectrum size of the sound source (21) And the speech paragraph, and transmitted to the processor (3) for analysis; the processor (3), and the image capture module (1), the sound capture module (2), the database (4) and The display (5) is connected, and has functions of logic operation, integration, input and output, and is matched to provide target image and tone, and is given a predetermined parameter according to a preset parameter provided by the database (4). And the score value is established in the database (4), and the score is displayed on a display (5); the database (4) provides storage and reading, including There is a sample audio, a speaker audio, the same speaker image and a speaker image. The database (4) can be used by the speaker to pre-record the same audio and the preset parameters of the present speaker image. The sound capture module and the image capture module receive the speaker audio and image, and feed the speaker audio and image back to the processor for cross comparison; thereby, the creation can be used for evaluation Whether the speaker's performance on the stage is stable, and then developed into a guiding principle of the training course. The audio-visual automatic scoring and training system of the presentation technique according to claim 1, wherein: the feature (12) of the target image is one of the following, a facial feature, a torso feature, and a gesture feature. An audio-visual automatic scoring and training system for a presentation skill according to claim 2, wherein: the feature (12) is a method for determining a facial feature by the processor (3) when the facial feature is One or a combination: whether the expression is stiff and unnatural, whether it is smiling, whether the eyes are flickering, whether the eyes are escaping from the audience, and whether the eyes take care of all the audience. An audio-visual automatic scoring and training system for a presentation skill according to claim 2, wherein: the feature (12) is a method for determining a torso feature when the processor (3) is a torso feature, and is one of the following or The combination: whether the standing posture is straight, whether the chest is in line with the shoulder, whether the upper body is relaxed but straight, whether the lower body is stable and stable, whether it is swaying at random, and whether the dress is officially decent. The audio-visual automatic scoring and training system of the presentation skill described in claim 2, wherein: the feature (12) is a method for determining a gesture feature when the gesture is a gesture feature, or is one of the following or The combination: whether the gesture is smooth, whether the hand is in contact with the facial features, and whether the hand is in contact with the torso features. The audio-visual automatic scoring and training system of the presentation technique according to claim 1, wherein the sound source (21) is composed of the following features, a tonal feature, a timbre feature, and a linguistic feature. The audio-visual automatic scoring and training system of the speaking skill according to claim 6, wherein: the processor (3), when determining the sound source (21), the determined features are subdivided into the following ways: whether the tone is Whether it is smooth, whether the bite is clear, the volume is sufficient, and whether the tone speed is maintained between 110wpm and 130wpm. The audio-visual automatic scoring and training system of the speaking skill described in claim 1, wherein: the database (4) can store a large number of speech recordings, and then the rating person sets a five-level rating, and the invention generates a group. The scoring unit allows scorers to quickly rate the presentation. An audio-visual automatic scoring and training system for presentation skills as claimed in claim 1, wherein: the database (4) is capable of recording features (12) that are often present in the target image and inputting them into a database (4) ), as the basis for the next rating. An audio-visual automatic scoring and training system for presentation skills as claimed in claim 1, wherein: The database (4) can record the characteristics of the target sound source (21) and input it into the database (4) as the basis for the next score. The audio-visual automatic scoring and training system for the presentation technique described in claim 1, wherein: the processor (3) and the database (4) are more connectable to the Internet for remote operation; The database (4) is further adapted to connect the remotely located camera device (11) and the sound capture module (2) with the Internet, and the remote end passes through the aforementioned camera device (11) and the sound capture module. (2), and the feature (12) captured by the image and the sound source (21) are fed back to the processor (3) for cross-comparison, and the score is displayed on the remotely located display (5). The audio-visual automatic scoring and training system of the speaking skill described in claim 1, wherein: the display (5) can not only display the score, but also synchronously record the image recorded by the image capturing module (1) on the display. (5) Play, let the speaker can correct his own manner.