WO2019196205A1

WO2019196205A1 - Foreign language teaching evaluation information generating method and apparatus

Info

Publication number: WO2019196205A1
Application number: PCT/CN2018/092777
Authority: WO
Inventors: 陈铿帆; 杨宁; 卢炀
Original assignee: 深圳市鹰硕技术有限公司
Priority date: 2018-04-09
Filing date: 2018-06-26
Publication date: 2019-10-17
Also published as: CN108537702A

Abstract

The present disclosure relates to a foreign language teaching evaluation information generating method and apparatus, an electronic device, and a storage medium. Said method comprises: outputting foreign language test information, detecting a video signal acquired by a video acquiring device, and extracting video image information and audio information in the video signal, respectively; positioning a user's lips in the video image information, and capturing mouth shape features of the user's lips; matching, in a pre-established foreign language test information and pronunciation information model, pronunciation information corresponding to the foreign language test information, the pronunciation information comprising a standard pronunciation and standard mouth shape features; calculating a first feature deviation value between the mouth shape features of the user's lip and the matched standard mouth shape features, and generating teaching evaluation information according to the first feature deviation value. The present disclosure can realize the evaluation of foreign language teaching by positioning and comparing mouth shape features in foreign language test information.

Description

Foreign language teaching evaluation information generating method and device

Technical field

The present disclosure relates to the field of computer technology, and in particular, to a foreign language teaching evaluation information generating method, apparatus, electronic device, and computer readable storage medium.

Background technique

Whether the pronunciation is accurate in foreign language learning is directly related to the teaching and learning effect. In the actual teaching, it is accurate to determine the correctness of the user's pronunciation and mouth shape each time, so that the user can know the direction to be corrected, which can improve the learning efficiency of the user and achieve more with less. Effect.

However, in the actual teaching, only one-to-one teaching can only achieve the observation and judgment of the user's pronunciation and mouth each time. This way requires a large amount of teaching resources, and the learning time and place are also Very restrictive factor.

In the prior art, there have been some patent applications in the prior art that have made useful attempts around the topic of display of annotated content, such as:

The patent application with the application number CN201610823063.6 discloses a voice mouth type animation recognition method and device, and extracts a voice feature from the voice to be recognized; and inputs the extracted voice feature into a pre-trained voice mouth recognition model; Determining a lip type corresponding to the voice feature output by the voice port recognition model; determining a lip animation corresponding to the lip type according to the lip type output of the voice port recognition model; A lip animation that recognizes the recognized voice. The application processes the extracted voice features in a pre-trained voice lip recognition model to obtain a corresponding lip shape, but does not collect and analyze the user's mouth shape.

Patent Application No. CN201611075466.3 discloses a lip language recognition method and apparatus for acquiring image information of a target human body object; acquiring a lip region image of the target human body object from the image information; and from the lip region image The lip features are extracted and the lip features are lip language recognized. The application first acquires image information of a human body object, and then acquires an image of a lip region therein, thereby extracting a lip feature, but the image of the lip region is obtained by positioning the feature of the nose tip, not through the contour or color of the lip. Features to achieve.

The patent application with the application number CN201611076381.7 discloses a lip image interaction method based on a depth image and a lip language interaction device for acquiring depth image information of a target human body object; and acquiring a lip of the target human body object from the depth image information The region image; the lip feature is extracted from the lip region image, and the lip language recognition is performed according to the lip feature; the result of the lip language recognition is converted into a corresponding operation instruction, and the interaction is performed according to the operation instruction. The application relies on image recognition to realize the positioning of the lip and the extraction of the lip feature, the purpose of which is to identify the corresponding lip language to realize the control of the interactive device, and can not display the lip feature to the user to enable the user to reach The purpose of foreign language learning.

In the prior art, there are the following problems regarding the processing of lip-type features in the method for generating foreign language teaching evaluation information:

1. After the mouth device is collected and analyzed by the video capture device, it is not displayed in the user's display terminal;

2, can not first through the body recognition, and then through the contours or color features of the lips to achieve the positioning of the lip;

3. In addition, there is no concern about the comparison between the user's mouth type and the standard mouth type.

Therefore, it is desirable to provide one or more technical solutions that at least solve the above problems.

It should be noted that the information disclosed in the Background section above is only for enhancement of understanding of the background of the present disclosure, and thus may include information that does not constitute prior art known to those of ordinary skill in the art.

Summary of the invention

An object of the present disclosure is to provide a foreign language teaching evaluation information generating method, apparatus, electronic device, and computer readable storage medium, thereby at least partially obviating one or more problems due to limitations and disadvantages of the related art.

According to an aspect of the present disclosure, a method for generating foreign language teaching evaluation information includes:

A method for generating foreign language teaching evaluation information, characterized in that the method comprises:

Outputting foreign language test information, detecting a video signal collected by the video capture device, and separately extracting video picture information and audio information in the video signal;

Positioning a user's lips in the video screen information and grabbing a lip shape of the user's lips;

Matching the pronunciation information corresponding to the foreign language test information in the pre-established foreign language test information and the pronunciation information model, the pronunciation information including standard pronunciation and standard lip type features;

Calculating a first feature deviation value of the lip shape of the user's lip and the matched standard lip shape feature, and generating teaching evaluation information according to the first feature deviation value.

In an exemplary embodiment of the present disclosure, the method further includes:

Analyzing the audio information to identify a user's pronunciation in the audio information;

Calculating a second feature deviation value of the user pronunciation and the matched standard pronunciation;

The teaching evaluation information is generated according to the first feature deviation value and the second feature deviation value.

In an exemplary embodiment of the present disclosure, generating the teaching evaluation information according to the first feature deviation value and the second feature deviation value includes:

Generating score information according to the first feature deviation value;

The score information is used as teaching evaluation information.

Determining whether the first feature deviation value satisfies a preset deviation condition;

And if the characteristic deviation value satisfies the preset deviation condition, generating teaching evaluation information according to the first characteristic deviation value, including:

Obtaining first deviation information according to the first characteristic deviation value;

Teaching evaluation information is generated based on the first deviation information and the matched standard pronunciation and standard lip type features.

Adding a prompt identifier to the lip-shaped feature area that satisfies the preset deviation condition;

The teaching evaluation information is generated according to the lip type feature of the increased prompt identifier, and the teaching evaluation information is controlled to be displayed on the user display device.

In an exemplary embodiment of the present disclosure, positioning a user's lips in the video picture information includes:

Identifying and locating a user's facial contour in the video picture information;

Use a lip filter to position and track the user's lips for a specified area of the facial contour.

In an exemplary embodiment of the present disclosure, the lip features include a lip contour, a lip diameter, a lip opening angle, a lip height, and a lip width.

In an exemplary embodiment of the present disclosure, matching the pronunciation information corresponding to the foreign language test information in the pre-established foreign language test information and the pronunciation information model includes:

Geometrically modeling according to at least one of the lip contour, the lip diameter, the lip opening angle, the lip height, and the lip width to form a vectorized lip shape feature;

The vectorized lip features are matched to the standard lip features.

Determining whether the score information is less than a preset score;

If yes, saving the score information and foreign language test information and pronunciation information corresponding to the score information;

Training a foreign language practice model according to the saved score information and corresponding foreign language test information and pronunciation information;

The foreign language test information and the pronunciation information in the foreign language practice model are periodically sent to the user for the foreign language intensive exercise.

In an aspect of the present disclosure, a device for generating foreign language teaching evaluation information includes:

a signal detecting module, configured to output a foreign language test information, detect a video signal collected by the video capture device, and respectively extract video image information and audio information in the video signal;

a feature capture module, configured to locate a user's lip in the video screen information, and capture a lip shape of the user's lip;

An information matching module, configured to match pronunciation information corresponding to the foreign language test information in a pre-established foreign language test information and a pronunciation information model, where the pronunciation information includes a standard pronunciation and a standard lip type feature;

And an information generating module, configured to calculate a first feature deviation value of the lip shape of the user's lip and the matched standard lip shape feature, and generate teaching evaluation information according to the first feature deviation value.

In an aspect of the disclosure, an electronic device is provided, comprising:

Processor;

A memory having stored thereon computer readable instructions that, when executed by the processor, implement the method of any of the above.

In an aspect of the present disclosure, a computer readable storage medium having stored thereon a computer program, the computer program being executed by a processor, implements the method of any of the above.

a method for generating a foreign language teaching evaluation information in an exemplary embodiment of the present disclosure, outputting foreign language test information, detecting a video signal collected by a video capture device, and extracting video image information and audio information in the video signal, respectively, for the video image Positioning the user's lip in the information, and grasping the lip shape of the user's lip, matching the pronunciation information corresponding to the foreign language test information in the pre-established foreign language test information and the pronunciation information model, the pronunciation information The standard pronunciation and the standard lip shape feature are included, and the first feature deviation value of the lip shape of the user's lip and the matched standard lip shape feature are calculated, and the teaching evaluation information is generated according to the first feature deviation value. On the one hand, through the face recognition of the user video signal and the processing algorithm of the specific area of the face, the rapid positioning of the lip is realized, and the speed and accuracy of the positioning are improved; on the other hand, the lip shape of the user's lips is adopted. Compared with the matching standard lip shape features and the display of the deviation area identification, the user can intuitively feel the gap between the mouth shape and the standard mouth shape, and realize the mouth shape adjustment more quickly to make the calibration more accurate. The pronunciation greatly enhances the user experience.

The above general description and the following detailed description are intended to be illustrative and not restrictive.

DRAWINGS

The above and other features and advantages of the present disclosure will become more apparent from the detailed description.

FIG. 1 illustrates a flowchart of a foreign language teaching evaluation information generating method according to an exemplary embodiment of the present disclosure;

2 illustrates a schematic diagram including a lip feature in accordance with an exemplary embodiment of the present disclosure;

FIG. 3 illustrates a schematic diagram of a lip-shaped feature contrast display image application scenario according to an exemplary embodiment of the present disclosure; FIG.

FIG. 4 shows a schematic block diagram of a foreign language teaching evaluation information generating apparatus according to an exemplary embodiment of the present disclosure;

FIG. 5 schematically illustrates a block diagram of an electronic device in accordance with an exemplary embodiment of the present disclosure;

FIG. 6 schematically illustrates a schematic diagram of a computer readable storage medium in accordance with an exemplary embodiment of the present disclosure.

detailed description

Example embodiments will now be described more fully with reference to the accompanying drawings. However, the exemplary embodiments can be embodied in a variety of forms and should not be construed as being limited to the embodiments set forth herein. To those skilled in the art. The same reference numerals in the drawings denote the same or similar parts, and the repeated description thereof will be omitted.

Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are set forth However, one skilled in the art will appreciate that the technical solution of the present disclosure may be practiced without one or more of the specific details, or other methods, components, materials, devices, steps, etc. may be employed. In other instances, well-known structures, methods, devices, implementations, materials, or operations are not shown or described in detail to avoid obscuring aspects of the present disclosure.

The block diagrams shown in the figures are merely functional entities and do not necessarily have to correspond to physically separate entities. That is, these functional entities may be implemented in software, or implemented in one or more software-hardened modules, or in different network and/or processor devices and/or microcontroller devices. Implement these functional entities.

In the present exemplary embodiment, a method for generating a foreign language teaching evaluation information is first provided, which can be applied to an electronic device such as a computer. Referring to FIG. 1, the foreign language teaching evaluation information generating method may include the following steps:

Step S110: outputting foreign language test information, detecting a video signal collected by the video capture device, and separately extracting video picture information and audio information in the video signal;

Step S120. Positioning a user's lip in the video screen information, and grasping a lip shape of the user's lip;

Step S130. Matching the pronunciation information corresponding to the foreign language test information in the pre-established foreign language test information and the pronunciation information model, the pronunciation information including standard pronunciation and standard lip type features;

Step S140. Calculate a first feature deviation value of the lip shape of the user's lip and the matched standard lip shape feature, and generate teaching evaluation information according to the first feature deviation value.

According to the foreign language teaching evaluation information generating method in the exemplary embodiment, on the one hand, the face recognition of the user video signal and the processing algorithm of the specific area of the face realize the rapid positioning of the lip, and the positioning speed and accuracy are improved. On the other hand, through the comparison between the lip characteristics of the user's lips and the matching standard lip features, and the display of the deviation region identification, the user can intuitively feel the mouth shape and the standard mouth shape. Gap, faster implementation of lip adjustment to give a more accurate pronunciation, greatly enhance the user experience.

Next, the foreign language teaching evaluation information generating method in the present exemplary embodiment will be further described.

In step S110, the foreign language test information may be output, the video signal collected by the video capture device is detected, and the video screen information and the audio information in the video signal are respectively extracted.

In the exemplary embodiment, the foreign language test information is displayed on the display device of the user, and then the corresponding pronunciation audio information of the foreign language test information and the video picture information of the facial feature including the pronunciation are obtained by the video collection device.

In step S120, the user's lip in the video screen information may be located and the lip shape of the user's lips may be grasped.

In this example, the lip of the user may be directly positioned according to the facial features of the user in the video picture information, so that the lip feature of the lip of the user may be collected, or the video image may be first The user's facial contour is positioned in the information to realize lip positioning. The specific method is as follows:

In this example embodiment, positioning the user's lip in the video picture information includes: identifying and locating a user's facial contour in the video picture information; using a lip color filter on the user's lips in a specified area of the facial contour The department performs positioning and tracking. Because the existing facial recognition algorithm based on video picture information is a mature technology, the user's facial contour is first recognized, and then the designated area is specified in the facial contour, and the lip of the user is positioned by using the lip color filter. And tracking, compared with directly positioning the user's lips, can greatly increase the recognition and enhance the robustness of the system.

In this exemplary embodiment, the lip features include a lip contour, a lip aperture, a lip opening angle, a lip height, and a lip width. As shown in FIG. 2, which is a schematic diagram of a lip-shaped feature, the lip-shaped feature parameter is extracted, and the digitized parameter identification can be implemented on the irregular lip-shaped feature.

In the example embodiment, matching the pronunciation information corresponding to the foreign language test information in the pre-established foreign language test information and the pronunciation information model includes: according to the lip contour, the lip diameter, the lip opening angle, and the lip height And geometric modeling of at least one of the lip widths to form a vectorized lip shape feature. Because in the foreign language test, it is often not the pronunciation of a single syllable, so the corresponding lip shape feature is also a change of the mouth shape feature parameter. Each of the mouth shape feature parameters after geometric modeling can establish each port when the mouth shape changes. The vectorized lip shape of the type parameter completely records the change process of each mouth shape characteristic parameter. Thereafter, the vectorized lip feature is matched to the standard lip feature, and the standard lip feature may also be a vectorized standard lip feature.

In step S130, the pronunciation information corresponding to the foreign language test information may be matched in the pre-established foreign language test information and the pronunciation information model, and the pronunciation information includes standard pronunciation and standard lip type features.

In the example embodiment, the pronunciation information corresponding to the foreign language test information may be matched in the pre-established foreign language test information and the pronunciation information model, and the standard pronunciation and the standard lip-shaped feature corresponding to the foreign language test information are obtained. Further, the pronunciation information corresponding to the user's pronunciation or the lip-shaped feature may be matched in the pre-established foreign language test information and the pronunciation information model, and used to check the correctness of the user's foreign language test information.

In step S140, a first feature deviation value of the lip shape of the user's lip and the matched standard lip shape feature may be calculated, and the teaching evaluation information is generated according to the first feature deviation value.

In this example embodiment, comparing the lip-shaped feature corresponding to the user's foreign language test information with the standard lip-shaped feature matched in the foreign language test information and the pronunciation information model, and separately calculating the deviation of each port type feature parameter, the comprehensive After the deviation, a first feature deviation value is generated, and the first feature deviation value reflects deviation information of each mouth shape characteristic parameter of the user, and the first feature deviation value may be used as a basis for generating teaching evaluation information.

In this example, the method further includes: determining whether the first feature deviation value satisfies a preset deviation condition; if the feature deviation value satisfies a preset deviation condition, generating a teaching evaluation according to the first characteristic deviation value The information, the teaching evaluation information may be an evaluation score of the foreign language test information, or may be an evaluation file value of the foreign language test information, such as "qualified", "unqualified" or "excellent", "good", "failed", and the like. Specifically, the method further includes: obtaining first deviation information according to the first characteristic deviation value; and generating teaching evaluation information according to the first deviation information and the matched standard pronunciation and standard lip type features.

In this example, the method further includes: adding a prompt identifier to the lip-shaped feature region that satisfies the preset deviation condition; generating teaching evaluation information according to the lip-shaped feature of the added prompt identifier, and controlling the teaching evaluation information to be Displayed on the user display device. As shown in FIG. 3, the solid line part is a lip-shaped feature of the foreign language test information “C” when the user tests in a foreign language, and the dotted line part is a standard lip-shaped feature corresponding to the foreign language test information “C”, in the solid line part and the dotted line. Part of the uncoincident area, the prompt identifier is added, which is convenient for the user to further correct the mouth shape feature according to the prompt identifier, and achieve a complete fitting of the user mouth type feature and the standard mouth shape feature, thereby issuing a more standard foreign language test information "C" pronunciation. .

In this example embodiment, the method further includes: analyzing the audio information, identifying a user's pronunciation in the audio information; calculating a second feature deviation value of the user's pronunciation and the matched standard pronunciation; The first feature deviation value and the second feature deviation value are used to generate teaching evaluation information. In order to enable the user to finally achieve the standard foreign language pronunciation, in addition to comparing the user's mouth shape characteristics, the user's pronunciation should also be judged, and the user's pronunciation in the collected audio information is compared with the matched standard pronunciation to obtain the second characteristic deviation. The value, the second characteristic deviation value reflects the deviation information of the user's pronunciation, and combined with the first characteristic deviation value, the teaching evaluation information of the user's foreign language test information may be comprehensively generated.

In the present exemplary embodiment, generating the teaching evaluation information according to the first feature deviation value and the second feature deviation value includes: generating the score information according to the first feature deviation value; and using the score information as the teaching evaluation information. As shown in FIG. 4, the teaching evaluation information of the score information corresponding to the foreign language test information "C" at the time of the user's foreign language test.

In this example embodiment, the method further includes: determining whether the score information is less than a preset score; if yes, saving the score information and foreign language test information and pronunciation information corresponding to the score information; The score information and the corresponding foreign language test information and the pronunciation information are used to train the foreign language practice model; the foreign language test information and the pronunciation information in the foreign language practice model are periodically sent to the user for the foreign language intensive exercise. The above method is a further enhancement of foreign language learning after the evaluation of foreign language teaching. By means of the preset score line of the facility, the user's foreign language pronunciation level can be quickly checked, and the number of foreign language pronunciation tests that do not satisfy the score line is increased, and the purpose of reinforcement learning is realized, which is beneficial to the purpose. Enhance the pronunciation of foreign language pronunciation.

It should be noted that, although the various steps of the method of the present disclosure are described in a particular order in the drawings, this does not require or imply that the steps must be performed in the specific order, or that all the steps shown must be performed. Achieve the desired results. Additionally or alternatively, certain steps may be omitted, multiple steps being combined into one step execution, and/or one step being decomposed into multiple step executions and the like.

Further, in the present exemplary embodiment, a foreign language teaching evaluation information generating apparatus is also provided. Referring to FIG. 4, the foreign language teaching evaluation information generating apparatus 400 may include: a signal detecting module 410, configured to output foreign language test information, detect a video signal collected by the video capturing device, and respectively extract video image information in the video signal and Audio information

The feature capture module 420 is configured to locate a user's lip in the video screen information, and capture a lip shape of the user's lip;

The information matching module 430 is configured to match the pronunciation information corresponding to the foreign language test information in the pre-established foreign language test information and the pronunciation information model, where the pronunciation information includes standard pronunciation and standard lip-shaped features;

The information generating module 440 is configured to calculate a first feature deviation value of the lip shape of the user's lip and the matched standard lip shape feature, and generate teaching evaluation information according to the first feature deviation value.

The specific details of the above-mentioned foreign language teaching evaluation information generating device modules have been described in detail in the corresponding audio paragraph identifying method, and therefore will not be described herein.

It should be noted that although several modules or units of the foreign language teaching evaluation information generating apparatus 400 are mentioned in the above detailed description, such division is not mandatory. Indeed, in accordance with embodiments of the present disclosure, the features and functions of two or more modules or units described above may be embodied in one module or unit. Conversely, the features and functions of one of the modules or units described above may be further divided into multiple modules or units.

Further, in an exemplary embodiment of the present disclosure, an electronic device capable of implementing the above method is also provided.

Those skilled in the art will appreciate that various aspects of the present invention can be implemented as a system, method, or program product. Accordingly, aspects of the present invention may be embodied in the form of a complete hardware embodiment, a complete software embodiment (including firmware, microcode, etc.), or a combination of hardware and software aspects, which may be collectively referred to herein. "Circuit," "module," or "system."

An electronic device 500 in accordance with such an embodiment of the present invention is described below with reference to FIG. The electronic device 500 shown in FIG. 5 is merely an example and should not impose any limitation on the function and scope of use of the embodiments of the present invention.

As shown in FIG. 5, electronic device 500 is embodied in the form of a general purpose computing device. The components of the electronic device 500 may include, but are not limited to, the at least one processing unit 510, the at least one storage unit 520, the bus 530 connecting the different system components (including the storage unit 520 and the processing unit 510), and the display unit 540.

Wherein the storage unit stores program code, which can be executed by the processing unit 510, such that the processing unit 510 performs various exemplary embodiments according to the present invention described in the "Exemplary Method" section of the present specification. The steps of the examples. For example, the processing unit 510 can perform steps S110 to S140 as shown in FIG. 1.

The storage unit 520 can include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 5201 and/or a cache storage unit 5202, and can further include a read only storage unit (ROM) 5203.

The storage unit 520 can also include a program/utility 5204 having a set (at least one) of the program modules 5205, such as but not limited to: an operating system, one or more applications, other program modules, and program data, Implementations of the network environment may be included in each or some of these examples.

Bus 530 may be representative of one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local area using any of a variety of bus structures. bus.

The electronic device 500 can also communicate with one or more external devices 570 (eg, a keyboard, pointing device, Bluetooth device, etc.), and can also communicate with one or more devices that enable the user to interact with the electronic device 500, and/or with Any device (eg, router, modem, etc.) that enables the electronic device 500 to communicate with one or more other computing devices. This communication can take place via an input/output (I/O) interface 550. Also, electronic device 500 can communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and/or a public network, such as the Internet) via network adapter 560. As shown, network adapter 560 communicates with other modules of electronic device 500 via bus 530. It should be understood that although not shown in the figures, other hardware and/or software modules may be utilized in conjunction with electronic device 500, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives. And data backup storage systems, etc.

Through the description of the above embodiments, those skilled in the art can easily understand that the exemplary embodiments described herein may be implemented by software, or may be implemented by software in combination with necessary hardware. Therefore, the technical solution according to an embodiment of the present disclosure may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (which may be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network. A number of instructions are included to cause a computing device (which may be a personal computer, server, terminal device, or network device, etc.) to perform a method in accordance with an embodiment of the present disclosure.

In an exemplary embodiment of the present disclosure, there is also provided a computer readable storage medium having stored thereon a program product capable of implementing the above method of the present specification. In some possible embodiments, aspects of the present invention may also be embodied in the form of a program product comprising program code for causing said program product to run on a terminal device The terminal device performs the steps according to various exemplary embodiments of the present invention described in the "Exemplary Method" section of the present specification.

Referring to FIG. 6, a program product 600 for implementing the above method, which may employ a portable compact disk read only memory (CD-ROM) and includes program code, and may be in a terminal device, is illustrated in accordance with an embodiment of the present invention. For example running on a personal computer. However, the program product of the present invention is not limited thereto, and in the present document, the readable storage medium may be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus or device.

The program product can employ any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (non-exhaustive lists) of readable storage media include: electrical connections with one or more wires, portable disks, hard disks, random access memory (RAM), read only memory (ROM), erasable Programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the foregoing.

The computer readable signal medium may include a data signal that is propagated in the baseband or as part of a carrier, carrying readable program code. Such propagated data signals can take a variety of forms including, but not limited to, electromagnetic signals, optical signals, or any suitable combination of the foregoing. The readable signal medium can also be any readable medium other than a readable storage medium that can transmit, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

Program code embodied on a readable medium can be transmitted using any suitable medium, including but not limited to wireless, wireline, optical cable, RF, etc., or any suitable combination of the foregoing.

Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C++, etc., including conventional procedural Programming language—such as the "C" language or a similar programming language. The program code can execute entirely on the user computing device, partially on the user device, as a stand-alone software package, partially on the remote computing device on the user computing device, or entirely on the remote computing device or server. Execute on. In the case of a remote computing device, the remote computing device can be connected to the user computing device via any kind of network, including a local area network (LAN) or wide area network (WAN), or can be connected to an external computing device (eg, provided using an Internet service) Businesses are connected via the Internet).

Further, the above-described drawings are merely illustrative of the processes included in the method according to the exemplary embodiments of the present invention, and are not intended to be limiting. It is easy to understand that the processing shown in the above figures does not indicate or limit the chronological order of these processes. In addition, it is also easy to understand that these processes may be performed synchronously or asynchronously, for example, in a plurality of modules.

Other embodiments of the present disclosure will be apparent to those skilled in the <RTIgt; The present application is intended to cover any variations, uses, or adaptations of the present disclosure, which are in accordance with the general principles of the disclosure and include common general knowledge or common technical means in the art that are not disclosed in the present disclosure. . The specification and examples are to be regarded as illustrative only,

It is to be understood that the invention is not limited to the details of the details and The scope of the disclosure is to be limited only by the appended claims.

Industrial applicability

On the one hand, through the face recognition of the user video signal and the processing algorithm of the specific area of the face, the rapid positioning of the lip is realized, and the speed and accuracy of the positioning are improved; on the other hand, the lip shape of the user's lips is adopted. Compared with the matching standard lip shape features and the display of the deviation area identification, the user can intuitively feel the gap between the mouth shape and the standard mouth shape, and realize the mouth shape adjustment more quickly to make the calibration more accurate. The pronunciation greatly enhances the user experience.

Claims

A method for generating foreign language teaching evaluation information, characterized in that the method comprises:

Outputting foreign language test information, detecting a video signal collected by the video capture device, and separately extracting video picture information and audio information in the video signal;

Positioning a user's lips in the video screen information and grabbing a lip shape of the user's lips;

Matching the pronunciation information corresponding to the foreign language test information in the pre-established foreign language test information and the pronunciation information model, the pronunciation information including standard pronunciation and standard lip type features;

Calculating a first feature deviation value of the lip shape of the user's lip and the matched standard lip shape feature, and generating teaching evaluation information according to the first feature deviation value.
The method of claim 1 wherein the method further comprises:

Analyzing the audio information to identify a user's pronunciation in the audio information;

Calculating a second feature deviation value of the user pronunciation and the matched standard pronunciation;

The teaching evaluation information is generated according to the first feature deviation value and the second feature deviation value.
The method according to claim 2, wherein the generating the teaching evaluation information according to the first feature deviation value and the second feature deviation value comprises:

Generating score information according to the first feature deviation value;

The score information is used as teaching evaluation information.
The method of claim 1 wherein the method further comprises:

Determining whether the first feature deviation value satisfies a preset deviation condition;

And if the characteristic deviation value satisfies the preset deviation condition, generating teaching evaluation information according to the first characteristic deviation value, including:

Obtaining first deviation information according to the first characteristic deviation value;

Teaching evaluation information is generated based on the first deviation information and the matched standard pronunciation and standard lip type features.
The method of claim 4, wherein the method further comprises:

Adding a prompt identifier to the lip-shaped feature area that satisfies the preset deviation condition;

And generating teaching evaluation information according to the lip type feature of the increased prompt identifier, and controlling the teaching evaluation information to be displayed on the user display device.
The method of claim 1, wherein locating the user's lips in the video picture information comprises:

Identifying and locating a user's facial contour in the video picture information;

Use a lip filter to position and track the user's lips for a specified area of the facial contour.
The method of claim 1 wherein said lip features include a lip contour, a lip diameter, a lip opening angle, a lip height, and a lip width.
The method according to claim 7, wherein the matching of the pronunciation information corresponding to the foreign language test information in the pre-established foreign language test information and the pronunciation information model comprises:

Geometrically modeling according to at least one of the lip contour, the lip diameter, the lip opening angle, the lip height, and the lip width to form a vectorized lip shape feature;

The vectorized lip features are matched to the standard lip features.
The method of claim 3, wherein the method further comprises:

Determining whether the score information is less than a preset score;

If yes, saving the score information and foreign language test information and pronunciation information corresponding to the score information;

Training a foreign language practice model according to the saved score information and corresponding foreign language test information and pronunciation information;

The foreign language test information and the pronunciation information in the foreign language practice model are periodically sent to the user for the foreign language intensive exercise.
A device for generating foreign language teaching evaluation information, characterized in that the device comprises:

a signal detecting module, configured to output a foreign language test information, detect a video signal collected by the video capture device, and respectively extract video image information and audio information in the video signal;

a feature capture module, configured to locate a user's lip in the video screen information, and capture a lip shape of the user's lip;

An information matching module, configured to match pronunciation information corresponding to the foreign language test information in a pre-established foreign language test information and a pronunciation information model, where the pronunciation information includes a standard pronunciation and a standard lip type feature;

And an information generating module, configured to calculate a first feature deviation value of the lip shape of the user's lip and the matched standard lip shape feature, and generate teaching evaluation information according to the first feature deviation value.
An electronic device, comprising:

Processor;

A memory having computer readable instructions stored thereon, the computer readable instructions being executed by the processor to implement the method of any one of claims 1 to 9.
A computer readable storage medium having stored thereon a computer program, the computer program being executed by a processor to implement the method of any one of claims 1 to 9.