JP2012073299A - Language training device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a language training device that evaluates a correct movement of a mouth of a user objectively and allowing the user to confirm the movement of the mouth to make comprehensive evaluation based on subjective evaluation and the objective evaluation to train the user efficiently.SOLUTION: A user is imaged by imaging means 10 so that a feature amount extraction unit 34 extracts an amount of features from a change in a shape of a lip part of the user. Evaluation means 35 compares an amount of features of a change in a shape of a lip part to be reference stored in feature storing means 33 with the amount of features extracted by the feature amount extraction unit 34. The evaluation means 35 quantitatively evaluates a difference between the amounts of features. Video display means 20 displays target video to be reference stored in target storing means 31. A half mirror is laminated on a display surface of the video display means 20 to display mirror video of the user together with the target video. Accordingly, the user can compare the mirror video of himself/herself with the target video.

Description

  The present invention relates to a language training apparatus used to acquire correct pronunciation when recovering a pronunciation function or learning a foreign language.

  Conventionally, as a language training device used for learning languages such as foreign languages, lip motion video and voice sampling are performed for standard pronunciation and practice subjects, and lip contour characteristics and frequency components are determined. A technique has been proposed (see, for example, Patent Document 1). Further, Patent Document 1 shows a diagram in which a standard utterer and a training subject are displayed side by side on a screen.

  On the other hand, language auditors train one-on-one when people who have reduced pronunciation or language function due to cerebral infarction or aftereffects of traffic accidents are not aiming to learn foreign languages. There are many cases.

Special Table 2008-158055

  In the technique described in Patent Document 1, since the person to be practiced is compared with the standard pronunciationr for both lip movement and voice, it is evaluated whether or not the lip movement for correct pronunciation is performed. In addition, it is also evaluated whether or not the correct pronunciation is actually performed.

  By the way, not only for the acquisition of foreign languages but also for the purpose of functional recovery, during language training, it is desirable for the trainer to check how to open his / her mouth on the spot using a mirror. However, Patent Literature 1 does not clearly indicate that a trainee (a person who is to practice) confirms how to open his / her mouth in a language training field. In the technique described in Patent Document 1, it is suggested that the standard pronunciation person and the practice subject are displayed side by side on the screen, but the practice subject compares the opening of the mouth with the standard pronunciation person on the spot. Rather, the focus is on evaluation and advice on the pronunciation of the subject. Therefore, the technique described in Patent Document 1 cannot be expected to have the same effect as training for confirming how to open the mouth on the spot by using a mirror.

  The present invention makes it possible to obtain an objective evaluation of how to open the mouth to acquire correct pronunciation, and to enable the user himself to confirm how to open the mouth on the spot. It is an object of the present invention to provide a language training apparatus that enables efficient language training by combining objective evaluation and objective evaluation.

  In order to achieve the above object, the present invention provides a video display means for displaying a video, a target presentation means for presenting a video including the shape of the lip as a model to the video display means, and a target video. A feature storage unit that stores a feature amount representing a shape change of a lip portion included in a predetermined period as a first feature amount, an imaging unit that captures a spatial region including the user's lip portion, and an imaging unit Mirror video presenting means for presenting a mirror image of a spatial area including at least a spatial area to a user together with a target video displayed on the video display means, and a user in a period corresponding to a predetermined period from an image captured by the imaging means A feature extracting means for extracting a feature quantity representing the shape change of the lip portion of the lip as a second feature quantity, an evaluation means for evaluating a difference of the second feature quantity with respect to the first feature quantity, and a result evaluated by the evaluation means. And a display processing means for determining the display content of the image display means in accordance with.

  The mirror image presenting means preferably has a configuration in which a half mirror is arranged in front of the image display means.

  Alternatively, it further includes video reversing means for generating a reversed video obtained by inverting the left and right of the image captured by the imaging means, and the target presentation means has a function of presenting the reversed video together with the target video to the video display means, However, the image reversing means and the target presenting means may be configured.

  Tongue position detecting means for detecting the position of the user's tongue is further provided, and the feature storage means stores a feature quantity representing a position change in a predetermined period with respect to the position of the tongue serving as a model as a third feature quantity, The extraction unit extracts a feature amount representing a change in the position of the user's tongue in a period corresponding to a predetermined period from the tongue position detected by the tongue position detection unit, and the evaluation unit includes a third feature amount. A configuration for evaluating the difference of the fourth feature amount with respect to the feature amount may be adopted.

  The feature storage means further comprises voice output means for outputting voice, voice control means for presenting the voice as a model to the voice output means as target voice, and voice acquisition means for acquiring the user's voice. The feature amount of the target speech is stored as the fifth feature amount, the feature extraction unit extracts the feature amount of the speech acquired by the speech acquisition unit as the sixth feature amount, and the evaluation unit stores the fifth feature amount. You may evaluate the difference of the 6th feature-value with respect to.

  In this case, a configuration may be adopted in which a recording unit that records the voice of the user acquired by the voice acquisition unit is further provided, and the voice control unit outputs the voice of the user recorded by the recording unit to the voice output unit. .

  The target video is an actual image captured by the imaging unit, and the feature extraction unit preferably has a function of extracting the first feature amount from the actual image and storing it in the feature storage unit.

  Alternatively, it is desirable that the target video is a model image in which the shape of the lip is expressed by a model.

  According to the configuration of the present invention, it is possible to obtain an objective evaluation from the feature amount of the shape change of the lip for how to open the mouth for acquiring correct pronunciation, and the mirror including the lip of the user Since the user can confirm how to open the mouth on the spot by presenting the video, efficient language training combining subjective evaluation and objective evaluation becomes possible.

It is a block diagram which shows embodiment. It is a side view which shows an external appearance same as the above. It is a perspective view which shows the usage example same as the above. It is operation | movement explanatory drawing which shows the example of a screen same as the above.

  In the embodiment described below, as shown in FIG. 2, an imaging unit 10 that captures a spatial region including a lip portion of a user 1 who is a subject of language training, and a video display unit 20 that displays a video. Prepare. Further, the user 1 usually uses the apparatus while sitting on the chair 2.

  The image pickup means 10 includes an image pickup device such as a solid-state image pickup device whose representative example is a CCD image sensor or a CMOS image sensor. Further, the image pickup means 10 is preferably configured to pick up a color image. However, as will be described later, it is desirable to adopt a configuration in which, in addition to a color image, three-dimensional information on a space is acquired and a distance image is output.

  Various technologies for outputting range images are known, but the intensity modulated light modulated by the intensity is projected, the intensity modulated light reflected by the object is received, and the phase difference between the intensity modulated light is received and received. It is desirable to use a technique that measures the distance to the object. When an active type configuration in which intensity-modulated light is projected is employed, information (light reception output) for generating a plurality of pixel values (distance values) corresponding to one screen can be obtained by one imaging. Therefore, it is possible to generate a moving image of a distance image, and to increase the temporal resolution of the moving image as compared with the case where three-dimensional information is acquired using a triangulation method or a stereo image method.

  The video display means 20 includes a display such as a flat panel display, typically a liquid crystal display or a plasma display. In the present embodiment, the half mirror 21 is arranged on the display surface for displaying the video in the video display means 20.

  Since the half mirror 21 is arranged on the video display surface of the video display means 20, information that the user 1 visually recognizes depends on the relationship between the brightness of the video display means 20 and the brightness of the space where the user 1 exists. Change. That is, when the brightness of the display surface of the video display means 20 is low, a mirror image of the space where the user 1 exists is visually recognized by the user 1. On the other hand, when the luminance of the display surface of the video display means 20 is high, the video displayed on the video display means 20 is visually recognized by the user 1 through the half mirror 21. The luminance when the user 1 visually recognizes the video on the video display means 20 can be appropriately determined by selecting the transmittance of the half mirror 21.

  From this operation, by adjusting the luminance for each region on the display surface of the video display means 20, it is possible to form a region that mainly shows a video, a region that mainly shows a mirror image, and a region that shows a video and a mirror image superimposed. . Thus, it is one feature of the configuration using the video display means 20 and the half mirror 21 that the appearance can be made different for each region.

  In the configuration example shown in FIG. 2, by using the half mirror 21, a mirror image of the space where the user 1 exists is presented to the user 1. However, even when an inverted image obtained by inverting the left and right of the image captured by the imaging device 10 without using the half mirror 21 is displayed on the video display means 20, the user 1 is provided with a display equivalent to a mirror image. become. Both the mirror image of the half mirror 21 and the inverted image generated by reversing the left and right of the image captured by the imaging means 10 mathematically correspond to the reflection of the space where the user 1 exists. Therefore, both can be referred to as “mirror images” in which the left and right sides of the space where the user 1 exists are reversed. That is, the half mirror 21 is used as a mirror image presentation means. Note that the reverse video is not strictly a mirror because the display surface is not the target surface, but can be handled equivalent to a mirror by enlarging or reducing.

  In the present embodiment, in addition to the imaging unit 10 and the video display unit 20, a voice acquisition unit 11 that acquires a voice uttered by the user 1 and a voice output unit 22 that performs a voice output described later are also provided. A microphone can be used as the sound acquisition unit 11, and a speaker can be used as the sound output unit 22. There are no particular restrictions on the configuration and arrangement of the microphone and speaker. The voice acquisition unit 11 and the voice output unit 22 are not essential, and a configuration not using them may be adopted.

  The imaging unit 10, the audio acquisition unit 11, the video display unit 20, and the audio output unit 22 are connected to the control unit 30. The control unit 30 includes a processor such as a microcomputer, a DSP, and an FPGA, and mainly includes a device that operates the processor according to a program.

  As shown in FIG. 1, the control unit 30 includes a target storage unit 31 that stores a video including the shape of the lip as an example of language training as a target video, and a target video input from the target storage unit 31. Is displayed on the video display means 20. That is, the target storage unit 31 and the display processing unit 32 constitute a target presentation unit.

  The user 1 can imitate the movement of the lip portion as a model displayed on the video display means 20 by watching it. The language training model is stored in advance in the target storage means 31 by a language auditor or the like in a predetermined format such as a single sound, a word, or a sentence. Further, the target storage unit 31 stores the shape change of the lip in a predetermined period corresponding to the above-described format.

  As described above, in the configuration of the present embodiment, by using the video display unit 20 and the half mirror 21, the mirror image of the spatial region captured by the imaging unit 10 is combined with the target video stored in the target storage unit 31. A mirror image presentation means to be presented to the user 1 is configured. That is, the mirror image presenting means has a configuration in which the half mirror 21 is arranged in front of the image display means 20.

  A feature amount (first feature amount) representing a shape change of the lip is extracted in advance from the example (the shape change of the lip during a predetermined period) stored in the target storage unit 31. The control means 30 is provided with a feature storage means 33 for storing this feature quantity. The feature amount of the lip is expressed as a set of a plurality of parameters representing the shape of the lip. Such parameters can be determined using known techniques.

  For example, it is possible to extract a plurality of feature points such as the position of the mouth corner from the image of the lip, normalize the size of the lip, and use a ratio representing the positional relationship of the feature points as a set of parameters. it can. In order to extract the feature points, it is necessary to specify the position of the lips. Therefore, not only the shape of the lips is recognized, but the color information in the color image is used to separate the lips and the peripheral portion of the lips. By extracting the position of the lips in this way, feature points can be extracted with the position of the lips as a reference.

  The feature storage unit 33 stores the time change of the parameter set described above as a feature amount. Note that the feature amount is normalized on the time axis. In addition, when the language training model is in the form of a word or sentence consisting of a plurality of sounds, it is desirable to perform feature extraction by speech recognition and divide the feature quantity for each sound element. In the following, it is assumed that the model is a single tone (one vowel, one consonant and one vowel).

  The reason why the feature storage means 33 stores the feature quantity of the model in the control means 30 is to evaluate the shape change of the lip of the user 1 in language training. That is, the control unit 30 extracts a feature amount (second feature amount) of the shape change of the lip from the image of the user 1 captured by the imaging unit 10, and the feature amount and the feature storage unit. And an evaluation means 35 for evaluating a difference from the feature quantity stored in 33.

  The feature extraction unit 34 normalizes the time axis of the extracted feature amount in order to compare the extracted feature amount with the feature amount stored in the feature storage unit 33. That is, the time axis of the feature amount extracted by the feature extraction unit 34 is matched with the time axis of the feature amount stored in the feature storage unit 33. Here, the evaluation means 35 adjusts the time axis so that the start time of the shape change of the lip of the user 1 coincides with the start time of the shape change corresponding to the feature amount stored in the feature storage means 33. To do.

  In the evaluation means 35, for example, a set of parameters as feature quantities is set as a multidimensional vector, and the Euclidean distance between the feature quantity stored in the feature storage means 33 and the feature quantity extracted by the feature extraction means 34 is obtained and obtained. It is evaluated that the smaller the Euclidean distance, the smaller the difference in feature amount. In other words, by using the Euclidean distance as an evaluation value, the degree of difference between the lip shape change of the user 1 and the lip shape change in the model is quantified.

  Here, not only the Euclidean distance, but also the distance of an appropriate parameter included in the feature amount, the evaluation means 35 can also evaluate the difference in the shape of the lip. When the evaluation means 35 evaluates the difference in the shape of the lip, the difference in shape and advice for approaching the model can be set in the evaluation means 35 as rules (knowledge). If such knowledge is set in the evaluation means 35, advice can be obtained as to which part should be shaped and which shape approaches the model according to the evaluation result.

  The evaluation result obtained by the evaluation unit 35 is output to the video display unit 20 through the display processing unit 32. That is, the video display means 20 indicates the degree of difference from the model regarding the shape change of the lip of the user 1. In addition, when a rule for giving advice is set in the evaluation unit 35, advice for approaching a model is displayed on the video display unit 20 with respect to the shape change of the lip of the user 1.

  In the above-described configuration example, it is assumed that a moving image is used, but a still image may be used. For example, a diagram showing the shape of the lip corresponding to pronunciation may be used in the initial stage of language learning, so that pronunciation practice can be performed by using a still image of a model instead of such a diagram. . Also in this case, the evaluation result by the evaluation unit 35 is displayed on the video display unit 20 as in the above example.

  When a still image is used as the target image, the still image captured by the imaging unit 10 while the user 1 is continuously generating the same sound can be used. In this case, since the data amount of the feature amount is smaller than that of the moving image and it is not necessary to match the time axis, the evaluation by the evaluation unit 35 can be easily performed.

  In order to create the target video stored in the target storage unit 31, it is desirable to use a real image obtained by capturing an expert such as a speech auditor. That is, the control unit 30 enables selection of an operation mode for creating a target video, and an image of the real image including the lip portion is captured for the expert using the imaging unit 10 in a state in which the operation mode is selected. Further, the feature amount is extracted by giving the real image stored in the target storage unit 31 to the special information extraction unit 34, and the extracted feature amount is stored in the feature storage unit 33. With such an operation, the actual image of the expert can be used as a model, and the feature amount of the model can be stored in the feature storage unit 33.

  Here, the target video to be stored in the target storage unit 31 is not necessarily a real image. For example, a human body model in a virtual three-dimensional space realized by computer graphics may be used, and the movement of the lip portion of the human body model may be used for the target video. In this case, the parameters used when creating the human body model can be used for the feature amount stored in the feature storage unit 33.

  The shape of the lip can also be expressed using a model simplified to a half-moon or rhombus. That is, the shape of the target image of the lip portion may be expressed using a simple figure as used in animation. The feature amount in this case may be set by using a database publicly available for the human body or extracted from an actual image. If such a model is used, the shape change of the lip portion cannot be accurately represented, but information necessary for noise can be omitted and only necessary information can be emphasized. It may be easier to understand than using.

  In actual use, it is desirable that a target video using a real image and a target video using a model can be selected.

  In language training, it is possible to evaluate the pronunciation more accurately by detecting not only the shape change of the lip but also the position of the tongue. Therefore, tongue position detecting means 12 for detecting the position of the tongue may be added.

  The tongue position detecting means 12 includes, for example, a minute body that can be attached to the tongue, a gyro sensor or a three-dimensional acceleration sensor, a power supply unit that receives power in a non-contact manner, and a transmission that transmits sensor output in a non-contact manner. It is comprised by accommodating a part in a container. This type of apparatus has a configuration similar to that of an RFID (IC tag), and is configured by providing a gyro sensor or a three-dimensional acceleration sensor made of a semiconductor instead of a memory in the RFID. With such a configuration, it is possible to detect a tongue position without hindering the movement of the tongue, since it is possible to use a several mm square body similar to RFID.

  When the tongue position detecting means 12 is used, a feature quantity (third feature quantity) relating to the position of the tongue serving as a model corresponding to the position of the tongue detected by the tongue position detecting means 12 is stored in the feature storage means 33 in advance. Keep it. Further, the feature extraction unit 34 extracts a feature amount (fourth feature amount) corresponding to the tongue position change detected by the tongue position detection unit 12. Both feature quantities are evaluated by the evaluation means 35 in the same manner as the feature quantities of the lip shape change, and the evaluation results are presented to the video display means 20 through the display processing means 32. Further, similarly to the feature amount with respect to the lip shape change, the feature amount with respect to the tongue position change is also extracted in a predetermined period and the time axes are matched.

  As described above, if the feature value of the tongue position change is evaluated by providing the tongue position detecting means 12, it is possible to compare the pronunciations distinguished by the position of the tongue such as consonants with the model. . That is, by using the tongue position detecting means 12, not only the lip shape change but also the tongue position change is detected, and language training can be performed with higher accuracy.

  By the way, it is desirable to evaluate speech in language training. Therefore, in the present embodiment, the voice acquisition unit 11 for acquiring the voice generated by the user 1 is provided, and the voice can be compared with the model. Further, a target voice serving as a model is registered in the target storage means 31 so that the target voice can be presented from the voice output means 22 via the voice control means 37. In addition, the feature storage unit 33 stores a feature amount (fifth feature amount) of speech corresponding to the target speech.

  The voice of the user 1 acquired by the voice acquisition unit 11 is input to the feature extraction unit 34, and the feature amount of the voice is extracted by the feature extraction unit 34. A technique known in the field of voice recognition can be used as a technique for extracting a feature amount of voice. The feature amount of the voice is also extracted in a predetermined period and the time axis is matched. The voice feature quantity acquired from the user 1 is compared with the feature quantity stored in the feature storage means 33 in the evaluation means 35, whereby the difference from the model is evaluated. The evaluation result is presented on the video display means 20.

  The control means 30 includes recording means 36 for recording the user's voice acquired by the voice acquisition means 11, and the voice recorded in the recording means 36 is output as voice output means through the voice control means 37 as necessary. 22 is output. In other words, the user 1 can confirm the sound produced while the user 1 is performing language training by the user 1 himself / herself after producing the sound. The voice control unit 37 has a function of outputting the voice of the user 1 recorded by the recording unit 36 to the voice output unit 22 together with the target voice stored in the target storage unit 31. With this function, the voice of the user 1 can be output with the voice of the model, and the difference between the voice of the user 1 and the voice of the model can be confirmed by the user 1 himself.

  Below, the usage example of the language training apparatus mentioned above is demonstrated. Here, as shown in FIG. 3, it is assumed that the user 1 uses the apparatus while sitting on the chair 2 in front of the half mirror 21. Specifically, the screen of FIG. 3 has the contents shown in FIG. This screen is set for training the user 1 who cannot remember a certain word, or the user 1 who cannot recall the word correctly even if he can remember the word.

  In the lower part of the screen shown in FIG. 4, a target area 42 for displaying a target video and a target area 43 for displaying a mirror video of the user 1 are provided. The target area 43 is an area for displaying a mirror image by the half mirror 21, and the portion corresponding to the target area 43 on the display surface of the video display means 20 is set to have a luminance of 0 or the front side of the half mirror 21. The brightness is set so as not to transmit light.

  In addition, a picture area 41 for displaying a picture or a figure related to a word to be recalled by the user 1 is provided at the top of the screen shown in FIG. A character area 44 is provided for displaying the character of the word to be recalled.

  By the way, in this apparatus, in order to enable interactive input by the user 1, it is possible to perform gesture input by recognizing the movement of the hand of the user 1 from the image captured by the imaging unit 10. Yes. When the imaging unit 10 generates only a two-dimensional image, the input according to the position of the hand is possible by recognizing the position of the hand in a plane at a certain distance from the half mirror 21. In this operation, the half mirror 21 is not touched, but the operation is similar to the operation using the touch panel. That is, the operation is defined according to the position on the screen, and the operation according to the position where the hand is brought close can be performed.

  On the other hand, when the imaging unit 10 generates a three-dimensional image, the three-dimensional position of the tip of the hand is extracted after recognizing that it is the tip of the hand, and according to the position. Make input. In this operation, a cursor (marker) indicating the position of the hand is displayed on the screen, and the operation specified according to the position on the screen can be performed by overlapping the cursor position on the desired position on the screen. .

  Although the above-described operation is an operation according to the position of the hand, a technique for performing an operation according to the movement of the hand may be employed. In some cases, the operation can be performed using not only the hand but also other parts of the user's 1 body. In this way, an input technique that uses the physical motion of the user 1 to perform an operation is called “gesture input”.

  The screen shown in FIG. 4 is provided with five buttons 45 to 49 for receiving gesture input. These buttons 45 to 49 are used as follows. The pictures and figures displayed in the picture area 41 described above are selected from a plurality of sets. Each set includes a plurality of pictures and drawings, and the set is selected according to the user 1 who performs language training.

  When a single picture or drawing is displayed in the picture area 41, nothing is displayed in the character area 44. Here, when the user 1 can recall a word indicating a picture or a figure displayed in the picture area 41, the user 1 presses an “answer” button 46 (selection by gesture input is hereinafter referred to as “press”). ). When the “answer” button 46 is pressed, a target video is displayed in the target area 42 and shows an example of correct pronunciation. At this time, since the mirror image of the user 1 is shown in the target area 43, the user 1 advances the practice of how to move the lip while confirming the shape change (movement) of the lip in the target image. be able to.

  The user 1 presses a “hint” button 45 when the user cannot recall a word even after looking at the picture or figure shown in the picture area 41. At this time, blank marks (circular portions) corresponding to the number of characters are displayed in the word area 44. However, at first, no characters are displayed in the blank mark, and only the number of characters is shown. If the user 1 can recall the words at this time, the user 1 presses the “answer” button 46, and if not, the user 1 presses the “hint” button 45.

  Each time the “hint” button 45 is pressed, hiragana characters are displayed one by one in the character area 44. That is, each time a “hint” button 45 is pressed, a word corresponding to the picture or figure shown in the picture area 41 is displayed in the character area one sound at a time. Here, even when the “hint” button 45 is not pressed, hiragana is desirably displayed one character at a time.

  In either case, the user 1 can press the “answer” button 46 when he / she recognizes that he / she can recall the words. If the “answer” button 46 is pressed, the target image is displayed in the target area 42 even when the words cannot be recalled. In this case, since the characters displayed in the character area 44 are insufficient, Person 1 will not get the reward of learning the correct words. Therefore, it is considered that the user 1 presses the “answer” button 46 only when the user can actually recall the word in order to obtain a reward for learning the word. If the “answer” button 46 is erroneously pressed, the “return” button 49 can be pressed to return to a state where the target video is not displayed.

  As described above, after practicing the movement of the lip portion for one word along with the target video, if the “Next” button 48 is pressed, the next picture or diagram in the set is displayed in the picture area 41. The Further, when returning to the previous figure or picture, the “Previous” button 47 may be pressed.

  The above-described operation example is effective for the user 1 who can understand figures and pictures but cannot recall words. In addition, for the user 1 who can recall words from a figure or picture but cannot pronounce, a character is displayed in the character area 44 at the same time as the figure or picture. 1) or display the target image in the target area 42. In this operation, evaluation by the evaluation means 35 can be performed, and a quantitative evaluation value can be obtained for the difference between the model and the user 1, so that the user is motivated by training by scoring the evaluation value. You may make it attach.

  Here, the target area 43 in which the mirror image of the user 1 is projected on the half mirror 21 can be moved by changing the display on the screen of the video display means 20, or the transparency can be changed. . Therefore, the position of the user 1 can be recognized using the image captured by the imaging unit 10, and the position of the target region 43 can be changed according to the position of the user 1. In this case, if the user 1 moves so as to overlap the target region 43 with the target region 42, the mirror image of the user 1 can be superimposed on the target image serving as a model. If such superposition is performed, the difference in movement of the lip portion between the model and the user 1 can be visually confirmed, and the training effect can be further enhanced.

  In the configuration described above, the case where the mirror image of the user 1 is projected using the half mirror 21 is illustrated. However, an inverted image obtained by inverting the left and right of the image captured by the imaging unit 10 without using the half mirror 21 is generated. Thus, a reverse video may be displayed in the screen of the video display means 20. In other words, the video reversing means 38 for reversing the left and right of the space area including the user 1 imaged by the imaging means 10 is provided (see FIG. 1). The reverse video generated by the video reversing means 38 is displayed on the video display means 20 together with the target video in the display processing means 32. Thus, even when a reverse image generated by image processing is used, a mirror image having the same effect as a mirror image using the half mirror 21 can be presented to the user 1.

  In the display processing means 32, if the target video and the reverse video are handled as different layers, the target video and the mirror video of the user 1 can be easily realized in a state where they are displayed side by side and a state where they are displayed in a superimposed manner. Can do.

DESCRIPTION OF SYMBOLS 10 Imaging means 11 Audio | voice acquisition means 12 Tongue position detection means 20 Image | video display means (mirror image presentation means)
21 half mirror (mirror image presentation means)
22 Voice output means 30 Control means 31 Target storage means (target presentation means)
32 Display processing means 33 Feature storage means 34 Feature extraction means 35 Evaluation means 36 Recording means 37 Audio control means 38 Video inversion means (mirror image presentation means)

Claims (8)

  Video display means for displaying video, target presentation means for presenting video including the shape of the lip portion as a model to the video display means as a target video, and shape change of the lip portion included in the target video during a predetermined period A feature storage means for storing a feature quantity representing a first feature quantity, an imaging means for imaging a spatial area including a user's lip, and a spatial area including at least a spatial area imaged by the imaging means Mirror video presentation means for presenting the mirror video to the user together with the target video displayed on the video display means, and the user's lip in a period corresponding to the predetermined period from the image captured by the imaging means A feature extraction unit that extracts a feature amount representing a shape change of the second feature amount, an evaluation unit that evaluates a difference of the second feature amount with respect to the first feature amount, and the evaluation unit Language training apparatus comprising: a display processing unit in accordance with the evaluation result determines the display contents of the image display unit.   2. The language training apparatus according to claim 1, wherein the mirror image presentation unit has a configuration in which a half mirror is disposed in front of the image display unit.   The image display unit further includes a video inversion unit that generates an inverted video obtained by inverting the left and right of the image captured by the imaging unit, and the target presentation unit has a function of presenting the inverted video together with the target video to the video display unit The language training apparatus according to claim 1, wherein the mirror video presentation unit includes the video inversion unit and the target presentation unit.   Tongue position detecting means for detecting the position of the user's tongue is further provided, and the feature storage means stores, as a third feature quantity, a feature quantity representing a change in position in the predetermined period with respect to the position of the tongue serving as a model. The feature extraction unit extracts, as a fourth feature amount, a feature amount representing a change in the position of the user's tongue in a period corresponding to the predetermined period from the tongue position detected by the tongue position detection unit. The language training apparatus according to claim 1, wherein the evaluation unit evaluates a difference of the fourth feature amount with respect to the third feature amount.   A voice output means for outputting a voice; a voice control means for presenting a voice as a model to the voice output means as a target voice; and a voice acquisition means for acquiring the voice of the user. Stores the feature amount of the target speech as a fifth feature amount, the feature extraction unit extracts the feature amount of the speech acquired by the speech acquisition unit as a sixth feature amount, and the evaluation unit includes: The language training apparatus according to claim 1, wherein a difference between the sixth feature quantity and the fifth feature quantity is evaluated.   The apparatus further comprises recording means for recording the voice of the user acquired by the voice acquisition means, and the voice control means causes the voice output means to output the voice of the user recorded by the recording means. The language training apparatus according to claim 5.   The target video is an actual image captured by the imaging unit, and the feature extraction unit has a function of extracting the first feature amount from the actual image and storing it in the feature storage unit. The language training apparatus according to any one of claims 1 to 6.   The language training apparatus according to claim 1, wherein the target video is a model image in which a shape of a lip portion is expressed by a model.

Images