GB2409098A

GB2409098A - A language learning device

Info

Publication number: GB2409098A
Application number: GB0328680A
Authority: GB
Inventors: Keiron Trott; Stephen Bird
Original assignee: APPLIED PSYCHOLINGUISTICS Ltd
Current assignee: APPLIED PSYCHOLINGUISTICS Ltd
Priority date: 2003-12-09
Filing date: 2003-12-09
Publication date: 2005-06-15
Also published as: GB0328680D0

Abstract

A language learning device comprises a means 3 for displaying an audio visual stream including subtitles, a means 9 for separating the subtitle information from the stream and an interface 5 which allows the user to select subtitle information which he has difficulty understanding, and to access explanatory information about it. The audio visual stream may be generated and displayed by a digital video disc (DVD) and player, a digital television signal or a signal derived by a land-line from a remote server via the internet. The separation of the subtitle information may be by optical character recognition software (OCR) which is linked to a computer and a DVD player with a VDU which may be an integral part of a television receiver or the computer. The user interface may be a mouse, a keyboard, a joystick or a standard hand held infra-red remote control device with associated software to display a control panel on the VDU.

Description

1 2409098

A LANGUAGE LEARNING DEVICE

This invention relates to a learning device, particularly, though not exclusively for teaching languages.

It is common practice for Digital Versatile Discs (DVDs) recorded with movies also to carry "within language" subtitles, i.e. transcripts of the soundtrack, in the same language, for use by people with impaired hearing. This invention arose from the recognition that this subtitle information can be accessed and used for language teaching purposes.

According to the invention there is provided a learning device comprising means for displaying an audio visual stream including subtitles, means for separating the subtitle information from the stream and an interface allowing the user to select subtitle information which the user has difficulty understanding and to access explanatory information about it.

The "means for displaying the audio visual stream is preferably a socalled "DVD" player or other device for reading digital information from an optically encoded disc. A DVD player is particularly suitable because it allows any part of the movie to be accessed quickly and because it allows the subtitle information to be easily separated. However other devices which meet these requirements could also be used and the audio visual stream does not need to be - 2 generated from a source at the same location as the user of the invention. For example, the audio visual stream could be a digital television signal, a signal from a client server in the same building but remote from the user, or a signal derived by land-line from a remote server eg via the internet.

In some streams, subtitle information is available in text form. In others it is only available in a digital picture format. The inventor has realised that, in this latter case, separation of the subtitle information can easily be effected using commonly available optical character recognition (OCR) software to inspect the digital image. It is envisaged that this could be loaded into a computer linked to a DVD player and a VDU. The VDU can be an integral part of a television receiver or the computer. The user interface is preferably provided by a standard hand-held infra-red remote control device having associated software to display a control panel on the VDU.

The explanatory information about a selected word or expression can be provided by an electronic dictionary of words and expressions. Alternatively this information may be available on the DVD or equivalent, or from the network. The explanatory information is preferably provided in the same language as the queried word or expression, even when the intention is to teach a language. This is because languages are often best taught by exposing the student to the language being learned. However, for beginners, it may be an advantage to provide the option of receiving the information in the student's native language. For that purpose, a multi-language dictionary may be included in the system.

S One way of performing the invention will now be described by way of example and with reference to the accompanying drawings in which: Fig 1 is a schematic block diagram of a language learning device constructed in accordance with the invention; Fig 2 is a flow chart illustrating the operations performed by the processor shown at 4 on Fig 1 when running the language content interface program indicated in Fig 1; and Fig.3 is a flow chart illustrating the operations performed by the "subtitle extractor" indicated at 9 in Fig 1 and also at 10.3 on Fig 2.

Referring to Fig 1, the system comprises a conventional DVD drive 1, associated control electronics 2 linked to a VDU 3, a processor 4 and a loudspeaker or other electrical-to-audio transducer (not shown). The processor 4 receives picture data from the control system 2 and control signals from a user interface 5 which, in this particular embodiment of the invention is a mouse and associated software but which could alternatively be a keyboard, joystick, infrared remote control unit or other control device. The processor 5 is connected to access information in a storage facility 6 containing a dictionary 7, session content store 8, subtitle extractor software 9, a language content interface 10, a language session analyser 1 1 and a language testing system 12.

The dictionary 7 is a multilanguage dictionary, but it is proposed that the multiple language function is only used as an option for beginner students.

To operate the system, the user places, in the DVD drive 1, a disc recorded with a motion picture with voice, in a language which the user wishes to learn. The disc must also carry subtitles in the same language as the voice information; as is normal for use by those who have impaired hearing. The user has an initial option of selecting, using the mouse, a simple mode of operation in which he can only access literal translations of individual words, or a "verbose" mode where more complex linguistic information can be accessed.

When a subtitle is displayed, which is not understood, the user presses a button on the mouse which has been preset to provide a "pause" function. In response to this, the processor 5 (a) produces a control signal instructing the DVD player control 2 to pause the playing of the DVD and (b) starts the language content interface program 9 which will now be described with reference to Fig 2.

Referring to Fig 2 it will be seen that, having detected the pause signal at 10.1, the program 10, through the processor 4, causes the movie to be paused on the - s - subtitle (as indicated at 10.2). It then initiates, at 10.3, the subtitle extraction program 9 which is explained in Fig 3.

Referring to Fig 3, the subtitle extractor 9 first makes the subtitle picture (caption bmp) opaque so that it is not complicated by unwanted background picture information. It then identifies the position of the caption and uses standard optical character recognition software to perform a conversion to text of that part of the picture. This text information is then passed by the processor 4 (Fig 1) to the player control system 2 and is displayed as an interactive overlay image on the VDU.

Referring again to Fig 2, at 10.4 the user is invited to select a word or phrase from the subtitle as displayed on the overlay image. At 10.5, contextual information is inspected to ascertain whether adjacent words are relevant to a correct translation of the selected word or phrase. If not, operation 10.6 causes a translation of the selected word or phrase to be displayed on the overlay. If, on the other hand, there is relevant contextual information, the latter is displayed (at 10.7) along with a translation providing the system has been detected (at 10.8) as being set to the "verbose" mode. The system then checks (at 10.9) whether another word has been chosen and, if not, a command is generated (at 10.10 and 10.1 1) instructing the processor to restart the movie slightly before the selected subtitle first appeared. - 6

During the course of a language learning session all the queries made by the student are recorded by a session content storage facility 8 (Fig 1). After completion of a learning session, the user can make a request, using the user interface 5, for a testing session. This is performed by program 12 in co operation with the language session analyser 11 and the information held in the session content store 8.

In practice it will probably be most convenient for the processor 4, the data store 6 (and possibly the VDU 3) to be embodied by a general purpose computer programmed to perform the described processes.

It will be appreciated that the illustrated system is just one example of many ways in which the invention can be implemented. For example the principle of the invention could be used for teaching subjects other than languages where the audio-visual material is an educational lecture, demonstration, or documentary accompanied by subtitles expressing important facts associated with the audio- visual material. The student could select any of these facts which he did not understand and access explanatory information before continuing viewing.

Although a preferred feature of the invention is its ability to be used with movie recordings originally prepared for entertainment rather than educational value, it would be possible to use the invention with specially prepared educational recordings in which case the function of the dictionary 7 could be provided on the same medium as the audio visual recording. However the invention is believed to be especially effective for language learning from recordings originally produced for entertainment purposes and a prototype system designed in the way described has proved that the invention has great potential to provide a rapid and effective language learning tool. - 8

Claims

1. A learning device comprising means for displaying an audio visual stream including subtitles, means for separating the subtitle information from the stream and an interface allowing the user to select subtitle information which he has difficulty understanding and to access explanatory information about it.

2. A learning device according to Claim 1 comprising a digital video disc player for generating the audio visual stream.

3. A learning device according to Claim 1 comprising means for aquiring an audio visual stream from a communications network

4. A learning device according to Claim 1, 2 or 3 in which the means for separating the subtitle information is a processor programmed with optical character reading software.

5. A learning device according to any preceding Claim comprising a control panel or means for simulating a control panel.

6. A learning device according to any preceding Claim in which the control panel is provided by a processor programmed to display a control panel on a display unit.

7. A learning device according to any preceding Claim comprising a database of the aforementioned explanatory information in the form of a dictionary of words and/or expressions.

8. A learning device according to any preceding Claim comprising: a memory; means whereby the user can select subtitle information during continuous viewing of the audio visual stream and store it in the memory; and means for accessing the explanatory information.