AU2003248013A1 - Storage of audio and text - Google Patents

Description

141553039

AUSTRALIA

Patents Act 1990 University of New England COMPLETE SPECIFICATION Invention Title: Storage of audio and text The invention is described in the following statement: 141553039 2 Storage of audio and text Field of the Invention The present invention relates broadly to a method and storage system for storing audio and associated text information.

Background to the Invention Many pursuits, be they recreational, commercial or academic, involve the storage and reproduction of sounds. The storage of sounds is particularly important in language teaching, where sounds need to be repetitively reproduced in order to be studied.

It is desirable to be able to rapidly locate stored sounds and reproduce them.

Particularly in language teaching, it is also desirable to facilitate an understanding of the meaning of the sounds, when the sounds represent words, sentences or phrases in a foreign language. In order to facilitate the location of stored sounds, the sounds may be associated with a text string. Methods for searching data for text strings are known.

European Patent No. EP 0 949 621 describes an audio data system which assigns keywords to audio data files which indicate the general content of the audio data file. The system provides an updateable database of audio data that is capable of being searched based on the keywords.

United States patent No. 5,742,816 describes a method and apparatus for identifying textual documents and multi-media files corresponding to a search topic. The multi-media records have multi-media information fields and associated text fields, each of the associated text fields representing text associated with one of the multi-media information fields. An index database is created including a table for associating each of the document and multi-media records with one or more search terms.

Summary of the Invention According to a first aspect of the present invention there is a method for storing audio and associated text information, the method comprising the steps of: 141553039 3 storing in a storage system at least one audio record and an associated text string, the audio record including information representing at least one pronunciation of the text string; and creating or maintaining a browseable list including an entry associated with the text string, whereby the audio record is locatable by browsing the browseable list.

According to a second aspect of the present invention there is a storage system for storing audio and text information, the storage system comprising an audio and text storage means storing at least one audio record and an associated text string, the audio record including information representing at least one pronunciation ofthe text string; and a browseable list storage means storing a browseable list including an entry associated with the text string, whereby the stored audio record is accessible by browsing the browseable list.

Preferably the entry associated with the text string includes an explanation of the meaning of the text string. It is thus understood that the browseable list may form a dictionary. It is also preferable that the at least one audio record is associated with a plurality of text strings, and stored associated with a plurality of text strings.

Preferably the step of creating or maintaining a browseable list includes the step of making the browseable list available over a computer network, and the browseable list is preferably stored in a form and location accessible over a computer network. The computer network may be the Internet.

Where the browseable list is made available over a computer network, it is preferable for the step of creating or maintaining a browseable list to include the step of creating or maintaining the browseable list in a form viewable by a World Wide Web browser. Preferably the browseable list is stored in a form viewable by a World Wide Web browser.

141553039 4 It is preferable that the method for storing audio and associated text information includes the step of creating or maintaining a transcription list having a transcription entry including a transcription of at least part of the audio record. Preferably, the storage system for storing audio and text information further includes a transcription list storage means storing a transcription list having a transcription entry including a transcription of at least part of the audio record.

The transcription entry in the transcription list may include a link to the audio record. Preferably, the link is coded with instructions to locate a temporal portion of the audio record.

The step of creating or maintaining a transcription list preferably also includes the step of making the transcription list available over a computer network. The transcription list is preferably stored in a form and location accessible over a computer network.

The computer network may be the Internet.

The step of creating or maintaining a transcription list preferably also includes the step of creating or maintaining the transcription list in a form viewable by a World Wide Web browser. Preferably, the transcription list is stored in a form viewable by a World Wide Web browser.

Preferably the method for storing audio and associated text information further includes the step of receiving audio records and transcriptions of the audio records for storage in the storage system. The audio records and transcriptions of the audio records may be received over a computer network, such as the Internet.

In a third aspect, the present invention provides a method of storing audio and associated text information, the method comprising the steps of: storing in a storage system an audio record; storing in the storage system one or more transcription text strings each being a transcription of at least a corresponding part of the audio record; and associating the one or more transcription text strings with the audio file 141553039 whereby a transcription of at least part of the audio record is locatable by selecting the audio record.

In a fourth aspect, the present invention provides a storage system for storing audio and text information, the storage system comprising: audio record storage means storing an audio record; text string storage means storing one or more transcription text strings each being a transcription of at least a corresponding part of the audio record, said text string storage means being in communication with the audio storage means, whereby the one or more transcription text strings are stored associated with the audio record and a transcription of at least a corresponding part of the audio record is locatable by selecting the audio record.

Preferably, the step of associating the one or more transcription text strings with the audio record includes the step of linking the one or more transcription text strings with one or more temporal portions of the audio record. In the storage system of the present invention, the transcription text strings are preferably stored associated with one or more temporal portions of the audio record.

The step of associating the one or more transcription text strings with the audio record may include the step of associating the audio record with a list of the one or more transcription text strings. Preferably the audio record is stored associated with a list of one or more stored transcription text strings.

The method of storing audio and text information preferably further includes the step of associating a part of any one or more of the transcription text strings with a corresponding secondary text string. Preferably a part of one or more of the transcription text strings is stored associated with a corresponding stored secondary text string.

Preferably the secondary text string is locatable by selection of either the part of the transcription text string or the whole of the transcription text string. The stored secondary text string preferably represents the meaning of the part of the transcription text string.

I

141553039 6 The transcription text strings may be sentences. Where this is the case, the parts of the transcription text strings may be words or phrases. Alternatively, the transcription text strings may be phrases. Where this is the case, the parts of the transcription text strings may be words.

Preferably, the steps of storing an audio record and storing one or more transcription text strings include the steps of storing the audio record or transcription text strings in a form and location accessible by World Wide Web browser software. The audio record, transcription text strings or secondary text strings are preferably stored in a form and location accessible by World Wide Web browser software.

It is also preferably that the audio records, transcription text strings or secondary text strings are selectable by clicking on hyperlinks associated with them using World Wide Web browser software. Preferably the audio records, transcription text strings or secondary text strings are stored in a location and form selectable by clicking on hyperlinks associated with them using World Wide Web browser software.

Brief Description of the Drawings In order to achieve a better understanding of the nature of present invention a preferred embodiment of a method for storing audio and associated text information and a storage system for storing audio and text information will now be described, by way of example only, with reference to the accompanying drawings in which: Figure 1 is a representation of a browseable list of a preferred embodiment of the invention.

Figure 2 is a representation of a transcription list of the preferred embodiment of the invention, obtained by selecting an entry in the browseable list of Figure 1.

Figure 3 is a representation of a list of audio records stored in the storage system of the preferred embodiment of the invention.

Figure 4 is an illustration of the list of transcription text strings associated with one of the audio records of Figure 3.

141553039 7 Figure 5 is an illustration of the secondary text strings associated with one of the transcription text strings of-Figure 4.

Detailed Description of the Preferred Embodiment In a preferred embodiment of the present invention the storage of audio and associated text information in a storage system is used for the purposes of linguistic study and analysis, and language teaching.

In this embodiment computer-readable audio records are stored in a storage system comprising a random access medium, such as an optical or magnetic disk, or other form of static memory. The audio information forming an audio record may be encoded in any computer-readable form. Preferably the audio information is stored in a pervasive, compressed form such as MP3 (MPEG-1, Layer 3) or WMA (Windows Media Audio).

The audio records and associated information stored in the storage system are accessible over the Internet through a World Wide Web browser, such as Internet Explorer, Netscape Communicator or Mozilla Firebird.

The audio records stored in the storage system of the preferred embodiment are recordings of speakers talking, preferably in a language foreign to the users of the system.

Each audio record may be arbitrarily long.

Each of the audio records stored on the system are associated with one or more text strings, the text strings representing either the whole or parts of the audio records.

Accordingly, text strings stored in the storage system the hard disk of a web server) are associated with one or more audio records.

For example, an audio record may be an MP3-encoded file which, when played, would reproduce the sounds made by a speaker saying the phrase "Wherefore art thou Romeo." This audio record may be associated with the text strings "Wherefore", "art", and "thou". As is clear from the above, the audio record includes information representing at least one pronunciation of the text string the pronunciation of the speaker.

In order to facilitate access to the audio records, the text strings stored in the hard disk are listed in a form which may be browsed by a person wishing to access the audio records. By creating a browseable list, the person need not enter a search term into a search 141553039 8 engine and hope that the correct form of the word has been stored. In the above example, if a search were conducted for the word "thee", no result would be returned. However, it may have been sufficient for the person wishing to access an audio record to have accessed an audio record containing the pronunciation of the word "thou". By creating a browseable list, a person wishing to access an audio record is able to conclusively determine whether a particular word, or any of its word forms, has an associated audio record.

The browseable list contains the stored text strings, and entries associated with the text strings. The entries may contain alternate word forms, or a link directly to the stored audio record. Figure 1 depicts the browseable list of the preferred embodiment, the entries associated with the text string including an explanation of the meaning of the text string.

When used in this fashion, the browseable list becomes a form of dictionary. The words in the dictionary are associated with one or more audio records, each associated audio record containing a pronunciation of the word or phrase.

In order to make the browseable list widely accessible, it is generated in Hyper-Text Markup Language (HTML) and made available over the Internet. This allows the browseable list to be viewed by anyone with sufficient access using a World Wide Web browser.

In the preferred embodiment of the present invention, the stored text strings do not link directly to the stored audio files. Clicking on the text strings using a mouse or other pointing device will result in a second HTML page being displayed. This HTML page contains a list of transcriptions of the stored audio files associated with the stored text string in the browseable list. Figure 2 depicts a transcription list which is displayed when a person clicks on the word 'biyirrh-dhu" on the list depicted in Figure 1. The first column of the transcription entry contains a transcription of the stored audio file, or part of a stored audio file, associated with the text string. The second column of the transcription entry contains an indication of the length (in seconds) of the audio file. This length indication is also a hyperlink, which, when activated, will result in the playing of the audio file or a relevant portion of the audio file.

As indicated above, each stored audio record may be of arbitrary length. The sentence "Wherefore art thou Romeo" may be part of an audio record of all of Act II Scene 2 of Romeo and Juliet. The stored text string "thou" would have an entry explaining its 141553039 9 meaning. Selecting the word "thou" would result in a transcription list being displayed, the transcription list having entries including transcriptions of parts of audio records containing the word "thou". For example, the transcription list would have an entry including the text "Wherefore art thou Romeo", but may also have entries with the text "Look in thy glass, and tell the face thou viewest" and "Gentle thou art and therefore to be won, beauteous thou art, therefore to be assailed". The entries would also contain links to audio records of the appropriate works of Shakespeare. However, because the audio record associated with the transcription is likely to be quite long, the link can be coded with instructions to locate only that temporal portion of the audio record that gives context to the use of "thou". Accordingly, while the whole of Shakespeare's 4 1 st sonnet may be stored as an MP3-encoded audio file, only the part of the file representing "gentle thou art and therefore to be won" will be accessed by selecting the link.

In the preferred embodiment the transcription list is an HTML document accessible over the Internet, and each entry in the transcription list is hyperlinked to an MP3-encoded file, the hyperlink specifying which temporal portion of the audio file is to be accessed.

In the preferred embodiment the browseable and transcription lists are updateable through the Internet. A person can upload to the server hosting the HTML lists and audio records an audio record and associated transcription. Once the audio record and associated transcription have been received by the server hosting the HTML lists and audio files, the transcription list and browseable list can be updated to allow a subsequent user to easily locate and access the audio record. In this fashion, the storage of audio records and associated text, as well as the creation and maintenance of a browseable list of text strings, may be administered remotely. The browseable and transcription lists and audio files may be stored on multiple servers, and access control may be implemented by known methods to ensure that different classes of users have access to different parts of the storage system.

Described above is a method for storing audio and text information, and a storage system for storing same, which enables a user to access an audio record by browsing a dictionary list containing words or phrases and their meanings, selecting an entry from the dictionary list and consequently being presented with a list of transcriptions containing the selected word or phrase, and utilising a link in the entry in the transcription list which identifies which part of an audio record is relevant to the selected word or phrase.

141553039 However, a user may wish to know what words or phrases are found in an audio record, and other ancillary information about the words or phrases found in the audio record. The third and fourth aspects of the invention relate to this access mechanism.

Figure 3 illustrates a list of all the audio records stored on the web server of the preferred embodiment of the invention. As can be seen from Figure 3, each of the audio records is an MP3-encoded audio file. Along with these files, the web server also contains one or more transcription text strings, each of which corresponds to at least a part of the audio record.

Selecting one of the audio records in Figure 3 (for example, by clicking on a hyperlink associated with the audio record using a mouse or other pointing device) results in the display of a list of transcription text strings, illustrated in Figure 4. This list of transcription text strings is similar to the transcription list illustrated in Figure 2, except that each of the transcription text strings in Figure 4 is associated with a different temporal portion of the selected audio record.

Each of the transcription text strings is associated with one or more secondary text strings, which can be accessed by selecting the transcription text string (using the radio buttons to the left of the text string) and clicking on the "Dictionary" button located to the bottom right. Alternatively, parts of the transcription text string may be selected by clicking on those parts which form hyperlink tags.

The result is illustrated in Figure 5. Words in the transcription text string are used as queries to generate a list containing secondary text strings. These secondary text strings may contain any ancillary information relating to the words in the selected transcription text string, but preferably include an explanation of the meaning of words in the selected transcription text string. The secondary text string list of Figure 5 is similar to the browseable list of Figure 1.

In this aspect of the invention, the storage system may contain an audio record of each of Shakespeare's sonnets, one audio record for each sonnet. Clicking on Sonnet 41 will cause a list of transcription text strings to be displayed. Each entry may be a sentence or phrase of the first and second verses of the sonnet. A user may click on the radio button next to the transcription text string "Beauteous thou art, therefore to be assailed". When the user clicks on the "Dictionary" button, a list of secondary text strings is displayed. This list 141553039 11 contains entries for "beauteous", "thou", "art' and "assailed". Each of these entries contains a meaning of these words. Accordingly, in this aspect of the invention a user can easily obtain a transcription of parts of an audio record, including the meaning of parts of the transcription.

Audio files and associated transcription text strings and secondary text strings can may be transmitted to the web server through the Internet, to be stored as part of the database containing existing audio record, transcription text strings and secondary text strings. In this way the database can be updated and managed remotely.

Now that a preferred embodiment of the present invention has been described in some detail it will be apparent to those skilled in the art that the preferred method of storing, and storage system for storing, audio and associated text information has at least the following advantages over the admitted prior art: 1. People wishing to access the stored audio records are able to do so without the complexity of using a search engine. Text associated with the audio records may be browsed, allowing a person to determine authoritatively whether the audio file they are seeking has been stored or not.

2. When used for the purposes of studying language, people are able to use the stored audio and text information to quickly and easily determine the meaning of a word, and obtain a pronunciation of a word in context.

3. New audio records and text information may be stored, and the browseable list regenerated, as a result of audio and text information being sent over a computer network from a remote location.

4. People studying a foreign language can quickly and easily obtain translations of spoken foreign language, hear its pronunciation, and determine its meaning.

It should be apparent to those skilled in the relevant art that the invention described herein is susceptible to variations and modifications other than those specifically described. For example, the audio and text information may be made available only within a local area network and/or by specialised access software. The entries associated with text strings in the browseable list may contain foreign-language equivalents of the word represented by 141553039 12 the text string. The HTML pages may be dynamically generated from the results of database queries using Structured Query Language (SQL).

All such variations and modifications are to be considered within the scope of the present invention the nature of which is to be determined from the foregoing description.

It is to be understood that any acknowledgment of prior art in this specification is not an admission that this prior art forms part of the common general knowledge in the art.

Claims (54)

1. A method for storing audio and associated text information, the method comprising the steps of: storing in a storage system at least one audio record and an associated text string, the audio record including information representing at least one pronunciation of the text string; and creating or maintaining a browseable list including an entry associated with the text string, whereby the audio record is locatable by browsing the browseable list.

2. A method as claimed in claim 1 wherein the entry associated with the text string includes an explanation of the meaning of the text string.

3. A method as claimed in claim 1 or 2 wherein said at least one audio record is associated with a plurality of text strings.

4. A method as claimed in any one of the preceding claims wherein the step of creating or maintaining a browseable list includes the step of making the browseable list available over a computer network.

A method as claimed in claim 4 wherein the computer network is the Internet.

6. A method as claimed in either of claims 4 or 5 wherein the step of creating or maintaining a browseable list includes the step of creating or maintaining the browseable list in a form viewable by a World Wide Web browser.

7. A method as claimed in any one of the preceding claims further including the step of creating or maintaining a transcription list having a transcription entry including a transcription of at least part of the audio record.

8. A method as claimed in claim 7 wherein the transcription entry in the transcription list includes a link to the audio record. 141553039 14

9. A method as claimed in claim 8 wherein the link is coded with instructions to locate a temporal portion of the audio record.

A method as claimed in any one of claims 7 to 9 wherein the step of creating or maintaining a transcription list includes the step of making the transcription list available over a computer network.

11. A method as claimed in claim 10 wherein the computer network is the Internet.

12. A method as claimed in claims 10 or 11 wherein the step of creating or maintaining a transcription list includes the step of creating or maintaining the transcription list in a form viewable by a World Wide Web browser.

13. A method as claimed in any one of the preceding claims further including the step of receiving audio records and transcriptions of the audio records for storage in the storage system.

14. A method as claimed in claim 13 wherein the audio records and transcriptions are received over a computer network.

15. A method as claimed in claim 14 wherein the computer network is the Internet.

16. A storage system for storing audio and text information, the storage system comprising an audio and text storage means storing at least one audio record and an associated text string, the audio record including information representing at least one pronunciation of the text string; and a browseable list storage means storing a browseable list including an entry associated with the text string, whereby the stored audio record is accessible by browsing the browseable list.

17. A storage system as claimed in claim 16 wherein the entry associated with the stored text string includes an explanation of the meaning of the stored text string. 141553039

18. A storage system as claimed in claim 16 or 17 wherein the at least one audio record is stored associated with a plurality of stored text strings.

19. A storage system as claimed in any one of claims 16 to 18 wherein the browseable list is stored in a form and location accessible over a computer network.

20. A storage system as claimed in claim 19 wherein the computer network is the Internet.

21. A storage system as claimed in either of claim 19 or 20 wherein the browseable list is stored in a form viewable by a World Wide Web browser.

22. A storage system as claimed in any one of claims 16 to 21 further including a transcription list storage means storing a transcription list having a transcription entry including a transcription of at least part of the audio record.

23. A storage system as claimed in claim 22 wherein the stored transcription entry includes a link to the audio record.

24. A storage system as claimed in claim 23 wherein the link is coded with instructions to locate a temporal portion of the audio record.

A storage system as claimed in any one of claims 22 to 24 wherein the transcription list is stored in a form and location accessible over a computer network.

26. A storage system as claimed in claim 25 wherein the computer network is the Internet.

27. A storage system as claimed in claims 25 or 26 wherein the transcription list is stored in a form viewable by a World Wide Web browser.

28. A method of storing audio and associated text information, the method comprising the steps of storing in a storage system an audio record; storing in the storage system one or more transcription text strings each being a transcription of at least a corresponding part of the audio record; and 141553039 16 associating the one or more transcription text strings with the audio file whereby a transcription of at least part of the audio record is locatable by selecting the audio record.

29. A method as claimed in claim 28 wherein the step of associating the one or more transcription text strings with the audio record includes the step of linking the one or more transcription text strings with one or more temporal portions of the audio record.

A method as claimed in claim 28 or 29 wherein the step of associating the one or more transcription text strings with the audio record includes the step of associating the audio record with a list of the one or more transcription text strings.

31. A method as claimed in any one of claims 28 to 30 further including the step of associating a part of any one or more of the transcription text strings with a corresponding secondary text string.

32. A method as claimed in claim 31 wherein the secondary text string is locatable by selection of either the part of the transcription text string or the whole of the transcription text string.

33. A method as claimed in claims 31 or 32 wherein the secondary text string represents the meaning of the part of the transcription text string.

34. A method as claimed in any one of claims 31 to 33 wherein the transcription text strings are sentences.

A method as claimed in claim 34 wherein the parts of the transcription text strings are words or phrases.

36. A method as claimed in any one of claims 31 to 33 wherein the transcription text strings are phrases.

37. A method as claimed in claim 36 wherein the parts of the transcription text strings are words. 141553039 17

38. A method as claimed in any one of claims 28 to 37 wherein the steps of storing an audio record, transcription text strings or secondary text strings includes the step of storing the audio record or transcription text string in a form accessible by World Wide Web browser software.

39. A method as claimed in claim 38 wherein the audio records, transcription text strings or secondary text strings are selectable by clicking on hyperlinks associated with them using World Wide Web browser software.

A method as claimed in any one of claims 28 to 39 further including the step of receiving an audio record for storage in the storage system.

41. A method as claimed in claim 40 wherein the audio record is received over a computer network.

42. A method as claimed in claim 41 wherein the computer network is the Internet.

43. A storage system for storing audio and text information, the storage system comprising: audio record storage means storing an audio record; text string storage means storing one or more transcription text strings each being a transcription of at least a corresponding part of the audio record, said text string storage means being in communication with the audio storage means, whereby the one or more transcription text strings are stored associated with the audio record and a transcription of at least a corresponding part of the audio record is locatable by selecting the audio record.

44. A storage system as claimed in claim 43 whereby the one or more transcription text strings are stored associated with one or more temporal portions of the audio record.

45. A storage system as claimed in claim 43 or 44 wherein the audio record is stored associated with a list of the one or more stored transcription text strings. 141553039 18

46. A storage system as claimed in any one of claims 43 to 45 wherein a part of any one or more of the stored transcription text strings is stored associated with a corresponding stored secondary text string.

47. A storage system as claimed in claim 46 wherein the secondary text string is locatable by selection of either the part of the transcription text string or the whole of the transcription text string.

48. A storage system as claimed in claims 46 or 47 wherein the stored secondary text string represents the meaning of the part of the transcription text string.

49. A storage system as claimed in any one of claims 43 to 48 wherein the stored transcription text strings are sentences.

A storage system as claimed in claim 49 wherein the parts of the stored transcription text strings are words or phrases.

51. A storage system as claimed in any one of claims 43 to 48 wherein the stored transcription text strings are phrases.

52. A storage system as claimed in claim 51 wherein the parts of the transcription text strings are words.

53. A storage system as claimed in any one of claims 43 to 52 wherein the audio record, transcription text strings or secondary text strings are stored in a form and location accessible by World Wide Web browser software.

54. A storage system as claimed in claim 53 wherein the audio records, transcription text strings or secondary text strings are stored in a location and form selectable by clicking on hyperlinks associated with them using World Wide Web browser software. Dated this Seventeenth day of September 2003 University of New England By their Patent Attorneys Blake Dawson Waldron Patent Services