KR100872191B1 - System and method for enhancing live speech with information accessed from the world wide web - Google Patents

Abstract

The present invention relates to a system, method, and computer program that enables an audience listening to a live speech to access complementary information related to a pronunciation term during the course of the presentation immediately after or after a period of time. do. The system associates hyperlinks (ie, URLs) with optional terms and words that are likely to be pronounced by the presenter during the presentation process. A speech recognition system operating on a presenter device (i.e., a computer system with a microphone connected) recognizes the pronunciation of one of the hyperlink terms by the presenter during the course of the presentation (i.e. word spotting). Record the time at which each recognized hyperlink term is pronounced. The system also provides the presenter device and multiple audience devices (e.g., workstations, portable computers, personal digital assistants, smart phones, or any other type) according to the same universal time. Portable computer devices, etc.) are synchronized so that they are transmitted by the presenter regardless of the relative position of the presenter and the audience, and the flow of information received by the audience is always synchronized. Whenever the audience recognizes a topic of interest during the presentation process, the audience simply selects the topic by pressing a key provided on the audience device. The world standard time at which the audience selects a topic is stored in the audience device.

Description

TECHNICAL AND METHOD FOR ENHANCING LIVE SPEECH WITH INFORMATION ACCESSED FROM THE WORLD WIDE WEB}

FIELD OF THE INVENTION The present invention generally relates to a technique for including complementary information access means in a live event, and more particularly to a live broadcast event such as a live speech or live program, in particular information, world wide. A system and method for improving with information accessible on the World Wide Web is disclosed.

In the present society, the desire for information and services is increasing. Most of the audience listening to the live event (eg, the audience listening to the conference presented by the presenter) or the audience listening to the live radio or television broadcast program would like to have access to complementary information. This complementary information may consist of a biography of the players of a football game, a historical background of the events mentioned in the news program, or a record of the games during the broadcast of the Olympic Games.

In fact, today people search for more information about what they hear as audiences of live events at close range and audiences of live shows at a distance.

The consumer wants to have access to special services associated with the product being advertised.

· Media providers expect to gain new sources of income by expanding the amount of services provided and the quality and information of services provided to consumers, more specifically audiences of live television or radio programs.

Advertisers are looking for new and more effective forms of advertising.

On-line Services over the Web

Regardless of the large scale development of radio and television, online services offered on the World Wide Web (ie, the Web) are rapidly emerging in the present society and are now widely available. Such online services, based on Internet technology, provide access to vast amounts of information on an interactive basis. The Internet is a global network of computers. The Internet uses a language called TCP / IP ("Transmission Control Protocol / Internet Protocol") to connect computers based on a variety of different operating systems or languages. As the size and complexity of the Internet has increased, tools have been developed to help users find information on the network. Such tools are often referred to as "navigators" or "navigation systems". The World Wide Web ("WWW" or "Web") is the latest superior navigation system. The web,

Internet based navigation system

Information distribution and management system through the Internet,

Dynamic format for communicating on the web.

The Internet and the Web are changing our society by allowing millions of users to access and exchange information and to communicate with each other. By integrating images, text, sound, and video, users on the web using a graphical user interface may, for example, have different computers on different systems, different system applications, different information formats, for example. Files and documents including text, sound, and graphics can be easily communicated. Currently, online systems on the web include, for example, personal message services, electronic commerce, news, real-time games, access to electronic databases, electronic newsletters, business-to-business transactions. ), Or a variety of services, such as a job placement service.

However, even though such online services are now widely available, searching and finding relevant information on the Web can be a daunting task, even for experienced users, sometimes even hours. Clearly, the Web is basically an open, multi-point to multi-point network, so that each user can select and retrieve different information from different servers. In fact, today, most web-based interactions use only textual input, such as entering a Uniform Resource Locator (URL) name, entering keywords on a search tool, or Hypertext Markup Language (HTML) documents. This is done by activating the text hyperlink for. In the near future, the development of audiovisual interfaces (e.g., human speech interfaces, Web-phone integration, etc.) may further reduce the importance of text input within the online environment. There is a good chance that the web will not be easy for users to use due to the massiveness, lack of organization, and randomness in the web. In short, within the web, there is no order or direction. Most of the information remains difficult to find, and finding the desired information that falls within the desired context in the foreseeable future will be more difficult.

Online service in live broadcast

Unlike multi-point-to-point web networks, live announcements to an audience (audience in the same location as the presenter or remotely located audience, such as those accessed through radio or television stations) are essentially from a single emitter. Communication to the receiver. All audiences receive the same content, whether close to the presenter or remote from the broadcaster.

Thus, in order to provide the same online service as can be accessed on the web, the first problem in live announcements is that information flows in the same direction from a single source to multiple receivers and from a provider to multiple audiences. It must be maintained. If there is no exchange of information with the audience, the communication flow is confined in one direction. Users cannot talk directly to received oral information to access additional information or services.

In addition, when an audience listens to a live presentation, the problem for the audience is to select a topic of interest and then access a network address (i.e. on the web) to access multimedia information or services related to the selected topic (i.e., URL). ). To this day, these problems have been partially solved.

A solution for providing web-like functionality for voice or radio information is to include information (e.g., URLs, etc.) in the transmitted broadcast sound signal or in a separate channel (simultcast). An example of such a system is US Pat. No. 6,125,172, entitled "Apparatus and method for initiating a transaction having acoustic data receiver that filters human voice," US Pat. No. 6,098,106, entitled "Method for controlling a computer with an audio signal." US Patent No. 5,841,978, entitled "System, method and device for automatic capture of Internet access information in a broadcast signal for use by an Internet access device." Patent No. 5,832,223, U.S. Patent No. 5,761,606, entitled "Media online services access via address embedded in video or audio program," U.S. Patent No. 5 entitled "Method for encoding and broadcasting information about live event using computer pattern matching techniques" 5,189,630, "Receiver apparatus and methods for identifying broadcast audio program selections in a radio broadcast syste m, US Pat. No. 5,119,507 or US Pat. No. 6,061,719 entitled "Synchronized presentation of television programming and web content."

The systems and methods described in these patents allow for the transmission of complementary information (e.g. URLs, etc.) that are encoded, inserted or modulated on the same sound or video signal, or transmitted on separate channels, simultaneously with the transmission of the main program. in need. The radio or television station must include means for encoding, modulating and transmitting this complementary information along with the acoustic signal. Audience or television viewers of the radio must have special receiver and decoder circuitry to recover this information.

Regardless of the device described above, the system allows the audience to “pre-select” a topic of interest (ie, a keyword or sentence), and select such a topic and a predetermined network address (ie, a URL). It has been developed to be relevant. This pre-designated network address is used to access multimedia information or services related to the preselected subject. Generally speaking, these systems are all based on speech recognition techniques. This technique is used to identify keywords (ie, selected words or sentences) that allow a particular action to be performed in response to recognition of a particular sound. Examples of such systems can be found in the patents below.

US Patent 5,946,050 entitled "Keyword listening device" discloses a method and system for monitoring the acoustic portion of a broadcast signal in which a relatively limited set of keywords is stored using a keyword listening device. . The keyword listening device monitors the broadcast signal for all of these keywords. Upon recognizing any one or more of the keywords, the broadcast sound signal is recorded at predetermined time periods and then fully analyzed. After the analysis, it is also possible, depending on the recorded and analyzed broadcast sound signal, to perform a number of different functions, such as connecting to an external network at a specific address or controlling a video cassette recorder.

U.S. Patent 6,011,854, entitled "Automatic recognition of audio information in a broadcast program," describes a sound processing system that retrieves information reports or updates (traffic, weather, time, sports, news, etc.) that are broadcast across one or more stations. Disclosed. The search is preselected by the user and is based on at least one keyword ("traffic", "weather", "time", "sports", "news", etc., depending on the desired report). The speech recognition software used by the sound processing system scans the radio station for the requested information report, but the user can listen to other sound sources (CDs, tapes, other radio stations, etc.), in which case this sound source There is no need to monitor (ie, listen to) the content of information from. When a requested information report used for a radio broadcast is detected based on the entered keyword, the sound processing system switches the sound output to a radio station that transmits the desired broadcast, so that the user can select traffic, weather, time, sports and / or Makes timely and convenient listening to news reports or updates.

US Pat. No. 6,332,120, entitled "Broadcast speech recognition system for keyword monitoring," discloses a system for automatically monitoring broadcast sound for information of interest. The system includes a computer processor having a memory for storing a vocabulary of a keyword of interest, an acoustic receiver for receiving acoustic broadcasts, and a time associated with the acoustic receiver and the computer processor in which one of the keywords appears in the received acoustic segment. It includes a speech recognition system for detecting the. A report generator coupled with the computer processor and responsive to the detection of the keyword generates a report on the detected keyword and details related to its content.

Although the above-mentioned system does not need to transmit complementary information embedded in an acoustic signal (or as a secondary signal transmitted at the same time as retransmission of the main program), the audience may still have a voice to detect the occurrence of a hyperlink term in the data stream. It must have a receiver with a recognition function.

In the field of presentation processing, the ability to identify the occurrence of a word or sentence in a stream of speech data is commonly referred to as "word spotting." The purpose of audio wordspotting is to identify a range of search terms within a digitized continuous presentation stream without requiring manual editing of the dictionary. Searching and indexing live broadcasts that must be voiced by any presenter is particularly problematic. This is largely due to the limited functionality of existing automatic speech recognition technology. In the system described above, it is important to note that the word extraction operation is done at the audience side in a manner that is independent of the presenter and uses a presentation model tuned to use speech data other than the data to be recognized.

In fact, the fundamental problem with any system is that "word extraction" (ie, identification of pre-specified keywords or terms) in a continuous manner independent of the presenter, based on unknown or common presentation styles, vocabulary, noise levels and language models. ), The operation of the state-of-the-art speech recognition technology is not reliable.

As expressed in the description above, in recent years, even in recent years, the level of conversation with the user has been increased and improved, and more information, learning or entertainment opportunities (eg, interactive television, WebTV, etc.) While interactive systems have been developed to provide the services, important sources of information, such as those that can be found on the Web, are available to audiences of live presentations (for example, live conferences or live interviews received from radio or television broadcasts). Is still inaccessible.

Thus, today, it would provide a convenient, universal, easy mechanism for allowing an audience to listen to live presentations (eg, instead of receiving live broadcast programs) to select and access complementary information. There is a need.

In addition, there is no need to include these hyperlinks as in conventional one-way broadcast signals, and more generally there is no need to physically transmit such hyperlinks and modify the conventional transmitter or receiver. Without, the presenter and producer of a live program will have relevant data on the web from selected terms (typically selected utterances, words or sentences) to be pronounced during the presentation (eg, during a conference or during a live radio or television program). There is a need to allow hyperlinks to be created.

The overall object of the present invention is to enhance sound information such as live broadcast announcements or live broadcast radio or television broadcast programs with complementary information or services related to the sound information.

Another object of the present invention is to create a hyperlink between a selected term or word to be pronounced by the presenter during the presentation process and complementary information related to the selected term.

Another object of the present invention is to identify hyperlink terms when they are pronounced by the presenter during the presentation process and to activate hyperlinks associated with these identified terms.

Another object of the present invention is to allow an audience to select one or more terms related to a subject of interest to them during the live presentation, and to access information related to the previously selected subject immediately after or after the announcement. It is to be.

It is yet another object of the present invention to allow an audience of a live broadcast to access information related to the subject matter that attracts their attention with minimal effort and participation on the part of the audience, while minimizing the complexity of the equipment required during the presentation. will be.

The present invention relates to a system, method and computer program that, as defined in the independent claims of the claims, immediately after the announcement or after a period of time, allows the audience of the live announcement to access information related to the pronounced term during the announcement.

The system associates hyperlinks (ie, URLs) with selected terms or words that are likely to be pronounced by the presenter during the presentation process. A speech recognition system operating on a presenter device (i.e., a computer system with a microphone connected) recognizes the presenter's pronunciation (ie, word extraction) for one of the hyperlink terms during the presentation, and recognizes each recognized hyperlink. Record the time the term is pronounced.

In addition, the system is based on the same universal time, with the presenter device and several audience devices (e.g., workstations, handheld computers, Personal Digital Assistants, PDAs or other smart phones). Kind of hand-held computing devices, etc.), based on always synchronizing the flow of information transmitted by the presenter and received by the audience, regardless of the relative location of the presenter and the audience. . Whenever an audience recognizes a topic of interest during the presentation, the audience immediately selects the topic by simply pressing a key provided on the audience device. The world standard time at which the audience has selected a topic is stored in the audience device.

In a preferred embodiment of the present invention, the synchronization between the presenter device and the audience device is a global standard such as Global Positioning System Time (GPS), Global Orbiting Navigational Satellite System (GLONASS), or any other suitable standard time based on a satellite system. By referencing the time. The GPS or GLONASS receiver is integrated or connected to the presenter device. A GPS or GLONASS receiver is integrated or connected to each audience device. Each audience device is independent and separate from the radio or television set used by the audience to receive the broadcasted announcement.

More specifically, the present invention discloses a system, method, and computer program for creating a Speech Hyperlink-Time Table from a presenter device while presenting one or more foils, wherein the Presentation Hyperlink Time A table can be accessed by one or more audiences. The method described above,

A presentation comprising means for identifying one or more predefined hyperlinked terms to be pronounced by the presenter, and means for locating and accessing information about each of the one or more predefined hyperlinked terms Navigation and access steps to navigate and access hyperlinked tables,

Searching for means of searching and accessing the presentation hyperlink-time table;

During the presentation,

When the presenter pronounces the hyperlink term, recognizing a predefined hyperlink term in the presentation hyperlink table using a speech recognition system connected to the presenter device,

For each recognized hyperlink term,

Determining a universal-time corresponding to the speaker presenting a recognized hyperlink term,

Creating a new record for the presentation hyperlink-time table,

A world standard time corresponding to the presenter pronounces the recognized hyperlink term, means for identifying a recognized hyperlink term retrieved from the presentation hyperlink table, and a recognized hyperlink term retrieved from the presentation hyperlink table Copying means for retrieving and accessing information related to the new record.

The present invention also provides a system for selecting one or more hyperlink terms pronounced by a presenter from an audience device during a presentation and accessing relevant information for each of the one or more selected hyperlink terms after a predetermined time has elapsed; A method and a computer program are also disclosed. This way,

Each time a selection command is received by the presenter to select the term pronounced at that time during the course of the presentation, determining the current world standard time and recording the current world standard time in the selection hyperlink time table;

Accessing a presentation hyperlink time table accessible by one or more of the audience devices—the presentation hyperlink time table includes a predefined hyperlink term for each of a plurality of predefined hyperlink terms pronounced by the presenter. Includes global standard time intervals as they are pronounced, means for identifying predefined hyperlink terms, and means for searching and accessing information related to predefined hyperlink terms;

For each world standard time recorded in the selection hyperlink time table,

Identifying within the presentation hyperlink time table the selected hyperlink term pronounced at recorded world standard time;

Retrieving, from the presentation hyperlink time table, means for identifying, navigating and accessing selected hyperlink terms;

Identifying a selected hyperlink term, retrieved in the search step above, and storing means in the selection hyperlink-time table for searching and accessing information related to the selected hyperlink term.

Other embodiments of the invention are set forth in the appended dependent claims.

By referring to the following specification, claims and drawings, the above-described objects, features and advantages of the present invention and other objects, features and advantages will be better understood.

New features that are believed to have the properties of the invention and features of the invention are set forth in the appended claims. However, the preferred mode of use, other objects and advantages thereof, as well as the present invention itself, will be best understood by reference to the following detailed description of exemplary and detailed embodiments in conjunction with the accompanying drawings.

1 illustrates that an audience of a live broadcast presentation, such as a conference, recognizes a subject of interest.

FIG. 2 shows that when listening to a live radio or television program, the audience perceives a subject of interest.

FIG. 3 illustrates that, in accordance with the present invention, the presenter workstation and audience device are synchronized according to the same world standard time, and a speech recognition system operating on the presenter device recognizes hyperlinked terms pronounced by the presenter.

4 is a diagram showing that a presenter prepares a presentation in advance according to the present invention.

5 illustrates, in accordance with the present invention, a presenter selecting a statement, word or term to generate a hyperlink (ie, an associated URL) to multimedia information or service.

FIG. 6 illustrates generating a presentation hyperlink table that associates an address allowing a presenter to access multimedia information or services for each hyperlink term, in accordance with the present invention.

7 illustrates a presenter training a speech recognition system prior to presentation, in accordance with the present invention.

8 illustrates, during a presentation, a speech recognition system operating on a presenter workstation that recognizes the hyperlink term (ie, word extraction) when the presenter pronounces the hyperlink term.

9 illustrates a communication network (e.g., the Internet) when a hyperlink term is recognized (word extracted) by a speech recognition system, in accordance with the present invention, a world standard time for the recognized term, the associated address and the recognized time. A diagram illustrating storing in a presentation hyperlink time table located on a presentation server connected to the system.

10 illustrates an example of a presentation hyperlink time table stored on a presentation server in accordance with the present invention.

FIG. 11 is a diagram illustrating that, in accordance with the present invention, an audience recognizing a term corresponding to a subject of interest during a presentation proceeds to simply select the term by pressing a key provided on the portable computer.

FIG. 12 illustrates storing, in accordance with the present invention, a world standard time of day in which an audience selected a term of interest during a presentation in a selection hyperlink time table located within the audience device.

13 is an optional hyperlink time table located within its workstation using information contained in a presentation hyperlink time table stored on a presentation server by connecting the audience device to a communication network, in accordance with the present invention. A diagram illustrating updating.

FIG. 14 illustrates that hyperlink terms, together with associated Uniform Resource Locators (URLs), are identified and copied from a presentation hyperlink time table stored on a presentation server to a selected hyperlink time table on an audience device, in accordance with the present invention. .

FIG. 15 illustrates that, in accordance with the present invention, an audience selects a hyperlink term (corresponding to a subject of interest selected by the audience during the presentation) from an updated selection hyperlink time table and activates the corresponding hyperlink. drawing.

FIG. 16 illustrates, in accordance with the present invention, hyperlinked multimedia information or services for selected terms are accessed via a communication network and retrieved on an audience device.

17 illustrates the steps of creating a presentation hyperlink table and training a speech recognition system to identify hyperlink terms in accordance with the present invention.

18 illustrates the steps of creating a presentation hyperlink time table on a presentation server and recognizing hyperlinked terms pronounced during presentation in accordance with the present invention.

FIG. 19 illustrates generating a selection hyperlink time table on an audience device and selecting a subject of interest during the presentation process in accordance with the present invention.

20 illustrates the steps of retrieving Uniform Resource Locators (URLs) and accessing information or services related to the selected hyperlink term.

As shown in Figures 1 and 2, the present invention is directed to the audience 100 of a live broadcast event (e.g., participation in a conference) 102, 202 or a radio audience or television viewer 200 participating in a live broadcast program. A system and method for selecting one or more of the topics 101, 201 that draw their attention or interest immediately after presentation or over time, thereby providing easy access to multimedia information related to the selected subjects 103, 203. We start about.

1 shows a typical situation according to the invention. An audience 100 watching a live broadcast presentation (eg, a conference on the topic “wine and health”) is interested in a topic 101 (eg, the term “Resveratrol”). You will want to have additional information about it. Under these circumstances, during the presentation process, the audience 100 can select subjects (eg, terms such as "Tannis", "Phenols", "Resveratrol"), and immediately or during the presentation. After this has elapsed, there is a need to provide a simple mechanism for allowing access to information related to the selected subject, for example, on a server connected to the Internet.

2 illustrates another typical situation. The television viewer 200 watches a live broadcast presentation (eg, a live broadcast television program on the theme of "wine and health").

As shown in FIG. 3, the hyperlink (ie, URL) 303 is associated with a term or word 304 that may be referred to by the presenter 301 during the presentation process. These hyperlink terms or words are associated with the related topic or item 302. Speech recognition system 305 operating on presenter workstation 306 (ie, a computer system to which microphone 307 is connected) detects that one of the hyperlink terms 308 is pronounced by the presenter during the presentation. The time 309 at which the detected hyperlink term is pronounced is recorded. The system may also present the presenter device 306 and several audience devices 311 (e.g., workstations, portable computers, personal digital assistants, smartphones, or any other) according to the same world standard time (309, 310). Other types of portable computing devices, etc.) to synchronize 312 information of information transmitted by the presenter 301 and received by the audience 300 regardless of the relative location of the presenter 301 and the audience 300. It is based on keeping the flow synchronized at all times. Each time the audience 300 recognizes a topic 313 of interest during the presentation, the audience immediately selects the topic by simply pressing a key 314 provided on the audience device 311. The world standard time 310 selected 314 by the audience 300 is stored on the audience device 311.

As shown in Fig. 3, the present invention is based on the following principle.

1. Regardless of the relative location of the presenter 301 and the audience 300, the presenter workstation according to the same world standard time 312 (e.g., GPS time provided by the GPS receivers 309, 310, etc.) 306 synchronization with the audience device 311.

2. The hyperlink term 304 (e.g., pronounced by the presenter 301 during the presentation 302) using the speech recognition system 305 operating on the presenter workstation 306 (to which the microphone is connected). For example, "resveratrol 308".

Universal Standard Timing Systems

Common timing sequences that are independent of the location of the presenter and the audience are, for example, Global Positioning System (GPS) time or Universal Time Co-ordinated (UTC) time (also known as GMT and ZULU time today). May be derived from an absolute timing reference.

To transmit accurate timing signals, GPS uses 24 satellites that orbit along an orbit with a 55 ° inclination 10,000 miles from Earth. Anywhere on earth, with a given GPS receiver, these timing signals can be used to determine their location. Transmission at 1575 MHz carries a 1-MHz bandwidth phase-modulated signal, referred to as a clear acquisition (C / A) code. When the GPS receiver receives this signal from at least three GPS satellites, the GPS receiver can determine its latitude and longitude with an accuracy of approximately 30 meters. Apart from determining geographic location, GPS is now widely used for the distribution of Precise Time and Time Intervals (PTTIs). The system uses time of arrival (TOA) measurements to determine the location. Since the time can be obtained in addition to the location by measuring the TOA of the four satellites entering the field of view at the same time, the user does not necessarily have to have a precisely set clock. If altitude is known, three satellites are sufficient. If the user is stationary at a known location, in principle, the time can be obtained by observing one satellite. Information about GPS visual services is available at http://tycho.usno.navy.mil/, operated by the "Time Service Department" at the US Naval Observatory in Washington, DC.

Today, GPS is a major source of accurate time around the world. GPS is widely used as a means of transmitting time from one location to another as well as a visual source. Three types of time are available from GPS, GPS time, UTC estimated and generated by the United States Naval Observatory, and time available from the atomic clocks of each free-navigating GPS satellite. The Master Control Station (MCS) at the Falcon Air Base near Colorado Springs, Colorado, USA, collects GPS satellite data from five monitor stations around the world. The Kalman filter software program finds time error, frequency error, frequency drift, and Kepplerian orbit parameters for each satellite and its operating clock. This information can be uploaded to each satellite and broadcast in real time. This process provides GPS time consistency in a few nanoseconds across the constellation and provides accurate positioning within a few meters by satellite.

Universal Time Co-ordinated (UTC), the second world standard time standard, introduces leaf seconds to keep the earth rotating and synchronized. In order to provide an estimate of UTC time derivable from the GPS signal, a set of UTC corrections is also provided as part of the GPS broadcast signal. This broadcast message includes the time difference between the total seconds of GPS time and the total seconds of UTC. This complicates the software to handle the gentle flow of the data stream or to calculate the time between data samples. Since the present system avoids the introduction of leaf seconds, the GPS time is preferred in the present invention, and the GPS time can be easily related to UTC. Information about UTC (GMT) time services can be found at http://time.greenwich2000.com/.

GPS receiver

Direct-to-Digital GPS receivers are described at the following Web site:

http://w3.research.ibm.com/present/gto200038.htm

This is an example of an inexpensive, compact chip that can integrate GPS into any device (eg, PDAs, mobile phones, wearable computers, video cameras, etc.). The receiver was jointly developed by IBM and Leica. When integrated with the CMOS technique, the high-speed analog performance of the SiGe technique allows the integration of this single chip directly into a digital Global Positioning System (GPS) receiver. It has been confirmed that location information derived from GPS can be used for a variety of purposes, from cartography and surveying to vehicle tracking of 911 telephone call callers, automated agricultural vehicles and robotic golf carts. This receiver chip reduces radio range and complexity. There are no analog mixer stages here, but expensive discrete components that require conventional two stage analog down conversion (such as high quality filters). This does not exist. Instead, the input GPS signal is fully digitized directly at the antenna and then digitally filtered within the CMOS based chip. This direct digitization is made possible by the ability of SiGe technology to run at high speed with very little power, and the core of the technology is a SiGe-based analog-to-digital data converter.

In accordance with the present invention, a GPS or GLONASS receiver may be a speaker workstation (typically a personal computer) and an audience device (e.g., a personal computer, a wearable computer, a personal digital assistant (PDA), a smartphone, an onboard mobile computer) or onboard mibile computers). World standard time timing signals received from GPS or GLONASS satellites are used to initialize and synchronize internal electronic clocking systems on presenter workstations and audience devices according to the same world standard time. During a period in which a GPS or GLONASS satellite is out of view (for example, the presenter or audience's device is in a building or not connected to an external antenna, etc.) and thus no timing signal is received from such satellite, the timing information is It must continue to be derived from the automatic electronic clocking system of such a device. Depending on the drift of the clocking system installed in the device, and also to maintain sufficient timing accuracy, and to ensure that the user's device is synchronized to the same world standard time with the presenter workstation and station, the periodic recognition of the satellite signal is somewhat Should be done frequently. Practically,

If the user device is a portable or in-vehicle device, it can receive satellite signals when the user is outdoors or traveling.

If the user device is fixed or fixed for a long period of time in a home or building, the user device must be connected to the outdoors with a GPS or GLONASS antenna (eg an antenna installed on the building roof).

System and method for detecting words during successive presentations

Speech recognition is the process by which a computer (or other type of system or device) identifies a pronounced word. Basically, a speech recognition system is a computer that can accurately recognize what a presenter is saying.

Speech recognition is an extremely difficult task. Unlike written documents, there is no clear gap between pronounced words. The complete sentence or sentences are typically pronounced without interruption. Also, acoustic diversity in the presentation signal typically hinders inaccurate matching of sequences of subword units, such as the pronunciation of words or consonants and vowels. The main cause of diversity in a presentation is due to the coarticulation, or the tendency for the acoustic characteristics of a given speech or phone sound to vary depending on the generated phonetic context.

Speech recognition systems can be classified according to discourse style, vocabulary and adapted language models. Isolated word recognizers require the presenter to insert a short comma between individual words. Continuous speech recognizers can operate on rich discourse, but typically use a strict language model or grammar to limit the number of acceptable word sequences.

Word spotters are a special kind of speech recognizer. This too can work for rich discourse. However, rather than providing a global notation, the word extractor selectively searches for related words or sentences. Word extraction is useful for retrieving information based on keyword indexing or for recognizing delimiters in voice command applications.

Today, many speech recognition systems can provide the word extraction functionality required to support the present invention. Such a system enables the detection of predefined words or sentences (hyperlink terms) that can be pronounced by the presenter during or during the presentation. Such a speech recognition system is disclosed in the following patent, for example.

U.S. Patent No. 5,199,077 entitled "Wordspotting for voice editing and indexing" discloses a word extraction technique based on hidden Markov models (HMM). This technique allows the presenter to dynamically specify keywords and train the associated HMM by a single iteration of the keywords. Non-keyword speech is modeled to use HMMs that have been trained with pre-recorded samples of successive speeches. Such word extractors are intended for interactive applications, such as editing voice mail or mixed-media documents, and for indexing keywords in a single presenter sound or video record.

US Patent No. 5,425,129, entitled "Method for word spotting in continuous speech," discloses a system and method for analyzing a digitized presentation data channel for the presence of words or sentences from a desired list. The system and method according to the invention, implemented in connection with the IBM Continuous Speech Recognition System (ICSRS), comprises a subsystem for extracting a pre-specified word or sentence when there is irrelevant phonetic data. Provided within the recognition system.

U.S. Patent 5,950,159, entitled "Word spotting using both filler and phone recognition", discloses a word extraction system and method for identifying keywords in acoustic data. The method includes a filler recognition phase and a keyword recognition phase, during which the acoustic data identifies a single tone, temporal delimiters and likelihood scores for the single note. to generate likelihood scores, and during the keyword recognition step, the acoustic data is processed to identify instances of the specified keyword comprising a sequence of monotones, where the temporal delimiter and likelihood scores generated in the filler recognition step are Used in the keyword recognition phase.

US Pat. No. 6,006,185, entitled "System and device for advanced voice recognition word spotting," discloses a speaker-independent, continuous presentation, word extraction speech recognition system and method. In pronunciation, the boundaries of phonemes must be quickly and accurately distinguished. Pronunciation is divided into wave segments based on phoneme boundaries. A voice recognition engine searches multiple wavelength segments several times and analyzes the results to identify words that are pronounced correctly.

U.S. Patent No. 6,073,095 entitled "Fast vocabulary independent method and apparatus for spotting words in speech" describes a pre-processing step and a coarse-to-detailed search strategy for extracting word / single sequences in a presentation. strategy), a rapid vocabulary-independent method of extracting words from a presentation is disclosed.

US Patent No. 6,185,527, entitled "System and method for automatic audio content analysis for word spotting, indexing, classification and retrieval", discloses a system and method for indexing an acoustic stream for subsequent information retrieval.

U. S. Patent No. 6,230, 126, entitled " Word-spotting speech recognition device and system, " discloses a speech recognition device that includes a dictionary that stores features of a recognition object. The device compares the characteristics of the input presentation with the characteristics of the recognition object and based on the input presentation when the matching device finds a substantial similarity between the input presentation and one of the recognition objects. The dictionary further includes a dictionary updating unit for updating the length of time for the phonemes in the dictionary.

A fundamental problem with speech recognition systems is that they can perform "word extraction" (ie, identification of pre-specified keywords or terms) in a continuous manner independent of the presenter, based on unknown or common presentation styles, vocabulary, noise levels, and language models. In doing so, the operation of the latest technology is not reliable. On the other hand, in the present invention, since the presenter can train and tune the system using his voice, his specific pronunciation style, and a specially adjusted vocabulary and language model, the automatic words required by the present invention are required. The extraction function is relatively easy to implement.

According to one aspect of the invention, hyper from preselected pronunciation 304 (typically spoken, term or sentence) to be pronounced by presenter 301 during presentation 302 to relevant data 303 on the web. A system and method are disclosed for training a speech recognition system 305 that creates a link and identifies (word extraction) the hyperlink term 304.

According to another aspect of the present invention, when the hyperlink term 304 is pronounced by the presenter 301 during live broadcast presentation, the hyperlink term 304 is automatically recognized (word extracted) and the hyper recognized on the network server. A system and method are disclosed for generating a table that includes a list of link terms 304, an associated network address (ie, URL) 303, and a world standard time 309 corresponding to the recognition of such hyperlink terms.

According to another aspect of the invention, during the presentation 302, the audience 300 receives a world standard time 310 corresponding to selecting 314 one or more topics of interest 313 of interest to the audience device ( A system and method for recording to 311) is disclosed.

According to another aspect of the present invention, a system and method are disclosed that enable an audience 300 to access and retrieve relevant information related to hyperlink term 304 from a server connected to a computer network.

How to create a presentation hyperlink table and train the speech recognition system with vocabulary of hyperlink terms

Referring to FIG. 17, prior to a presentation, the present invention may be used by the presenter 301 to select from the selected pronunciation 304 (typically a pronounced word or sentence) to be pronounced by the presenter 301 during the presentation 302. Speech recognition system 305 to generate hyperlinks to relevant data 303 on one or more servers connected to a computer network, preferably web, and extract words for presentation with the vocabulary of hyperlink term 304. A system, method, and computer program that can train) are disclosed. This way,

During the creation or editing of a presentation,

Editing the presentation or draft text 400 (1701),

Selecting and marking a plurality of related terms or words 501 in which the hyperlink should be generated in the presentation 500, 1702;

Creating a presentation hyperlink table 600 for the presentation (1703),

Defining in the presentation hyperlink table 600 a hyperlink between the selected term or word 501 and multimedia information or services located on one or more servers 909 connected to the computer network 908. The step includes assigning a name and / or description 601 (preferably a brief description) 1704 for each of the selected hyperlink terms or words 501 and a destination address 602 within the network 908. Step 1705 to access the desired multimedia information or service by assigning (eg, a URL, etc.), and an assigned name (or description) 601 and / or a destination address 602 are presented. 1704)

Once a hyperlink is defined within the presentation hyperlink table 600,

The speech recognition system 701 running on the presenter workstation 702 so that the hyperlink term is automatically recognized when the hyperlink term is pronounced by the presenter 700 during the presentation. Training 1707 with a vocabulary of.

4 shows a presentation 400 of an exemplary presentation (eg, a conference entitled “Wine and Health”, etc.) performed in a conference room or broadcast on a radio or television. During a live presentation, the presenter 301 may read the entirety of such a statement, or the presenter may intentionally omit some parts of the reading, change the order, or introduce a replacement or additional commentary for convenience.

5 shows that the presenter, the program editor (or the person entrusted with the editing) obtains the presentation 500 and the predetermined terms 501 ("phenol", "resveratrol", "tanin", "ellagic acid"). ", Words or phrases such as" Hydroxycinnamates "," Free radicals "," Anthocyanin "," Gallic acid ", etc.) The method of associating with information is shown. Each of these selected terms, referred to as hyperlink terms, must be associated with a network address (ie, a corresponding URL) on the web that allows searching for relevant information or services.

6 shows that the presenter, program editor (or entrusted editor), each selected hyperlink term 601 on the presenter workstation 306 (i.e., the word or sentence selected in the announcement, eg, "resveratrol", etc.). To generate a presentation hyperlink table 600 that associates a... With a corresponding URL on the web 602 (eg, http://www.ag.uiuc.edu/~ffh/resvera.html, etc.). .

FIG. 7 illustrates a speech recognition system 701 having a word extraction function installed on its workstation 702 by the presenter 700 prior to the presentation of the hyperlink term 703 (eg, “phenol” “resveratrol”). Training with a vocabulary of ", " tannin. &Quot; In a particular embodiment of the invention, the word extraction function is implemented using an IBM Continuous Speech Recognition System (ICSRS) running on an IBM ViaVoice software product.

How to recognize hyperlink terminology during a presentation and create a presentation hyperlink timetable on the presentation server

As shown in FIG. 18, the present invention is also used within the presenter workstation 802 to recognize the hyperlink term 803 when the hyperlink term 803 is pronounced by the presenter 800 during a live broadcast presentation. And, on a presentation server 907 connected to the network 908, generates a presentation hyperlink time table 906, a sequence 905 of recognized hyperlink terms 304, a corresponding network address (ie, a URL). 303 and a system, method, and computer program for updating this table with a record comprising a world standard time 309 where such hyperlink terms are recognized. More specifically, the method,

Generating (1801) a presentation hyperlink time table 906 for presentation on a presentation server 907 connected to the network 908,

During the presentation,

Hyperlink terminology by the presenter 800 by performing a "word extraction" on the presentation using a speech recognition system 801 running on the presenter workstation 802 and trained for that purpose. Recognizing a pronunciation of 1803 (1802),

For each recognized hyperlink term 803,

Determining, using a world standard time device, a world standard time 904 corresponding to the recognition of the hyperlink term 903, 1803;

The world standard time 904 at which the hyperlink term 903 has been recognized, the name or short description 601 of the hyperlink term 803 recognized using the presentation hyperlink table 600, and the presentation hyperlink table. Generating (1804) a record comprising a network address 1005 (ie, a URL) corresponding to the recognized hyperlink term 803 using 600;

Storing said record in a presentation hyperlink time table 906, 1000 stored on a presentation server 907 accessible from the presenter workstation 902 via a network 908.

8 shows an IBM continuous speech recognition running on a speech recognition system 801 installed on the presenter workstation 802 (eg, on an IBM ViaVoice software product) while the presenter 800 presents his presentation. Implemented using the system (ICSRS), a "word extraction" function, previously trained by the presenter, illustrates how to automatically detect the hyperlink term 803 from the speech stream.

9 recognizes a hyperlink term 903 (e.g., "resveratrol", etc.) using a speech recognition system 901 running on the presenter workstation 902,

The world standard time 904 corresponding to the recognition of the hyperlink term 903 (eg 12/05/2001 14:23:18),

A hyperlink term name or short description 601 (eg, "resveratrol"),

The associated URL 602 (eg, http://www.ag.uiuc.edu/~ffh/resvera.html) (the last two are extracted from the presentation hyperlink table 600) and the presentation server ( Forwarding 905 and storing to a presentation hyperlink time table 906 located on 907.

FIG. 10 shows an example of a presentation hyperlink time table 1000, 906 stored on a presentation server 907, displayed at the end of a presentation (eg, a presentation on the subject “Wine and Health” 1001). . The header of this table may be a URL 1002 (http://www.directbuyer.com/conference-0173.htm/) or a network address where the announcement hyperlink time table 1000 is stored within the announcement server 907 (e.g., For example, www.directbuyer.com. The URL where the presentation hyperlink time table 1000 can be found within the network must be known to the audience in advance. Each column in this table corresponds to a hyperlink term 903 released by the presenter during the presentation and recognized by the speech recognition system 901. Each row is a separate

A world standard time 1003 corresponding to the recognition of the speech recognition system 901 to the pronounced hyperlink term 903 during the presentation,

A name and / or short description 601, 1004 for the recognized hyperlink term 903, copied from the presentation hyperlink table 600, and

Copied from the presentation hyperlink table 600 and corresponding to URLs 602 and 1005 corresponding to the recognized hyperlink term 903.

How to create a selection hyperlink timetable and select topics of interest

As shown in FIG. 19, the present invention also creates an optional hyperlink time table 1106 on the audience device 1102 to receive additional information related to the subject immediately after the announcement or after time elapses. Disclosed are a system, method, and computer program for recording in this table a sequence of world standard time 1105 corresponding to the selection 1104 of an audience 1100 of interest during the presentation. More specifically, this method used in an audience device,

Generating 1901 a selection hyperlink time table 1200 on an audience device 1102;

Recording 1902 the network address (i.e., URL) 1201 of the presentation hyperlink time table 906 stored on the presentation server 907 in the selection hyperlink time table 1200;

Listening to the announcement 1107 announced by the presenter 1101, and

During the presentation, recognizing 1904 a subject of interest 1103 that requires additional information or services, and

Selecting 1905 a subject of interest 1103 by inputting a selection command 1104 on the audience device 1102, and

Determining (1906) the world standard time 1105 at that time using a world standard time device (e.g. a GPS receiver, etc.) integrated or connected to the audience device;

Recording 1907 the global standard time 1105 at this time in the selection hyperlink time tables 1106 and 1202.

FIG. 11 shows that during the presentation, the audience 1100 operates in a stand-alone mode (ie, isolated and not connected to the network) using the portable audience device 1102. According to the particular embodiment described in detail in FIG. 11, whenever the audience 1100 recognizes a topic 1103 of interest during the presentation, the audience 1100 is immediately provided 1104 on the audience device 1102. Select these topics instantly by simply pressing a key. The world standard time 1105 corresponding to the topic selection by the audience is stored in the selection hyperlink time table 1106 on the audience device 1102.

12 shows a typical selection hyperlink time table 1200 generated on the audience device 1102 during the presentation. Each row in this table corresponds to a different world standard time that corresponds to selecting the topic of interest to the audience during the presentation. The header of this table contains the URL 1201 (eg, http://www.directbuyer.com/conference-0173,htm/) of the announcement hyperlink time tables 906 and 1000 corresponding to the announcement. The presentation hyperlink time tables 906 and 1000 on the presentation server 907 (eg, www.directbuyer.com, etc.) are updated from the presenter workstation 902 during the presentation. As mentioned above, the URL of the presentation server must be provided to the audience in advance so that the audience can generate a locally selected hyperlink time table 1200.

How to Retrieve Hyperlinks and Access Information

As shown in FIG. 20, the present invention also provides that the audience 300 accesses and retrieves from the web the corresponding data or information related to the hyperlink term 304 that is activated when the audience performs the corresponding selection 314. Disclosed are a system, method, and computer program that make it possible. More specifically, this method used on audience device 1300,

Accessing the presentation server 1303 (2001);

For each world standard time 1202 recorded in a selection hyperlink time table 1304, 1200 located on an audience device 1300,

Forwarding (2002) the world standard time 1306 recorded in the presentation server 1303;

A hyperlink term 1004 (hyperlink term activated at that world standard time) associated with the world standard time 1306 within the presentation hyperlink time tables 1302 and 1000 located on the presentation server 1303. Identifying (2003),

Retrieving the name (or description) 1004 and the destination address (URL) 1005, 1307 of the selected hyperlink term from the presentation hyperlink time tables 1302, 1000 located within the presentation server 1303 ( 2004),

Storing (2005) the retrieved hyperlink name and destination address (URL) 1307 in the selection hyperlink time tables 1304, 1402 located on the audience device 1300;

Selecting (2006) the hyperlink 1501 using the hyperlink name (or description) retrieved from the presentation server 1507 or the associated destination address in the selection hyperlink time table 1502, and

Activating (2007) the hyperlink 1501 using a browser program running on the audience device 1500,

Accessing (2008) information and / or services located on a server 1506 connected to the network 1505 using the retrieved destination addresses 1503 and 1504 associated with the selected hyperlink 1501,

Retrieving (2009) information and / or services 1604, 1601 from an accessed server 1603 on the network 1602;

Displaying the retrieved information and / or service 1604 on the audience device 1600 using a browser program.

FIG. 13 illustrates an audience by connecting portable device 1300 of audience 1100 to communication network 1301 (eg, an Internet network) and accessing a presentation hyperlink time table 1302 on presentation server 1303. Updating information on the selection hyperlink time table 1304 of the device 1300 is shown.

14 details the process of updating the selection hyperlink time table 1400 on the audience device 1300 using the presentation hyperlink time tables 1401 and 1302 on the presentation server 1303. Basically, using this process, the active hyperlink term (or short name and / or short description) and the URL of the hyperlink term are selected at the presentation server ( It is identified within the presentation hyperlink time table 1401 located on 1303 and copied into the selection hyperlink time table 1402 located on the audience device 1300.

FIG. 15 illustrates selecting a hyperlinked web page for a topic selected by an audience during an announcement from an audience device 1500 and gaining access to that web page. Basically, the audience points and selects the hyperlink term 1501 on the updated selection hyperlink time table 1502, activates the web browser using software on the audience device 1500, and selects the selected item 1501. Triggers a hyperlink to URL 1503. In the example shown in this figure, the audience selects the hyperlink term "elazic acid" (which is, in fact, a chemical compound found in wine), and URL 1504, i.e., http://www.hopeforcancer.com Trigger a hyperlink that points to /ellagicacid.htm.

16 illustrates a web server 1603 to which a web page 1601 (eg, a document of ellagicacid.htm, etc.) associated with the selected hyperlink term 1501 (eg, "Ellagic Acid", etc.) is accessed. Received via network 1602 from http://www.hopeforcancer.com (eg, http://www.hopeforcancer.com) and shown on display 1604 or playback on audience device 1600.

While the invention has been particularly shown and described with reference to preferred embodiments, it will be understood that various modifications may be made to the form and details of the embodiments without departing from the spirit and scope of the invention.

Claims (20)

A method of generating a Speech Hyperlink-Time table in connection with a universal time system that provides absolute time. Providing a presentation hyperlink table comprising a plurality of entities, each of the plurality of entities comprising a hyperlinked term and a corresponding network address of a network, the network address associated with the hyperlink term Linking the hyperlink terminology to information, the information being on a server in the network; While the presenter is presenting, recognizing each hyperlink term in the presentation hyperlink table pronounced by the presenter, wherein the recognition step is performed by a speech recognition system on a computing device; For each recognized hyperlink term, Determining a universal time in which the hyperlink term was recognized; Identifying a network address corresponding to the recognized hyperlink term from the published hyperlink table; Generating a record in the announcement hyperlink time table comprising the universal time, the recognized hyperlink term and the network address corresponding to the recognized hyperlink term. How to create a presentation hyperlink timetable. Claim 2 was abandoned when the setup registration fee was paid. The method of claim 1, The universal time system is derived from a Global Positioning System (GPS) time system, a Universal Time Co-ordinated time (UTC time) system, a Greenwich Mean Time (GMT) system, and a free-running atomic clock of GPS satellites. One is selected from a group of visual systems How to create a presentation hyperlink timetable. The method of claim 1, Determining the universal time includes determining the universal time by a GPS receiver How to create a presentation hyperlink timetable. The method of claim 1, Providing the presentation hyperlink table, Selecting the hyperlink term from the presentation; For each selected hyperlink term, Identifying a network address corresponding to the selected hyperlink term, and storing an entity comprising the selected hyperlink term and the identified corresponding network address in the presentation hyperlink table. How to create a presentation hyperlink timetable. Claim 5 was abandoned upon payment of a set-up fee. The method of claim 1, The network is the Internet, the network address is a URL (Universal Resource Locator), the information is a web page, and the server is a web server. How to create a presentation hyperlink timetable. delete The method of claim 1, Prior to the recognition step, further comprising training the speech recognition system to recognize the hyperlink term when the hyperlink term is pronounced during the presentation. How to create a presentation hyperlink timetable. Claim 8 was abandoned when the registration fee was paid. The method of claim 1, The announcement may be included in a radio program or television program. How to create a presentation hyperlink timetable. The method of claim 1, The determining, verifying and generating step is executed by a computer program consisting of instruction code that is executed on the computing device How to create a presentation hyperlink timetable. delete delete As a method of processing a presentation in relation to a universal time system that provides absolute perspective, Inputting at least one selection command on an auditor device, wherein the audience device is a computing device, each selection command being input in real time in response to a hyperlink term pronounced at the presentation, each pronounced The hyperlink term appears in a record of one of a plurality of records of a presentation hyperlink time table included in a presentation server, wherein each record in the presentation hyperlink time table is associated with the hyperlink term of the presentation and the hyperlink term during the presentation. A network address linking said hyperlink term to information relating to said universal time and said hyperlink term, said information being on a server of a network; For each selection command entered, determining the universal time in which the selection command was entered and recording the determined universal time in a record of a selections Hyperlink-Time table included in the audience device. Containing How to handle a presentation. The method of claim 12, The audience device is coupled to the presentation server via the network, Selecting at least one universal time from the selection hyperlink time table; For each selected universal time, Delivering the selected universal time to the presentation server; Receiving a hyperlink term and an associated network address appearing in a record of the announcement hyperlink timetable from the announcement server, wherein the universal time included in the announcement hyperlink time table is closest to the selected universal time and the selected universal time. Not exceeding-with, Storing the received hyperlink term and associated network address in the selected hyperlink time table that includes the selected universal time. How to handle a presentation. The method of claim 13, Selecting a record of the selection hyperlink time table; Connecting to information related to a hyperlink term in said selected record using a network address in said selected record; Retrieving the information from a server in the network including the information; Displaying the retrieved information on the audience device; How to handle a presentation. Claim 15 was abandoned upon payment of a registration fee. The method of claim 12, The universal time system is selected from the group consisting of a GPS vision system, a Coordinated Universal Time (UTC) system, a Greenwich Mean Time (GMT) system, and a vision system derived from a free-driven atomic clock of GPS satellites. How to handle a presentation. Claim 16 was abandoned upon payment of a setup registration fee. The method of claim 12, Determining the universal time includes determining the universal time by a GPS receiver at the audience device. How to handle a presentation. Claim 17 was abandoned upon payment of a registration fee. The method of claim 12, The audience device is selected from the group consisting of workstations, portable computers, PDAs, smartphones and other types of portable computing devices. How to handle a presentation. delete Include audience devices that process presentation in relation to a universal time system that provides absolute perspective, The audience device is a computing device having a computer program therein, wherein the computer program comprising instruction code for implementing the method of any one of claims 12 to 17 is executed. Computing system. When executed, a computer program comprising instruction code for performing the method of claim 1 or 12 is recorded. Computer-readable recording media.

Images