JP7210939B2 - INFORMATION PROVIDING METHOD, TERMINAL DEVICE OPERATING METHOD, DISTRIBUTION SYSTEM AND PROGRAM - Google Patents

Description

The present invention relates to technology for providing various types of information about programs broadcast by broadcasting stations.

2. Description of the Related Art Conventionally, techniques have been proposed in which a terminal device acquires various types of information using identification information transmitted by acoustic communication using sound waves as a transmission medium. For example, Patent Document 1 discloses a technique in which sound superimposed with identification information is reproduced by a reproduction device such as a television receiver, and a terminal device that collects the reproduced sound acquires various information using the identification information. is disclosed.

JP 2012-155706 A

For example, under the configuration of Patent Document 1, according to the configuration in which identification information for identifying a broadcasting station is transmitted to a terminal device by acoustic communication, information on programs broadcast by the broadcasting station is provided to the terminal device. It is possible to However, when playing back a past program recorded in a recording device, there is a problem that information about the program cannot be provided to the terminal device. In view of the above circumstances, it is an object of the present invention to provide terminal devices with information not only about programs currently being broadcast, but also about recorded past programs.

In order to solve the above problems, an information providing method according to a preferred aspect of the present invention transmits broadcast data including, as an acoustic component, time information representing the time of a program broadcast by a broadcasting station, The terminal device that collected the reproduced sound receives the time information extracted from the reproduced sound from the terminal device, and the time information received from the terminal device represents the time information received from the terminal device among the plurality of related information related to the program of the broadcasting station. Related information corresponding to the time is transmitted to the terminal device.

A method of operating a terminal device according to a preferred aspect of the present invention extracts the time information from a reproduced sound containing, as an acoustic component, time information representing time relating to audio of a program broadcast by a broadcasting station, Related information corresponding to the time represented by the time information is acquired from among the plurality of related information.

An information providing system according to a preferred aspect of the present invention includes: a broadcasting system that transmits broadcast data including, as an acoustic component, time information representing the time of a program broadcast by a broadcasting station; and a reproduced sound that corresponds to the broadcast data. the terminal device receives the time information extracted from the reproduced sound from the terminal device, and related information corresponding to the time represented by the time information received from the terminal device, among the plurality of related information related to the program of the broadcasting station to the terminal device.

A terminal device according to a preferred aspect of the present invention includes: an information extracting unit for extracting time information from reproduced sound containing, as an acoustic component, time information representing time relating to audio of a program broadcast by a broadcasting station; and an operation control unit that acquires related information corresponding to the time represented by the time information, from among a plurality of related information related to.

1 is a block diagram illustrating the configuration of an information providing system according to a first embodiment; FIG. 1 is a block diagram illustrating the configuration of a broadcasting system; FIG. It is a block diagram which illustrates the structure of a processing apparatus. FIG. 3 is an explanatory diagram of the relationship between an acoustic signal, identification information, and time information; It is a block diagram which illustrates the structure of a terminal device. 1 is a block diagram illustrating the configuration of a delivery system; FIG. 4 is a flowchart illustrating the operation of the first embodiment; 4 is a flowchart illustrating the operation of the first embodiment; FIG. 11 is an explanatory diagram of a storage device of a distribution system in the second embodiment; 9 is a flowchart illustrating operations of a terminal device and a distribution system in the second embodiment; 11 is a flow chart illustrating the operation of a terminal device in the third embodiment;

<First Embodiment>
FIG. 1 is a block diagram illustrating the configuration of an information providing system 10 according to the first embodiment of the invention. The information providing system 10 is a computer system for providing various types of information to users U who view programs provided by broadcasting. A program in the first embodiment is content provided by television broadcasting such as terrestrial broadcasting or satellite broadcasting, and is composed of video and audio. The program may include CM (Commercial Message).

A user U uses the reproduction system 40 and the terminal device 50 . The reproduction system 40 reproduces programs by receiving broadcast waves. The terminal device 50 is, for example, a portable information terminal such as a mobile phone, a smart phone, a tablet terminal, or a personal computer. A guidance display terminal such as an electronic signboard (for example, a digital signage) installed in facilities such as commercial facilities may be used as the terminal device 50 .

As illustrated in FIG. 1, the information providing system 10 includes a broadcasting system 20 and a distribution system 30. FIG. Broadcast system 20 broadcasts programs. The distribution system 30 transmits information (hereinafter referred to as “related information”) C regarding programs broadcast by the broadcasting system 20 to the terminal device 50 . The related information C of the first embodiment is a character string (that is, subtitles) representing the contents of the audio in the program (hereinafter referred to as "program audio"). The broadcasting system 20 and the distribution system 30 communicate with each other via a communication network such as the Internet. Also, the distribution system 30 communicates with the terminal device 50 via a communication network 90 such as a mobile communication network or the Internet.

<Broadcast system 20>
FIG. 2 is a block diagram illustrating the configuration of the broadcasting system 20. As shown in FIG. As illustrated in FIG. 2, the broadcasting system 20 of the first embodiment is a computer system comprising a management device 21 and a transmission device 22. FIG. The management device 21 is installed, for example, for each broadcasting station, and the transmitting device 22 is shared by, for example, a plurality of broadcasting stations.

Broadcast data D representing a program is supplied to the management device 21 . The broadcast data D includes an audio signal A1 representing the sound of the program and a video signal V representing the video of the program. A program represented by the broadcast data D is a program recorded in the past or a program recorded in parallel with the broadcast. The management device 21 executes various processes on the broadcast data D. FIG. Broadcast data D processed by the management device 21 is supplied to the transmission device 22 . The transmission device 22 is broadcasting equipment for transmitting the broadcast data D over a wide range. The transmission device 22 includes, for example, a radio tower that transmits broadcast waves representing the broadcast data D. FIG. A transmitter that transmits the broadcast data D via a wired line may be used as the transmitter 22 .

As illustrated in FIG. 2 , the management device 21 includes a communication device 211 , a timer device 212 and a processing device 213 . The clock device 212 generates time information T representing the current time. The time indicated by the time information T includes, for example, the date and the hour, minute, and second. Generation of the time information T by the clock device 212 is repeated periodically, for example. The processing device 213 is realized by a processing circuit such as a CPU (Central Processing Unit) or a DSP (Digital Signal Processor), and processes the broadcast data D, for example. The processing device 213 of the first embodiment generates an acoustic signal A2 from the acoustic signal A1 of the broadcast data D. FIG. Broadcast data D after processing including the audio signal A2 and the video signal V is supplied to the transmission device 22 .

FIG. 3 is a block diagram illustrating the configuration of elements of the processing device 213 that generate the acoustic signal A2 from the acoustic signal A1. As illustrated in FIG. 3 , the processing device 213 of the first embodiment includes a signal generator 25 and a signal combiner 26 . The signal generator 25 generates a modulated signal M containing, as acoustic components, the identification information B of the broadcasting station and the time information T generated by the timer 212 .

The identification information B is a code uniquely given to each broadcasting station. Modulated signal M is generated by frequency-modulating a carrier wave of a predetermined frequency with identification information B and time information T, for example. The modulated signal M may be generated by sequentially performing spread modulation of the identification information B and the time information T using a spread code and frequency conversion using a carrier wave of a predetermined frequency. The frequency band of the sound represented by the modulated signal M is the frequency band in which sound can be emitted by the reproduction system 40 and collected by the terminal device 50, and the frequency band of the sound that the user U listens to in a normal environment. A higher frequency band (for example, 18 kHz or more and 20 kHz or less) is set. However, the frequency band of the sound represented by the modulated signal M is arbitrary.

The signal synthesizer 26 in FIG. 3 synthesizes the acoustic signal A1 of the broadcast data D and the modulated signal M generated by the signal generator 25 to generate the acoustic signal A2. The signal synthesizing unit 26 generates the acoustic signal A2 by, for example, adding the acoustic signal A1 and the modulated signal M (for example, weighted addition).

FIG. 4 is an explanatory diagram of the acoustic signal A2. As illustrated in FIG. 4, the acoustic signal A2 is a time signal over a plurality of unit periods Q (Q1, Q2, . . . ). Each unit period Q corresponds to, for example, a variable-length period during which a series of utterances continues in a program (that is, a period for each utterance). Any one unit period Q repeatedly includes identification information B and time information T (T1, T2, . . . ) multiple times. The time information T differs for each unit period Q. FIG. That is, the time represented by the time information T advances every unit period Q every moment. As can be understood from the above description, the audio signal A2 of the broadcast data D includes the time information T representing the time of the program broadcast by the broadcasting station indicated by the identification information B as an audio component. A configuration in which the acoustic signal A2 includes one piece of time information T for each unit period Q is also assumed. Also, the unit period Q may be a period of a predetermined length that is set independently of the speech.

The communication device 211 in FIG. 2 is a communication device that transmits the acoustic signal A1 of the broadcast data D to the distribution system 30. As shown in FIG. Time information T generated by the clock device 212 is added to the acoustic signal A1 transmitted by the communication device 211 every unit period Q. FIG. The time information T added to any one unit period Q represents the time corresponding to the unit period Q (for example, the time of the starting point of the unit period Q). Note that the communication device 211 may transmit the acoustic signal A2 processed by the processing device 213 to the distribution system 30 .

<Reproduction system 40>
The reproduction system 40 of FIG. 1 reproduces the program represented by the broadcast data D transmitted from the broadcasting system 20. FIG. A playback system 40 of the first embodiment includes a recording device 41 and a playback device 42 . The recording device 41 is video equipment that records the broadcast data D transmitted from the broadcasting system 20 . A user U selects a program to be recorded in the recording device 41 . For example, a recorder for recording broadcast data D on a recording medium such as a magnetic recording medium or a semiconductor recording medium is used as the recording device 41 . Note that the recording device 41 and the reproducing device 42 may be integrated.

The playback device 42 is video equipment that plays back the program represented by the broadcast data D. FIG. For example, a television receiver is preferably used as the playback device 42 . A user U selects a program to be played back by the playback device 42 . The reproduction device 42 includes a display device 421 that displays video represented by the video signal V of the broadcast data D, and a sound emitting device 422 that emits sound represented by the audio signal A2 of the broadcast data D. FIG. The display device 421 is, for example, a liquid crystal display panel, and the sound emitting device 422 is, for example, a speaker. Note that the display device 421 and the sound emitting device 422 may be configured separately. The reproducing device 42 reproduces the program represented by the broadcast data D transmitted by the broadcasting system 20 in parallel with the broadcast, and can also reproduce the past program represented by the broadcast data D recorded in the recording device 41. .

As described above, the acoustic signal A2 includes the identification information B and the time information T as acoustic components. Therefore, the sound emitting device 422 of the reproducing device 42 functions as an acoustic device that reproduces the program sound, and also as a transmitter that transmits the identification information B and the time information T by acoustic communication using sound waves as air vibrations as a transmission medium. also works. As can be understood from the above description, in parallel with the reproduction of the program audio, the identification information B of the broadcasting station of the program and the time information T representing the time regarding the program audio are transmitted around the reproducing device 42. .

<Terminal device 50>
FIG. 5 is a block diagram illustrating the configuration of the terminal device 50. As shown in FIG. As illustrated in FIG. 5 , the terminal device 50 includes a control device 51 , a storage device 52 , a communication device 53 , a sound pickup device 54 and a reproduction device 55 .

The sound pickup device 54 is an acoustic device (microphone) that picks up ambient sound. Specifically, the sound pickup device 54 picks up the sound emitted by the sound emission device 422 of the playback device 42 and generates the sound signal X representing the waveform of the sound. Acoustic signal X includes acoustic components representing identification information B and time information T. FIG. Therefore, the sound collecting device 54 is used not only for recording voice during voice communication or video recording, but also as a receiver for receiving identification information B and time information T through acoustic communication using sound waves as air vibrations as a transmission medium. Function. For the sake of convenience, the illustration of an A/D converter that converts the acoustic signal X generated by the sound collecting device 54 from analog to digital is omitted. Further, instead of the sound pickup device 54 configured integrally with the terminal device 50, a separate sound pickup device 54 may be connected to the terminal device 50 by wire or wirelessly.

The storage device 52 stores programs executed by the control device 51 and various data used by the control device 51 . For example, a known recording medium such as a semiconductor recording medium and a magnetic recording medium, or a combination of multiple types of recording media can be arbitrarily adopted as the storage device 52 . The communication device 53 communicates with the distribution system 30 under the control of the control device 51 .

The reproduction device 55 is an output device that reproduces the related information C. FIG. The playback device 55 of the first embodiment includes a display device (for example, a liquid crystal display panel) that displays the related information C. FIG. That is, the character string representing the content of the program audio reproduced by the reproduction device 42 is displayed as the related information C on the reproduction device 55 . Therefore, the hearing-impaired person can grasp the content of the program reproduced by the reproduction device 42 . Note that a sound emitting device that emits a sound representing the content of the related information C may be used as the reproducing device 55 .

The control device 51 is composed of a processing circuit such as a CPU, for example, and controls each element of the terminal device 50 in an integrated manner. As illustrated in FIG. 5, the control device 51 of the first embodiment implements multiple functions (an information extractor 511 and an operation controller 512) by executing programs stored in the storage device 52. FIG. A part of the functions of the control device 51 may be realized by a dedicated electronic circuit. Also, the functions of the control device 51 may be installed in a plurality of devices.

The information extraction unit 511 extracts the identification information B and the time information T from the acoustic signal X generated by the sound pickup device 54 by collecting the reproduced sound by the reproduction device 42 . Specifically, the information extraction unit 511 performs, for example, filter processing for emphasizing the frequency band containing the acoustic components of the identification information B and the time information T in the acoustic signal X, and modulation processing when generating the acoustic signal A2. The identification information B and the time information T are extracted from the acoustic signal X by the demodulation processing.

The operation control unit 512 acquires the relevant information C from the distribution system 30 and causes the reproduction device 55 to reproduce the relevant information C. FIG. Specifically, the operation control unit 512 generates an information request R including the identification information B and the time information T extracted by the information extraction unit 511 and transmits the information request R from the communication device 53 to the distribution system 30 . The information request R is transmitted each time the time information T extracted from the acoustic signal X changes (that is, every unit period Q). Further, the operation control unit 512 receives the related information C transmitted from the distribution system 30 in response to the information request R by the communication device 53 and causes the playback device 55 to play back the related information C. FIG.

<Distribution system 30>
FIG. 6 is a block diagram illustrating the configuration of distribution system 30. As shown in FIG. As illustrated in FIG. 6, the distribution system 30 of the first embodiment includes a control device 31, a storage device 32, and a communication device 33. FIG. The storage device 32 stores programs executed by the control device 31 and various data used by the control device 31 . For example, a known recording medium such as a semiconductor recording medium and a magnetic recording medium, or a combination of multiple types of recording media can be arbitrarily adopted as the storage device 32 .

Communication device 33 communicates with each of broadcasting system 20 and terminal device 50 . For example, the communication device 33 receives the acoustic signal A 1 and the time information T transmitted from the communication device 211 of the broadcasting system 20 and the information request R transmitted from the terminal device 50 . Also, the communication device 33 transmits related information C corresponding to the information request R to the terminal device 50 .

The control device 31 is composed of, for example, a processing circuit such as a CPU, and controls each element of the delivery system 30 in an integrated manner. As illustrated in FIG. 6, the control device 31 of the first embodiment implements multiple functions (a speech recognition section 311 and an information processing section 312) by executing programs stored in the storage device 32. FIG. A part of the functions of the control device 31 may be realized by a dedicated electronic circuit. Also, the functions of the control device 31 may be installed in a plurality of devices.

The speech recognition unit 311 performs speech recognition on the acoustic signal A1 received by the communication device 33 . That is, the speech recognition unit 311 identifies a character string representing the content of the program speech represented by the acoustic signal A1. For speech recognition of the acoustic signal A1, a known technique such as recognition processing using an acoustic model such as HMM (Hidden Markov Model) and a language model indicating linguistic constraints can be arbitrarily adopted. Speech recognition by the speech recognition unit 311 is performed for each unit period Q of the acoustic signal A1. A character string specified by speech recognition for an arbitrary unit period Q in the acoustic signal A1 is stored in the storage device 32 as related information C. FIG. As illustrated in FIG. 6, time information T (T1, T2, . . . ) of the unit period Q is added to the related information C of each unit period Q. In FIG.

As can be understood from the above description, the related information C representing the contents of the program audio and the time information T representing the time of utterance of the program audio are time-series for each utterance of the program audio (that is, each unit period Q). A reference table Z registered in is generated in the storage device 32 . Speech recognition for the acoustic signal A1 is executed in real time in parallel with the broadcast of the program. Therefore, the related information C and the time information T are sequentially added to the reference table Z for each utterance of the program voice in parallel with the broadcasting of the program. A reference table Z is generated in the storage device 32 for each broadcasting station. In the reference table Z of each broadcasting station, related information C and time information T relating to programs broadcast by the broadcasting station are registered.

The information processing section 312 identifies the relevant information C from the reference table Z according to the information request R received from the terminal device 50 . Specifically, the information processing unit 312 identifies the related information C corresponding to the time indicated by the time information T in the information request R from the reference table Z of the broadcasting station indicated by the identification information B in the information request R. . For example, the information processing unit 312 searches the reference table Z for the time information T representing the time closest to the time represented by the time information T in the information request R, and identifies the related information C corresponding to the time information T. FIG. Further, the information processing section 312 transmits the related information C specified from the reference table Z from the communication device 33 to the terminal device 50 which is the transmission source of the information request R. FIG. As can be understood from the above description, the distribution system 30 of the first embodiment includes related information C corresponding to the time represented by the time information T received from the terminal device 50, among the plurality of related information C related to the program of the broadcasting station. is transmitted to the terminal device 50 concerned.

<Action>
FIG. 7 is a flowchart illustrating a specific procedure of operation of the first embodiment. FIG. 7 assumes the case where the reproducing device 42 reproduces the program in parallel with the broadcast of the program (that is, the case where the broadcast data D is not recorded in the recording device 41). The process of FIG. 7 is executed for each utterance of the program voice (that is, for each unit period Q).

The communication device 211 of the broadcasting system 20 transmits the acoustic signal A1 of the unit period Q to the distribution system 30 together with the time information T generated by the clock device 212 (Sa1). The communication device 33 of the distribution system 30 receives the acoustic signal A1 and the time information T transmitted from the broadcasting system 20 (Sa2). The speech recognition unit 311 generates related information C by speech recognition of the acoustic signal A1 received by the communication device 33 (Sa3). The control device 31 associates the related information C generated by speech recognition with the time information T and registers them in the reference table Z (Sa4).

The processing device 213 of the broadcasting system 20 generates an acoustic signal A2 containing, as acoustic components, the identification information B of the broadcasting station and the time information T generated by the clock device 212 from the acoustic signal A1 of the unit period Q (Sa5). The broadcasting system 20 transmits the broadcasting data D including the audio signal A2 and the video signal V from the transmitting device 22 (Sa6). The order of transmission of the acoustic signal A1 by the communication device 211 (Sa1) and transmission of the broadcast data D by the transmission device 22 (Sa5, Sa6) may be reversed.

The reproducing device 42 of the reproducing system 40 reproduces the program represented by the broadcast data D received from the broadcasting system 20 (Sa7). That is, the video represented by the video signal V of the broadcast data D is displayed on the display device 421 and the program sound represented by the audio signal A2 of the broadcast data D is emitted from the sound emitting device 422 .

The sound collecting device 54 of the terminal device 50 generates the acoustic signal X by collecting the program sound emitted by the sound emitting device 422 of the reproducing device 42 (Sa8). The information extraction unit 511 extracts the identification information B and the time information T from the sound signal X generated by the sound collection device 54 (Sa9). The operation control unit 512 transmits an information request R including the identification information B and the time information T extracted by the information extraction unit 511 from the communication device 53 to the distribution system 30 (Sa10).

The communication device 33 of the distribution system 30 receives the information request R from the terminal device 50 (Sa11). The information processing unit 312 identifies the related information C corresponding to the time indicated by the time information T in the information request R from the reference table Z of the broadcasting station indicated by the identification information B in the information request R (Sa12). Specifically, the latest related information C registered in the reference table Z is identified. The information processing section 312 transmits the relevant information C identified from the reference table Z from the communication device 33 to the terminal device 50 (Sa13).

The communication device 53 of the terminal device 50 receives the related information C from the distribution system 30 (Sa14). The operation control unit 512 causes the reproduction device 55 to reproduce the related information C received by the communication device 53 (Sa15). That is, following or in parallel with the sound emission (Sa7) of the program sound by the sound emission device 422, the character string representing the content of the program sound is displayed on the reproduction device 55. FIG. As a result of executing the above operation for each unit period Q, the related information C representing the content of each program's audio is displayed on the reproducing device 55 in real time in parallel with the reproducing of the program by the reproducing system 40 .

FIG. 8 is a flow chart illustrating the operation of reproducing a past program using the broadcast data D recorded in the recording device 41. As shown in FIG. The operation procedure (Sa1 to Sa4) for registering the related information C and the time information T in the reference table Z is the same as in FIG.

The processing device 213 of the broadcasting system 20 generates an acoustic signal A2 including the identification information B and the time information T as acoustic components (Sa5), and transmits broadcast data D including the acoustic signal A2 and the video signal V to the transmitting device 22. (Sa6). The recording device 41 of the reproduction system 40 records the broadcast data D (Sb1). By repeating the above operations, the related information C covering the entire program is registered in the reference table Z, and the broadcast data D covering the entire program is recorded in the recording device 41 .

The user U instructs the recording device 41 to reproduce the program recorded in the recording device 41 after the broadcast of the program ends (for example, at a later date). Triggered by the instruction from the user U, the recording device 41 sequentially outputs the broadcast data D of the past programs, thereby causing the reproducing device 42 to reproduce the programs (Sb2). That is, the video of the past program is displayed on the display device 421 and the program sound of the program is emitted from the sound emitting device 422 .

Subsequent operations are the same as in FIG. That is, the extraction of the identification information B and the time information T by the information extraction unit 511 (Sa8, Sa9) and the transmission of the information request R by the communication device 53 (Sa10) are executed every unit period Q. FIG. When the communication device 33 of the distribution system 30 receives the information request R (Sa11), the information processing unit 312 identifies the related information C corresponding to the information request R (Sa12), and transmits the related information C from the communication device 33 to the terminal. It is transmitted to the device 50 (Sa13). Of the plurality of related information C registered in the reference table Z, the related information C corresponding to the past time represented by the time information T is identified. The communication device 53 of the terminal device 50 receives the related information C transmitted from the distribution system 30 (Sa14), and the reproducing device 55 reproduces the related information C (Sa15).

As described above, according to the first embodiment, the related information C corresponding to the program currently being broadcast can be provided to the terminal device 50 in parallel with the reproduction of the program. Related information C corresponding to a past program can be provided to the terminal device 50 in parallel with reproduction of the program. In addition to the time information T, the broadcast station identification information B is included as an acoustic component in the broadcast data D, and the related information C corresponding to the broadcast station represented by the identification information B extracted by the terminal device 50 is transmitted to the terminal device 50. be done. Therefore, it is possible to provide the terminal device 50 with the related information C related to the programs of the broadcasting stations selected around the terminal device 50 among the plurality of broadcasting stations.

<Second embodiment>
A second embodiment of the present invention will be described. It should be noted that, in each of the following illustrations, the reference numerals used in the description of the first embodiment are used for the elements whose functions are the same as those of the first embodiment, and detailed description of each will be omitted as appropriate.

FIG. 9 is an explanatory diagram of information stored in the storage device 32 of the distribution system 30 in the second embodiment. As illustrated in FIG. 9, the reference table Z stored by the storage device 32 of the second embodiment for each broadcasting station is classified into a reference table Z1 and a reference table Z2. The reference table Z1 is a data table in which related information C and time information T corresponding to programs currently being broadcast are registered in chronological order. On the other hand, the reference table Z2 is a data table in which related information C and time information T corresponding to past programs that have been broadcast are registered in chronological order. Related information C and time information T registered in the reference table Z1 for a specific program are transferred to the reference table Z2 when the broadcast of the program ends.

The reference table Z1 is stored in the first storage area 32A of the storage device 32, as illustrated in FIG. The reference table Z2 is stored in the second storage area 32B of the storage device 32. FIG. The first storage area 32A and the second storage area 32B are separate areas. Although FIG. 9 illustrates a case where both the first storage area 32A and the second storage area 32B are set in the single storage device 32, the first storage area 32A and the second storage area 32B are separate storage areas. It may be set in the device 32 .

FIG. 10 is a flowchart illustrating specific procedures of operations of the terminal device 50 and the distribution system 30 in the second embodiment. When the identification information B and the time information T are extracted (Sa9) from the collected sound (Sa8) of the reproduced sound by the reproduction system 40, the operation control unit 512 refers to the time represented by the time information T, and reproduces the reproduction by the reproduction system 40. It is determined whether the current program (hereinafter referred to as "playback program") is currently being broadcast or is a program broadcast in the past (that is, a program recorded in the recording device 41) (Sc1). For example, the operation control unit 512 compares the time indicated by the time information T with the current time measured by a clock circuit (not shown). When the time of the time information T is within a predetermined range including the current time, the operation control unit 512 determines that the reproduced program is a program being broadcast, and the time of the time information T is outside the range. If so, it is determined that the reproduced program is a program that was broadcast in the past.

The operation control unit 512 generates an information request R including determination information F corresponding to the determination result described above in addition to the identification information B and the time information T, and transmits the information request R to the distribution system 30 (Sa10). When determining that the reproduced program is a program being broadcast, the operation control unit 512 generates information (for example, an address) specifying the first storage area 32A of the distribution system 30 as the determination information F. FIG. On the other hand, when determining that the reproduced program is a past program, the operation control section 512 generates information designating the second storage area 32B of the distribution system 30 as determination information F. FIG. As can be understood from the above description, the determination information F can also be rephrased as information representing the acquisition destination of the related information C (either the first storage area 32A or the second storage area 32B). However, the content of the determination information F is not limited to the above examples. For example, a flag may be generated as the determination information F to indicate whether the program to be reproduced is the program being reproduced or the program in the past.

When the communication device 33 receives the information request R transmitted by the terminal device 50 (Sa11), the information processing unit 312 of the distribution system 30 determines that the determination information F included in the information request R is the currently broadcast program and the past program. It is determined which program is represented (Sc2). If the determination information F indicates a program being broadcast (Sc2: YES), the information processing section 312 retrieves related information corresponding to the identification information B and the time information T of the information request R from the first storage area 32A of the storage device 32. Identify C (Sc3). On the other hand, if the determination information F indicates a past program (Sc2: NO), the information processing section 312 retrieves the related information corresponding to the identification information B and the time information T of the information request R from the second storage area 32B of the storage device 32. Information C is specified (Sc4). The above process is the process of specifying the related information C corresponding to the information request R (Sa12). The processing (Sa13) for transmitting the related information C specified by the information processing unit 312 to the terminal device 50 and the processing (Sa14, Sa15) for the terminal device 50 to receive and reproduce the related information C are the same as in the first embodiment. be.

The same effects as in the first embodiment are achieved in the second embodiment. Further, in the second embodiment, the terminal device 50 transmits the judgment information F according to the result of judging whether the reproduced program from which the time information T is extracted is a program being broadcast or a past program. Therefore, whether to search for the related information C from the first storage area 32A storing the related information C related to the program being broadcast or from the second storage area 32B storing the related information C related to the past programs is It can be determined in the distribution system 30 . That is, compared with the configuration in which the information processing unit 312 searches for the related information C from the entire range of the reference table Z, the search range of the related information C by the information processing unit 312 is reduced. Therefore, there is an advantage that the processing load of the information processing unit 312 is reduced.

<Third Embodiment>
In the first embodiment and the second embodiment, the configuration in which the terminal device 50 receives the related information C from the distribution system 30 via the communication network 90 is exemplified. In the third embodiment, the operation control unit 512 identifies the related information C corresponding to the identification information B and the time information T from the reference table Z stored in the storage device 52 of the terminal device 50 . Therefore, the operation of identifying the related information C from the reference table Z by the information processing unit 312 of the distribution system 30 is omitted.

The related information C generated by the speech recognition unit 311 and the time information T representing the time of the utterance are transmitted to the terminal device 50 each time the speech recognition unit 311 executes speech recognition. When the communication device 53 of the terminal device 50 receives the related information C and the time information T, the operation control unit 512 adds the related information C and the time information T to the reference table Z stored in the storage device 52 . That is, the related information C and the time information T are sequentially added to the reference table Z of the storage device 52 for each utterance of the program voice in parallel with the broadcast of the program. The reference table Z is stored in the storage device 52 for each broadcasting station. Note that when a recorded program is broadcast, the reference table Z may be generated and stored in the storage device 52 of the terminal device 50 before the program is broadcast.

FIG. 11 is a flowchart illustrating a specific procedure of operation of the terminal device 50 in the third embodiment. When the identification information B and the time information T are extracted (Sa9) by collecting the reproduced sound (Sa8) by the reproduction system 40, the operation control unit 512 specifies the related information C from the reference table Z stored in the storage device 52. (Sd1). Specifically, the operation control unit 512 identifies the related information C corresponding to the time indicated by the time information T from the reference table Z of the broadcasting station indicated by the identification information B. FIG. The operation control unit 512 causes the reproduction device 55 to reproduce the related information C specified from the reference table Z (Sa15). That is, the transmission of the information request R to the distribution system 30 (Sa10) and the reception of the related information C transmitted by the distribution system 30 (Sa14) are omitted.

The third embodiment also achieves the same effect as the first embodiment. The operation of the first embodiment in which the operation control unit 512 of the terminal device 50 receives the related information C from the distribution system 30 (Sa14) and the operation of the third embodiment in which the terminal device 50 identifies the related information C from the storage device 52 (Sd1) is comprehensively expressed as the operation of acquiring the related information C corresponding to the time represented by the time information T. That is, the relevant information C may be obtained from either the storage device 32 of the distribution system 30 or the storage device 52 of the terminal device 50 .

According to the configuration of the first embodiment for receiving the related information C from the distribution system 30, the terminal device 50 does not need to hold the reference table Z. FIG. Therefore, compared with the third embodiment in which the reference table Z is stored in the storage device 52, there is an advantage that the required storage capacity of the storage device 52 of the terminal device 50 is reduced.

<Modification>
Specific modified aspects added to the above-exemplified aspects will be exemplified below. A plurality of aspects arbitrarily selected from the following examples may be combined as appropriate within a mutually consistent range.

(1) In the above embodiments, the time information T is added to the acoustic signal A1 transmitted from the broadcasting system 20 to the distribution system 30. Sending is not required. For example, time information T representing the time at which the communication device 33 of the distribution system 30 received the acoustic signal A1 from the broadcasting system 20 may be registered in the reference table Z in association with the related information C. FIG. Also, time information T representing the time at which the related information C is stored in the storage device 32 may be registered in the reference table Z in association with the related information C. FIG. As can be understood from the above description, the time represented by the time information T transmitted to the terminal device 50 by acoustic communication and the time represented by the time information T registered in the reference table Z do not need to be exactly the same. Of the plurality of related information C registered in the reference table Z, the related information C corresponding to the time represented by the time information T received from the terminal device 50 is transmitted from the distribution system 30 to the terminal device 50 .

(2) When an emergency such as a disaster (natural disaster, man-made disaster) or crime occurs, emergency information representing the occurrence of the emergency may be transmitted from the distribution system 30 to the terminal device 50 . For example, when the information processing unit 312 of the distribution system 30 recognizes the occurrence of an emergency from a notification from an external device or an administrator, it transmits the emergency information together with the related information C to the terminal device 50 . Emergency information is information received from an external device or information entered by an administrator. The operation control unit 512 of the terminal device 50 causes the reproducing device 55 to reproduce the emergency information. For example, the operation control unit 512 switches the display of the received related information C to the display of the emergency information. The playback device 55 may display the related information C and the emergency information in parallel. Note that the time information T may be omitted in a configuration for reproducing emergency information.

If an emergency occurs while the playback device 42 is playing back a past program recorded in the recording device 41, the playback device 42 will replace the past program with the current broadcast program (e.g., an emergency event). A configuration for switching to the playback of an emergency program that reports the For example, upon receiving emergency information from the distribution system 30, the terminal device 50 instructs the reproduction device 42 to reproduce the current program being broadcast by a specific broadcasting station. In response to an instruction from the terminal device 50, the playback device 42 switches playback of the recorded past program to playback of the current broadcast program. Note that the program switching instruction may be directly transmitted from the distribution system 30 to the playback device 42 . Also, in the above description, the configuration for switching the reproduction of the past program to the reproduction of the program being broadcast was exemplified, but the past program and the program being broadcast may be reproduced in parallel.

(3) In each of the above embodiments, programs provided by television broadcasting such as terrestrial broadcasting or satellite broadcasting were exemplified, but the programs are not limited to the above examples. For example, a configuration similar to that of each of the above embodiments is applied to provision of a program by IP (Internet Protocol) broadcasting in which broadcast data D is transmitted via a communication network such as the Internet. Further, in each of the above-described forms, a program composed of video and sound was exemplified, but a configuration similar to that of each of the above-described forms is also applied to the provision of a radio broadcast program composed of only sound. That is, the video signal V is not essential for the broadcast data D. Therefore, display device 421 may be omitted from playback system 40 .

(4) In each of the above-described embodiments, the configuration in which the speech recognition unit 311 of the distribution system 30 performs speech recognition was exemplified. It may be installed separately from system 30 . The communication device 211 of the broadcasting device transmits the acoustic signal A1 to the speech recognition device, and the control device 31 of the distribution system 30 stores the related information C generated by the speech recognition device together with the time information T in the storage device 32 . Further, it is not essential to execute speech recognition (that is, generation of related information C) for the acoustic signal A1 in parallel with the broadcast of the program. For example, for a recorded program, related information C representing program audio may be generated and registered in the reference table Z before broadcasting.

(5) In each of the above embodiments, the result of speech recognition for the acoustic signal A1 is used as the related information C. That is, the language of the program audio and the language of the related information C are common. The character string in the first language (that is, the language of the program audio) specified by speech recognition for the acoustic signal A1 is translated into the second language by, for example, machine translation, and is registered in the reference table Z as the related information C. good too. That is, the language of the program audio and the language of the related information C may be different. According to the above configuration, foreigners who have difficulty understanding the language of the program audio can understand the content of the program audio from the related information C expressed in their native language, for example. Alternatively, a plurality of related information C expressed in different languages may be registered in the reference table Z, and the related information C in the language specified by the terminal device 50 may be provided to the terminal device 50 .

If the content of pronunciation of the program voice can be predicted in advance, a character string similar to the utterance content of the program voice among a plurality of character strings prepared in advance may be registered in the reference table Z as the related information C. That is, the terminal device 50 is provided with a character string as the related information C that is similar in content but does not strictly match the content of the program audio.

Also, the related information C is not limited to a character string representing the content of the program audio. For example, any information about the program, such as information about the performers of the program, information about the places or products introduced in the program, information about other programs related to the program, or future notices about the program, can be used as related information C. provided to person U. In addition, the form of the related information C is arbitrary, and is not limited to a character string. good too. Information (for example, URL) indicating the location of information about a program may be used as related information C. FIG.

(6) In each of the above embodiments, the identification information B representing the broadcasting station was exemplified, but a code for identifying the program may be used as the identification information B. A plurality of pieces of related information C are registered in the reference table Z for each program. The information processing section 312 of the distribution system 30 transmits the related information C corresponding to the time information T among the plurality of related information C corresponding to the program indicated by the identification information B in the information request R to the terminal device 50 . As can be understood from the above description, the acoustic signal A2 of the broadcast data D contains the identification information B of the broadcasting station or program as an acoustic component.

(7) In the above embodiments, the distribution system 30 transmits the related information C to the terminal device 50 each time the information request R is received from the terminal device 50. As a trigger, the related information C may be transmitted to the terminal device 50 each time the related information C is registered.

(8) The broadcasting system 20 in each of the above-described forms is realized by a plurality of devices configured separately from each other. The distribution system 30 is also realized by a plurality of devices configured separately from each other. A part or all of the functions of distribution system 30 may be installed in broadcasting system 20 . Some or all of the functions of broadcasting system 20 may be installed in distribution system 30 .

(9) The functions of the information providing system 10 (especially the distribution system 30) and the terminal device 50 according to each of the above-described embodiments are, as illustrated in each embodiment, the control device 51 or the processing circuit such as the control device 31 and the program. Realized through collaboration. The program according to each of the forms described above can be provided in a form stored in a computer-readable recording medium and installed in a computer. The recording medium is, for example, a non-transitory recording medium, and an optical recording medium (optical disc) such as a CD-ROM is a good example. Also included are recording media in the form of It should be noted that the non-transitory recording medium includes any recording medium other than transitory, propagating signals, and does not exclude volatile recording media. Alternatively, the program may be provided to the computer in the form of distribution via a communication network.

<Appendix>
For example, the following configuration can be grasped from the form illustrated above.

An information providing method according to a preferred aspect (first aspect) of the present invention transmits broadcast data including, as an audio component, time information representing time relating to a program broadcast by a broadcasting station, and reproduces sound corresponding to the broadcast data. The terminal device that has picked up the sound receives the time information extracted from the reproduced sound from the terminal device, and corresponds to the time represented by the time information received from the terminal device among a plurality of pieces of related information related to the program of the broadcasting station. Related information is transmitted to the terminal device. In the above aspect, among a plurality of pieces of related information relating to a program of a broadcasting station, related information corresponding to the time indicated by the time information extracted from the reproduced sound by the terminal device is transmitted to the terminal device. Therefore, in addition to being able to provide related information corresponding to a program currently being broadcast to a terminal device in parallel with the reproduction of the program, related information corresponding to a past program recorded in a recording device, for example, can be provided during the reproduction of the program. It can be provided to the terminal device in parallel.

In a preferred example of the first aspect (second aspect), the broadcast data includes identification information of the broadcasting station or the program as an acoustic component, and includes the identification information and the time information extracted from the reproduced sound by the terminal device. Related information corresponding to the time represented by the time information received from the terminal device, among a plurality of related information received from the terminal device and corresponding to the broadcasting station or the program indicated by the identification information received from the terminal device , to the terminal device. In the above aspect, the identification information of the broadcasting station or the program is included in the broadcast data together with the time information as the acoustic component, and the relevant information corresponding to the broadcasting station or the program indicated by the identification information received from the terminal device is transmitted to the terminal device. be. Therefore, it is possible to provide the terminal device with related information about the programs of the broadcasting stations selected around the terminal device among the plurality of broadcasting stations.

A method of operating a terminal device according to a preferred aspect (third aspect) of the present invention extracts the time information from a reproduced sound containing, as an acoustic component, time information representing time relating to audio of a program broadcast by a broadcasting station, Related information corresponding to the time represented by the time information is acquired from among a plurality of related information related to the program of the broadcasting station. In the above aspect, the terminal device acquires the related information corresponding to the time represented by the time information extracted from the reproduced sound, among the plurality of related information related to the program of the broadcasting station. Therefore, in addition to being able to reproduce the related information corresponding to the program currently being broadcast on the terminal device in parallel with the reproduction of the program, for example, the related information corresponding to the past program recorded in the recording equipment can be reproduced during the reproduction of the program. Terminal devices can be played in parallel.

In a preferred example of the third mode (fourth mode), in acquiring the related information, an information request including the time information is transmitted to an information providing device, and the information is transmitted from the information providing device in response to the information request. Receive relevant information. In the above aspect, since the related information is transmitted from the information providing device to the terminal device in response to a request from the terminal device, a plurality of related information serving as search candidates according to the time information are stored in the terminal device. No need to leave. Therefore, the required storage capacity of the storage device is reduced.

In a preferred example of the fourth aspect (fifth aspect), determining whether the program is a program being broadcast or a program broadcast in the past according to the time information extracted from the reproduced sound, transmitting the information request including determination information according to the result of the determination; In the above aspect, the judgment information is transmitted according to the result of judging whether the program from which the time information is extracted is a program being broadcast or a past program. Therefore, which of the first storage area storing related information about the program being broadcast and the second storage area storing related information about past programs should be searched for the related information to be transmitted to the terminal device? can be determined at the receiver of the information request. The determination information is, for example, information (information acquisition destination address information) specifying either the first storage area or the second storage area.

As an information providing system that executes the information processing method according to the first aspect or the second aspect, or as a terminal device that executes the operation method according to any one of the third to fifth aspects, a preferred aspect of the present invention is Realized.

DESCRIPTION OF SYMBOLS 10... Information provision system 20... Broadcasting system 21... Management apparatus 211... Communication apparatus 212... Time-measurement apparatus 213... Processing apparatus 22... Transmission apparatus 25... Signal generation part 26... Signal synthesis part 30... Distribution system 31 Control device 311 Speech recognition unit 312 Information processing unit 32 Storage device 33 Communication device 40 Reproduction system 41 Recording device 42 Reproduction device 421 Display device 422... Sound emission device 50... Terminal device 51... Control device 511... Information extraction unit 512... Operation control unit 52... Storage device 53... Communication device 54... Sound collection device 55... Playback device 90 …Communication network.

Claims (8)

The time information extracted by the terminal device from the reproduced sound of the program containing as an acoustic component time information representing the time of the program broadcast by the broadcasting station , and whether the program is being broadcast or has been broadcast in the past receive determination information according to the result of determining whether there is according to the time information from the terminal device;
A method for transmitting, to the terminal device, related information corresponding to the time represented by the time information received from the terminal device, among a plurality of pieces of related information related to the program of the broadcasting station, comprising:
In transmitting the related information, if the determination information represents a program being broadcast, the related information is acquired from a first storage area, and if the determination information represents a past program, the first storage area is Acquiring the related information from a different second storage area
An information providing method implemented by a computer system. the reproduced sound of the program includes identification information of the broadcasting station or the program as an acoustic component;
In the reception, the terminal device receives the identification information extracted from the reproduced sound , the time information, and the determination information from the terminal device,
In the transmission of the related information, related information corresponding to the time represented by the time information received from the terminal device, among a plurality of related information corresponding to the broadcasting station or program represented by the identification information received from the terminal device. to the terminal device. transmitting broadcast data including a second acoustic signal representing audio of the program;
The second acoustic signal includes the identification information and time information corresponding to each unit period in each of a plurality of unit periods obtained by dividing the voice of the program for each utterance.
3. The information providing method according to claim 2. Extracting the time information from the reproduced sound of the program containing time information representing the time related to the program broadcast by the broadcasting station as an acoustic component,
determining whether the program is currently being broadcast or has been broadcast in the past according to the time information extracted from the reproduced sound;
transmitting an information request including the time information and determination information according to the result of the determination;
A method of operating a terminal device, acquiring related information corresponding to a time represented by the time information, from among a plurality of related information related to the program of the broadcasting station. When determining that the program is a program that is being broadcast, generating the determination information that specifies a first storage area as an acquisition destination of the related information, and determining that the program is a program that was broadcast in the past and generating the determination information designating a second storage area different from the first storage area as an acquisition destination of the related information.
5. A method of operating the terminal device according to claim 4. The time information represents the time regarding the audio of the program.
6. The operating method of the terminal device according to claim 4 or 5. The time information extracted by the terminal device from the reproduced sound of the program containing as an acoustic component time information representing the time of the program broadcast by the broadcasting station , and whether the program is being broadcast or has been broadcast in the past and determination information corresponding to the result of determining whether there is a program according to the time information from the terminal device, and the time represented by the time information received from the terminal device among the plurality of related information related to the program of the broadcasting station An information processing unit that transmits relevant information corresponding to the terminal device to the terminal device,
The information processing unit acquires the related information from a first storage area when the determination information represents a program being broadcast, and obtains the related information from a second storage area different from the first storage area when the determination information represents a past program. 2 Acquire the related information from the storage area
delivery system . terminal computer,
an information extraction unit that extracts the time information from the reproduced sound of the program that includes time information representing the time of the program broadcast by the broadcasting station as an acoustic component ;
Determining whether the program is currently being broadcast or has been broadcast in the past according to the time information extracted from the reproduced sound, and determining information according to the time information and the result of the determination. and acquires related information corresponding to the time represented by the time information from among a plurality of related information related to the program of the broadcasting station ,
A program that acts as a

Images