JPH10149193A - Device and method for processing information - Google Patents

Abstract

PROBLEM TO BE SOLVED: To exactly generate text data corresponding to words from audio signals. SOLUTION: A voice recognizing part 120 recognizes the voices of audio signals of a center channel containing word signals in audio data for 5.1 channels reproduced from a DVD 111 and generates the text data. At an address generating part 121, an address corresponding to these text data is generated and bit map data corresponding to a text are outputted from a bit map data ROM 122 and stored as superimposed characters in a frame memory 114 while being superimposed on the image of video data outputted from an MPEG 2 decoder 113. The video data written in the frame memory 114 are encoded by a video encoder 115, converted to NTSC system video signals, outputted to a display 133 to display.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

The present invention relates to an information processing apparatus and method, and more particularly to an information processing apparatus and method capable of generating text data from an audio signal corresponding to a video signal.

[0002]

2. Description of the Related Art Recently, a DVD (Digital Versatile Disc) has been developed and is becoming popular. In this DVD, besides the video data and the corresponding audio data,
Sub picture data can be recorded.
Then, by recording subtitle data as the sub-picture data, the subtitle can be superimposed on the original image and displayed as necessary.

[0003]

However, in many cases, for example, when a spoken English language is spoken,
The language of the subtitles is generally a language other than English, for example, a language for viewers who cannot understand English, such as Japanese and French. As a result, for example, there is a problem that a hearing-impaired person or the like who cannot understand voices can understand English by using letters but cannot enjoy the program.

[0004] In the United States, it is mandatory to display English subtitles for the hearing impaired. However, if English is added to subpicture data, other languages can be used as subpictures accordingly. I can't afford to add
It will be difficult to distribute the same DVD to many countries.

Therefore, for example, it is conceivable to generate text data by voice-recognizing an audio signal accompanying a video signal and display it as subtitles. However, in general, not only speech but also music and sound effects are synthesized in the audio signal, and there is a problem that it is difficult to correctly recognize only speech from this audio signal.

The present invention has been made in view of such a situation, and it is an object of the present invention to easily and surely generate text data corresponding to dialogue.

[0007]

An information processing apparatus according to claim 1, comprising: a separating unit for separating a speech signal from a multiplexed signal in which a video signal and a speech signal corresponding to the video signal are multiplexed; It is characterized by comprising speech recognition means for speech recognition of the separated speech signal, and generation means for generating text data in accordance with the speech recognition result.

According to a sixth aspect of the present invention, there is provided an information processing method, comprising: a separating step of separating a speech signal from a multiplexed signal in which a video signal and a speech signal corresponding to the video signal are multiplexed; The method includes a voice recognition step for voice recognition and a generation step for generating text data in accordance with the voice recognition result.

In the information processing apparatus according to the first aspect and the information processing method according to the sixth aspect, a speech signal is separated from a multiplexed signal in which a video signal and a speech signal corresponding to the video signal are multiplexed. Then, the separated speech signal is subjected to speech recognition. Then, text data is generated corresponding to the speech recognition result. As a result, accurate speech recognition becomes possible, and accurate text data can be obtained.

[0010]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiments of the present invention will be described below. In order to clarify the correspondence between each means of the invention described in the claims and the following embodiments, each means is described. When the features of the present invention are described by adding the corresponding embodiment (however, an example) in parentheses after the parentheses, the result is as follows. However, of course, this description does not mean that each means is limited to those described.

According to a first aspect of the present invention, there is provided an information processing apparatus for separating a speech signal from a multiplexed signal in which a video signal and a speech signal corresponding to the video signal are multiplexed (for example, the demultiplexer shown in FIG. 1). 112), voice recognition means for voice recognition of the separated speech signal (for example, voice recognition unit 120 in FIG. 1), and generation means for generating text data corresponding to the voice recognition result (for example, voice in FIG. 1) And a recognition unit 120).

According to another aspect of the present invention, there is provided an information processing apparatus comprising: decoding means for decoding a video signal;
2 decoder 113) and synthesizing means (for example, the frame memory 114 in FIG. 1) for synthesizing text corresponding to text data with the decoded image.

According to a third aspect of the present invention, there is provided an information processing apparatus which stores text data (for example, the HDD
0) is further provided.

FIG. 1 shows an information processing apparatus according to the present invention.
3 illustrates a configuration example of a DVD player. This DVD player 1
In 00, the signal reproduced from the DVD 111 is supplied to the demultiplexer 112, and is separated into video data, audio data, and sub-picture data. MPEG (Moving Pictures Experts Gro
up) The 2 decoder 113 decodes the input video data and outputs it to the frame memory 114.

The sub-picture decoder 118 decodes the sub-picture data supplied from the demultiplexer 112 and supplies the decoded data to the frame memory 114 via the contact A of the switch 119. The data read from the frame memory 114 is input to the video encoder 115, converted into a video signal of the NTSC system or the PAL system, etc.
To be supplied.

The audio data output from the demultiplexer 112 is encoded according to the Dolby AC-3 (trademark) system, and the AC-3 decoder 116 decodes the encoded data to produce 5.1-channel multi-track audio data. The output has been made. D / A
The converter 117 D / A-converts the 5.1-channel audio data output from the AC-3 decoder 116 and outputs the data to the multi-channel amplifier 131.
The multi-channel amplifier 131 amplifies the input 5.1-channel audio signal and outputs the amplified signal to the speaker system 132. Speaker system 132
Is a speaker for five channels, a front left speaker 141, a front right speaker 142, a rear left speaker 143, a rear right speaker 144, and a front center speaker 145, and a 0.1 channel speaker. Of the speaker (woofer) 146 for heavy bass.

The speech recognition unit 120 is provided with the AC-3 decoder 1
16 out of the 5.1 channel data output
The audio data of the center channel (audio data supplied to the speaker 145 of the speaker system 132) is subjected to voice recognition, text data is generated in accordance with the voice recognition result, and the text data is output to the address generation unit 121. The address generation unit 121 generates an address corresponding to the text data, and generates an address corresponding to the text data.
Output to Bitmap data corresponding to text data is stored in the bitmap data ROM 122, and the bitmap data is supplied to the frame memory 114 via the contact B of the switch 119.

For example, a control circuit 124 constituted by a microcomputer or the like switches a switch 119 in response to an input from the operation unit 123 and controls each unit.

Next, the operation will be described. When the user operates the operation unit 123 to instruct reproduction of the DVD 111, the control circuit 124 controls each unit to start reproduction. The signal reproduced from the DVD 111 is input to the demultiplexer 112,
From this playback signal, video signal, audio signal,
And the sub-picture signal are separated and output to the MPEG2 decoder 113, the AC-3 decoder 116, or the sub-picture decoder 118, respectively.

The MPEG2 decoder 113 decodes the input video data according to the MPEG2 system, decodes it as 4: 2: 2 digital video data, and
4 for storage.

On the other hand, the sub-picture decoder 118 decodes the input sub-picture data and
The data is output to the frame memory 114 via the 19 contact points A and stored. As a result, an image in which subtitles are superimposed on the original image is obtained in the frame memory 114, and this image is input to the video encoder 115 and converted into, for example, an NTSC video signal.
33 and is displayed.

The AC-3 decoder 116 decodes the input 5.1 channel audio data,
Output to the D / A converter 117. The D / A converter 117 performs D / A conversion on the input audio data and outputs the audio data to the multi-channel amplifier 131. The multi-channel amplifier 131 amplifies the input 5.1-channel audio signals and outputs the amplified signals to the corresponding speakers 141 to 146, respectively.

Normal reproduction is performed as described above. When the user does not need the subtitle by the sub-picture, the user operates the operation unit 123 to switch the switch 119 to the contact C side via the control circuit 124 so that the subtitle is not displayed. Can be. Alternatively, if subtitles in a plurality of independent languages are prepared as sub-picture data, the operation unit 123
By operating the subtitle, one desired subtitle can be selected from the subtitles, decoded by the sub-picture decoder 118, and displayed on the display 133.

Further, the user can use the speaker system 13
If the user wants to display the text corresponding to the language of the audio output from 2 as subtitles, the operation unit 123 is controlled,
The control circuit 124 switches the switch 119 to the contact B side. At this time, the voice recognition unit 120 performs voice recognition processing of voice data corresponding to the center channel among the 5.1-channel audio data output from the AC-3 decoder 116. The center channel usually includes only speech among audio components corresponding to the image displayed on the display 133, and music and sound effects are transmitted to the other speakers 141 to 144 and the speaker 146. Only included in the corresponding channel, not in the center channel. Therefore, the voice recognition unit 120 can accurately recognize only the speech without being affected by music or sound effects.

The voice recognition unit 120 generates text data corresponding to the result of the voice recognition, and supplies the text data to the address generation unit 121. The address generation unit 121 generates an address corresponding to the input text data, and outputs the generated address to the bitmap data ROM 122. The bitmap data ROM 122 reads out bitmap data corresponding to the address input from the address generation unit 121,
Output. For example, when the address generation unit 121 outputs an address corresponding to the alphabetical character A,
The bitmap data corresponding to the alphabetical character A is read and output.

The bitmap data output from the bitmap data ROM 122 is supplied to the frame memory 114 via the contact B of the switch 119, and is superimposed as subtitles on the image data supplied from the MPEG2 decoder 113. . Then, the data on the frame memory 114 is converted into video data of the NTSC system or the PAL system by the video encoder 115 and output to the display 133 for display.

As described above, even if the person is deaf, the user can enjoy the program by operating the operation unit 123 while watching the subtitles in the same language as the language of the sound output from the speaker system 132. be able to.

FIG. 2 shows an example of the configuration of an AV system to which the information processing apparatus of the present invention is applied. In this embodiment, a personal computer 1 is an AV device 2 such as a tuner, an amplifier, and a video disc player.
In addition, it is connected to the television receiver 3. The television receiver 3 has a CRT 4 for displaying an image and a speaker 5 for outputting an audio signal.

The keyboard 11 includes a plurality of keys 12.
And a touch pad 13, and an infrared signal corresponding to the operation is emitted from the infrared transmitting unit 14 to the personal computer 1.

FIG. 3 shows the external shape of the personal computer 1. The personal computer 1 has a width of 225 mm, a height of 94 mm, and a depth of 350 mm. Also, on the front of the personal computer 1,
An openable and closable door 21 is provided.
A surface 22 is provided. In the figure, a power switch 23 that is operated to turn on or off the power and an infrared receiving unit 24 that receives infrared light emitted from the infrared transmitting unit 14 of the keyboard 11 are formed on the left surface 22.

Also, on the upper surface of the personal computer 1, when peripheral devices to be connected to the personal computer 1 are mounted, the peripheral devices are arranged so that the legs of the peripheral devices are stably arranged on the upper surface. A recess 25 is formed at a position corresponding to the leg of the device.

FIG. 4 shows a state where the door 21 of the personal computer 1 is opened. As shown in the figure,
When the door 21 is opened, a DVD (Digital Versatile Disc)
The drive 33 is exposed. A USB terminal 31 as a serial interface and an IEEE (Institute of Electric
l and Electronics Engineers) 1394 standard 1394
A terminal 32 is provided.

FIG. 5 shows a state in which the door 41 on the back of the personal computer 1 is opened. As shown in the figure, when the door 41 is opened, the PC card insertion slot 42 is exposed. Also, the PC card insertion slot 42
A USB terminal 43 and a 1394 terminal 44, a printer terminal 45 for connecting a printer, and a VGA terminal 46 for outputting computer graphics data are provided below the USB terminal 43.

FIG. 6 shows an example of the internal configuration of the personal computer 1. CPU (Central Processing Unit)
For example, Pentium (trademark) of Intel Corporation is used for 71. This CPU operates with an internal clock having a frequency of 166 MHz or an external clock having a frequency of 66 MHz. R
The AM 72 is a 16 MB main memory, and stores data, programs, and the like processed by the CPU 71 as appropriate. The ROM 73 stores programs necessary for the CPU 71 to execute various processes. EEPROM (Electri
cally Erasable Programmable Read Only Memory) 74
Stores data that needs to be stored even after the power of the personal computer 1 is turned off.

The graphics processing unit 75 performs a moving image process (color space conversion for converting a YUV signal, which is a display format of moving image data, to an RGB signal, which is a graphics signal data format).
In addition to performing scaling (enlargement or reduction) processing for displaying with a desired screen size, etc., three-dimensional graphics processing (for example, rasterization processing for projecting a three-dimensional object on a secondary screen screen, object Glow shading processing to make the surface look smooth, alpha blending processing to represent translucent objects, etc.), and further, to write the processing result to the display memory 76 and output it to the synthesizing circuit 85. Do. The graphics processing unit 75 also generates bitmap data corresponding to the text data.

[0036] The MPEG2 video decoder 77 decodes video data reproduced from the DVD by the DVD drive 33 and outputs the decoded data to the synthesizing circuit 85. The sub-picture decoder 88 decodes the sub-picture data reproduced from the DVD and outputs it to the synthesizing circuit 85. The digital sound processing unit 81 has an ADPCM (Adaptive Difference Pulse Code M
odulation) sound source expansion, MPEG audio data expansion,
FM (Frequency) for generating sound effects such as reverberation and surround
ency Modulation) sound configuration (ie, the process of generating an audio signal by synthesizing multiple sine waves of different frequencies and amplitudes) or MIDI (Musical Instr.
ument Digital Interface) Wavetable synthesis processing,
An AC-3 decoding process is performed. The MIDI wave table synthesizing process is a process in which a built-in synthesizer reproduces MIDI data using a wave table that stores digital data serving as phonemes of musical instrument sounds. Each processed audio signal is mixed by a built-in audio mixer, converted into an analog audio signal, and converted into a speaker 5 of the television receiver 3.
Is output to The speech recognition circuit 87 performs a speech recognition process.

The Intercast (trademark) board 78 is a board that receives an intercast broadcast via an antenna 91 and performs processing for demodulation. In the intercast, the World Wide Web (W
HTML (Hyper Text Markup Langua)
ge) Insert data and send. The data received is
It is stored on a hard disk driven by a hard disk drive (HDD) 80. Hard disk drive 80
By traversing through the HTML data, the user can have a pseudo-interactive environment.

For example, in the case of a sports program, a score, a still image of a definitive scene, a video clip, and the like are transmitted by the intercast in accordance with the contents of the program.
These still images and video clips are linked to related information. For example, the link destination is accessed via an analog telephone line so that the related information can be obtained. This intercast was developed by Intel.

DSVD (Digital Simultaneous Voice & Dat)
a) The modem 79 time-division multiplexes voice and data using the DSVD method developed by Intel Corporation, outputs the time-division multiplexed signal to the telephone line via the modular jack 92, and outputs the DSVD signal input via the telephone line. Then, a process of demodulating and separating the audio signal and the data is performed. In this scheme,
The digitally-compressed audio signal and the normal audio signal are converted into V.V. It is multiplexed using a header of 42 protocol. If no audio signal is present, the maximum data rate is 28.8 kbit / sec, and if there is an audio signal, 19.2 kbit / sec. Further, the transmission speed of the audio signal is 9.6 kbit / sec. As the compression / expansion method of the audio signal, Rockwell's DigiTalk (trademark), DSP Group's TrueSpeech (trademark), or the like is used.

The keyboard controller 84 receives a signal from the infrared receiving section 24 and outputs a signal corresponding to the received signal to the CPU 71.

The synthesizing circuit 85 includes the graphics processing unit 7
5, the output of the MPEG2 video decoder 77 and the output of the sub-picture decoder 88 are synthesized as necessary.
An output is provided to the NTSC encoder 86. NT
The SC encoder 86 converts the video data input from the synthesizing circuit 85 into an NTSC analog video signal,
The data is output to the television receiver 3.

Although only one bus is shown for convenience, a local bus connecting the CPU 71 and the RAM 72 and an ISA connected to the keyboard controller 84 are actually used.
(Industry Standard Architecture) bus, and other PCI (Periph
eral Component Interconnect) bus. The ISA bus is an 8-bit or 16-bit bus, and the PCI bus is a 32-bit or 64-bit bus. The PCI bus operates at a speed between 25 MHz and 66 MHz and achieves a maximum of 528 KB / sec throughput. This speed is more than 42 times faster than the ISA bus.

The expansion slot 82 is an expansion slot for a PCI bus, and the expansion slot 83 is an expansion slot for an ISA bus. In this expansion slot,
A desired function can be realized by connecting a peripheral circuit of a predetermined function (for example, a SCSI board).

Note that dedicated bus bridge circuits (not shown) are provided between the local bus and the PCI bus, and between the PCI bus and the ISA bus.

FIG. 7 shows an example of the internal configuration of the keyboard 11. The detection circuit 141 detects a key operated from the keys 12. Further, the detection circuit 141 detects the coordinate data (X, Y) of the operated point P on the touch pad 13. Then, the detection circuit 14
1 outputs the detection result to the transmission module 142. The transmission module 142 converts the signal input from the detection circuit 141 into a transmission signal, outputs the signal to the infrared transmitting unit 14,
Transmit as an infrared signal.

The battery 143 supplies a predetermined power to the power supply circuit 144. The power supply circuit 144 includes a detection circuit 141
And the necessary power to the transmission module 142. The power switch 145 is operated when the keyboard 11 is used or stopped.

Next, the operation of the personal computer 1 when a command to reproduce a DVD is issued will be described. When one of the keys 12 of the keyboard 11 for inputting a DVD playback command is operated, the operation is performed by the detection circuit 1.
It is detected at 41. The transmission module 142 outputs a signal corresponding to the detected key to the infrared transmitting unit 14 and outputs the signal as an infrared signal.

This infrared signal is received by the infrared receiving section 24 of the personal computer 1. When receiving the detection signal from the infrared receiver 24, the keyboard controller 84 outputs the detection result to the CPU 71 via the bus. When receiving the instruction to reproduce the DVD in this way, the CPU 71 controls the DVD driver 33 to reproduce the DVD mounted thereon.

The video data of the reproduction signal reproduced from the DVD by the DVD driver 33 is the MPEG2 video decoder 7.
7 and decoded. Also, the sub-picture data reproduced from the DVD is
And decoded. The synthesizing circuit 85 is an MPEG2
The video data output from the video decoder 77 and the subtitle data output from the sub-picture decoder 88 are combined, and the combined data is output to the NTSC encoder 86. The NTSC encoder 86 converts the input data into NTSC video data,
Output to T4 and display.

The audio data reproduced from the DVD is input to the digital sound processing section 81,
It is decoded by three methods. Then, after the decoded audio signal is converted into an analog signal, it is output to the speaker 5 of the television receiver 3 and emitted.

However, in this case, the digital sound processing section 81 outputs only the two-channel audio signals of the front left and front right to the speaker 5.

As described above, it is possible to enjoy the image and the sound reproduced from the DVD.
You can also see the subtitles recorded in.

On the other hand, when the user operates the keyboard 11 to instruct the display of subtitles corresponding to the dialogue of the center channel, the CPU 71 sets the digital sound processing unit 8
1 of the 5.1 channel audio data input thereto, and the center channel audio data is supplied to the speech recognition circuit 87. The speech recognition circuit 87 performs speech recognition on the input speech signal of the center channel, generates text data, and outputs the text data to the graphics processing unit 75. Graphics processing unit 75
Generates bitmap data corresponding to the input text data, outputs the bitmap data to the synthesizing circuit 85, and superimposes it as subtitles on the image output from the MPEG2 video decoder 77.
As a result, an image in which the dialog of the center channel is superimposed as subtitles is displayed on the CRT 4 of the television receiver 3.

Further, when the keyboard 11 is operated to instruct recording of the text data obtained as a result of the speech recognition, the CPU 71 supplies the text data output from the speech recognition circuit 87 to the HDD 80 and records it on the hard disk. .

In this way, the CPU 71 translates the text data recorded on the hard disk according to the translation application software, outputs the text data to a printer (not shown), prints the text data, and outputs the text data via the DSVD modem 79.
Use it by transmitting it over a network.

[0056]

As described above, according to the information processing apparatus according to the first aspect and the information processing method according to the sixth aspect, the speech signal separated from the multiplexed signal is subjected to speech recognition, and the speech recognition result is output. Since the corresponding text data is generated, the dialogue can be accurately converted to text data without being affected by music or sound effects. Therefore, it can be displayed as subtitles, translated into other languages,
It can be reused.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration example of a DVD player to which an information processing device according to the present invention is applied.

FIG. 2 is a perspective view showing a configuration example of an AV system to which the information processing apparatus of the present invention is applied.

FIG. 3 is a perspective view showing a configuration of an external appearance of the personal computer of FIG. 2 as viewed from the front.

FIG. 4 is a perspective view showing a state in which a door of the personal computer of FIG. 3 is opened.

5 is a perspective view showing a state where a door on a rear surface of the personal computer in FIG. 2 is opened.

FIG. 6 is a block diagram showing an example of the internal configuration of the personal computer shown in FIG. 2;

FIG. 7 is a block diagram showing an example of the internal configuration of the keyboard shown in FIG. 2;

[Explanation of symbols]

1 personal computer, 3 television receiver, 4 CRT, 5 speakers, 11 keyboard,
12 keys, 13 touch pad, 14 infrared transmitter, 24 infrared receiver, 33 DVD drive, 75 graphics processor, 77 MPEG2 video decoder, 85 synthesis circuit, 86 NTSC encoder, 111 DVD, 112 demultiplexer,
113 MPEG2 decoder, 114 frame memory,
115 video encoder, 116 AC-3 decoder, 118 sub-picture decoder, 120 voice recognition unit, 122 bitmap data ROM

Claims (6)

[Claims] 1. A separating means for separating a speech signal from a multiplexed signal in which a video signal and a speech signal corresponding to the video signal are multiplexed, and a speech recognition means for recognizing the separated speech signal as speech. An information processing apparatus comprising: a generation unit configured to generate text data in response to a speech recognition result. 2. The apparatus according to claim 1, further comprising: decoding means for decoding the video signal; and synthesizing means for synthesizing a text corresponding to the text data with an image corresponding to the decoded video signal. 2. The information processing device according to 1. 3. The information processing apparatus according to claim 1, further comprising a storage unit that stores the text data. 4. The information processing apparatus according to claim 1, wherein the multiplex signal is a reproduction signal from a DVD. 5. The information processing apparatus according to claim 1, wherein the speech signal is a signal of a center channel among audio signals encoded by Dolby AC-3. 6. A separating step of separating the speech signal from a multiplexed signal in which a video signal and a speech signal corresponding to the video signal are multiplexed, and a voice recognition step of recognizing the separated speech signal as speech. And a generating step of generating text data in response to a speech recognition result.