JP2012074961A - Video shooting device and method of adding captions to audio - Google Patents

Abstract

PROBLEM TO BE SOLVED: To add captions to audio in a user-desired language.SOLUTION: A video shooting device according to the present invention comprises: a camera module configured to shoot a video of a subject; a microphone configured to collect audio in the neighborhood of the subject while the camera module shoots a video; an audio-to-text conversion module configured to convert audio data of audio collected by the microphone into text data; a text translation module configured to translate text data included in the text data converted by the audio-to-text conversion module and written in a language other than a predetermined language into text data in the predetermined language; and a video text superimposition module configured to superimpose the text data in the predetermined language converted by the audio-to-text conversion module and the text data in the predetermined language translated by the text translation module on video data of the video shot by the camera module as captions.

Description

  The present invention relates to a moving image photographing apparatus and an audio subtitle conversion method.

  On TV and the like, comments of performers are often displayed as subtitles.

  However, in order for a general user to add subtitles to a video shot by a video shooting device equipped with a video shooting function such as a digital camera or a mobile phone, it is necessary to add subtitles using a personal computer or the like. Very troublesome work was necessary.

  Therefore, recently, a technique for converting audio data collected during moving image shooting into text data in a moving image shooting apparatus and synthesizing the converted text data as moving image data has been proposed (for example, Patent Document 1). 2, 3).

JP 2004-120279 A JP 2006-197115 A JP 2007-027990 A

  However, the techniques described in Patent Literatures 1, 2, and 3 do not particularly mention whether the voice data is in Japanese or another language, and therefore, voice data of a person who speaks in English. Is considered to be converted into English text data.

  Then, for users who are not good at English, there is a problem that even if the audio data is converted into subtitles, the contents of the subtitles are not understood and it is very inconvenient. Therefore, it is considered convenient if the audio can be captioned in the language desired by the user.

  Accordingly, an object of the present invention is to solve the above-described problems and provide a moving image shooting apparatus and a voice captioning method capable of captioning voice in a user-desired language.

The moving image photographing apparatus of the present invention is
A camera unit that shoots the subject as a movie,
A microphone that collects sound around the subject at the time of shooting a video by the camera unit;
A voice text converting unit that converts voice data of voice collected by the microphone into text data;
A text translation unit that translates text data of a language other than a predetermined language out of the text data converted by the speech text unit into the text data of the predetermined language;
The text data of the predetermined language converted by the voice text unit and the text data of the predetermined language translated by the text translation unit are combined as subtitles with the moving image data of the video shot by the camera unit. A moving image text composition unit.

The audio subtitle conversion method of the present invention
An audio subtitle method by a video shooting device,
Shooting a subject with a video,
Collecting sound around the subject at the time of shooting the video;
Converting voice data of the collected voice into text data;
Translating text data of a language other than a predetermined language out of the converted text data into text data of the predetermined language;
Synthesizing the converted text data in the predetermined language and the translated text data in the predetermined language as subtitles with the moving image data of the captured moving image.

  Since the present invention is configured as described above, it is possible to obtain an effect that the user can convert audio into a desired language.

It is a block diagram which shows the structure of the moving image imaging device of the 1st and 2nd embodiment of this invention. It is a sequence diagram explaining the operation | movement at the time of the video recording of the video recording device of the 1st Embodiment of this invention. It is a sequence diagram explaining the operation | movement at the time of editing of the text data of the moving image imaging device of the 1st Embodiment of this invention. It is a sequence diagram explaining the operation | movement at the time of the moving image reproduction of the moving image imaging device of the 1st Embodiment of this invention. It is a sequence diagram explaining the operation | movement at the time of the video recording of the video recording device of the 2nd Embodiment of this invention.

EMBODIMENT OF THE INVENTION Below, the form for implementing this invention is demonstrated with reference to drawings.
(1) First Embodiment FIG. 1 is a block diagram illustrating a configuration of a moving image shooting apparatus according to a first embodiment of the present invention.

  As shown in FIG. 1, the moving image shooting apparatus of the present embodiment includes a control unit 101, an operation unit 102, a moving image shooting unit 103, a storage unit 106, an audio text conversion unit 107, a text translation unit 108, A text editing unit 109, a moving image text synthesis unit 110, a display 111, and a speaker 112 are included.

  The control unit 101 performs various processes by controlling the components in the moving image photographing apparatus.

  The operation unit 102 performs various operations by the user.

  The moving image shooting unit 103 includes a camera unit 104 and a microphone 105.

  The camera unit 104 captures a subject with a moving image.

  The microphone 105 collects sound around the subject when the camera unit 104 captures a moving image.

  The storage unit 106 stores various data.

  The voice text converting unit 107 recognizes voice data collected by the microphone 105 and converts the voice data into text data.

  The text translation unit 108 translates text data in a language other than a predetermined language (for example, Japanese) out of the text data converted by the speech text conversion unit 107 into text data in a predetermined language. To do.

  The text editing unit 109 edits the text data of a predetermined language converted by the voice text converting unit 107 and the text data of a predetermined language translated by the text translation unit 108.

  The moving image text synthesizing unit 110 uses the moving image moving image data of the moving image captured by the camera unit 104 using the text data of the predetermined language converted by the voice text converting unit 107 and the text data of the predetermined language translated by the text translating unit 108. Composite as subtitles.

  If the text editing unit 109 has edited the text data, the moving image text synthesizing unit 110 synthesizes the edited text data with the moving image data.

  The display 111 is a display unit that displays moving image data photographed by the camera unit 104 and text data of a predetermined language synthesized as captions on the moving image data.

  The speaker 112 outputs the sound data collected by the microphone 105 as sound.

  Hereinafter, the operation of the moving image shooting apparatus of the present embodiment will be described. In the following description, it is assumed that the predetermined language is set to Japanese via the operation unit 102 by the user.

  First, the operation at the time of moving image shooting will be described with reference to FIG.

  As shown in FIG. 2, when the moving image photographing unit 103 starts moving image photographing (step A101), the moving image photographing unit 103 notifies the storage unit 106 that “synchronization data (e.g., elapsed time from the start of photographing is necessary for synchronization). "Moving image data (moving image data shot by the camera unit 104. the same applies hereinafter)" and "audio data (voice data collected by the microphone 105 during shooting of moving images) to which" data. The following is the same) (step A102), and the storage unit 106 writes “moving image data” and “audio data” to which “synchronized data” is added (step A103).

  When the moving image shooting unit 103 finishes moving image shooting (step A104), the moving image shooting unit 103 notifies the voice text converting unit 107 to that effect (step A105).

  When receiving the notification of the end of shooting from the moving image shooting unit 103, the voice text conversion unit 107 requests the storage unit 106 to read “voice data” with “synchronization data” added (step A106). The unit 106 reads out “voice data” to which “synchronization data” is added, and delivers it to the voice text unit 107 (step A107).

  When the “speech data” to which “synchronization data” is added is delivered from the storage unit 106, the speech text unit 107 recognizes the “speech data” and converts it into “text data” (step A108). The “text data” to which “synchronized data” is added is delivered to the text translation unit 108 (step A109). Several techniques for recognizing voice data and converting it into text data are already known, and any of them can be used in this embodiment.

  When the “text data” to which “synchronization data” is added is delivered from the speech text conversion unit 107, the text translation unit 108 converts a part of the “text data” described in a language other than Japanese into Japanese. Translate into words (step A110). Several translation techniques are already known, and any of them can be used in this embodiment.

  At this time, it is not necessary to translate all other languages in the “text data” into Japanese. For example, when “text data” is delimited in sentence units, and the delimited sentence does not contain Japanese at all ( That is, if the sentence is composed entirely of a language other than Japanese), treat the sentence as “a part written in a language other than Japanese” and translate only that sentence into Japanese. Good. Several techniques for dividing text data into sentences are already known, and any of them can be used in the present embodiment.

  Then, the text translation unit 108 requests the storage unit 106 to write the translated “text data” with “synchronization data” added (step A111), and the storage unit 106 stores “synchronization data”. The added translated “text data” is written (step A112), and the process is terminated. At this time, the text translation unit 108 hands over the portion of the “text data” that has not been translated to the storage unit 106 as it is passed from the speech text conversion unit 107.

  Next, the operation at the time of editing text data will be described with reference to FIG.

  As shown in FIG. 3, when editing the “text data” stored in the storage unit 106, the text editing unit 109 reads “text data” with “synchronous data” added to the storage unit 106. A request is made (step B101), and the storage unit 106 reads “text data” to which “synchronization data” is added, and delivers it to the text editing unit 109 (step B102).

  When the “text data” to which “synchronization data” is added is delivered from the storage unit 106, the text editing unit 109 edits the “text data” (step B 103), and sends the “synchronization data” to the storage unit 106. Is requested to write the edited “text data” (step B104), and the storage unit 106 writes the edited “text data” to which “synchronized data” is added (step B105). finish.

  Next, the operation at the time of moving image reproduction will be described with reference to FIG.

  As shown in FIG. 4, when reproducing a moving image, the moving image text synthesizing unit 110 adds “synchronous data” to “audio data”, “moving image data”, and “text data” to the storage unit 106. Is read out (step C101), and the storage unit 106 reads out “audio data”, “moving image data”, and “text data” to which “synchronous data” has been added, and delivers them to the moving image text synthesizing unit 110. (Step C102).

  When the “speech data”, “moving image data”, and “text data” to which “synchronization data” has been added are delivered from the storage unit 106, the video text synthesis unit 110 receives “text data” based on the “synchronization data”. “Data” is combined with “Video data” as subtitles to generate “Video data with subtitles”, “Video data with subtitles” is displayed on the display 111, and “Audio data” is converted based on “Synchronization data”. Then, audio is output from the speaker 112 in synchronization with the “moving image data with caption” (step C103).

In addition, when storing the “moving image data with subtitles” in the storage unit 106, the moving image text synthesis unit 110 requests the storage unit 106 to write “moving image data with subtitles” to which “synchronization data” is added. (Step C104), the storage unit 106 writes “moving image data with subtitles” added with “synchronization data” (step C105), and ends the process.
(2) Second Embodiment The moving image capturing apparatus according to the present embodiment is similar in structure to the first embodiment but operates differently.

  That is, in the first embodiment, the subtitles in which the audio data is converted into text are displayed on the display 111 at the time of moving image reproduction, whereas in the present embodiment, the audio data is converted into text at the time of moving image shooting. The subtitles are displayed on the preview screen of the display 111.

  Hereinafter, the operation at the time of moving image shooting of the moving image shooting apparatus of the present embodiment will be described with reference to FIG. In the following description, it is assumed that the predetermined language is set to Japanese via the operation unit 102 by the user.

  As shown in FIG. 5, when the moving image shooting unit 103 starts moving image shooting (step D101), writing of “moving image data” and “audio data” with “synchronization data” added to the storage unit 106, respectively. The request is made (step D102), and the storage unit 106 writes “moving image data” and “audio data” to which “synchronization data” is added (step D103).

  In addition, the moving image photographing unit 103 makes the above write request to the storage unit 106, and at the same time, delivers the “voice data” to which “synchronization data” is added to the voice text unit 107 (step D104).

  When the “speech data” to which “synchronization data” is added is delivered from the moving image photographing unit 103, the speech text conversion unit 107 converts “speech data” into “text data” (step D105), and “synchronization data”. The “text data” added with is handed over to the text translation unit 108 (step D106).

  When the “text data” to which “synchronization data” is added is delivered from the speech text conversion unit 107, the text translation unit 108 converts a part of the “text data” described in a language other than Japanese into Japanese. The text is translated into words (step D107), and the translated "text data" is delivered to the moving image text synthesis unit 110 (step D108). At this time, the text translation unit 108 hands over the portion of the “text data” that has not been translated to the video text synthesis unit 110 as it is passed from the speech text conversion unit 107.

  When the “text data” is delivered from the text translation unit 108, the moving image text synthesizing unit 110 synthesizes “text data” into “moving image data” as a caption and displays it on the preview screen of the display 111 (step D109). At this time, the “voice data” is output as voice from the speaker 112 without going through the moving image text synthesis unit 110.

  Further, the text translation unit 108 delivers the translated “text data” to the moving image text synthesis unit 110, and at the same time, the translated “text data” to which “synchronization data” is added to the storage unit 106. Is written (step D110), and the storage unit 106 writes the translated “text data” to which “synchronous data” is added (step D111). At this time, the text translation unit 108 hands over the portion of the “text data” that has not been translated to the storage unit 106 as it is passed from the speech text conversion unit 107.

  Thereafter, when the moving image shooting by the moving image shooting unit 103 ends (step D112), the process ends.

In the present embodiment, the operation at the time of editing text data and the operation at the time of reproducing a moving image are the same as those in the first embodiment, and thus description thereof is omitted (see FIGS. 3 and 4 respectively).
(3) Other Embodiments Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above embodiments. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

  For example, in the first and second embodiments, when moving image data and text data are displayed on the display 111, the moving image data is displayed on the entire display screen of the display 111, and the text data is superimposed on the moving image data. May be displayed. Alternatively, the display screen of the display 111 may be divided into two screens, moving image data may be displayed on one screen, and text data may be displayed on the other screen.

  In the first and second embodiments, when text data is displayed on the display 111, the text data already in Japanese at the time of conversion by the speech text conversion unit 107 and the text translation unit 108 are displayed. The text data translated into Japanese may be displayed in a different color or font.

  In the first and second embodiments, an example has been described in which the predetermined language is Japanese and the speech data of a person speaking in a language other than Japanese is converted into Japanese text data. Of course, other languages can be used. For example, when the predetermined language is English, voice data of a person speaking in a language other than English can be converted into English text data.

  As described above, in the moving image capturing apparatus of the present invention, voice data collected at the time of moving image shooting is converted into text data, and text data in a language other than a predetermined language among the text data is converted into text in a predetermined language. After being translated into data, the text data is combined with moving image data as subtitles.

  Therefore, voice data in a language other than the predetermined language among the voice data is translated into text data in a predetermined language and then synthesized into moving image data.

  Therefore, the user can set the desired language as the predetermined language, thereby obtaining the effect that the audio can be subtitled in the desired language.

  The present invention can be applied to a moving image shooting apparatus such as a video camera, a digital camera, a mobile phone, a PHS (Personal Handyphone System), a PDA (Personal Digital Assistant) equipped with a moving image shooting function.

DESCRIPTION OF SYMBOLS 101 Control part 102 Operation part 103 Movie imaging | photography part 104 Camera part 105 Microphone 106 Memory | storage part 107 Speech text conversion part 108 Text translation part 109 Text editing part 110 Movie text synthesis part 111 Display 112 Speaker

Claims (7)

A camera unit that shoots the subject as a movie,
A microphone that collects sound around the subject at the time of shooting a video by the camera unit;
A voice text converting unit that converts voice data of voice collected by the microphone into text data;
A text translation unit that translates text data of a language other than a predetermined language out of the text data converted by the speech text unit into the text data of the predetermined language;
The text data of the predetermined language converted by the voice text unit and the text data of the predetermined language translated by the text translation unit are combined as subtitles with the moving image data of the video shot by the camera unit. A moving image photographing device comprising: a moving image text synthesizing unit; The moving image text synthesis unit
The text data of the predetermined language is synthesized as subtitles with the video data when the video shot by the camera unit is played back, and the video data and the text data of the predetermined language are displayed. Movie shooting device. The moving image text synthesis unit
The text data of the predetermined language is combined with the video data as subtitles when capturing a video by the camera unit, and the video data and the text data of the predetermined language are displayed on a preview screen. Movie shooting device. The moving image text synthesis unit
When displaying the moving image data and the text data of the predetermined language,
Divide the display screen into two screens,
Display the video data on one screen,
The moving image photographing apparatus according to claim 2 or 3, wherein text data of the predetermined language is displayed on the other screen. The moving image text synthesis unit
When displaying the moving image data and the text data of the predetermined language,
The text data of the predetermined language converted by the speech text conversion unit and the text data of the predetermined language translated by the translation unit are displayed in different colors. The moving image photographing device according to the item. The moving image text synthesis unit
When displaying the moving image data and the text data of the predetermined language,
The text data of the predetermined language converted by the speech text conversion unit and the text data of the predetermined language translated by the translation unit are displayed in different fonts. The moving image photographing device according to the item. An audio subtitle method by a video shooting device,
Shooting a subject with a video,
Collecting sound around the subject at the time of shooting the video;
Converting voice data of the collected voice into text data;
Translating text data of a language other than a predetermined language out of the converted text data into text data of the predetermined language;
Synthesizing the converted text data of the predetermined language and the translated text data of the predetermined language as subtitles with the video data of the captured moving image.

Images