JP2008065653A - Video translation device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To translate voice or characters contained in a video into another language according to the video and position information. <P>SOLUTION: This video translation device includes: a voice information extracting part 104 for extracting sound from video data 103 containing position information 101 and video 102; an image information extracting part 105 for extracting images; a position information extracting part 106 for extracting positions; a voice analysis part 107 for analyzing voice contained in the sound; an image analysis part 108 for analyzing characters contained in the images; an input language selecting part 109 for selecting an input language corresponding to the position information from a position language kind DB 110; an output language selecting part 111 for selecting an output language; a voice translating part 112 for translating the voice; an image translating part 113 for translating the characters; a voice information dubbing part 114 for dubbing the voice with the translated voice; an image information superposing part 115 for superposing a translated caption; and a video output part 116 for displaying the video after the translation. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a video translation apparatus, and more particularly to a video translation apparatus capable of performing translation in accordance with position information included in captured video data.

The language used varies depending on the country or region where you travel, such as conversations and broadcasts that you see and hear in the country you are traveling to, and the characters on the signboard. Traditionally, based on the language corresponding to the user's location, There has been an information processing apparatus that performs translation without resetting the language each time a place is moved. (For example, refer to Patent Document 1).
JP 2000-194698 A

  However, in the conventional information processing apparatus that performs translation, there is a problem that it takes time to carry a translation apparatus separately from a photographing apparatus such as a digital video camera when photographing an image that needs to be translated in a travel destination country. Had.

  When shooting a video, there is a shooting device that records the positional information acquired by GPS in association with each other so that the positional information of the video can be known during reproduction.

  The present invention is an apparatus that performs translation based on a language corresponding to position information included in photographed video data. Since data after photographing is input, it is possible to eliminate the need to carry the translation apparatus during photographing.

  The video translation device according to claim 1 is an extraction means for extracting a position from video data including position information, an input language selection means for selecting an input language corresponding to the position extracted by the extraction means, and a user arbitrarily Output language selection means for selecting another type of output language, and translation means for performing video translation processing based on the input language selected by the input language selection means and the output language selected by the output language selection means It is characterized by providing.

  The video translation device according to claim 2 is selected by the voice extraction means for extracting the voice from the video data, the analysis means for analyzing the voice included in the voice extracted by the voice extraction means, and the output language selection means. And voice translation means for translating the voice analyzed by the analysis means based on the output language.

  According to a third aspect of the present invention, there is provided the video translation apparatus further comprising output means for outputting a video in which the voice translated by the voice output means is dubbed into the voice included in the original video.

  The video translation device according to claim 4 includes output means for translating the voice translated by the voice translation means into the output language selected by the output language selection means and outputting a video superimposed as a telop on the original video. It is further provided with the feature.

  The video translation apparatus according to claim 5, further comprising: an image extraction unit that extracts an image from the video data; and a character analysis unit that analyzes a character included in the image extracted by the image extraction unit. Characters analyzed by the character analysis means are translated.

  According to a sixth aspect of the present invention, there is provided the video translation apparatus further comprising output means for outputting a video in which the characters translated by the translation means are superimposed as a telop on the original video.

  The video translation apparatus according to claim 7, wherein the character translated by the translation unit is converted into voice based on the output language selected by the output language selection unit, and a video obtained by adding the voice to the original video is output. Output means is further provided.

  The video translation device according to claim 8 is characterized in that the position information extracted by the extraction means is acquired by GPS.

  According to the present invention, at the time of shooting, the video can be translated after shooting without the photographer being aware of the video shot in which country and region. Even when shooting is performed across countries and regions where the language used is complicated, the language can be determined according to the position information that changes every moment, and the video can be translated. This has the advantage that it can be viewed and translated into its own language or other languages based on video data taken at home and abroad, such as for travel, business, and reporting purposes.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

(Embodiment 1)
FIG. 1 is a block diagram showing a configuration of a video translation apparatus according to Embodiment 1 of the present invention. In FIG. 1, reference numeral 103 denotes video data recorded by associating a video 102 shot by a shooting device such as a digital video camera with position information 101 acquired by GPS at the time of shooting the video. Here, the position information is information including latitude and longitude that change every moment depending on the shooting point. An audio information extracting unit 104 extracts audio from the video data 103. A voice analysis unit 107 analyzes and extracts voices such as conversation and broadcast included in the voice extracted by the voice information extraction unit 104. An image information extraction unit 105 extracts an image from the video data 103. An image analysis unit 108 analyzes and extracts characters such as a signboard included in the image extracted by the image information extraction unit 105. Reference numeral 106 denotes a position information extraction unit that extracts position information for specifying a language type to be input for translation from the video data 103. Reference numeral 109 denotes an input language selection unit. A position language type DB 110 in which the position information extracted by the position information extraction unit 106 and the language types corresponding to the position information are collected is input to the input language selection unit 109 and used as an input for translation. Search and select the language type. Reference numeral 111 denotes an output language selection unit, in which the user selects an arbitrary language type to be output for translation by means such as key input or voice input. A voice translation unit 112 translates the voice extracted by the voice analysis unit 107 from the input language selected by the input language selection unit 109 to the output language selected by the output language selection unit 111. Reference numeral 114 denotes a voice information dubbing unit that dubbes the voice extracted by the voice information extraction unit 104 into a voice translated by the voice translation unit 112. An image translation unit 113 translates the characters extracted by the image analysis unit 108 from the input language selected by the input language selection unit 109 into the output language selected by the output language selection unit 111. Reference numeral 115 denotes an image information superimposing unit that superimposes characters translated by the image translating unit 113 on the image extracted by the image information extracting unit 105 as a telop. Reference numeral 116 denotes a video output unit that synthesizes the audio dubbed by the audio information dubbing unit 114 and the image superimposed by the video information superimposing unit 115 and outputs the synthesized video.

  FIG. 2 is a diagram illustrating an example of information stored in video data according to Embodiment 1 of the present invention. Video data shot by a shooting device that records the position information acquired by GPS in correspondence with the image is composed of position information 203 including the latitude and longitude of the shooting point in addition to the image 201 and sound 202 for each shooting scene. Has been. When moving to a country or region with a different language type by moving the shooting point, the language type of the voice 202 changes. For example, as shown in FIG. 2, the point at 40 degrees north latitude and 74 degrees west longitude is the United States of America, and the image 201 of the scene 1 photographed here contains English characters, and the voice 202 contains English voices. include. The point at 43 degrees north latitude and 79 degrees west is Canada. The image 201 of the scene 2 taken here contains French characters, and the voice 202 contains French words. Image information, audio information, and position information are extracted from the video data thus captured.

  Operation for automatically determining the language used in the country and region indicated by the position information and selecting the input language type based on the position information extracted from the video data in the video translation apparatus configured as described above Will be described with reference to FIGS. FIG. 3 is a diagram illustrating an example of information stored in the position language type DB. First, the location language type DB 110 is searched based on location information represented by a combination of latitude, longitude, and the like extracted by the location information extraction unit 106. In the position language type DB 110, the language type used in the country and region indicated by each position information is recorded in advance. For example, as shown in FIG. 3, the position information that the language type corresponding to the position information of 40 degrees north latitude and 74 degrees west longitude is English and the data paired with the type of language used there are accumulated. Has been. Thus, the language type paired with the position information in the position language type DB 110 that matches the position information extracted by the position information extraction unit 106 is selected.

  Next, the operation of selecting the output language type will be described with reference to FIG. FIG. 4 is a diagram illustrating an example of an output language selection screen. The video translation apparatus of the present invention has an output language selection screen as shown in FIG. 4, for example, and selects an output language type from the language types displayed in the output language selection list 402 by key input, voice input, or the like. By directly inputting the language type into the output language input area 401, the user arbitrarily selects the output language type.

  Next, the operation when the speech translation process is executed will be described with reference to FIGS. FIG. 5 is a diagram illustrating an example of video output information translated using the video translation apparatus according to the first embodiment. The speech translation unit 112 translates the voice extracted by the speech analysis unit 107 from the input language selected by the input language selection unit 109 to the output language selected by the output language selection unit 111. For example, as shown in scene 1 of FIG. 5, when the input language type of the voice 505 included in the voice 502 is English and the output language type is Japanese, the voice information dubbing unit 114 extracts voice information. The English voice “Welcome to Hourwood” extracted by the section 104 is dubbed into the Japanese voice “Welcome to Hollywood” translated by the speech translation section 112. Similarly, in scene 2, the French voice “Bienvenue vers Canada” is dubbed into the Japanese voice “Welcome to Canada”.

  Next, the operation when the video translation process is executed will be described with reference to FIGS. The image translation unit 113 translates the characters extracted by the image analysis unit 108 from the input language selected by the input language selection unit 109 into the output language selected by the output language selection unit 111. For example, as shown in scene 1 of FIG. 5, when the input language type of the character 503 included in the image 501 is English and the output language type is Japanese, the image information superimposing unit 115 extracts the image information. The Japanese character “Hollywood” translated by the image translating unit 113 is superimposed as a telop 504 on the image including the English character “HOOOYWOOD” extracted by the unit 105. Similarly, in the scene 2, the Japanese character “Canada” is superimposed as a telop 504 on an image including the French character “Canada”.

  With the above processing, in the video translation apparatus according to Embodiment 1 of the present invention, the language used in the country and region indicated by the location information is automatically determined from the video data having the location information corresponding to the video, This makes it possible to select a language as an input for translation, translate an image and sound into an output language selected by the user, and output the translation result as video information.

  It should be noted that when performing speech information translation processing, the voice of the person and the accompanying person is often different from the input language type, and thus has a function of not translating languages other than the input language type. Similarly, the characters included in the image information have a function that does not translate languages other than the input language type.

  Further, in the correspondence relationship between the position information and the language type shown in FIG. 3, the latitude and longitude information are assumed to have numerical ranges, and the correspondence relationship between the position information and the shooting points in each country and region can be maintained. It is supposed to be.

  In addition, in the case of a DB configuration that holds a plurality of candidate language types used in the country and region for the location information of the shooting location and the location information of one location shown in FIG. 3, it is the highest among the candidate language types. It is assumed that it has a function of automatically discriminating an optimum language type that can obtain translation accuracy.

  In addition, the video translation apparatus of the present invention is not limited to moving images, and the translation and output of images can be performed by similar means for still images taken with a digital still camera or the like.

  In addition, the video translation device of the present invention can be connected to or incorporated in a shooting device that records and records position information acquired by GPS in correspondence with the video, so that it can be taken on the spot while shooting the video. It is possible to view the output video after translation processing.

  Also, by implementing the video translation apparatus of the present invention as a program for a personal computer, the output video after translation processing is delivered to video editing software such as digital video video editing software “MotionDV STUDIO” for personal computers of Matsushita Electric Industrial Co., Ltd. Alternatively, by incorporating it as a function of the software, it becomes possible to add or modify post-translational audio information or telop information in addition to conventional nonlinear editing.

  In addition, it will be used as a video communication tool between different countries, such as video translation emails and video phone applications.

(Embodiment 2)
In Embodiment 1, the translation result is output as video information by switching the voice of the input language included in the voice to the voice of the output language. In the second embodiment, a method of superimposing voice of an input language included in speech on an image as character information such as subtitles is introduced.

  FIG. 6 is a block diagram showing the configuration of the video translation apparatus according to Embodiment 2 of the present invention. In FIG. 6, 101 is position information, 102 is video, 103 is video data, 104 is an audio information extraction unit, 105 is an image information extraction unit, 106 is a position information extraction unit, 107 is an audio analysis unit, and 108 is an image analysis unit. 109 is an input language selection unit, 110 is a location language type DB, 111 is an output language selection unit, 112 is a speech translation unit, 113 is an image translation unit, 615 is a speech / image information superimposition unit, and 116 is a video output unit. . Position information, video, video data, audio information extraction unit, image information extraction unit, position information extraction unit, audio analysis unit, image analysis unit, input language selection unit, position language type DB, output language selection unit, speech translation unit, The image translation unit and the video output unit perform the same operations as in the first embodiment. The audio / image information superimposing unit 615 superimposes the characters translated by the image translating unit 113 as telops, converts the voice translated by the audio translating unit 112 into character information such as subtitles, and superimposes them on the image.

  Next, the operation when the voice / image translation process is executed will be described with reference to FIGS. FIG. 7 is a diagram illustrating an example of video output information translated using the video translation apparatus according to the second embodiment. The speech translation unit 112 translates the voice extracted by the speech analysis unit 107 from the input language selected by the input language selection unit 109 to the output language selected by the output language selection unit 111. For example, as shown in scene 1 of FIG. 7, when the input language type is English and the output language type is Japanese, the audio / image information superimposing unit 615 extracts the English language extracted by the audio analyzing unit 107. The voice “Welcome to Hourwood” is converted into Japanese text information “Welcome to Hollywood” translated by the speech translation unit 112 and superimposed on the image as subtitle 704. Similarly, in the scene 2, the French voice “Bienvenue vers Canada” is called “Welcome to Canada” in Japanese. The caption 704 is superimposed on the image. Further, the processing for the image is performed in the same manner as in the first embodiment so that the display of the telop is at a position where it does not overlap with the caption.

  As described above, in the second embodiment, the voice of the input language included in the voice is superimposed on the image as text information such as subtitles, so that it can be output as video information with which the translation result can be understood only by the image. Become.

(Embodiment 3)
In the first embodiment, the translation result is output as video information by superimposing an input language character included in the image as an output language character telop on the image. In the third embodiment, a method of converting characters of an input language included in an image into a voice and synthesizing it with the voice is introduced.

  FIG. 8 is a block diagram showing the configuration of the video translation apparatus according to Embodiment 3 of the present invention. In FIG. 8, 101 is position information, 102 is video, 103 is video data, 104 is an audio information extraction unit, 105 is an image information extraction unit, 106 is a position information extraction unit, 107 is an audio analysis unit, and 108 is an image analysis unit. 109 is an input language selection unit, 110 is a position language type DB, 111 is an output language selection unit, 112 is a speech translation unit, 113 is an image translation unit, 814 is a voice / image information dubbing unit, and 116 is a video output unit. . Position information, video, video data, audio information extraction unit, image information extraction unit, position information extraction unit, audio analysis unit, image analysis unit, input language selection unit, position language type DB, output language selection unit, speech translation unit, The image translation unit and the video output unit perform the same operations as in the first embodiment. The voice / image information dubbing unit 814 changes the voice extracted by the voice information extraction unit 104 to a voice translated by the voice translation unit 112, and converts the characters translated by the image translation unit 113 into voice and synthesizes the voice. .

  Next, the operation when the voice / image translation process is executed will be described with reference to FIGS. FIG. 9 is a diagram illustrating an example of video output information translated using the video translation device according to the third embodiment. The image translation unit 113 translates the characters extracted by the image analysis unit 108 from the input language selected by the input language selection unit 109 into the output language selected by the output language selection unit 111. For example, as shown in scene 1 of FIG. 9, when the input language type is English and the output language type is Japanese, the audio / image information dubbing unit 814 performs image processing on the image information superimposing unit 115. The image including the English character “HOOOLYWOOD” extracted by the information extraction unit 105 is converted into a voice guidance 903 such as “There is a display of“ Hollywood ”” translated by the image translation unit 113, and converted into voice. Let's synthesize. Similarly, in the scene 2, an image including the French-language character “Canada” is converted into a voice guidance 903 such as “There is a display of“ Canada ”” in Japanese, and synthesized into a voice. In the processing for voice, the same operation as in the first embodiment is performed so that voice guidance and voice of voice-over do not overlap.

  As described above, in the third embodiment, characters in an input language included in an image are synthesized with voice as voice information such as voice guidance, so that the translation result can be output only by voice as video information that can be understood. It becomes possible.

(Embodiment 4)
In the first to third embodiments, the user arbitrarily selects the output language type. In the fourth embodiment, the current location information acquired by GPS is input to the output language selection unit, and the language used in the current country / region where the user uses the video translation device of the present invention is automatically selected. Output language selection means for selecting an output language is provided.

  FIG. 10 is a block diagram showing the configuration of the output language selection unit in the fourth embodiment of the present invention. In FIG. 10, 110 is a location language type DB, 111 is an output language selection unit, and 1001 is location information. Next, the operation when the output language selection process is executed will be described with reference to FIGS. For example, when the video translation apparatus of the present invention in Embodiment 4 is operated at a point of 40 degrees north latitude and 135 degrees east longitude, latitude and longitude information can be acquired as position information 1001 by GPS, and as shown in FIG. Japanese is selected as the language type of the output language from the data accumulated in the position language type DB 110.

  Through the above processing, the output language selection unit according to Embodiment 4 of the present invention automatically determines the language used in the country and region indicated by the current position information, and selects the language to be output for translation. It becomes possible.

  The video translation apparatus of the present invention can determine the language of the country and region in which the image was taken, and translate the image into any other type of language according to the position information that changes every moment included in the image data after shooting. As a result, it is possible to translate and view in the native language or other languages based on video data taken at home and abroad, such as for travel, business, and reporting purposes.

1 is a block diagram showing a configuration of a video translation apparatus according to Embodiment 1 of the present invention. A figure showing an example of information stored in video data The figure showing the example of the information stored in position language classification DB Diagram showing an example of the output language selection screen Example of effect of implementation of embodiment 1 The block diagram which shows the structure of the video translation apparatus in Embodiment 2 of this invention. Example of effect of implementation of embodiment 2 The block diagram which shows the structure of the video translation apparatus in Embodiment 3 of this invention. Example of effect of implementation of embodiment 3 The block diagram which shows the structure of the output language selection part in Embodiment 4 of this invention.

Explanation of symbols

101 Position Information 102 Video 103 Video Data 104 Audio Information Extraction Unit 105 Image Information Extraction Unit 106 Position Information Extraction Unit 107 Audio Analysis Unit 108 Image Analysis Unit 109 Input Language Selection Unit 110 Location Language Type DB
DESCRIPTION OF SYMBOLS 111 Output language selection part 112 Speech translation part 113 Image translation part 114 Voice information dubbing part 115 Image information superimposition part 116 Video output part 201 Image 202 Voice 203 Position information 401 Output language input area 402 Output language selection list 501 Image 502 Voice 503 Image 504 Telop 505 Voice 615 Audio / image information superimposing unit 814 Audio / image information dubbing unit 903 Audio guidance 1001 Position information

Claims (8)

Extraction means for extracting a position from video data including position information, input language selection means for selecting an input language corresponding to the position extracted by the extraction means, and a user arbitrarily selecting another type of output language Output language selection means, and an input language selected by the input language selection means and a translation means for performing video translation processing based on the output language selected by the output language selection means Video translation device. Based on the output language selected by the output language selection means, the voice extraction means for extracting voice from the video data, the analysis means for analyzing the voice included in the voice extracted by the voice extraction means, The video translation apparatus according to claim 1, further comprising speech translation means for translating the voice analyzed by the analysis means. 3. The video translation apparatus according to claim 2, further comprising output means for outputting a video in which the voice translated by the voice translation means is dubbed into a voice included in the original video. The apparatus further comprises output means for translating the voice translated by the speech translation means into the output language selected by the output language selection means and outputting a video superimposed as a telop on the original video. Item 3. The video translation apparatus according to Item 2. Image extraction means for extracting an image from video data; and character analysis means for analyzing characters included in the image extracted by the image extraction means; and the translation means for analyzing characters analyzed by the character analysis means The video translation apparatus according to claim 1, wherein the video translation apparatus translates the video. 6. The video translation apparatus according to claim 5, further comprising output means for outputting a video in which characters translated by the translation means are superimposed as a telop on the original video. It further comprises output means for converting the character translated by the translation means into sound based on the output language selected by the output language selection means, and outputting a video obtained by adding the sound to the original video. The video translation apparatus according to claim 5, wherein the video translation apparatus is characterized. The video translation apparatus according to claim 1, wherein the position information extracted by the extraction unit is acquired by GPS.

Images