JP2019047391A - Device, method and program for distributing content information with caption - Google Patents

Abstract

To provide a content information distribution device with caption which can select an appropriate caption automatically.SOLUTION: A voice-to-text conversion unit 107 converts voice information included in content information into first text information. A text translation unit 108 translates the first text information into second text information. A viewer attribute analysis unit 102 analyzes the race of the viewer on the basis of an imaged image obtained by imaging the view viewing the content information, so as to generate the attribute information of the viewer. A caption display mode selection unit 103 generates caption display mode information which identifies a caption language to be superposed with video information included in the content information on the basis of the attribute information. A caption generation unit 104 selects either one of the first text information and the second text information on the basis of caption display mode information, so as to generate caption information to be superposed with the video information. A distribution control unit 105 generates content information with caption to distribute to a viewing client.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a subtitled content information distribution apparatus that distributes subtitled content information to a viewing client, a subtitled content information distribution method, and a subtitled content information distribution program.

  Patent Document 1 describes a subtitle distribution system which selects subtitles corresponding to reproduced video according to the attribute of the viewer and distributes the selected subtitles to the viewing client.

JP, 2014-200002, A Unexamined-Japanese-Patent No. 2007-94840 JP, 2016-161830, A

  According to the subtitle delivery system described in Patent Document 1, the viewer has to set the type of subtitle, which is cumbersome. Moreover, according to the subtitle delivery system described in Patent Document 1, when a plurality of viewers view a video in one viewing client, it is not possible to deliver appropriate subtitles.

  The present invention relates to a subtitled content information distribution apparatus capable of automatically selecting subtitles appropriate for a viewer who views content information in a viewing client and distributing subtitled content information to the viewing client It is an object of the present invention to provide a content information distribution method and a subtitled content information distribution program.

  The present invention converts a content receiving unit for receiving content information transmitted from a content information transmitting apparatus, and audio information in a first language contained in the content information into first text information in a first language. A voice / text conversion unit, a text translation unit for translating the first text information into second text information in a second language, and a captured image obtained by capturing a viewer viewing content information in a viewing client A language of subtitles to be superimposed on video information included in the content information based on the attribute information and a viewer attribute analyzing unit that analyzes the race of the viewer and generates attribute information of the viewer; A subtitle display mode selection unit for generating subtitle display mode information for identifying the first text information and the second text information based on the subtitle display mode information; A subtitle generation unit that selects any one of text information and generates subtitle information to be superimposed on the video information, adds the subtitle information to the video information to generate subtitled content information, and A subtitled content information distribution apparatus comprising: a distribution control unit for distributing to a client.

  The present invention receives content information transmitted from a content information transmitting apparatus, converts audio information in a first language included in the content information into first text information in a first language, and The text information of 1 is translated into second text information in a second language, and the race of the viewer is analyzed based on the captured image obtained by capturing the viewer viewing the content information in the viewing client. An attribute information of a viewer is generated, subtitle display mode information for specifying a language of a subtitle to be superimposed on video information included in the content information is generated based on the attribute information, and the subtitle display mode information is generated based on the subtitle display information. One of the first text information and the second text information is selected, subtitle information to be superimposed on the video information is generated, and the video information is selected as the subtitle information. By adding curtain information to generate a captioned content information provides captioned content information distribution method characterized by distributing the viewing client.

  The present invention comprises the steps of: receiving a content information transmitted from a content information transmitting apparatus to a computer; and audio information in a first language contained in the content information into first text information in a first language. The step of converting, the step of translating the first text information into second text information in a second language, and the viewing based on a captured image obtained by capturing a viewer who views the content information at a viewing client Analyzing the race of the person to generate the attribute information of the viewer, and based on the attribute information, generating subtitle display mode information specifying the language of the subtitle to be superimposed on the video information included in the content information And, based on the subtitle display mode information, any one of the first text information and the second text information. Or one of them is selected to generate subtitle information to be superimposed on the video information, and the subtitle information is added to the video information to generate subtitled content information and distributed to the viewing client A subtitled content information distribution program is provided.

  According to the subtitled content information distribution apparatus, subtitled content information distribution method, and subtitled content information distribution program of the present invention, appropriate subtitles are automatically selected according to the viewer who views the content information in the viewing client. Thus, subtitled content information can be distributed to viewing clients.

FIG. 1 is a diagram showing a schematic configuration of a subtitled content information delivery system configured to include a subtitled content information delivery device of an embodiment. It is a block diagram which shows the concrete structure of the content information delivery apparatus with subtitles of one Embodiment. 6 is a flowchart showing an operation of a subtitled content information distribution apparatus of an embodiment, a process executed by a subtitled content information distribution method of an embodiment, and a process of causing a computer to execute a subtitled content information distribution program of an embodiment. . It is a block diagram showing roughly a server provided with a computer which runs a subtitles content information distribution program of one embodiment.

  Hereinafter, a subtitled content information distribution apparatus, a subtitled content information distribution method, and a subtitled content information distribution program according to an embodiment will be described with reference to the attached drawings.

  First, a schematic configuration and operation of a subtitled content information distribution system including the subtitled content information distribution device will be described using FIG. 1. In FIG. 1, a server 10 is connected to the Internet 11. The server 10 is a so-called cloud server which is used via the Internet 11 and provided in a cloud environment.

  A storage device 21 and a video camera 22 capable of network communication are connected to the Internet 11. The storage device 21 and the video camera 22 are an example of a content information transmission apparatus that transmits content information including video and audio information. The content information transmission apparatus may be only the storage device 21 or only the video camera 22.

  The storage device 21, the video camera 22, or any other content information transmission device will be referred to as a content information transmission device 20. The content information transmitting apparatus 20 transmits content information to the server 10 via the Internet 11.

  As described later, the server 10 superimposes subtitles on video information included in the content information and distributes the subtitles to the viewing client. The server 10 constitutes a subtitled content information distribution apparatus of the present embodiment.

  A television receiver 31 and a personal computer 32 capable of network communication are connected to the Internet 11, respectively. The television receiver 31 and the personal computer 32 are examples of viewing clients that receive the subtitled content information distributed from the server 10 and display them on the display units 310 and 320, respectively.

  The viewing client may be only the television receiver 31 or only the personal computer 32. The viewing client may be a projector device that projects the received subtitled content information on a screen. A screen on which image light is projected is also included in the display unit. The television receiver 31, the personal computer 32, the projector device, or any other viewing client will be referred to as a viewing client 30.

  An external camera 311 is attached to the television receiver 31. A plurality of viewers may view the subtitled content information displayed on the display unit 310 of the television receiver 31. The camera 311 shoots a plurality of viewers. The television receiver 31 transmits, to the server 10, photographed images obtained by photographing the plurality of viewers by the camera 311.

  The television receiver 31 transmits the setting information set in the television receiver 31 to the server 10. The camera 311 may be incorporated in the housing of the television receiver 31.

  The personal computer 32 incorporates a camera 321. Similarly, a plurality of viewers may view the subtitled content information displayed on the display unit 320 of the personal computer 32. The camera 321 shoots a plurality of viewers. The personal computer 32 transmits, to the server 10, photographed images in which the camera 321 photographed a plurality of viewers.

  The personal computer 32 transmits setting information set in the personal computer 32 to the server 10. The camera 321 may be an external camera separate from the personal computer 32.

  When the viewing client is a projector device, dozens or 100 or more viewers can view the subtitled content information projected on the screen. In this case, several tens or 100 or more viewers may be photographed by the camera, and the projector device may transmit the photographed image to the server 10.

  The specific configuration and operation of the server 10 will be described with reference to FIG. In FIG. 2, the photographed image receiving unit 101 receives a photographed image transmitted from the viewing client 30. As described above, the photographed image is an image obtained by photographing a plurality of viewers. The photographed image may be an image obtained by photographing one viewer. The photographed image is supplied to the viewer attribute analysis unit 102.

  The viewer attribute analysis unit 102 analyzes the attributes of each of a plurality of viewers based on the captured image. The attributes of the viewer are at least the race of the viewer, preferably the race and the age. In the present embodiment, the audience attribute analysis unit 102 analyzes race and age.

  The technology described in Patent Document 2 can be employed for analyzing the race of the viewer. The technology described in Patent Document 3 can be adopted for analyzing the age of the viewer. Techniques for analyzing attributes such as race or age are provided by, for example, Google Inc. (Alphabet Inc.), Microsoft Corporation, Kairos AR, Inc., and techniques provided by these companies can also be adopted.

  The viewer attribute analysis unit 102 analyzes the attributes of each of a plurality of viewers, and generates attribute information indicating the ratio of a predetermined race and the ratio of a predetermined age group among all the plurality of viewers. The race may be roughly classified into races such as Asians, Westerners, whites, blacks, etc. As the age group, it is sufficient to roughly classify the age like a child, an adult excluding an old man, and an old man. The attribute information analyzed by the viewer attribute analysis unit 102 is supplied to the subtitle display mode selection unit 103.

  If the photographed image is an image obtained by photographing one viewer, the viewer attribute analysis unit 102 analyzes the race and age of the one viewer to generate attribute information.

  The subtitle display mode selection unit 103 determines the subtitle display mode when displaying on the viewing client 30 superimposed on the video information according to the input attribute information, and supplies subtitle display mode information to the subtitle generation unit 104. . Subtitle display mode information is information for specifying at least a language of subtitles. The subtitle display mode information preferably includes, in addition to the information specifying the language, information for selecting a subtitle according to the age group of the viewer.

  The subtitle mode according to the viewer's age group is one of the subtitle size, the kanji included in the subtitle, the subtitle display time, and the maximum number of characters or the maximum number of lines of subtitles displayed at one time, Or any combination of these.

  The content receiving unit 106 receives the content information transmitted from the content information transmitting apparatus 20. The video and audio information included in the content information is supplied to the distribution control unit 105. Of the video and audio information, audio information is supplied to the audio / text conversion unit 107. The speech / text conversion unit 107 converts speech information in the first language into text information (first text information) in the first language. The text information is supplied to the subtitle generation unit 104 and the text translation unit 108.

  As technology for converting speech information into text information, technology provided by Google Inc. or Microsoft Corporation can be adopted.

  The text translation unit 108 translates the input text information in the first language into a second language different from the first language to generate translated text information (second text information). The language to which the text translation unit 108 translates the text information input may be designated by a control signal supplied from the distribution control unit 105. The translated text information is supplied to the subtitle generation unit 104. As a technology for translating text information into another language, a technology provided by Google Inc. or Microsoft Corporation can be adopted.

  For example, the text translation unit 108 can translate English text information into Japanese, and translate Japanese text information into English text information. The language before translation and the language after translation are not particularly limited. When the language after translation is designated by the control signal supplied from the distribution control unit 105, the text translation unit 108 translates the input text information into the designated language.

  The subtitle generation unit 104 selects one of the text information supplied from the speech / text conversion unit 107 and the translated text information supplied from the text translation unit 108 based on the subtitle display mode information. When the subtitle display mode information includes information for specifying a subtitle display mode other than the information for specifying the language, the subtitle generation unit 104 determines the subtitle display mode based on the information included in the subtitle display mode information. The subtitle information generated by the subtitle generation unit 104 in this manner is supplied to the distribution control unit 105.

  The distribution control unit 105 superimposes subtitle information on the input video information to generate subtitled content information, and distributes subtitled content information to the viewing client 30.

  The subtitles displayed by the viewing client 30 may be set manually. The setting information set in the view client 30 is supplied to the distribution control unit 105. The delivery control unit 105 supplies a control signal to the subtitle generation unit 104 for controlling to generate subtitle information set manually when the subtitle generation unit 104 generates subtitle information. Further, the delivery control unit 105 supplies the text translation unit 108 with a control signal for controlling to translate the text information into the manually set subtitle information.

  When the text information converted by the voice / text conversion unit 107 is different from the language set in the viewing client 30, the text translation unit 108 translates the text information into the language specified by the control signal to generate translated text information. Just do it.

  The operation of the server 10 which is the subtitled content information distribution apparatus of the present embodiment and the processing executed by the subtitled content information distribution method of the present embodiment will be described using the flowchart shown in FIG.

  In FIG. 3, the server 10 determines whether or not the subtitle setting in the viewing client 30 is automatic based on the setting information supplied from the viewing client 30 in step S1. If the subtitle setting is automatic (YES), the server 10 calculates the attribute information of the viewer based on the photographed image supplied from the viewing client 30 in step S2.

  The server 10 determines in step S3 whether or not the attribute information of the viewer has been calculated. If the attribute information of the viewer can be calculated (YES), the server 10 determines whether or not the ratio of Asians among the plurality of viewers is a predetermined ratio or more in step S4. The predetermined ratio may be appropriately set to, for example, 50%.

  If the percentage of Asians is equal to or greater than the predetermined percentage (YES), the server 10 selects Japanese subtitles in step S5, and shifts the processing to step S7. If the percentage of Asians is not the predetermined percentage or more (NO), the server 10 selects English subtitles in step S6 and shifts the processing to step S8.

  In addition, although the example which determines the ratio of Asian as a ratio of a race and selects Japanese subtitles or English subtitles was demonstrated, the ratio of a race other than Asians was determined and Japanese and English You may select subtitles other than subtitles. For example, Swahili may be selected if the ratio of blacks is greater than a predetermined ratio, and English subtitles may be selected if the ratio of blacks is not greater than a predetermined ratio. In short, it may be determined whether a predetermined ratio of races is equal to or greater than a predetermined ratio, and in accordance with the determination, a predetermined subtitle may be selected.

  When the process proceeds from step S4 to step S5, the server 10 continues to step S5, and determines in step S7 whether the ratio of children is equal to or more than a predetermined ratio. Also here, the predetermined ratio may be set appropriately, for example, 50%. If the percentage of children is equal to or greater than the predetermined percentage (YES), the server 10 selects a subtitle with many hiragana characters in step S11, and ends the processing.

  If the percentage of children is not the predetermined percentage or more in step S7 (NO), or if English subtitles are selected in step S6, the server 10 determines that the percentage of old people is the predetermined percentage or more in step S8. Determine if there is. Also here, the predetermined ratio may be set appropriately, for example, 50%.

  If the ratio of old people is equal to or higher than the predetermined ratio (YES), the server 10 selects a subtitle with a large subtitle size in step S9, and sets the subtitle display time to be longer in step S10, and processing End the If the ratio of old people is not the predetermined ratio or more (NO), the server 10 ends the process as it is.

  On the other hand, if the subtitle setting is not automatic in step S1 (NO), and if the attribute information of the viewer can not be calculated in step S3 (NO), the server 10 starts the viewing client 30 in step S12. Select the set subtitles and end the process.

  If the attribute information of the viewer is only the race, steps S7 to S11 in FIG. 3 are unnecessary. When the attribute information of the viewer is only the age group, steps S4 to S6 in FIG. 3 are unnecessary. Only one of step S9 and step S10 in FIG. 3 may be used.

  The photographed image reception unit 101 to the text translation unit 108 of the server 10 shown in FIG. 2 may be configured by hardware, may be configured by software (computer program), or may be configured to be a mixture of both. Good.

  FIG. 4 shows a schematic configuration in the case where the functional configuration of the server 10 is realized by a computer program. As shown in FIG. 4, the server 10 includes a computer 110. The storage unit 112, which is a non-temporary storage medium, stores a computer program (content information distribution program with subtitles).

  The central processing unit (CPU) 111 can execute the processing shown in FIG. 3 by executing the subtitled content information distribution program. The subtitled content information distribution program may be provided to the computer 110 via a communication line such as the Internet 11 or the like.

  The present invention is not limited to the embodiment described above, and various modifications can be made without departing from the scope of the present invention. The delivery control unit 105 adds subtitle information to the video information so as to superimpose the subtitle information on the video information in the viewing client 30, instead of generating subtitled content information in which the subtitle information is superimposed on the video information. Content information may be generated.

  That is, the distribution control unit 105 may generate subtitle-added content information in which the subtitle information is added to the video information regardless of whether the subtitle information is superimposed on the video information.

  If the distribution control unit 105 generates subtitled content information in a state where subtitle information is not superimposed on video information and distributes it to the viewing client 30, the viewer can appropriately select subtitle superposition and non-superimposition. It becomes. The distribution control unit 105 may add subtitle information as additional information to a stream for distributing video information and distribute it.

  Alternatively, the distribution control unit 105 may separately distribute the video information and the subtitle information without adding the video information to the stream to be distributed. In that case, the video information and the subtitle information may be distributed through the same transmission path, or the video information and the subtitle information may be distributed through different transmission paths. Then, the video information and the subtitle information may be superimposed in the viewing client 30.

  In the configuration shown in FIG. 2, the speech / text conversion unit 107 is provided inside the server 10, but it is also possible to convert speech information into text information using an external speech / text conversion device. Further, in the configuration shown in FIG. 2, the text translation unit 108 is provided inside the server 10, but it is also possible to generate translated text information using an external text translation device.

10 Server 11 Internet 20 Content Information Transmission Device 30 Viewing Client 101 Photographed Image Reception Unit 102 Viewer Attribute Analysis Unit 103 Subtitle Display Mode Selection Unit 104 Subtitle Generation Unit 105 Distribution Control Unit 106 Content Reception Unit 107 Voice / Text Conversion Unit 108 Text Translation Part 110 Computer 111 Central Processing Unit (CPU)
112 storage unit

Claims (5)

A content receiving unit that receives the content information transmitted from the content information transmitting apparatus;
A voice-to-text conversion unit that converts voice information in a first language included in the content information into first text information in a first language;
A text translation unit which translates the first text information into second text information in a second language;
A viewer attribute analysis unit that analyzes the race of the viewer and generates attribute information of the viewer based on a captured image obtained by capturing a viewer who views content information in a viewing client;
A subtitle display mode selection unit that generates subtitle display mode information that specifies a language of subtitles to be superimposed on video information included in the content information, based on the attribute information;
A subtitle generation unit which selects any one of the first text information and the second text information based on the subtitle display mode information, and generates subtitle information to be superimposed on the video information;
A distribution control unit that adds the subtitle information to the video information to generate subtitled content information, and distributes the information to the viewing client;
A subtitled content information distribution apparatus comprising: The viewer attribute analysis unit analyzes races of the plurality of viewers based on captured images obtained by capturing a plurality of viewers who view content information in the viewing client, and the viewer attribute analyzing unit analyzes the races of the plurality of viewers. Generate attribute information indicating the percentage of a given race,
The subtitle-added content information distribution device according to claim 1, wherein the subtitle display mode selection unit specifies a language of subtitles to be superimposed on the video information according to the ratio of the predetermined race. The viewer attribute analysis unit analyzes the age in addition to the race of the viewer based on the photographed image, and generates attribute information of the viewer.
The subtitle display mode selection unit specifies, based on the attribute information, a language of subtitles to be superimposed on the video information, and generates subtitle display mode information for selecting subtitles of an aspect according to the age group of the viewer. ,
The subtitle generation unit selects any one of the first text information and the second text information based on the subtitle display mode information, and the aspect according to the age group of the viewer The subtitled content information distribution apparatus according to claim 1 or 2, characterized in that subtitle information is generated. Receiving content information transmitted from the content information transmitting apparatus;
Converting voice information in a first language included in the content information into first text information in a first language;
Translating the first text information into second text information in a second language;
Based on a photographed image obtained by photographing a viewer who views content information in a viewing client, the race of the viewer is analyzed to generate attribute information of the viewer.
Subtitle display mode information for specifying a language of subtitles to be superimposed on video information included in the content information based on the attribute information;
One of the first text information and the second text information is selected based on the subtitle display mode information, and subtitle information to be superimposed on the video information is generated.
The subtitled content information is generated by adding the subtitle information to the video information to generate subtitled content information, and the information is distributed to the viewing client. On the computer
Receiving content information transmitted from the content information transmitting device;
Converting voice information in a first language included in the content information into first text information in a first language;
Translating the first text information into second text information in a second language;
Analyzing a race of the viewer based on a captured image obtained by capturing a viewer watching content information in a viewing client, and generating attribute information of the viewer;
Generating subtitle display mode information that specifies a language of subtitles to be superimposed on video information included in the content information, based on the attribute information;
Selecting any one of the first text information and the second text information based on the subtitle display mode information, and generating subtitle information to be superimposed on the video information;
Adding the subtitle information to the video information to generate subtitled content information and distributing it to the viewing client;
A subtitled content information distribution program characterized by performing.

Images