JP6950379B2 - Content information distribution device with subtitles, content information distribution method with subtitles, and content information distribution program with subtitles - Google Patents

Description

The present invention relates to a subtitled content information distribution device that distributes subtitled content information to a viewing client, a subtitled content information distribution method, and a subtitled content information distribution program.

Patent Document 1 describes a subtitle distribution system that selects subtitles corresponding to the reproduced video according to the attributes of the viewer and distributes them to the viewing client.

Japanese Unexamined Patent Publication No. 2014-20002 JP-A-2007-94840 Japanese Unexamined Patent Publication No. 2016-161830

According to the subtitle distribution system described in Patent Document 1, the viewer has to set the type of subtitle, which is complicated. Further, according to the subtitle distribution system described in Patent Document 1, it is not possible to distribute appropriate subtitles when a plurality of viewers view a video on one viewing client.

The present invention is a content information distribution device with subtitles, which can automatically select appropriate subtitles according to the viewer who views the content information on the viewing client and distribute the content information with subtitles to the viewing client. An object of the present invention is to provide a content information distribution method and a content information distribution program with subtitles.

The present invention converts a content receiving unit that receives content information transmitted from a content information transmitting device and audio information composed of a first language included in the content information into first text information composed of a first language. A voice / text conversion unit that translates the first text information into a second text information composed of a second language, and a shooting of a plurality of viewers viewing the content information on the viewing client. before Symbol analyzing a plurality of viewers each race and age, viewer attribute analysis unit generates attribute information including the race and age of the plurality of viewers based on the image, the human out of the attribute information The language of the subtitle to be superimposed on the video information included in the content information is specified based on the information indicating the species, and the subtitle to be superimposed on the video information is displayed based on the information indicating the age group of the attribute information. A subtitle display mode selection unit that generates subtitle display mode information that determines the length of time, and one of the first text information and the second text information based on the subtitle display mode information. A subtitle generator that generates subtitle information to be selected and superimposed on the video information so that the subtitles are displayed at a subtitle display time determined according to the age group, and a subtitle by adding the subtitle information to the video information. per generates content information, to provide subtitles with content information distribution apparatus Ru and a distribution control section for distributing the viewing client.

The present invention receives the content information transmitted from the content information transmitting device, converts the audio information in the first language included in the content information into the first text information in the first language, and the above-mentioned first. one of text information translated into a second text information consisting of the second language, each of the race before Symbol plurality of viewers based on the captured image obtained by photographing a plurality of viewers to view the content information in the view client And age are analyzed , attribute information including the race and age group of the plurality of viewers is generated, and the subtitles superimposed on the video information included in the content information based on the information indicating the race among the attribute information. Based on the information indicating the age group of the attribute information, the subtitle display mode information for determining the length of the display time of the subtitle to be superimposed on the video information is generated, and the subtitle display mode information is used as the subtitle display mode information. Based on this, either one of the first text information and the second text information is selected, and the video information is displayed so that the subtitles are displayed at the subtitle display time determined according to the age group. generates subtitle information to be superimposed, the added the subtitle information on the video information to generate a captioned content information, provide subtitles with content information distribution method delivers to the viewing client.

The present invention comprises a step of receiving content information transmitted from a content information transmitting device to a computer, and converting audio information in a first language included in the content information into first text information in a first language. The step of converting, the step of translating the first text information into the second text information in the second language , and the previous step based on the captured images of a plurality of viewers viewing the content information on the viewing client. serial analyzes a plurality of viewers each race and age, and generating the attribute information including the race and age of the plurality of viewers, based on the information indicating the race of the attribute information, wherein A subtitle display mode in which the language of the subtitles superimposed on the video information included in the content information is specified, and the length of the display time of the subtitles superimposed on the video information is determined based on the information indicating the age group of the attribute information. A subtitle determined according to the age group by selecting either one of the first text information and the second text information based on the step of generating the information and the subtitle display mode information. A step of generating subtitle information superimposed on the video information so that the subtitles are displayed at the display time, and a step of adding the subtitle information to the video information to generate content information with subtitles and distributing the content information to the viewing client. to provide a Ru allowed to run subtitles with content information distribution program.

According to the content information distribution device with subtitles, the content information distribution method with subtitles, and the content information distribution program with subtitles of the present invention, an appropriate subtitle is automatically selected according to the viewer viewing the content information on the viewing client. Therefore, the content information with subtitles can be distributed to the viewing client.

It is a figure which shows the schematic structure of the content information distribution system with subtitles which is configured including the content information distribution device with subtitles of one Embodiment. It is a block diagram which shows the specific structure of the content information distribution apparatus with subtitles of one Embodiment. It is a flowchart showing the operation of the content information distribution device with subtitles of one embodiment, the process executed by the content information distribution method with subtitles of one embodiment, and the process of causing the computer to execute the content information distribution program with subtitles of one embodiment. .. It is a block diagram which shows schematicly the server which includes the computer which executes the content information distribution program with subtitles of one Embodiment.

Hereinafter, the subtitled content information distribution device, the subtitled content information distribution method, and the subtitled content information distribution program of one embodiment will be described with reference to the attached drawings.

First, with reference to FIG. 1, a schematic configuration and operation of a subtitled content information distribution system including a subtitled content information distribution device will be described. In FIG. 1, a server 10 is connected to the Internet 11. The server 10 is a so-called cloud server that is used via the Internet 11 and is provided in a cloud environment.

A storage device 21 and a video camera 22 capable of network communication are connected to the Internet 11. The storage device 21 and the video camera 22 are examples of a content information transmitting device that transmits content information including video / audio information. The content information transmitting device may be only the storage device 21 or only the video camera 22.

The storage device 21, the video camera 22, or any other content information transmitting device will be referred to as a content information transmitting device 20. The content information transmission device 20 transmits content information to the server 10 via the Internet 11.

As will be described later, the server 10 superimposes subtitles on the video information included in the content information and distributes the subtitles to the viewing client. The server 10 constitutes the content information distribution device with subtitles of the present embodiment.

A television receiver 31 and a personal computer 32 capable of network communication are connected to the Internet 11, respectively. The television receiver 31 and the personal computer 32 are examples of viewing clients that receive subtitled content information distributed from the server 10 and display it on the display units 310 and 320, respectively.

The viewing client may be only the television receiver 31 or only the personal computer 32. The viewing client may be a projector device that projects the received content information with subtitles on the screen. A screen on which video light is projected is also included in the display unit. The television receiver 31, the personal computer 32, the projector device, or any other viewing client will be referred to as the viewing client 30.

An external camera 311 is attached to the television receiver 31. A plurality of viewers may view the content information with subtitles displayed on the display unit 310 of the television receiver 31. The camera 311 captures a plurality of viewers. The television receiver 31 transmits the captured images taken by the camera 311 to a plurality of viewers to the server 10.

The television receiver 31 transmits the setting information set in the television receiver 31 to the server 10. The camera 311 may be built in the housing of the television receiver 31.

The personal computer 32 has a built-in camera 321. Similarly, a plurality of viewers may view the subtitled content information displayed on the display unit 320 of the personal computer 32. The camera 321 captures a plurality of viewers. The personal computer 32 transmits the captured images taken by the camera 321 to a plurality of viewers to the server 10.

The personal computer 32 transmits the setting information set in the personal computer 32 to the server 10. The camera 321 may be an external camera separate from the personal computer 32.

When the viewing client is a projector device, dozens or 100 or more viewers can view the subtitled content information projected on the screen. In this case, dozens or 100 or more viewers may be photographed by the camera, and the projector device may transmit the photographed image to the server 10.

A specific configuration and operation of the server 10 will be described with reference to FIG. In FIG. 2, the captured image receiving unit 101 receives the captured image transmitted from the viewing client 30. As described above, the captured image is an image captured by a plurality of viewers. The captured image may be an image captured by one viewer. The captured image is supplied to the viewer attribute analysis unit 102.

The viewer attribute analysis unit 102 analyzes the attributes of each of the plurality of viewers based on the captured image. The attributes of the viewer are at least the race of the viewer, preferably race and age. In this embodiment, the viewer attribute analysis unit 102 analyzes race and age.

The technique described in Patent Document 2 can be adopted for the analysis of the race of the viewer. The technique described in Patent Document 3 can be adopted for the analysis of the age of the viewer. The analysis technology of attributes such as race or age is provided by, for example, Google Inc. (Alphabet Inc.), Microsoft Corporation, Kairos AR, Inc., and the technology provided by these companies can also be adopted.

The viewer attribute analysis unit 102 analyzes the attributes of each of the plurality of viewers, and generates attribute information indicating the ratio of a predetermined race and the ratio of a predetermined age group among the plurality of viewers as a whole. The race may be broadly classified into Asians, Westerners, whites, blacks, and the like. The age group may be broadly classified into age groups such as children, adults excluding the elderly, and the elderly. The attribute information analyzed by the viewer attribute analysis unit 102 is supplied to the subtitle display mode selection unit 103.

If the captured image is an image of one viewer, the viewer attribute analysis unit 102 analyzes the race and age of the one viewer and generates attribute information.

The subtitle display mode selection unit 103 determines the subtitle display mode when it is superimposed on the video information and displayed by the viewing client 30 according to the input attribute information, and supplies the subtitle display mode information to the subtitle generation unit 104. .. The subtitle display mode information is at least information that specifies the language of the subtitle. The subtitle display mode information preferably includes information for selecting subtitles in a mode according to the age group of the viewer, in addition to the information for specifying the language.

The mode of subtitles according to the viewer's age group is one of the size of the subtitle, the number of Chinese characters included in the subtitle, the display time of the subtitle, the maximum number of characters or the maximum number of lines of the subtitle to be displayed at one time. Or any combination of these.

The content receiving unit 106 receives the content information transmitted from the content information transmitting device 20. The video / audio information included in the content information is supplied to the distribution control unit 105. Of the video / audio information, the audio information is supplied to the audio / text conversion unit 107. The voice / text conversion unit 107 converts the voice information composed of the first language into the text information (first text information) composed of the first language. The text information is supplied to the subtitle generation unit 104 and the text translation unit 108.

As a technology for converting voice information into text information, a technology provided by Google Inc. or Microsoft Corporation can be adopted.

The text translation unit 108 translates the input text information consisting of the first language into a second language different from the first language, and generates translated text information (second text information). Which language the text translation unit 108 translates the input text information into may be specified by a control signal supplied from the distribution control unit 105. The translated text information is supplied to the subtitle generation unit 104. As a technology for translating text information into other languages, a technology provided by Google Inc. or Microsoft Corporation can be adopted.

For example, the text translation unit 108 can translate English text information into Japanese, and can translate Japanese text information into English text information. The language before translation and the language after translation are not particularly limited. When the translated language is specified by the control signal supplied from the distribution control unit 105, the text translation unit 108 translates the input text information into the specified language.

The subtitle generation unit 104 selects either the text information supplied from the voice / text conversion unit 107 or the translated text information supplied from the text translation unit 108 based on the subtitle display mode information. When the subtitle display mode information includes information that specifies the display mode of the subtitle other than the information that specifies the language, the subtitle generation unit 104 determines the subtitle display mode based on the information included in the subtitle display mode information. The subtitle information generated by the subtitle generation unit 104 in this way is supplied to the distribution control unit 105.

The distribution control unit 105 superimposes the subtitle information on the input video information to generate the subtitled content information, and distributes the subtitled content information to the viewing client 30.

There are times when the subtitles displayed by the viewing client 30 are set manually. The setting information set by the viewing client 30 is supplied to the distribution control unit 105. When the subtitle generation unit 104 generates the subtitle information, the distribution control unit 105 supplies the subtitle generation unit 104 with a control signal for controlling the generation of the manually set subtitle information. Further, the distribution control unit 105 supplies the text translation unit 108 with a control signal for controlling the translation of the text information into the manually set subtitle information.

When the text information converted by the voice / text conversion unit 107 is different from the language set by the viewing client 30, the text translation unit 108 translates the text information into the language specified by the control signal to generate the translated text information. Just do it.

Using the flowchart shown in FIG. 3, the operation of the server 10 which is the subtitled content information distribution device of the present embodiment and the processing executed by the subtitled content information distribution method of the present embodiment will be described.

In FIG. 3, the server 10 determines in step S1 whether or not the subtitle setting in the viewing client 30 is automatic based on the setting information supplied from the viewing client 30. If the subtitle setting is automatic (YES), the server 10 calculates the attribute information of the viewer based on the captured image supplied from the viewing client 30 in step S2.

The server 10 determines in step S3 whether or not the attribute information of the viewer can be calculated. If the attribute information of the viewers can be calculated (YES), the server 10 determines in step S4 whether or not the ratio of Asians among the plurality of viewers is equal to or higher than the predetermined ratio. The predetermined ratio may be appropriately set, for example, 50%.

If the proportion of Asians is equal to or greater than the predetermined proportion (YES), the server 10 selects Japanese subtitles in step S5 and shifts the process to step S7. If the percentage of Asians is not greater than or equal to the predetermined percentage (NO), the server 10 selects English subtitles in step S6 and shifts the process to step S8.

In addition, as an example of determining the ratio of Asians and selecting Japanese subtitles or English subtitles as the ratio of races, the example of selecting non-Asian subtitles was explained, but the ratio of races other than Asians was determined and Japanese and English Subtitles other than subtitles may be selected. For example, Swahili may be selected if the proportion of blacks is greater than or equal to a predetermined proportion, and English subtitles may be selected if the proportion of blacks is not greater than or equal to a predetermined proportion. In short, it may be determined whether or not the proportion of races determined in advance is equal to or greater than the predetermined proportion, and the predetermined subtitles may be selected according to the determination.

When shifting from step S4 to step S5, the server 10 determines in step S7 whether or not the proportion of children is equal to or greater than a predetermined proportion, following step S5. Here, too, the predetermined ratio may be appropriately set, for example, 50%. If the proportion of children is equal to or greater than a predetermined proportion (YES), the server 10 selects subtitles with many hiragana characters in step S11 and ends the process.

If the proportion of children is not equal to or greater than the predetermined proportion in step S7 (NO), or if English subtitles are selected in step S6, the server 10 determines that the proportion of elderly people is equal to or greater than the predetermined proportion in step S8. Determine if it exists. Here, too, the predetermined ratio may be appropriately set, for example, 50%.

If the proportion of elderly people is equal to or greater than a predetermined proportion (YES), the server 10 selects a subtitle with a large subtitle size in step S9, sets the subtitle display time to be longer in step S10, and processes it. To end. If the ratio of old people is not equal to or higher than a predetermined ratio (NO), the server 10 terminates the process as it is.

On the other hand, if the subtitle setting is not automatic in step S1 (NO), or if the attribute information of the viewer cannot be calculated in step S3 (NO), the server 10 is set by the viewing client 30 in step S12. Select the set subtitle and end the process.

If the attribute information of the viewer is only race, steps S7 to S11 in FIG. 3 are unnecessary. When the attribute information of the viewer is limited to the age group, steps S4 to S6 in FIG. 3 are unnecessary. Only one of step S9 and step S10 in FIG. 3 may be used.

The captured image receiving unit 101 to the text translation unit 108 of the server 10 shown in FIG. 2 may be configured by hardware, software (computer program), or a mixture of both. good.

FIG. 4 shows a schematic configuration when the functional configuration of the server 10 is realized by a computer program. As shown in FIG. 4, the server 10 includes a computer 110. The storage unit 112, which is a non-temporary storage medium, stores a computer program (content information distribution program with subtitles).

When the central processing unit (CPU) 111 executes the content information distribution program with subtitles, the computer 110 can execute the process shown in FIG. The subtitled content information distribution program may be provided to the computer 110 via a communication line such as the Internet 11.

The present invention is not limited to the present embodiment described above, and various modifications can be made without departing from the gist of the present invention. The distribution control unit 105 has subtitles by adding subtitle information to the video information in order to superimpose the subtitle information on the video information in the viewing client 30 instead of generating the content information with subtitles on which the subtitle information is superimposed on the video information. Content information may be generated.

That is, the distribution control unit 105 may generate the content information with subtitles in which the subtitle information is added to the video information regardless of whether or not the subtitle information is superimposed on the video information.

If the distribution control unit 105 generates content information with subtitles in a state where the subtitle information is not superimposed on the video information and distributes it to the viewing client 30, the viewer can appropriately select whether to superimpose the subtitles or not. It becomes. The distribution control unit 105 may add subtitle information as additional information to the stream for distributing the video information and distribute the video information.

Alternatively, the distribution control unit 105 may distribute the video information and the subtitle information separately without adding the video information to the stream to which the video information is distributed. In that case, the video information and the subtitle information may be distributed via the same transmission line, or the video information and the subtitle information may be distributed via different transmission lines. Then, the video information and the subtitle information may be superimposed on the viewing client 30.

In the configuration shown in FIG. 2, the voice / text conversion unit 107 is provided inside the server 10, but it is also possible to convert voice information into text information using an external voice / text conversion device. Further, in the configuration shown in FIG. 2, although the text translation unit 108 is provided inside the server 10, it is also possible to generate translated text information using an external text translation device.

10 Server 11 Internet 20 Content information transmitter 30 Viewing client 101 Captured image receiver 102 Viewer attribute analysis unit 103 Subtitle display mode selection unit 104 Subtitle generation unit 105 Distribution control unit 106 Content reception unit 107 Voice / text conversion unit 108 Text translation Part 110 Computer 111 Central processing unit (CPU)
112 Memory

Claims (5)

A content receiver that receives content information transmitted from the content information transmitter, and a content receiver
A voice / text conversion unit that converts voice information in a first language included in the content information into first text information in a first language, and
A text translation unit that translates the first text information into a second text information composed of a second language, and
It analyzes each race and age prior Symbol plurality of viewers based on the imaged image obtained by imaging a plurality of viewers to view the content information in the view client, attributes including race and age of the plurality of viewers Audience attribute analysis unit that generates information,
Based on the information indicating the race of the attribute information, the language of the subtitles superimposed on the video information included in the content information is specified, and the video information is based on the information indicating the age group of the attribute information. A subtitle display mode selection unit that generates subtitle display mode information that determines the length of the display time of the superposed subtitles,
Based on the subtitle display mode information, either one of the first text information and the second text information is selected, and the subtitle is displayed at the subtitle display time determined according to the age group. a caption generation unit which generates a caption information to be superimposed on so that the video information,
A distribution control unit that adds the subtitle information to the video information to generate content information with subtitles and distributes the content information to the viewing client.
Ru subtitles with content information distribution apparatus equipped with a. The viewer attribute analysis unit generates the attribute information indicating the percentage of a given race of the previous SL plurality of viewers,
The subtitle display mode selection unit specifies the language of the subtitle to be superimposed on the video information according to the ratio of the predetermined race.
Subtitled content information distribution apparatus according to請Motomeko 1. The viewer attribute analysis unit generates attribute information indicating the ratio of a predetermined age group among the plurality of viewers, and generates attribute information.
The caption display mode selection unit determines the length of display time of a subtitle to be superimposed on the video information in accordance with the ratio of the predetermined ages definitive on the attribute information
Subtitled content information distribution apparatus according to請Motomeko 1 or 2. Receives the content information transmitted from the content information transmitter and receives the content information.
The audio information in the first language included in the content information is converted into the first text information in the first language.
The first text information is translated into a second text information composed of a second language.
It analyzes each race and age prior Symbol plurality of viewers based on the imaged image obtained by imaging a plurality of viewers to view the content information in the view client, attributes including race and age of the plurality of viewers Generate information,
Based on the information indicating the race of the attribute information, the language of the subtitles superimposed on the video information included in the content information is specified, and the video information is based on the information indicating the age group of the attribute information. Generates subtitle display mode information that determines the length of the display time of the superposed subtitles,
Based on the subtitle display mode information, either one of the first text information and the second text information is selected, and the subtitle is displayed at the subtitle display time determined according to the age group. It generates subtitle information to be superimposed on so that the video information,
The subtitle information is added to the video information to generate the content information with subtitles, which is distributed to the viewing client.
Subtitles with content information delivery method. On the computer
The step of receiving the content information transmitted from the content information transmission device, and
A step of converting audio information composed of a first language included in the content information into first text information composed of a first language, and
The step of translating the first text information into the second text information consisting of the second language, and
It analyzes each race and age prior Symbol plurality of viewers based on the imaged image obtained by imaging a plurality of viewers to view the content information in the view client, attributes including race and age of the plurality of viewers Steps to generate information and
The language of the subtitles superimposed on the video information included in the content information is specified based on the information indicating the race of the attribute information, and the video information is based on the information indicating the age group of the attribute information. A step of generating subtitle display mode information that determines the length of the display time of the subtitles to be superimposed, and
Based on the subtitle display mode information, either one of the first text information and the second text information is selected, and the subtitle is displayed at the subtitle display time determined according to the age group. generating a caption information to be superimposed on so that the video information,
A step of adding the subtitle information to the video information to generate content information with subtitles and distributing the content information to the viewing client.
Ru allowed to run subtitles with content information distribution program.

Images