JP6401505B2 - Sign language CG generation apparatus and program thereof - Google Patents

Description

  The present invention relates to a sign language CG generation apparatus that generates a sign language CG and a program thereof.

  For congenital deaf people, sign language is a native language and information presentation in sign language is important, but there are very few broadcast programs with sign language translation. The reason is the problem of securing sign language interpreters. The absolute number of sign language interpreters is small, and it is very difficult to secure resident sign language interpreters who can handle special news programs and night news programs in addition to ordinary broadcast programs. However, such highly urgent programs are thought to require barrier-free information presentation.

  Regarding such problems, an invention has been proposed in which automatic translation from Japanese text into sign language and sign language animation using a 3DCG model are automatically generated by a computer (see Patent Document 1).

  In the invention described in Patent Literature 1, the result of language translation from Japanese to sign language notation is used to connect the motion of sign language words corresponding to each sign language notation (operation data of each sign language word acquired by optical motion capture). The sign language animation is generated by letting the character perform the motion.

JP 2011-175598 A

  However, in the invention described in Patent Document 1, in each word of the sign language animation output as the translation result, the facial expression when the person is motion-captured is used as the facial expression of the character performing the sign language animation. It is not always an appropriate expression that matches the content of For example, motion capture data of a person with a happy expression may be used in a news program in which a celebrity has died. As a result, even though a sad expression is a suitable news program, a character performing sign language animation has a pleasant expression.

  It is an object of the present invention to provide a sign language CG generation apparatus that generates a sign language CG with a character having a facial expression that matches the content, and a program therefor.

  In view of the above-described problems, a sign language CG generation device according to the present invention is a sign language CG generation device that generates a sign language CG with a character with a facial expression, and includes a word extraction unit, a first database, an impression value determination unit, And a second database, a character / facial expression determination unit, and a sign language CG generation unit.

According to this configuration, the sign language CG generation device extracts the first word of the content item name of the data broadcast content or web content selected by the viewer by the word extraction unit. In addition, the sign language CG generation device stores, in advance, first word-impression data in which the first word is associated with an impression value representing the strength of the impression received by the viewer from the first word. . Then, in the sign language CG generation device, the impression value determination unit refers to the first word-impression data and determines an impression value corresponding to the first word extracted by the word extraction unit.
In this way, the impression received from the content is determined.

  The sign language CG generation apparatus stores in advance impression-character-expression data in which the impression value, the character type, the expression type of the character, and the expression value representing the strength of the expression are associated with each other by the second database. . In the sign language CG generation device, the character / expression determination unit refers to the impression-character-expression data and determines the character type and expression value corresponding to the impression value determined by the impression value determination unit. The sign language CG generation device generates a sign language CG based on the character type and facial expression determined by the character / expression determination unit by the sign language CG generation unit.

  In the present invention, an impression is determined from the content, and a sign language CG is generated with a character having a facial expression based on the impression. Thus, according to the present invention, the expression of the character can be matched with the content, so that the sign language CG can be less uncomfortable.

It is a block diagram which shows the structure of the broadcast system which concerns on 1st Embodiment of this invention. It is explanatory drawing explaining the text data analysis part of FIG. It is explanatory drawing explaining the data structure of various data, (a) is content item name-impression data, (b) is word-impression data, (c) is impression-character-expression data. It is explanatory drawing of the character imitating a girl, (a) represents an expressionless expression, (b) represents a sad expression. It is explanatory drawing of the character imitating a man, (a) represents an expressionless expression, (b) represents the happy expression. It is explanatory drawing of an emergency character. It is explanatory drawing explaining the 1st example of the facial expression determination method of a character. It is explanatory drawing explaining the 2nd example of the facial expression determination method of a character. It is explanatory drawing explaining the 3rd example of the facial expression determination method of a character. It is a flowchart which shows operation | movement of the digital broadcast receiver of FIG. It is a block diagram which shows the structure of the broadcast system which concerns on 2nd Embodiment of this invention. It is a block diagram which shows the structure of the broadcast system which concerns on 3rd Embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings as appropriate. In each embodiment, means having the same function are denoted by the same reference numerals and description thereof is omitted.

(First embodiment)
A broadcasting system 100 according to a first embodiment of the present invention will be described with reference to FIG.
As shown in FIG. 1, the broadcast system 100 provides viewers with broadcast video and sign language CG representing the content of the broadcast video. The broadcast station 1 and a digital broadcast receiver (sign language CG generation device). 2 and a display 3.

[Broadcasting station configuration]
The broadcasting station 1 performs digital broadcasting such as terrestrial digital broadcasting and satellite digital broadcasting, and transmits data necessary for generating sign language CG to the digital broadcasting receiver 2. As shown in FIG. 1, the broadcasting station 1 includes a multiplexing unit 11, a modulator 13, and a data transmission unit 15.

The multiplexing unit 11 receives broadcast video, data broadcast, and caption data from the outside, multiplexes the input broadcast video, data broadcast, and caption data in a transport stream (MPEG2-TS) format. Is output.
In addition, since the subtitle data is not input when it is not a broadcast program with subtitles, the multiplexing unit 11 multiplexes only the broadcast video and the data broadcast in the transport stream format.

  The modulator 13 receives a transport stream from the multiplexing unit 11, modulates the input transport stream, and sends it to the broadcast receiver 2 as a digital broadcast wave.

  The data transmission unit 15 receives database update data (content item name-impression data, word-impression data, impression-character-expression data) necessary for generating sign language CG from the outside, and digitally broadcasts the various types of input data This is transmitted to the receiver 2. Here, the data transmission unit 15 transmits the database update data to the digital broadcast receiver 2 via the network N.

[Configuration of digital broadcast receiver]
The digital broadcast receiver 2 receives a digital broadcast wave transmitted from the broadcast station 1 and displays a broadcast video and a sign language CG representing the content of the broadcast video on the display 3.

  As shown in FIG. 1, the digital broadcast receiver 2 includes a demodulator 21, a demultiplexing unit 22, a text data analysis unit 23, a text-impression database 24, a content impression extraction unit 25, and an impression-character-expression. A database 26, a character / expression determining unit 27, a sign language notation translation unit 28, and a sign language CG generation unit 29 are provided.

  Here, the text data analysis unit 23 corresponds to the “word extraction unit” in the claims, the text-impression database 24 corresponds to the “first database” in the claims, and the content impression extraction unit 25 patents The impression-character-expression database 26 corresponds to the “impression value determination unit” in the claims, and the “second database” in the claims.

  The demodulator 21 receives a digital broadcast wave from the broadcast station 1 and demodulates the received digital broadcast wave as a transport stream. Further, the demodulator 21 outputs the demodulated transport stream to the demultiplexing unit 22.

  The demultiplexing unit 22 receives the transport stream from the demodulator 21 and separates the broadcast video, the data broadcast, and the caption data multiplexed in the input transport stream. The demultiplexing unit 22 also outputs the separated data broadcast and caption data to the text data analysis unit 23 and the sign language notation translation unit 28. Further, the demultiplexing unit 22 outputs the separated broadcast video to the sign language CG generation unit 29.

Note that the demultiplexing unit 22 outputs only the broadcast video and the data broadcast since the subtitle data is not included in the transport stream when the subtitle data is not a subtitle broadcast program.
Further, since a technique for decoding and descrambling broadcast video is common, a description thereof will be omitted.

  The text data analysis unit 23 extracts the first word included in the content item name of the data broadcast content selected by the viewer. Then, the text data analysis unit 23 outputs the extracted first word to the content impression extraction unit 25.

<Extraction of the first word>
The extraction of the first word by the text data analysis unit 23 will be described with reference to FIG. 2 (see FIG. 1 as appropriate).
Assume that the digital broadcast receiver 2 displays the data broadcast screen 30 as shown in FIG. On this data broadcast screen 30, broadcast video content 31 and data broadcast content 32 are displayed.

The broadcast video content 31 is video content such as “XX News”, for example.
The data broadcast content 32 is content provided by data broadcast, and is a button or title (heading) displayed on the data broadcast screen 30.
For example, the data broadcast content 32A is data broadcast content with high urgency related to a disaster, and is displayed with a content item name “tsunami warning”. The data broadcast content 32A is hatched to indicate that it is highly urgent.
In addition, for example, the data broadcast content 32B is data broadcast content related to sports, and is displayed with a content item name of “Senbat △△ Loses in the first game”.
Further, for example, the data broadcast content 32C is data broadcast content related to news, and is displayed with a content item name “What is sold before the consumption tax increase”.

  Here, the digital broadcast receiver 2 can select the data broadcast content 32 that the viewer is interested in by operating a remote controller (not shown) on the data broadcast screen 30. Then, the digital broadcast receiver 2 displays details (not shown) of the selected data broadcast content 32 on the data broadcast screen 30. At this time, the text data analysis unit 23 receives identification information (for example, button ID and title ID) of the data broadcast content 32 selected by the viewer via a remote control unit (not shown). Therefore, the text data analysis unit 23 acquires the data broadcast content 32 corresponding to the input identification information from the data broadcast separated by the demultiplexing unit 22.

  And the text data analysis part 23 extracts the content item name itself of the acquired data broadcast content 32 as a 1st word. For example, when the data broadcast content 32A is selected, the text data analysis unit 23 extracts “tsunami warning” of the content item name of the data broadcast content 32A. In addition, when the data broadcast content 32B is selected, the text data analysis unit 23 extracts “Senbat △ Δ Lose in the first game” of the content item name of the data broadcast content 32B as the first word. Then, the text data analysis unit 23 outputs the extracted content item name (first word) to the content impression extraction unit 25.

  In this case, the content item name itself “Senbatsu Δ △ lost in the first game” is not registered in the content item name-impression data 40 described later. For this reason, the content impression extraction part 25 mentioned later cannot determine the impression value corresponding to content item name itself (1st word). Therefore, the text data analysis unit 23 receives from the content impression extraction unit 25 a content item name unregistered notification indicating that the impression value could not be determined from the content item name itself.

  In response to the content item name unregistered notification, the text data analysis unit 23 performs morphological analysis on the acquired content item name of the data broadcast content 32 and sets the word included in the content item name of the data broadcast content 32 as the first word. Extract as For example, when the data broadcast content 32B is selected, the text data analysis unit 23 loses “senbat”, “ΔΔ”, “first game”, “de” and “defeat” included in the content item name of the data broadcast content 32B as the first word. Is extracted. Then, the text data analysis unit 23 outputs the word included in the content item name to the content impression extraction unit 25.

  Furthermore, the word included in the content item name may not be registered in the content item name-impression data 40. Also in this case, the content impression extraction unit 25 cannot determine the impression value. Therefore, the text data analysis unit 23 receives an impression value undetermined notification indicating that the impression value corresponding to the word included in the content item name could not be determined from the content impression extraction unit 25.

In response to the impression value undetermined notification, the text data analysis unit 23 extracts the second word included in the text data of the data broadcast content 32 selected by the viewer. Specifically, the text data analysis unit 23 extracts the text data of the data broadcast content 32 selected by the viewer from the data broadcast input from the demultiplexing unit 22. Then, the text data analysis unit 23 performs morphological analysis on the extracted text data to extract the second word. Thereafter, the text data analysis unit 23 outputs the extracted second word to the content impression extraction unit 25.
Note that, in the case of a broadcast program with captions, the text data analysis unit 23 performs the same processing on the caption data input from the demultiplexing unit 22.

  The text-impression database 24 is a database that stores content item name-impression data 40 and word-impression data 42 in advance (FIG. 3). The text-impression database 24 updates the content item name-impression data 40 and the word-impression data 42 using the database update data received from the broadcasting station 1.

  Here, the content item name-impression data 40 corresponds to “first word-impression data” in the claims, and the word-impression data 42 corresponds to “second word-impression data” in the claims. .

<Content item name-impression data>
The content item name-impression data 40 will be described with reference to FIG. 3A (see FIGS. 1 and 2 as appropriate).
The content item name-impression data 40 is data in which the first word 401 and the impression value 403 are associated with each other.
The first word 401 corresponds to a word included in the content item name of the data broadcast content 32.
The impression value 403 represents the strength of the impression received by the viewer from the first word 401.

  When the first word 401 is “losing” or “losing”, the impression value 403 is “dark 0.3”. Of the impression value 403 of “dark 0.3”, the word portion of “dark” represents the type of impression. Also, out of the impression value 403 of “dark 0.3”, the numerical value portion of “0.3” is obtained by quantifying the strength of the impression. The numerical value portion of the impression value 403 ranges from “0.0” indicating that the impression is weakest to “1.0” indicating that the impression is strongest.

Note that these two content item name-impression data 40 can be generated using the text impression mining technique described in Reference Document 1 below.
Reference 1: Tadahiko Kumamoto, Yukiko Kawai et al., “Design and Evaluation of Text Impression Mining Techniques for Newspaper Articles”, IEICE Transactions, Vol.J94-D No.3 pp.540-548

Since the first word 401 “tsunami warning” corresponds to the content item name of the data broadcast content 32 with high urgency, it is also called an urgent first word. Here, the first word 401 “tsunami warning” has an impression value 403 of “emergency 1.0”. This “emergency 1.0” is an impression value (emergency impression value) indicating that the urgency is high, and the numerical value portion is always “1.0”.
Note that the content item name-impression data 40 is manually generated instead of the text impression mining method described above.
Needless to say, the content item name-impression data 40 is not limited to the example of FIG.

<Word-impression data>
With reference to FIG. 3B, the word-impression data 42 will be described (see FIGS. 1 and 2 as appropriate).
The word-impression data 42 is data in which the second word 421 and the impression value 423 are associated with each other.
The second word 421 corresponds to a word extracted from text data or caption data of the data broadcast content 32.
Similar to the impression value 403 in FIG. 3A, the impression value 423 represents the strength of the impression received by the viewer from the second word 421.

For example, when the second word 421 is “lively”, the impression value 423 is “bright 0.2”.
Note that the word-impression data 42 can be generated using the text impression mining method described in the above-mentioned Reference Document 1.
Needless to say, the word-impression data 42 is not limited to the example of FIG.

Returning to FIG. 1, the description of the digital broadcast receiver 2 will be continued.
The content impression extraction unit 25 receives the first word (the content item name or the word included in the content item name) from the text data analysis unit 23, and calculates the impression value corresponding to the input first word as the content item name− This is determined with reference to the impression data 40.

  First, the content impression extraction unit 25 determines whether or not an impression value corresponding to the content item name input from the text data analysis unit 23 is registered in the content item name-impression data 40.

  When the content item name is registered, the content impression extraction unit 25 refers to the content item name-impression data 40 to determine an impression value corresponding to the content item name, and determines the determined impression value as a character / facial expression. To the unit 27. In other words, the content impression extraction unit 25 searches the content item name-impression data 40 using the input content item name as a search key, and acquires an impression value associated with the content item name as a search result.

  When the content item name is not registered, the content impression extraction unit 25 outputs a content item name unregistered notification to the text data analysis unit 23. Then, the content impression extraction unit 25 receives a word included in the content item name from the text data analysis unit 23. Then, the content impression extraction unit 25 determines whether or not an impression value corresponding to the word of the content item name is registered in the content item name-impression data 40.

  When the word of the content item name is registered, the content impression extraction unit 25 refers to the content item name-impression data 40 to determine an impression value corresponding to the word of the content item name, and uses the determined impression value. The data is output to the character / expression determining unit 27. In other words, the content impression extraction unit 25 searches the content item name-impression data 40 using the input word of the content item name as a search key, and uses the impression value associated with the word of the content item name as the search result. get.

  When the word of the content item name is not registered, the content impression extraction unit 25 outputs an impression value undetermined notification to the text data analysis unit 23. Then, the content impression extraction unit 25 receives the second word from the text data analysis unit 23.

  Then, the content impression extracting unit 25 refers to the word-impression data 42, determines an impression value corresponding to the second word, and outputs the determined impression value to the character / expression determining unit 27. In other words, the content impression extraction unit 25 searches the word-impression data 42 using the input second word as a search key, and acquires an impression value associated with the second word as a search result.

  The impression-character-expression database 26 is a database that stores in advance impression-character-expression data 44 to be described later (FIG. 3). The impression-character-expression database 26 updates the impression-character-expression data 44 using the database update data received from the broadcast station 1.

<Impression-Character-Facial expression data>
The impression-character-expression data 44 will be described with reference to FIG. 3C (refer to FIGS. 1 and 2 as appropriate).
The impression-character-expression data 44 is data in which an impression value 441, a character type 443, and an expression value 445 are associated with each other.

The impression value 441 is similar to the impression values 403 and 423.
The character type 443 represents the type of character, such as “Character A” or “Character B”. For example, “Character A” is a girl character as shown in FIG. “Character B” is a male character as shown in FIG.

  The expression value 445 represents the type of expression of the character and the strength of the expression. For example, of the facial expression value 445 of “sadness 0.3”, the word portion of “sadness” represents the type of facial expression of the character. The types of facial expressions of the character include Ekman's six facial expressions (joy, sadness, surprise, fear, anger, and disgust). Of the expression value 445 of “sadness 0.3”, the numerical value portion of “0.3” is obtained by quantifying the strength of the expression. The numerical value part of the expression value 445 is in a range from “0.0” indicating no expression to “1.0” indicating that the expression is strongest.

In the first impression-character-expression data 44, the impression value 441 “dark 0.3” corresponds to the character type 443 “character A” and the expression value 445 “sadness 0.3”. Indicates.
In the second impression-character-expression data 44, the impression value 441 of "dark 0.3" corresponds to the character type 443 of "character B" and the expression value 445 of "sadness 0.2". Indicates to do.

Even if the impression value 441 is the same value, since the impression received by the viewer differs depending on the expression level of each character, the expression value 445 is set to a different value.
Needless to say, the impression-character-expression data 44 is not limited to the example of FIG.

Here, a method of generating impression-character-expression data 44 will be described.
The impression-character-expression data 44 can be generated by determining which kind of character and which expression is appropriate for the impression value 441 by a subjective evaluation experiment. Specifically, for a sentence with an impression value 441 of “dark 0.3”, a sign language CG in which the expression is changed like “sadness 0.1”, “sadness 0.2”, etc. for each character is created. . Then, each sign language CG is shown to the subject, the impression of each character is evaluated by a questionnaire system, and the evaluation result is subjected to factor analysis / analysis to determine an appropriate facial expression value 445 for each character.

Returning to FIG. 1, the description of the digital broadcast receiver 2 will be continued.
The character / expression determining unit 27 receives the impression value from the content impression extracting unit 25 and determines the character type and expression value corresponding to the input impression value with reference to the impression-character-expression data 44. It is.

First, the character / expression determining unit 27 determines whether or not the input impression value matches a preset emergency impression value (that is, “emergency 1.0”).
If the input impression value matches the emergency impression value, the character / facial expression determination unit 27 outputs an emergency character notification to the sign language CG generation unit 29. The emergency character notification is a notification indicating that the determined character type and facial expression value are for emergency.

  If the input impression value does not match the emergency impression value, the character / expression determination unit 27 refers to the impression-character-expression data 44 to determine the character type and expression value corresponding to the impression value, and Output to the sign language CG generation unit 29. In other words, the character / expression determining unit 27 searches the impression-character-expression data 44 using the input impression value as a search key, and acquires the character type and expression value associated with the impression value as the search result. To do.

  The sign language notation translation unit 28 translates Japanese text into sign language notation. For example, the sign language notation translation unit 28 uses text data or subtitle data included in the data broadcast input from the demultiplexing unit 22 as Japanese text. Then, the sign language notation translation unit 28 translates the Japanese text into sign language notation using the sign language translation method described in Reference Documents 2 and 3 below, and outputs the translation result to the sign language CG generation unit 29.

Reference 2: Japanese Patent Application Laid-Open No. 2011-175598 Reference 3: Nobuyuki Hiruma, Toshihiro Shimizu et al., “Automatic Sign Language Animation Generation System Using CG”, Journal of the Institute of Image Electronics Engineers of Japan, Vol. 41, p.

The sign language CG generation unit 29 receives the character type and expression value from the character / expression determination unit 27 and generates a sign language CG based on the input character type and expression value. As illustrated in FIG. 1, the sign language CG generation unit 29 includes a sign language CG control command generation unit 291, a sign language animation generation unit 293, and a composite display unit 295.
The sign language CG generation unit 29 includes a CG data storage unit (not shown) that stores CG data of each character.

The CG data storage unit stores motion capture data and CG data of each character in advance.
The motion capture data is data representing the sign language action of the character generated by the motion capture.

Here, the CG data storage unit stores, for each character, no-expression CG data and CG data when Ekman's six expressions are strongest.
That is, for the “character A”, the CG data storage unit has no facial expression CG data as shown in FIG. 4A, “joy 1.0” CG data, “sadness 1.0” CG data, CG data of “surprise 1.0”, CG data of “anger 1.0”, and CG data of “disgust 1.0” are stored in total.
The CG data storage unit stores a total of seven CG data for “character B” having no expression as shown in FIG. 5A and CG data when Ekman's six expressions are strongest.
In addition, since the expression of the emergency character does not change, the CG data storage unit stores only one CG data of the emergency character as shown in FIG.

  The sign language CG control command generation unit 291 generates a control command for animating a character. In the present embodiment, the sign language CG control command generation unit 291 generates a TVML (TV program Making Language) script as a control command.

Specifically, the sign language CG control command generation unit 291 designates the character type and expression value input from the character / expression determination unit 27 using the “expression” command of the TVML script. For example, if the “expression” command is “character: expression (name = CharacterA, type = sad, degree = 0.3, time = 1.0);”, the type of character to be animated is “Character A” and its expression The value is “sadness 0.3”.
When an emergency character notification is input, the sign language CG control command generation unit 291 designates emergency as the character type and facial expression value.

  Subsequently, the sign language CG control command generation unit 291 designates motion capture data in which the same sign language operation as the translation result input from the sign language notation translation unit 28 is performed by the “openkeyframe” command of the TVML script. For example, if the “openkeyframe” command is “openkey character: openkeyframe (name = CharacterA, keyframename = 0001, filename =“ test.bvh ”)”, the motion capture data is “test.bvh”.

  Then, the sign language CG control command generation unit 291 generates a TVML script including an “expression” command and an “openkeyframe” command, and outputs the generated TVML script to the sign language animation generation unit 293.

In the TVML script, settings other than the character type and expression value can be set, but the description is omitted because it is not directly related to the present invention.
The details of TVML are described in Reference Document 4 below.
Reference 4: “TVML”, [online], [Search May 7, 2014], Internet <URL: http://www.nhk.or.jp/strl/tvml/japanese/player2/index.html >

  The sign language animation generation unit 293 receives a TVML script from the sign language CG control command generation unit 291 and generates a sign language animation according to the input TVML script.

First, the case where the character type is other than emergency is described.
The sign language animation generation unit 293 reads the expressionless CG data from the CG data storage unit for the type of character specified by the “expression” command. Furthermore, the sign language animation generation unit 293 reads from the CG data storage unit the CG data corresponding to the facial expression value specified by the “expression” command and having the strongest facial expression. Then, the sign language animation generation unit 293 synthesizes the expressionless CG data and the CG data when the expression is strongest at the ratio of the expression value specified by “degree” in the “expression” command. A general three-dimensional morphing technique can be used to synthesize the facial expression.

  For example, it is assumed that “character A” and “sadness 0.3” are designated by the “expression” command. In this case, the sign language animation generation unit 293 reads out CG data in which the character A has no expression and CG data in which the character A has the most sad expression (CG data of “sadness 1.0”). Then, the sign language animation generation unit 293 has a facial expression of “sadness 0.3” by synthesizing the CG data so that it approaches 30% of the CG data of “sadness 1.0” from the CG data of no expression. Character A can be synthesized.

  Then, the sign language animation generation unit 293 generates a sign language CG of the character synthesized with the facial expression using the motion capture data specified by the “openkeyframe” command. At this time, the sign language animation generation unit 293 can generate a sign language CG using the function of the TVML player.

Next, a case where the character type is emergency will be described.
The sign language animation generation unit 293 reads the CG model of the emergency character from the CG data storage unit. Then, the sign language animation generation unit 293 generates an animation of an emergency character using the motion capture data specified by the “openkeyframe” command.
Thereafter, the sign language animation generation unit 293 outputs the generated sign language CG of the character to the composite display unit 295 as the sign language CG.

  The composition display unit 295 synthesizes the sign language CG input from the sign language animation generation unit 293 with the broadcast video input from the demultiplexing unit 22. Then, the composite display unit 295 outputs the broadcast video combined with the sign language CG to the display 3.

  Hereinafter, three specific examples of the method for determining the facial expression of the character will be described in order with reference to FIGS. 7 to 9 (see FIGS. 1 to 6 as appropriate).

<Character Expression Determination Method: First Example>
As shown in FIG. 7, in the first example, it is assumed that the viewer has selected data broadcasting content 32 </ b> A with high urgency.
The text data analysis unit 23 extracts “tsunami warning” included in the content item name of the data broadcast content 32A as the first word. Then, the text data analysis unit 23 outputs the extracted first word “tsunami warning” to the content impression extraction unit 25.

  The content impression extraction unit 25 searches the content item name-impression data 40 using the first word “tsunami warning” input from the text data analysis unit 23 as a search key. Then, the content impression extraction unit 25 acquires the impression value “emergency 1.0” as the search result, and outputs the acquired impression value “emergency 1.0” to the character / facial expression determination unit 27.

  The character / expression determining unit 27 determines that the impression value “emergency 1.0” input from the content impression extracting unit 25 matches the emergency impression value. Therefore, the character / expression determination unit 27 outputs an emergency character notification to the sign language CG generation unit 29. In response to the emergency character notification, the sign language CG generation unit 29 generates a sign language CG with the emergency character of FIG.

<Character Expression Determination Method: Second Example>
As shown in FIG. 8, in the second example, it is assumed that the viewer has selected the data broadcast content 32B.
The text data analysis unit 23 extracts, as the first word, the content item name “Senbat △△ lost in the first game” of the data broadcast content 32B as the first word. Then, the text data analysis unit 23 outputs the extracted first word “Senbat △ Δ lost in the first game” to the content impression extraction unit 25.

  The content impression extraction unit 25 searches the content item name-impression data 40 using the first word “Senbat ΔΔ lost in the first game” input from the text data analysis unit 23 as a search key. Here, in the content item name-impression data 40, the first word “Senbat △△ Loses in the first match” is not registered. Therefore, the content impression extraction unit 25 outputs a content item name unregistered notification to the text data analysis unit 23.

  In response to this content item name unregistered notification, the text data analysis unit 23 performs morphological analysis on the content item name “Senbat △△ Loss in the first game” of the data broadcast content 32B, and “Senbat” “△” is used as the first word. △ ”,“ First Battle ”,“ De ”,“ Lose ”are extracted. Then, the text data analysis unit 23 outputs the extracted first words “senbat”, “ΔΔ”, “first game”, “de” and “defeat” to the content impression extraction unit 25, respectively.

  The content impression extraction unit 25 uses each of the first words “Sembatsu”, “ΔΔ”, “First Battle”, “De”, and “Lose” input from the text data analysis unit 23 as search keys. The impression data 40 is searched. Here, in the content item name-impression data 40, the impression value “dark 0.3” corresponds to the first word “losing”. Therefore, the content impression extraction unit 25 acquires the impression value “dark 0.3” as a search result, and outputs the acquired impression value “dark 0.3” to the character / expression determining unit 27.

  When each of the first words is used as a search key, a plurality of impression values may be acquired from the content item name-impression data 40. Here, the content item name of the data broadcast content 32 often simply represents the content. For this reason, it is considered that the possibility that different impressions such as “bright 0.5” and “dark 0.5” are extracted from the content item name-impression data 40 is low. Therefore, when the content impression extracting unit 25 acquires a plurality of impression values such as “dark 0.3” and “dark 0.5”, for example, the content impression extracting unit 25 averages the acquired impression values and obtains the impression value “dark”. 0.4 ".

  The character / expression determination unit 27 determines that the impression value “dark 0.3” input from the content impression extraction unit 25 does not match the emergency impression value. Accordingly, the character / expression determining unit 27 searches the impression-character-expression data 44 using the impression value “dark 0.3” as a search key. Here, in the impression-character-expression data 44, two characters “character A” and “character B” correspond to the same impression value “dark 0.3”. Therefore, the character / facial expression determination unit 27 refers to use character data described later, and acquires “character A” and an expression value “sadness 0.3”.

  Here, the digital broadcast receiver 2 allows the viewer to select in advance a character type to be used for sign language CG in case two or more types of characters correspond to the same impression value. Also, the digital broadcast receiver 2 may cause the viewer to operate the remote control and select the character type each time. Then, the digital broadcast receiver 2 sets the character type selected by the viewer as the used character data.

  Thereafter, the character / expression determination unit 27 outputs the acquired “character A” and the expression value “sadness 0.3” to the sign language CG generation unit 29. In this manner, the sign language CG generation unit 29 generates a sign language CG with “character A” having a sad expression as shown in FIG.

<Character Expression Determination Method: Third Example>
As shown in FIG. 9, in the third example, it is assumed that the viewer has selected the data broadcast content 32C.
In this third example, “Character B” is set as the used character data.
The text data analysis unit 23 extracts all the first words included in the content item name of the data broadcast content 32C as the first words. Then, the text data analysis unit 23 outputs the extracted first words to the content impression extraction unit 25, respectively.

  The content impression extraction unit 25 searches the content item name-impression data 40 using each of the first words input from the text data analysis unit 23 as a search key. Here, it is assumed that the content impression extraction unit 25 cannot acquire an impression value corresponding to the first word from the content item name-impression data 40. Therefore, the content impression extraction unit 25 outputs an impression value undetermined notification to the text data analysis unit 23.

  In response to the impression value undetermined notification, the text data analysis unit 23 performs morphological analysis on the text data of the data broadcast content 32C and extracts the second word. For example, the text data analysis unit 23 determines “consumption”, “tax rate”, “no”, “raise”, “next”, “ni”, “reserve”, “sendai”, “city” from the text data of the data broadcasting content 32C. “No” “Home Appliances” “Mass Sales” “Store” “Ya” “Department Store” “De” “Ha” “Now” “No” “House” “Ni” “Price” “No” “High” “Product” “ The second words “buy” “shi” “te” “ko” “u” “to” “shopping” “customer” “de” “busy” “masashi” “ta” “.” Are extracted. Then, the text data analysis unit 23 outputs the extracted second word to the content impression extraction unit 25.

  The content impression extraction unit 25 searches the word-impression data 42 using each of the second words input from the text data analysis unit 23 as a search key. Here, in the word-impression data 42, the impression value “bright 0.2” corresponds to the second word “Nigiwai”. Therefore, the content impression extraction unit 25 acquires the impression value “bright 0.2” from the word-impression data 42 and outputs the acquired impression value “bright 0.2” to the character / expression determination unit 27.

  The character / expression determining unit 27 determines that the impression value “bright 0.2” input from the content impression extracting unit 25 does not match the emergency impression value. Accordingly, the character / expression determining unit 27 searches the impression-character-expression data 44 using the impression value “bright 0.2” as a search key. Then, the character / expression determining unit 27 refers to the used character data and acquires “character B” and an expression value “joy 0.2”.

  Thereafter, the character / expression determination unit 27 outputs the acquired “character B” and the expression value “joy 0.2” to the sign language CG generation unit 29. In this manner, the sign language CG generation unit 29 generates a sign language CG with a “character B” with a happy expression as shown in FIG.

[Operation of digital broadcast receiver]
The operation of the digital broadcast receiver 2 will be described with reference to FIG. 10 (see FIG. 1 as appropriate).
In FIG. 10, it is assumed that the demodulator 21 demodulates the transport stream, and the demultiplexing unit 22 separates the data broadcast and the caption data from the transport stream.

The digital broadcast receiver 2 uses the text data analysis unit 23 to extract the content item name of the data broadcast content 32 selected by the viewer or the word included in the content item name as the first word (step S1).
The digital broadcast receiver 2 refers to the content item name-impression data 40 by the content impression extraction unit 25 (step S2), and determines whether or not an impression value corresponding to the first word extracted in step S1 is registered. Determine (step S3).

When the impression value is registered (Yes in step S3), the digital broadcast receiver 2 determines the impression value from the content item name-impression data 40 by the content impression extraction unit 25 (step S4).
The digital broadcast receiver 2 determines whether or not the impression value determined in step S4 matches the emergency impression value set in advance by the character / facial expression determination unit 27 (step S5).

If it matches the emergency impression value (Yes in step S5), the digital broadcast receiver 2 uses the character / facial expression determination unit 27 to determine the emergency type as the character type and facial expression value (step S6).
If it does not match the urgent impression value (No in step S5), the digital broadcast receiver 2 uses the character / expression determining unit 27 to refer to the used character data and the impression-character-expression data 44 (steps S7 and S8). The character type and expression value are determined (step S9).

  The digital broadcast receiver 2 uses the sign language CG generation unit 29 to generate a sign language CG based on the character type and expression value determined in step S6 or step S9 (step S10), and ends the process.

  When the impression value is not registered (No in step S3), the digital broadcast receiver 2 performs morphological analysis on the text data of the selected data broadcast content 32 by the text data analysis unit 23 (step S11). A second word is extracted (step S12).

  The digital broadcast receiver 2 refers to the word-impression data 42 by the content impression extracting unit 25 (step S13), determines an impression value corresponding to the second word extracted in step S12 (step S4B), and step 5 The subsequent processing is executed.

[Action / Effect]
As described above, the digital broadcast receiver 2 determines an impression from the content of the data broadcast content 32, and generates a sign language CG with a facial expression character based on the impression. In this way, the digital broadcast receiver 2 can adjust the facial expression of the character to the content content, thereby reducing the sense of discomfort in the sign language CG.

(Second Embodiment)
With reference to FIG. 11, a difference from the first embodiment in a broadcasting system 100A according to the second embodiment of the present invention will be described.
The broadcast system 100A is different from the first embodiment in that the translation from Japanese text to sign language is performed not by the digital broadcast receiver 2A but by the broadcast station 1A.

[Broadcasting station configuration]
As shown in FIG. 11, the broadcasting station 1A includes a multiplexing unit 11A, a modulator 13, a data transmission unit 15, and a sign language notation translation unit 17.
Since the configuration other than the multiplexing unit 11A and the sign language notation translation unit 17 is the same as that of the first embodiment, description thereof is omitted.

  The sign language notation translation unit 17 translates Japanese text into sign language notation, similar to the sign language notation translation unit 28 of FIG. For example, the sign language notation translation unit 17 uses text data or caption data included in a data broadcast input from the outside as Japanese text. The sign language notation translation unit 17 translates the Japanese text into sign language notation and outputs the translation result to the multiplexing unit 11A.

The multiplexing unit 11A multiplexes the translation result input from the sign language notation translation unit 17 in the transport stream format in addition to the broadcast video, the data broadcast, and the caption data, and outputs the result to the modulator 13.
That is, the translation result multiplexed by the multiplexing unit 11A is transmitted to the digital broadcast receiver 2A via the digital broadcast wave.

[Configuration of digital broadcast receiver]
As shown in FIG. 11, the digital broadcast receiver 2A includes a demodulator 21, a demultiplexing unit 22A, a text data analyzing unit 23, a text-impression database 24, a content impression extracting unit 25, an impression-character-expression. A database 26, a character / expression determining unit 27, and a sign language CG generating unit 29A are provided.
The configurations other than the demultiplexing unit 22A and the sign language CG generation unit 29A are the same as those in the first embodiment, and thus the description thereof is omitted.

  The demultiplexing unit 22A receives the transport stream from the demodulator 21, and separates the broadcast video, the data broadcast, the caption data, and the translation result multiplexed into the input transport stream. Further, the demultiplexing unit 22A outputs the separated data broadcast and caption data to the text data analysis unit 23. Further, the demultiplexing unit 22A outputs the separated broadcast video and the translation result to the sign language CG generation unit 29A.

The sign language CG generation unit 29A includes a sign language CG control command generation unit 291A, a sign language animation generation unit 293, and a composite display unit 295.
The sign language CG control command generation unit 291A designates the motion capture data in which the same sign language operation as the translation result input from the demultiplexing unit 22A is performed by the “openkeyframe” command of the TVML script.
In other respects, the sign language CG control command generation unit 291A is the same as the sign language CG control command generation unit 291 in FIG.

[Action / Effect]
As described above, the digital broadcast receiver 2A can reduce the sense of discomfort in the sign language CG, as in the first embodiment. Furthermore, since the digital broadcast receiver 2A does not need to include the sign language notation translation unit 28 of FIG. 1, a simple configuration can be achieved.

(Third embodiment)
With reference to FIG. 12, a difference from the second embodiment will be described regarding a broadcasting system 100B according to a third embodiment of the present invention.
The broadcast system 100B is different from the second embodiment in that the translation result of sign language notation is transmitted to the digital broadcast receiver 2B via the network N instead of the digital broadcast wave.

[Broadcasting station configuration]
As shown in FIG. 12, the broadcasting station 1B includes a multiplexing unit 11, a modulator 13, a data transmission unit 15, and a sign language notation translation unit 17B.
Since the configuration other than the sign language notation translation unit 17B is the same as that of the second embodiment, the description thereof is omitted.

  The sign language notation translation unit 17B translates Japanese text into sign language notation, similar to the sign language notation translation unit 17 of FIG. Then, the sign language notation translation unit 17B transmits the translation result of the sign language notation to the digital broadcast receiver 2 via the network N.

[Configuration of digital broadcast receiver]
As shown in FIG. 12, the digital broadcast receiver 2B includes a demodulator 21, a demultiplexer 22B, a text data analyzer 23, a text-impression database 24, a content impression extractor 25, an impression-character-expression. A database 26, a character / expression determining unit 27, and a sign language CG generating unit 29B are provided.
Note that the configurations other than the demultiplexing unit 22B and the sign language CG generation unit 29B are the same as those in the second embodiment, and thus the description thereof is omitted.

  The demultiplexing unit 22B receives the transport stream from the demodulator 21, and separates the broadcast video, the data broadcast, and the caption data that are multiplexed in the input transport stream. Further, the demultiplexing unit 22B outputs the separated data broadcast and caption data to the text data analysis unit 23. Further, the demultiplexing unit 22B outputs the separated broadcast video to the sign language CG generation unit 29B.

The sign language CG generation unit 29B includes a sign language CG control command generation unit 291B, a sign language animation generation unit 293, and a composite display unit 295.
The sign language CG control command generation unit 291B specifies motion capture data in which the same sign language operation as the translation result input from the broadcast station 1B is performed via the network N by the “openkeyframe” command of the TVML script.
In other respects, the sign language CG control command generation unit 291B is the same as that of the second embodiment, and thus further description thereof is omitted.

[Action / Effect]
As described above, similarly to the first and second embodiments, the digital broadcast receiver 2B can reduce the sense of discomfort in the sign language CG and can realize a simple configuration. Furthermore, since the digital broadcast receiver 2B receives the translation result of sign language notation via the network N, the data amount of the digital broadcast wave can be reduced.

(Modification)
The present invention is not limited to the above-described embodiments, and modifications can be made without departing from the spirit of the invention.
In the above-described embodiment, the text data and caption data of data broadcasting are exemplified as the Japanese text, but the present invention is not limited to this. For example, the Japanese text may be web content (FIG. 1) presented on a web server (not shown). In this case, the web content is translated into sign language notation by any of the broadcast station 1, the digital broadcast receiver 2, or the web server.

  In the above-described embodiment, the digital broadcast receiver 2B has been described as independent hardware. However, the present invention is not limited to this. For example, the digital broadcast receiver 2B can also be realized by a sign language CG generation program that causes hardware resources such as a CPU, a memory, and a hard disk included in a computer to operate cooperatively as the above-described means. This program may be distributed through a communication line, or may be distributed by writing in a recording medium such as a CD-ROM or a flash memory.

100, 100A, 100B Broadcast system 1, 1A, 1B Broadcast station 11, 11A, 11B Multiplexer 13 Modulator 15 Data transmitter 17, 17B Sign language notation translator 2 Digital broadcast receiver 21 Demodulator 22, 22A, 22B Multiplex Separation unit 23 Text data analysis unit (word extraction unit)
24 text-impression database (first database)
25 Content impression extraction unit (impression value determination unit)
26 Impression-character-expression database (second database)
27 Character / Expression Determination Unit 28 Sign Language Notation Translation Unit 29, 29A, 29B Sign Language CG Generation Unit 3 Display

Claims (7)

A sign language CG generation device for generating a sign language CG with a character having a facial expression,
A word extraction unit for extracting the first word of the content item name of the data broadcast content or web content selected by the viewer;
A first database that stores in advance first word-impression data in which the first word is associated with an impression value representing the strength of an impression received by the viewer from the first word;
An impression value determining unit that determines an impression value corresponding to the first word extracted by the word extracting unit with reference to the first word-impression data;
A second database that stores in advance impression-character-expression data in which the impression value, the character type, the expression type of the character, and an expression value representing the strength of the expression are associated;
A character / expression determination unit that determines the type and expression value of the character corresponding to the impression value determined by the impression value determination unit with reference to the impression-character-expression data;
A sign language CG generation unit that generates the sign language CG based on the character type and expression value determined by the character / expression determination unit;
A sign language CG generation apparatus comprising: The word extraction unit, when the impression value determination unit cannot determine an impression value corresponding to the first word from the first word-impression data, a second word included in text data or subtitle data included in the data broadcast Extract
The first database further stores second word-impression data in which the impression value and the second word are associated with each other;
The sign language according to claim 1, wherein the impression value determination unit determines an impression value corresponding to the second word extracted by the word extraction unit with reference to the second word-impression data. CG generator. In the first database, an emergency first word preset as a first word indicating high urgency is associated with an emergency impression value preset as an impression value indicating high urgency. Storing the first word-impression data;
The impression value determination unit refers to the first word-impression data, and when the word extraction unit extracts the emergency first word, determines the emergency impression value,
The character / facial expression determining unit determines the character type to be a preset emergency character when the impression value determining unit determines the emergency impression value, and sets the character's facial expression to a preset emergency character. The sign language CG generation apparatus according to claim 1, wherein the sign language CG generation apparatus is determined to be a facial expression. The word extraction unit performs morphological analysis for each section delimited by punctuation marks in the text data or the caption data , and extracts the second word,
The sign language CG generation device according to claim 2 or 3, wherein the impression value determination unit determines an impression value corresponding to the second word extracted by the word extraction unit for each of the sections. The impression value determination unit
When two or more first words are extracted by the word extraction unit, the impression value is extracted from the first word-impression data for each first word, and the extracted impression value for each first word is calculated. Average
When two or more second words are extracted by the word extraction unit, the impression value is extracted from the second word-impression data for each second word, and the extracted impression value for each second word is calculated. sign language CG generating apparatus according to any one of claims 4 be determined from claim 2, wherein.   2. The character / facial expression determining unit, when two or more types of characters correspond to the impression value determined by the impression value determining unit, determines the character type according to the selection of the viewer. The sign language CG generation device according to any one of claims 1 to 5.   A sign language CG generation program for causing a computer to function as the sign language CG generation device according to any one of claims 1 to 6.

Images