JP5213572B2 - Sign language video generation system, server, terminal device, information processing method, and program - Google Patents

Description

  The present invention relates to a sign language video generation system that generates and outputs sign language video information corresponding to subtitle information.

Conventionally, an apparatus for automatically generating a sign language video has been developed (see, for example, Patent Documents 1 to 4).
JP-A-11-184370 JP 2000-190442 A JP 2003-296653 A JP 2007-133609 A

  Sign language video is generated from program subtitles, etc. for broadcasting stations that broadcast video for hearing-impaired people, television receivers for hearing-impaired people, and set-top / It is thought that it is often a box. In such a case, when a sign language video is generated and broadcast in a broadcasting station, there is a problem that the capacity of the data to be broadcast becomes large. On the other hand, when a sign language image is generated by a television receiver or the like possessed by a hearing impaired person, a high load is imposed on the television receiver or the like.

  The present invention has been made to solve the above-described problems, and an object thereof is to provide a sign language video generation system and the like that can perform appropriate load distribution between a server and a terminal device.

  To achieve the above object, a sign language video generation system according to the present invention is a sign language video generation system including a server and a terminal device, wherein the server receives subtitle information that is information indicating a subtitle of a program. An information receiving unit, a term extracting unit for extracting a term that is an independent word from the subtitle information received by the subtitle information receiving unit, and a term string change for changing a term string that is a sequence of terms extracted by the term extracting unit And a transmitting unit that transmits the term string changed by the term string changing unit, and the terminal device relates to a receiving unit that receives the term string, a term, and a sign language operation corresponding to the term A sign language term correspondence information storage unit that stores sign language term correspondence information that is information that associates sign language action information that is information, and each term included in the term sequence received by the reception unit, the sign language term A sign language video information generating unit that generates sign language video information that is information indicating a video of a sign language corresponding to the term sequence using the sign language action information associated with the response information, and the sign language generated by the sign language video information generating unit And an output unit for outputting video information.

  With such a configuration, appropriate load distribution can be performed between the server and the terminal device. That is, since the preprocessing for generating sign language video information is performed in the server, it is possible to reduce the load on the terminal device as compared with the case where the terminal device generates sign language video information from the caption information. In addition, since the server does not transmit sign language video information but transmits a term string, there is an advantage that the data capacity transmitted by the server can be reduced.

  In the sign language video generation system according to the present invention, the term string changing unit includes term correspondence information that is information that associates a conversion target term that is a conversion target term with a converted term that is a converted term. A term correspondence information storage means to be stored; and a term conversion means for converting the conversion target term into a post-conversion term corresponding to the conversion target term when the conversion target term is included in the term string. Also good.

  With such a configuration, for example, a term transmitted by the server can be converted into a term used for generating sign language video information in the terminal device. As a result, in the terminal device, for example, it is not necessary to include many synonyms of terms in the sign language term correspondence information, and the capacity of the sign language term correspondence information can be reduced. Further, if the capacity of the sign language term correspondence information is reduced, the processing load for reading the sign language action information corresponding to the term string is also reduced.

In the sign language video generation system according to the present invention, the term string changing unit includes term order information storage means for storing term order information, which is information indicating the order of terms, and the order of terms indicated by the term order information. The term rearrangement means which rearranges the order of the term of the said term sequence may be provided.
With such a configuration, the terms can be rearranged so that appropriate sign language video information is generated by the terminal device. As a result, it is possible to improve the quality of sign language video information.

Further, in the sign language video generation system according to the present invention, the term string changing unit detects a term missing in the term string, and when the detecting means detects a term missing, There may be provided term supplementing means for supplementing the term string with the term.
With such a configuration, missing terms can be supplemented, and as a result, the quality of sign language video information can be improved.

  Further, in the sign language video generation system according to the present invention, the term string change unit includes an attribute acquisition unit that acquires an attribute of the term string, and attribute information that is information indicating an attribute acquired by the attribute acquisition unit. Attribute information adding means for adding to the sign language action information, the sign language action information includes at least information for each attribute indicated by the attribute information, and the sign language video information generation unit includes the attribute information included in the term string. Sign language video information may be generated using the corresponding sign language action information.

  With such a configuration, attribute information can be included in the term string, and the terminal device can generate sign language video information according to the attribute information. Therefore, it is possible to make the sign language image closer to the image when the actual sign language interpreter interprets the sign language.

In the sign language video generation system according to the present invention, the terminal device further includes a program related information receiving unit that receives program related information that is information related to a program corresponding to the term string received by the receiving unit, The video information generating unit may generate sign language video information using the program related information received by the program related information receiving unit.
With such a configuration, the program related information can be used for the process of generating sign language video information.

In the sign language video generation system according to the present invention, the sign language term correspondence information is information that associates a term with sign language action information and program related information, and the sign language video information generation unit includes the program related information reception unit. The sign language video information may be generated using sign language action information corresponding to the program related information received by.
With such a configuration, sign language video information can be generated using sign language motion information corresponding to the program related information, and sign language video information harmonized with the program can be generated.

In the sign language video generation system according to the present invention, the terminal device further includes a cache storage unit that can be accessed at a higher speed than the sign language term correspondence information storage unit, and the sign language video information generation unit receives the program related information. Using the program-related information received by the department, sign language motion information is stored in the cache storage unit in advance, and sign language video information is also generated using the sign language motion information stored in the cache storage unit. Good.
With such a configuration, the speed of generating sign language video information can be improved.

In the sign language video generation system according to the present invention, the server acquires subtitle information by performing a voice recognition process on the voice information received by the voice information receiving unit and the voice information receiving unit receiving the voice information of the program. A speech recognition unit, and the caption information receiving unit may receive caption information that is a result of the speech recognition unit performing a speech recognition process.
With such a configuration, for example, even when caption information is not included in program information, a term string can be generated from audio information included in program information.

In the sign language video generation system according to the present invention, the server further includes a program information storage unit that stores program information that is program information, and the transmission unit corresponds to the term string together with the term string. Program information is read from the program information storage unit and transmitted, the receiving unit also receives program information transmitted by the transmitting unit together with the term string, and the output unit includes sign language video information and the receiving unit. The program information received by may also be output.
With such a configuration, program information corresponding to the term string can be transmitted together with the term string.

  According to the sign language video generation system and the like according to the present invention, it is possible to perform appropriate load distribution by extracting and changing a term in the server and generating a sign language video using the term in the terminal device.

  Hereinafter, a sign language video generation system according to the present invention will be described using embodiments. In the following embodiments, components and steps denoted by the same reference numerals are the same or equivalent, and repetitive description may be omitted.

(Embodiment 1)
A sign language video generation system according to Embodiment 1 of the present invention will be described with reference to the drawings. The sign language video generation system according to the present embodiment performs distributed processing of generating a sign language video from caption information in a server and a terminal device.

  FIG. 1 is a block diagram showing a configuration of a sign language video generation system according to the present embodiment. The sign language video generation system according to the present embodiment includes a server 1 and a terminal device 2. Although FIG. 1 shows the case where information is broadcast from the server 1 to the terminal device 2, this need not be the case. For example, information may be transmitted from the server 1 to the terminal device 2 via a wired or wireless communication line. The communication line may be, for example, the Internet, an intranet, a public telephone line network, or the like. Moreover, although FIG. 1 shows the case where one terminal device 2 exists, two or more terminal devices 2 may exist.

  FIG. 2 is a block diagram showing a configuration of the server 1 according to the present embodiment. In FIG. 2, the server 1 according to the present embodiment includes a program information storage unit 11, a caption information receiving unit 12, a term extracting unit 13, a term string changing unit 14, and a transmitting unit 15.

  The program information storage unit 11 stores program information that is program information. The program information is video / audio information. The program information may be information on programs such as news, dramas, movies, sports, documentaries, and the like. Moreover, the data format of program information is not ask | required. For example, the program information may be analog data or digital data. In the latter case, the program information format may be, for example, MPEG (Moving Picture Experts Group), AVI (Audio Video Interleave), or any other format. Further, when the program information is compressed, the compression format is not limited.

  The process in which program information is stored in the program information storage unit 11 does not matter. For example, program information may be stored in the program information storage unit 11 via a recording medium, and program information transmitted via a communication line or the like is stored in the program information storage unit 11. Alternatively, program information input via a camera or the like may be stored in the program information storage unit 11. Storage in the program information storage unit 11 may be temporary storage in a RAM or the like, or may be long-term storage. The program information storage unit 11 can be realized by a predetermined recording medium (for example, a semiconductor memory, a magnetic disk, an optical disk, etc.).

  The subtitle information receiving unit 12 receives subtitle information that is information indicating a subtitle of a program. The caption information may be, for example, text information corresponding to an utterance included in the program information, which is program information, and further text describing the situation in the program (for example, “a sound that breaks the window glass”, etc. ) Or onomatopoeia text corresponding to sound (for example, “Gashan”, “Bakun”, etc.). Note that the caption information is preferably distinguishable for each unit displayed on the screen. For example, a delimiter or the like may be entered for each unit, or may be superimposed on the program information for each unit displayed on the screen.

  The caption information receiving unit 12 may receive, for example, caption information input from an input device (for example, a keyboard, a mouse, a touch panel, etc.) and receives caption information transmitted via a wired or wireless communication line. Alternatively, subtitle information read from a predetermined recording medium (for example, an optical disk, a magnetic disk, a semiconductor memory, etc.) may be received. In the present embodiment, the caption information receiving unit 12 reads the caption information from the program information storage unit 11. Therefore, it is assumed that the program information stored in the program information storage unit 11 includes subtitle information in advance. The subtitle information is included in the program information, for example, the subtitle information may be included in the user-defined area of the program information, and the subtitle information is multiplexed in the vertical blanking interval of the program information. It may be to do. Note that the caption information receiving unit 12 may or may not include a device (for example, a modem or a network card) for receiving. The caption information receiving unit 12 may be realized by hardware, or may be realized by software such as a driver that drives a predetermined device.

  The term extraction unit 13 extracts terms that are independent words from the subtitle information received by the subtitle information reception unit 12. For example, the term extraction unit 13 may perform morphological analysis on the text of the caption information and extract a term that is an independent word using the morphological analysis result. Autonomous words are verbs, adjectives, adjective verbs, nouns, conjunctions, adverbs, conjunctions, and emotional verbs. Autonomous words can be considered as words other than particles and auxiliary verbs. As a morphological analysis system, for example, “ChaSen” (http://chasen.naist.jp) developed at Nara Institute of Science and Technology is known. Extracting a term means that it is only necessary to distinguish between an extracted term and a term that has not been extracted as a result. For example, the term to be extracted may be stored in a recording medium, or A mark such as a flag may be given to a term to be extracted.

  The term string changing unit 14 changes a term string that is a sequence of terms extracted by the term extracting unit 13. Here, various methods of changing the term string by the term string changing unit 14 will be described. The term string changing unit 14 may (1) convert terms included in the term string, (2) rearrange the order of the terms included in the term string, and (3) supplement the terms to the term string. (4) Information indicating the attribute of the term string may be added to the term string, and changes regarding other term strings may be made.

(1) When converting terms included in a term string The term string changing unit 14 includes a term correspondence information storage unit 31 and a term conversion unit 32, as shown in FIG.

  The term correspondence information storage means 31 stores term correspondence information. The term correspondence information is information that associates the conversion target term with the converted term. The term to be converted is a term to be converted. The term after conversion is a term after conversion. Here, “associating the conversion target term with the converted term” means that the other term may be acquired from one of the conversion target term and the converted term. Therefore, the term correspondence information may include information including a conversion target term and a converted term as a set, or information that links the conversion target term and the converted term. In the latter case, the term correspondence information may be, for example, information that associates the conversion target term with a pointer or address indicating the position where the post-conversion term is stored. In the present embodiment, the former case will be described. Further, the conversion target term and the converted term may not be directly associated with each other. For example, the third information may correspond to the conversion target term, and the converted term may correspond to the third information. The same applies when other two or more pieces of information are associated.

  The process in which the term correspondence information is stored in the term correspondence information storage means 31 does not matter. For example, the term correspondence information may be stored in the term correspondence information storage means 31 via a recording medium, and the term correspondence information transmitted via a communication line or the like is stored in the term correspondence information storage means 31. Alternatively, the term correspondence information input via the input device may be stored in the term correspondence information storage unit 31. Storage in the term correspondence information storage means 31 may be temporary storage in a RAM or the like, or may be long-term storage. The term correspondence information storage means 31 can be realized by a predetermined recording medium (for example, a semiconductor memory, a magnetic disk, an optical disk, etc.).

  The term conversion means 32 receives a term string that is a sequence of terms extracted by the term extraction unit 13, and when the term string includes a conversion target term, the conversion target term is converted into the conversion target term using term correspondence information. Is converted to a post-conversion term associated with and passed to the transmission unit 15. Therefore, when the conversion target term is not included in the term string, the term conversion by the term conversion unit 32 is not performed.

  The term conversion by the term conversion means 32 may be performed so as to be a term associated with sign language action information in sign language term correspondence information used when generating sign language video information to be described later, for example. Good. Specifically, in the sign language term correspondence information, when there is no sign language motion information corresponding to the term “OOKINI”, there is sign language motion information corresponding to the term “Thank you”, the term conversion means 32 The conversion target term “OOKINI” may be converted to the converted term “Thank you”. In this way, the term correspondence information associates, for example, a conversion target term that is not associated with sign language action information in sign language term correspondence information and a converted term that is associated with sign language action information in sign language term correspondence information. It may be information.

  As described above, the term string changing unit 14 uses the term correspondence information storage unit 31 and the term conversion unit 32 to convert the conversion target term that is a term that is not associated with the sign language action information in the sign language term correspondence information into the sign language. You may convert into the term after conversion which is the term matched with sign language action information by term correspondence information. The relationship between the conversion target term and the converted term may be, for example, the relationship between a dialect term and a standard term, or a relationship between a non-terminated usage term and a termination term. It may be other relationships. By performing the term conversion, it may be possible to reduce the data capacity of sign language term correspondence information included in the terminal device 2 described later.

(2) When rearranging the order of the terms included in the term string The term string changing unit 14 includes a term order information storage unit 33 and a term rearrangement unit 34, as shown in FIG.

  The term order information storage means 33 stores term order information. The term order information is information indicating the order of terms. The term order information indicates the order of terms for generating more appropriate sign language video information when generating sign language video information to be described later. The order of terms may be, for example, the order of part of speech of terms, the subject / predicate of terms, or the order of subject / predicate / object, or the order of other term attributes. Good. The term order information may indicate the order of terms by arranging the attributes of terms, for example, as shown below.

Term order information: {noun} {verb}
Term order information: {subject} {object} {predicate}

  In the case of the above example, the term order information indicates, for example, that the terms are arranged in the order of nouns and verbs. In the case of the example below, the term order information includes, for example, the terms of the subject, object, and predicate. It shows that they are arranged in order.

  The process in which the term order information is stored in the term order information storage means 33 does not matter. For example, the term order information may be stored in the term order information storage means 33 via a recording medium, and the term order information transmitted via a communication line or the like is stored in the term order information storage means 33. Alternatively, the term order information input via the input device may be stored in the term order information storage means 33. Storage in the term order information storage means 33 may be temporary storage in a RAM or the like, or may be long-term storage. The term order information storage means 33 can be realized by a predetermined recording medium (for example, a semiconductor memory, a magnetic disk, an optical disk, etc.).

  The term rearranging unit 34 receives a term string that is a sequence of terms extracted by the term extracting unit 13, and the terms are arranged in the order of terms indicated by the term order information stored in the term order information storing unit 33. The order of the terms in the column is rearranged and passed to the transmitter 15. Therefore, when the order of the terms included in the term sequence is the order of the terms indicated by the term order information, that is, when it is not necessary to rearrange the order of the terms, the rearrangement of terms by the term rearranging means 34 is not performed. It will not be broken.

  The rearrangement of terms by the term rearranging means 34 is rearranged so that more appropriate sign language video information can be generated when sign language video information to be described later is generated. For example, the word order may be disturbed in colloquial language. Specifically, “I will buy! That flower.” In this case, the term extraction unit 13 can extract the term string “I”, “buy”, “that flower”, but the term rearrangement means 34 selects “I”, “that flower”, “buy”. You may rearrange to. In that case, the term order information stored in the term order information storage means 33 may be {subject} {object} {predicate}, for example.

  Here, a method will be briefly described in which the term rearranging means 34 determines the attributes (for example, part of speech, subject, predicate, etc.) of terms included in the term string. The term rearranging unit 34 may determine, for example, the part of speech, subject, predicate, etc. of each term using the term string received from the term extracting unit 13. However, since the term string does not include particles or auxiliary verbs, it may be difficult to determine the part of speech, subject, predicate, and the like. Therefore, the term rearranging unit 34 may refer to the caption information corresponding to the term string, and determine the part of speech, subject, predicate, and the like of each term using the caption information. In this case, it is assumed that caption information is also stored in advance in a recording medium (not shown). In addition, it is assumed that the caption information can be associated with a term string extracted from the caption information. Alternatively, each term in the term string may have an attribute of the term, such as part of speech, subject, predicate, object, etc. Then, the term rearranging means 34 may rearrange the terms using the attribute of the term added to each term in the term string. In this case, the term extraction unit 13 and other components may perform processing for acquiring the attribute of each term included in the caption information.

  A method for determining the part of speech, subject, predicate, object, etc. of each term using caption information will be briefly described. The term part of speech can be known by performing morphological analysis as described above. For the subject, predicate, and object, for example, a system that automatically determines the subject, predicate, object, etc. may be used, or the subject, predicate, object, etc. may be determined using a clue phrase. . For example, when a clue phrase “ha” or “ga” is attached to the end of the term, it may be determined that the term is the subject. Specifically, in the case of "I am today ..." or "Yamada is yesterday ...", it may be determined that "I" or "Mr. Yamada" is the subject. In addition, when “to”, “ni”, and “to” are attached to the rear end of the term, it may be determined that the term is an object. Specifically, in the case of “… buy flowers” or “… go to Tokyo”, it may be determined that “flower” or “Tokyo” is the object. Further, a term that is a verb may be determined to be a predicate. Note that the method for determining the part of speech, subject, predicate, object, etc. of the term is not limited to these, and as a result, other methods can be used as long as the part of speech of the term can be determined. May be.

(3) When a term is supplemented to a term string The term string changing unit 14 includes a detection unit 35 and a term supplementing unit 36, as shown in FIG.

  The detecting means 35 detects that a term is missing in the term string. For example, the detection unit 35 may detect a missing term when a term having a specific attribute is missing in a term string that is a sequence of terms extracted by the term extracting unit 13. The specific attribute may be a subject or other attribute. Therefore, for example, the detection unit 35 may detect that the subject is missing when the subject is not included in the term string. That the subject string is not included in the term string may be detected using the term string itself or may be detected using caption information corresponding to the term string, for example. For example, as described in “(2) When rearranging the order of terms included in a term string”, each term, each morpheme, etc. is the subject, predicate, purpose, etc. for the term string and caption information corresponding to the term string. It may be determined whether the word is a word or the like, and then the detecting unit 35 may detect the lack of the subject by determining whether the subject string is included in the term string or the caption information. If the subject is included, the subject omission is not detected. If the subject is not included, the subject omission is detected.

  When the detecting unit 35 detects a missing term, the term supplementing unit 36 supplements the term to the term string in which the missing term is detected. Supplementary terms are missing terms. For example, when the detecting means 35 detects a missing subject, the term supplementing means 36 supplements the subject. When supplementing the subject, for example, the term supplementing means 36 supplements the subject included in the term string before the subject string in which the subject omission is detected as the subject of the term string in which the subject omission is detected. May be. Specifically, when a subject is missing in a certain term sequence, the term supplementing means 36 is a term sequence in which the subject preceding the term sequence exists and the subject sequence is missing. The subject may be acquired from the closest term string. For example, as described above, in the term string and the caption information corresponding to the term string, the subject (for example, the clue phrase “ha” is determined for each term, each morpheme, and the like, as described above. Or the term “ga” attached to the back end). Then, the term supplementing means 36 acquires the subject term from the term sequence in which the subject exists, and supplements the term sequence in which the missing term is detected. The position where the term is supplemented may be, for example, the head of the term string.

(4) When adding information indicating the attribute of the term string to the term string The term string changing unit 14 includes an attribute obtaining unit 37 and an attribute information adding unit 38, as shown in FIG.

  The attribute acquisition unit 37 receives a term string that is a sequence of terms extracted by the term extraction unit 13 and acquires attributes of the term string. The attribute of the term string may be emotion or reputation corresponding to the term string. The attribute acquisition unit 37 can acquire attributes such as emotion and reputation of a term string by using, for example, a semantic understanding engine or a reputation analysis engine. The emotion may be, for example, “fun”, “sad”, “normal”, or the like. The reputation may be, for example, “positive”, “negative”, “normal”, and the like. Semantic understanding engines, reputation analysis engines, and the like are already known and will not be described in detail.

  The attribute information adding unit 38 adds attribute information, which is information indicating the attribute acquired by the attribute acquiring unit 37, to a term string that is a list of terms extracted by the term extracting unit 13 and passes the attribute information to the transmitting unit 15. It is preferable that the attribute information adding means 38 adds the attribute information to the term string so that it can be distinguished from other terms. For example, if the term string is “I”, “that flower”, “buy”, and the attribute “fun” is acquired as a result of the analysis by the semantic understanding engine, the attribute information “fun” is included in the term string. The term string may be “I”, “that flower”, “buy”: “fun”. In this case, information after the colon “:” is attribute information.

  In addition, although the change method of the term string of (1)-(4) was demonstrated here, it cannot be overemphasized that you may change the term string other than that.

  The transmission unit 15 transmits the term string that has been changed by the term string changing unit 14. Moreover, the transmission part 15 may read and transmit the program information corresponding to the term string from the program information storage part 11 together with the term string. The program information corresponding to the term string is program information corresponding to the caption information corresponding to the term string. In the present embodiment, a case will be described in which the transmission unit 15 also transmits program information together with a term string. When transmitting the term string and the program information, the transmission unit 15 may transmit the term string including the term string in the user-defined area or the vertical blanking interval of the program information. In addition, when subtitle information is included in program information, the transmission unit 15 may or may not transmit the subtitle information.

  The transmission unit 15 may directly transmit the term string or the like to the terminal device 2 or may indirectly transmit the term string or the like via a server or the like. Further, the transmission unit 15 may or may not include a transmission device (for example, a modem or a network card) for performing transmission. The transmission of the term string or the like by the transmission unit 15 may be broadcast. This broadcast may be, for example, a terrestrial broadcast, a satellite broadcast, or a wired broadcast via a cable. In the present embodiment, a case where the transmission unit 15 broadcasts program information including a term string will be described. The transmission unit 15 may be realized by hardware, or may be realized by software such as a driver that drives the transmission device.

  FIG. 7 is a block diagram showing the configuration of the terminal device 2 according to the present embodiment. In FIG. 7, the terminal device 2 according to the present embodiment includes a receiving unit 21, a program related information receiving unit 22, a sign language term correspondence information storage unit 23, a cache storage unit 24, a sign language video information generation unit 25, And an output unit 26. The terminal device 2 may or may not be an STB (Set Top Box) connected to a television receiver or the like, for example.

  The receiving unit 21 receives the term string transmitted from the server 1. When the program information is transmitted together with the term string from the server 1, the receiving unit 21 may also receive the program information transmitted from the server 1 together with the term string.

  The receiving unit 21 may or may not include a wired or wireless receiving device (for example, a modem or a network card) for receiving. The receiving unit 21 may receive a broadcast. The receiving unit 21 may be realized by hardware, or may be realized by software such as a driver that drives the receiving device. The term string and program information received by the receiving unit 21 may be temporarily stored in a recording medium (not shown).

  The program related information receiving unit 22 receives program related information which is information related to a program corresponding to the term string received by the receiving unit 21. The program related information may include, for example, information indicating the genre of the program, may include information indicating a term relating to the content of the program, or may include other information. The term relating to the contents of the program may be, for example, a program topic or a title, a program name, or a description of the program. The program-related information may be, for example, EPG (Electronic Program Guide) information itself, or may be a part of the information. This EPG information may be called, for example, SI information (official program information).

  The program related information receiving unit 22 may receive program related information input from an input device (for example, a keyboard, a mouse, a touch panel, etc.), for example, and acquires the program related information via a wired or wireless communication line. Alternatively, broadcast-related program-related information may be received, and program-related information read from a predetermined recording medium (for example, an optical disk, a magnetic disk, a semiconductor memory, etc.) may be received. The program related information may be transmitted by the server 1, for example. In addition, the program related information receiving unit 22 may or may not include a device (for example, a modem or a network card) for receiving. The program related information receiving unit 22 may be realized by hardware, or may be realized by software such as a driver that drives a predetermined device. The program related information received by the program related information receiving unit 22 may be temporarily stored in a recording medium (not shown).

  The sign language term correspondence information storage unit 23 stores sign language term correspondence information. The sign language term correspondence information is information that associates a term with sign language action information corresponding to the term. The sign language action information is information related to the sign language action. The information related to the sign language operation may be, for example, a sign language image itself, or information used to generate a sign language image (for example, information on a hand or face image, a hand or face trajectory, or the like). Information, which is not part of sign language video information), or a combination thereof. When the sign language motion information is a sign language image itself, the sign language motion information may be, for example, a CG (Computer Graphics) image, an animation image, or a live-action image. . Further, when the sign language motion information is a sign language video itself, the sign language motion information itself constitutes a part of the sign language video information generated by the sign language video information generation unit 25 described later. On the other hand, when the sign language motion information is information used to generate a sign language video and not the sign language video itself, sign language video information such as CG is generated using the sign language motion information. Become.

  As described above, when the attribute information is added to the term string by the term string changing unit 14, the sign language action information may include at least information for each attribute indicated by the attribute information. That is, the entire sign language action information may be information for each attribute, or a part of the sign language action information may be information for each attribute. For example, when the attribute information is “fun”, “sad”, or “normal”, the sign language action information includes sign language action information corresponding to the attribute information “fun” and sign language action information corresponding to the attribute information “sad”. Operation information and sign language operation information corresponding to the attribute information “normal” may be included. For example, only sign language action information related to facial expressions may be set for each of the attribute information “fun”, “sad”, and “normal”, or not.

  In addition, when sign language video information is generated using the program related information received by the program related information receiving unit 22, the sign language term correspondence information is information for associating terms with sign language action information and program related information. It may be. In this case, the sign language video information generation unit 25 described later generates sign language video information more suitable for the program by using the sign language operation information corresponding to the program related information received by the program related information reception unit 22. Can become possible. For example, when the program related information is information indicating the genre of the program, sign language operation information corresponding to the program related information “news” “documentary” and sign language operation information corresponding to the program related information “drama” “variety” May be different. In the former case, the sign language movement information in a serious atmosphere, and in the latter case, the sign language movement information in an entertaining and fun atmosphere (for example, the sign language movement may be slightly exaggerated). Good.

  The process in which the sign language term correspondence information is stored in the sign language term correspondence information storage unit 23 does not matter. For example, sign language term correspondence information may be stored in the sign language term correspondence information storage unit 23 via a recording medium, and sign language term correspondence information transmitted via a communication line or the like is stored in the sign language term correspondence information storage unit. 23, or the sign language term correspondence information input via the input device may be stored in the sign language term correspondence information storage unit 23. The storage in the sign language term correspondence information storage unit 23 may be temporary storage in a RAM or the like, or may be long-term storage. The sign language term correspondence information storage unit 23 can be realized by a predetermined recording medium (for example, a semiconductor memory, a magnetic disk, an optical disk, or the like).

  The cache storage unit 24 is accessible at a higher speed than the sign language term correspondence information storage unit 23. For example, when the sign language term correspondence information storage unit 23 is a hard disk, the cache storage unit 24 may be a memory, for example. Further, when the sign language term correspondence information storage unit 23 is a main memory, for example, the cache storage unit 24 may be a cache memory, for example. The sign language video information generation unit 25 described later may store sign language motion information in the cache storage unit 24 in advance, and generate sign language video information using the stored sign language motion information. In this way, sign language video information can be generated at higher speed.

  The storage in the cache storage unit 24 is basically a temporary storage, but this need not be the case. The cache storage unit 24 can be realized by a predetermined recording medium (for example, a semiconductor memory).

  The sign language video information generation unit 25 uses the sign language action information associated with each term included in the term string received by the reception unit 21 in the sign language term correspondence information, and indicates the sign language video corresponding to the term string. The sign language video information is generated. When the sign language action information is, for example, a sign language image itself, the sign language image information generation unit 25 may generate sign language image information by reading the sign language image. In addition, when the sign language motion information is information used to generate a sign language video and not the sign language video itself, the sign language video information generation unit 25 uses the sign language motion information and the like. Sign language video information such as CG is generated. The method for generating the sign language video information is described in the above-mentioned Patent Documents 1 to 4 and the like, and since it is already known, its detailed description is omitted.

  As described above, when the attribute information is added to the term string by the term string changing unit 14, the sign language video information generating unit 25 uses the sign language action information according to the attribute information included in the term string. Sign language video information may be generated. Specifically, as described above, the sign language action information also includes at least information for each attribute, and the sign language video information generation unit 25 uses the sign language action information corresponding to the attribute information added to the term string. Thus, sign language video information may be generated.

  In addition, the sign language video information generation unit 25 may generate sign language video information using the program related information received by the program related information reception unit 22. For example, in the sign language term correspondence information, when the term, the sign language action information, and the program related information are associated with each other, the sign language video information generation unit 25 receives the program related information ( The program related information may be generated using sign language motion information corresponding to the program related information of the program corresponding to the term string received by the receiving unit 21.

  Further, the sign language video information generation unit 25 stores the sign language action information in the cache storage unit 24 in advance using the program related information received by the program related information reception unit 22, and is stored in the cache storage unit 24. Sign language video information may also be generated using existing sign language action information. For example, when the program related information is information indicating terms related to the contents of the program, the sign language video information generating unit 25 stores sign language action information corresponding to the terms indicated by the program related information in the cache storage unit 24. Also good. Since the term indicated by the program related information is considered to appear frequently in the term string, the sign language operation information corresponding to the term is stored in the cache storage unit 24 so that the sign language operation information is accessed at high speed. As a result, sign language video information can be generated more quickly. For example, when the program related information is information indicating the genre of the program, the sign language video information generation unit 25 stores the sign language action information corresponding to the genre of the program indicated by the program related information in the cache storage unit 24. It may be left. In addition, when sign language action information is stored in the cache storage unit 24, a term corresponding to the sign language action information may be stored together with the sign language action information in the cache storage unit 24. It may not be so.

  The output unit 26 outputs the sign language video information generated by the sign language video information generation unit 25. The output unit 26 also outputs the program information received by the receiving unit 21 together with the sign language video information. By outputting the sign language video information and the program information, a person who views the output information can see the sign language video corresponding to the program together with the program. When program information and sign language video information are displayed, they may be output side by side, vertically, diagonally, etc. within one screen, and the other screen may be part of one screen. It may be output in a displayed form, and the form in which program information and sign language video information are displayed does not matter.

  Here, this output is, for example, display of sign language video information video and program information video on a display device (for example, CRT or liquid crystal display) and audio output of program information to a speaker. There may be. Alternatively, this output may be, for example, transmission of video / audio via a communication line to a predetermined device, accumulation in a recording medium, or delivery to another component. However, even in these cases, the video is ultimately displayed and the audio is output as audio. In addition, when only a hearing impaired person sees the image | video which the terminal device 2 output, audio | voice output does not need to be made. The output unit 26 may or may not include a device that performs output (for example, a display device or a printer). The output unit 26 may be realized by hardware, or may be realized by software such as a driver that drives these devices.

  Needless to say, the server 1 and the terminal device 2 may appropriately digitize or analogize information. Therefore, in the server 1 and the terminal device 2, there may be a DAC (Digital Analog Converter) for converting information into analog and an ADC (Analog Digital Converter) for digitizing information at necessary places.

Next, the operation of the sign language video generation system according to the present embodiment will be described using a flowchart.
FIG. 8 is a flowchart showing the operation of the server 1 according to this embodiment. In this flowchart, a case where the caption information receiving unit 12 receives the caption information by reading it from the program information storage unit 11 will be described. Also, a case will be described in which a term string is generated from the caption information, temporarily stored in a recording medium (not shown), and the temporarily stored term string is transmitted together with program information.

  (Step S101) The caption information receiving unit 12 determines whether or not to perform processing for generating a term string. And when performing the process which produces | generates a term string, it progresses to step S102, and when that is not right, it progresses to step S107. For example, when the server 1 receives an instruction to generate a term string, the caption information receiving unit 12 may determine that the term string is to be generated, and the program information storage unit 11 has the program information having the caption information. It may be determined that the term string is generated at the timing when the is newly accumulated, or it may be determined that the term string is generated at other timing. In addition, although it has been described here that the caption information receiving unit 12 performs the determination of this step, it is needless to say that other components may perform the determination.

  (Step S <b> 102) The caption information receiving unit 12 reads the caption information included in the program information stored in the program information storage unit 11. Note that the subtitle information read at a time is a unit that forms one subtitle. The unit may be, for example, the amount of subtitles displayed at a time on the program information screen, may be a single sentence, or may be other units.

  (Step S103) The term extraction unit 13 extracts a term that is an independent word from the caption information read by the caption information receiving unit 12.

  (Step S104) The term string changing unit 14 appropriately changes the term string extracted by the term extracting unit 13. Details of this process will be described later.

  (Step S105) The term string changing unit 14 temporarily stores the changed term string on a recording medium (not shown).

  (Step S106) The caption information receiving unit 12 determines whether or not a process of generating a term string has been performed for all caption information included in a certain program information. Then, when the process of generating the term string is performed for all the caption information, that is, when the process of generating all the term strings is completed, the process returns to step S101, and otherwise, the process returns to step S102. .

  (Step S107) The transmission unit 15 determines whether to transmit program information. And when transmitting, it progresses to step S108, and when that is not right, it returns to step S101. For example, the transmission unit 15 may determine to transmit the program information when the server 1 receives an instruction to transmit the program information, and has a schedule table related to the transmission of the program information. When it is time to transmit certain program information, it may be determined that program information is transmitted, or it may be determined that program information is transmitted at other timing.

  (Step S <b> 108) The transmission unit 15 reads program information from the program information storage unit 11. Note that the program information read in this step may be, for example, one frame of program information or a plurality of consecutive frames. Moreover, the subtitle information included in the program information may or may not be read.

  (Step S109) The transmission unit 15 reads a term string temporarily stored in a recording medium (not shown). This term string is a term string corresponding to the frame of the program information read in step S108. The term string corresponding to the frame of the program information means the term string generated from the caption information corresponding to the frame of the program information.

  (Step S110) The transmission unit 15 transmits the term string read out in step S109 by including it in the frame of the program information read out in step S108.

  (Step S111) The transmission unit 15 determines whether all the program information determined to be transmitted in Step S107 has been transmitted. If a frame that has not yet been transmitted remains, the process returns to step S108. If not, the process returns to step S101.

  If there is no term string corresponding to the frame of the program information read in step S108, the term string need not be read in step S109, and only the program information is transmitted in step S110. Also good.

  In addition, when transmitting a term string including program information in steps S109 and S110, the term string may be included in all frames of program information in which captions corresponding to the term string are displayed. Alternatively, the program information may be included and transmitted only in the first frame of the program information in which the caption corresponding to the term string is displayed, and the method for including the term string in the program information is not limited.

  Further, in the flowchart of FIG. 8, the processes of steps S101 to S106 and the processes of steps S107 to S111 may be executed in parallel. In that case, the generation of the term string and the transmission of the program information can be executed in parallel. For example, these processes may be processed as separate threads or separate processes.

In the flowchart of FIG. 8, a case has been described in which a term string is first generated and then program information including the term string is transmitted. However, the term string generation and the program information transmission may be performed simultaneously. Good. That is, a process for generating a certain term string, transmitting program information including the term string, and then generating a next term string and transmitting program information including the term string is repeatedly executed. May be.
Further, in the flowchart of FIG. 8, the processing is ended by powering off or interruption for aborting the processing.

  FIG. 9 is a flowchart showing details of the term string change processing (step S104) in the flowchart of FIG. The flowchart of FIG. 9 corresponds to the case where the term string changing unit 14 is the one shown in FIG.

  (Step S201) The term conversion means 32 of the term string changing unit 14 sets the counter i to 1.

  (Step S202) The term conversion means 32 determines whether the i-th term in the term string matches any of the conversion target terms stored in the term correspondence information storage means 31. If they match, the process proceeds to step S203, and if not, the process proceeds to step S204.

  (Step S203) The term conversion means 32 replaces the i-th term with a post-conversion term corresponding to the term.

  (Step S204) The term conversion means 32 increments the counter i by 1.

  (Step S205) The term conversion means 32 determines whether or not the i-th term exists in the term string. And when it exists, it returns to step S202, and when that is not right, it returns to the flowchart of FIG.

  FIG. 10 is a flowchart showing details of the term string change processing (step S104) in the flowchart of FIG. The flowchart in FIG. 10 corresponds to the case where the term string changing unit 14 is the one shown in FIG.

  (Step S301) The term rearrangement unit 34 of the term sequence changing unit 14 matches the order of terms included in the term sequence information storage unit 33 with the order of terms included in the term sequence. Judge whether or not. If they match, the terms need not be rearranged, so the process returns to the flowchart of FIG. 8. If they do not match, the terms need to be rearranged, and the process proceeds to step S302. For example, when the term order information is {noun} {verb} and the order of the terms included in the term string is “noun, verb, noun”, the order of the terms in the term string is the term It is determined that the order of the terms indicated by the order information does not match.

  (Step S302) The term rearranging means 34 rearranges the order of the terms included in the term string. For example, in step S301, rearrangement such as replacement of adjacent terms may be performed for a portion where it is determined that the order of terms does not match that indicated by the term order information. Then, in order to determine whether the rearranged term string is in an appropriate term order, the process returns to step S301. For example, in the case of the above-mentioned example, since the order of the last two “verbs and nouns” among the terms included in the term string is different, the term rearranging means 34 determines the two terms. The terms may be rearranged so that the order is “noun, noun, verb”. Even if a plurality of terms of the same part of speech are arranged, it may be determined that the term order information is used as one term of the part of speech. In that case, the order of “noun, noun, verb” after the rearrangement matches the term order information.

  FIG. 11 is a flowchart showing details of the term string change processing (step S104) in the flowchart of FIG. The flowchart in FIG. 11 corresponds to the case where the term string changing unit 14 is the one shown in FIG.

  (Step S401) The detection means 35 of the term string changing unit 14 determines whether a term is missing in the term string. And when missing, it progresses to step S402, and when that is not right, it returns to the flowchart of FIG.

  (Step S402) The term supplementing means 36 acquires a term to be supplemented. For example, when the term to be supplemented is the subject, the term supplementing means 36, as described above, sets the subject included in the term string before the term string to be changed as a term to be supplemented. You may get it.

  (Step S403) The term supplementing means 36 supplements the term string acquired in Step S402 to the term string. For example, when the term to be supplemented is the subject, the term supplementing means 36 may add the term to be supplemented to the head of the term string. And it returns to the flowchart of FIG.

  FIG. 12 is a flowchart showing details of the term string change processing (step S104) in the flowchart of FIG. The flowchart of FIG. 12 corresponds to the case where the term string changing unit 14 is the one shown in FIG.

  (Step S501) The attribute acquisition unit 37 of the term string changing unit 14 analyzes the term string. The analysis may be, for example, an analysis by a semantic understanding engine or an analysis by a reputation analysis engine.

  (Step S502) The attribute acquisition unit 37 acquires attribute information based on the analysis result. For example, this attribute information may indicate emotions such as “fun”, “sad”, and “normal”, or may indicate reputation such as “positive”, “negative”, and “normal”. .

  (Step S503) The attribute information adding unit 38 adds the attribute information acquired by the attribute acquiring unit 37 to the term string. And it returns to the flowchart of FIG.

  FIG. 13 is a flowchart showing the operation of the terminal device 2 according to this embodiment. In this flowchart, it is assumed that the program related information includes information indicating the genre of the program and information indicating terms relating to the contents of the program.

  (Step S601) The program related information receiving unit 22 determines whether program related information has been received. If accepted, the process proceeds to step S602; otherwise, the process proceeds to step S603.

  (Step S602) The program related information receiving unit 22 temporarily stores the received program related information in a recording medium (not shown). Then, the process returns to step S601.

  (Step S603) The receiving unit 21 determines whether to start receiving program information. And when starting, it progresses to step S604, and when that is not right, it returns to step S601. For example, when the terminal device 2 receives an instruction to receive a program, the receiving unit 21 may determine to start receiving program information, and the program of the channel received by the terminal device 2 ends. Then, when a new program starts, it may be determined to start receiving program information, or may be determined to start receiving program information at other timing.

  (Step S604) The sign language video information generation unit 25 reads, from the sign language term correspondence information storage unit 23, sign language action information corresponding to information indicating terms related to the contents of the program included in the program related information corresponding to the program to start reception. . Then, the sign language video information generation unit 25 stores the sign language motion information and the term corresponding to the sign language motion information in the cache storage unit 24 in association with each other. When there are a plurality of sign language action information corresponding to terms related to the contents of the program included in the program related information, the sign memory action information corresponding to the genre of the program included in the program related information is selected and the cache storage unit 24 is selected. You may accumulate in When the capacity of the cache storage unit 24 is large, the sign language video information generation unit 25 reads sign language action information corresponding to information indicating the genre of the program included in the program related information from the sign language term correspondence information storage unit 23. Then, it may be accumulated in the cache storage unit 24.

  (Step S605) The receiving unit 21 determines whether program information has been received. If the program information has been received, the process proceeds to step S606, and if not, the process proceeds to step S611. Note that the receiving unit 21 may receive the program information for each frame, or may receive it for each of a plurality of consecutive frames.

  (Step S606) The output unit 26 determines whether sign language video information corresponding to the program information received by the receiving unit 21 has already been generated. If it has already been generated, the process proceeds to step S610. If not, the process proceeds to step S607.

  (Step S607) The sign language video information generation unit 25 acquires a term string included in the program information.

  (Step S608) The sign language video information generation unit 25 reads the sign language action information corresponding to the acquired term string from the cache storage unit 24 and the sign language term correspondence information storage unit 23, and uses the sign language action information to sign language video information. Is generated.

  (Step S609) The sign language video information generation unit 25 temporarily stores the generated sign language video information in a recording medium (not shown).

  (Step S610) The output unit 26 reads sign language video information corresponding to the program information received by the receiving unit 21 from a recording medium (not shown), and outputs the sign language video information and the program information. Note that the output unit 26 may read and output sign language video information of the same number of frames as the program information received by the receiving unit 21 in order from the oldest.

  (Step S611) The receiving unit 21 determines whether to end the reception of program information. If the process is to end, the process returns to step S601; otherwise, the process returns to step S605. For example, when the terminal device 2 receives an instruction to end the reception of the program, the receiving unit 21 may determine that the reception of the program information is ended, and the program of the channel received by the terminal device 2 May be determined to end the reception of the program information.

  In the flowchart of FIG. 13, when a new term string is acquired, the sign language video information corresponding to the term string is continuously generated (steps S607 to S609). Generation of video information and reception of program information and output of program information and sign language video information may be performed in parallel.

  In the flowchart of FIG. 13, even if it is determined in step S606 that there is no sign language video information, if there is no term string corresponding to the program information received in step S605, that is, the program information is supported. When there is no subtitle information, it is only necessary to perform the process of outputting the program information without acquiring the term string or generating the sign language video information.

In the flowchart of FIG. 13, the processes in steps S601 and S602 and the processes in steps S603 to S611 may be executed in parallel. In that case, reception of program related information and output of program information and sign language video information can be executed in parallel. For example, these processes may be processed as separate threads or separate processes.
In the flowchart of FIG. 13, the process is terminated by powering off or a process termination interrupt.

  Next, the operation of the sign language video generation system according to the present embodiment will be described using a specific example. In this specific example, a case where attribute information is added as a change in term string will be described. Therefore, it is assumed that the term string changing unit 14 is as shown in FIG. The server 1 is assumed to be a broadcasting station that broadcasts program information.

  First, it is assumed that program information identified by the program ID “P001” is accumulated in the program information storage unit 11 in the server 1. Then, at that timing, the subtitle information reception unit 12 determines to generate a term string (step S101), and the first subtitle information “I am added to the program information identified by the program ID“ P001 ”. “I bought a flower” is read out and passed to the term extraction unit 13 (step S102). The term extraction unit 13 performs a morphological analysis on the received caption information “I bought a flower”, and uses the result of the morphological analysis to arrange an independent word sequence “I”, “flower”, “buy” Is extracted and passed to the term string changing unit 14 (step S103).

  The term string changing unit 14 performs a change process on the term string (step S104). Specifically, the attribute acquisition unit 37 of the term string changing unit 14 analyzes the received term string (step S501), acquires attribute information “normal” corresponding to the term string, and sends it to the attribute information adding unit 38. (Step S502) The attribute information adding means 38 adds the attribute information “normal” to the term strings “I”, “flower”, and “buy” (step S503). As a result, the term sequence is as follows. Information following the colon is attribute information.

  “I” “Flowers” “Buy”: Normal

  The term string changing unit 14 temporarily stores the changed term string in a recording medium (not shown). The first record in FIG. 14 is a diagram illustrating an example of a term string stored in this way. In FIG. 14, a program ID, a time code (start), a time code (end), and a term string are associated with each other. The program ID is a program ID for identifying program information that includes subtitle information that is a basis for generating a term string. The time code (starting time) is the time code of the program information at the time when the caption information that is the basis for generating the term string appears for the first time in the program information. The time code (end) is the time code of the program information at the time when the caption information that is the basis for generating the term string appears last in the program information. Accordingly, the subtitle information “I bought a flower” starts from the time code “00 (hours): 00 (minutes): 00 (seconds): 01 (frames)” and the time code “00: 00: 05: 02”. "Will exist.

  Thereafter, the process of generating the term string from the caption information of the program information identified by the program ID “P001” is continuously performed, and the term string is accumulated as shown in FIG. (Steps S102 to S106).

  Next, it is assumed that the transmission unit 15 refers to a schedule table (not shown) indicating the schedule of the program to be broadcast and determines that it is the timing to broadcast the program information identified by the program ID “P001” (step S107). Then, the transmission unit 15 reads a frame of program information identified by the program ID “P001” from the program information storage unit 11 (step S108). Further, the transmission unit 15 refers to the table shown in FIG. 14 and determines whether there is a record in which the time code of the read frame is included from the time code (start) to the time code (end). . If it exists, the term string included in the record is read (step S109). Thereafter, the transmission unit 15 broadcasts program information including the read term string (step S110). The transmission unit 15 repeatedly executes the process of transmitting the program information until all the program information has been transmitted (steps S108 to S111).

  Next, a specific example of the operation of the terminal device 2 will be described. First, processing for receiving program-related information will be described. The program related information reception unit 22 transmits a program related information transmission request to a predetermined server on the Internet every day at 2 am, and receives the program related information transmitted from the server in response to the transmission. (Step S601). Then, the program related information receiving unit 22 accumulates the received program related information in a recording medium (not shown) (step S602). FIG. 15 is a diagram showing an example of program-related information accumulated in this way. In FIG. 15, one record corresponds to one program, and in each record, a program ID for identifying a program corresponding to the record, a genre of a program corresponding to the record, and a program corresponding to the record Are associated with terms related to the contents of Terms related to the contents of a program are terms that appear in the program. For example, the first record in FIG. 15 shows that terms such as “flower”, “Nao”, and “Daisuke” appear in the program.

  Assume that the terminal device 2 is set so that the user of the terminal device 2 receives the program information identified by the program ID “P001”. Then, at the timing when transmission of the program ID “P001” is started, the receiving unit 21 determines to start receiving the program information (step S603), and is identified by the sign language video information generating unit 25 with the program ID “P001”. To start receiving program information. Then, the sign language video information generation unit 25 refers to the program related information shown in FIG. 15, and the program genre “drama” corresponding to the program ID “P001” and the terms “flower” “Nao” related to the content of the program. Read "Daisuke" and so on. Then, out of the sign language term correspondence information associated with these terms, the sign language term correspondence information corresponding to the program genre “drama” is read from the sign language term correspondence information storage unit 23 and stored in the cache storage unit 24. (Step S604). It is assumed that the sign language term correspondence information storage unit 23 stores sign language term correspondence information shown in FIG. In FIG. 16, terms, program genres, attribute information, and sign language action information are associated with each other. For example, when the sign language video information generation unit 25 caches the sign language term corresponding information corresponding to the term “flower”, the attribute information corresponding to the term “flower” and the program genre “drama” shown in FIG. And sign language action information are read out and stored in the cache storage unit 24.

  Thereafter, the receiving unit 21 of the terminal device 2 receives the program information identified by the program ID “P001”, and passes the program information to the sign language video information generating unit 25 and the output unit 26 (step S605). The sign language video information generation unit 25 determines whether the sign language video information has already been generated and stored (step S606). In this case, it is assumed that sign language video information has not yet been stored. Then, the sign language video information obtains the term string “I” “flower” “buy”: normal ”included in the received program information (step S607). Further, the sign language video information generation unit 25 refers to the information in FIG. 15 and acquires the genre “drama” of the program identified by the program ID “P001”. Then, the sign language video information generation unit 25 searches the cache storage unit 24 and the sign language term correspondence information storage unit 23 using the acquired term string, the attribute information “ordinary”, and the program genre “drama”, The sign language motion information “D003” or the like is read to generate sign language video information (step S608), and is temporarily stored in a recording medium (not shown) (step S609). When sign language video information is temporarily stored, the program ID “P001” and the term string “I” “flower” “buy”: normal ”may be stored in association with each other. By doing so, it becomes possible to know which sign language video information corresponds to which term string of which program.

  The output unit 26 reads out from the recording medium (not shown) as many sign language video information frames as the number of frames of the program information received from the receiving unit 21 and outputs the frames included in the program information received from the receiving unit 21 (step S610). ). For example, the program information and sign language video information are as shown in FIG. Note that the output of the program information identified by the program ID “P001” is continued until the program ends (steps S605 to S611).

  In this specific example, as shown in FIG. 17, a case has been described in which sign language video information is output in an image of program information, but this need not be the case. For example, as shown in FIG. 18, program information and sign language video information may be displayed side by side with the same size.

  Further, in this specific example, a case has been described in which a term string is transmitted by being included in a frame in which caption information corresponding to the term string is superimposed, but this need not be the case. For example, if the time required to generate sign language video information from a term sequence is large, that is, if it is difficult to generate and display sign language video information in real time, only the term sequence is transmitted in advance. In addition, the terminal device 2 may generate sign language video information corresponding to the term string in advance and output the generated sign language video information and the program information in synchronization. . In this case, it is preferable that information used for synchronizing the sign language video information and the program information is transmitted from the server 1 to the terminal device 2 together with the term string. The information used to synchronize the sign language video information and the program information may be information that can specify a frame in which caption information corresponding to the term string is displayed, for example.

  As described above, according to the sign language video generation system according to the present embodiment, after the term is extracted from the caption information in the server 1 or the term sequence is changed, the term sequence is transmitted to the terminal device 2. As a result, the amount of information transmitted to the terminal device 2 is reduced and the processing amount in the terminal device 2 is reduced. As a result, the time required to generate sign language video information in the terminal device 2 can be reduced. In this manner, appropriate load distribution can be performed between the server 1 and the terminal device 2.

  In the present embodiment, the case has been described where subtitle information is included in the program information stored in the program information storage unit 11, but this need not be the case. For example, the program information stored in the program information storage unit 11 does not include subtitle information, and the subtitle information may be acquired by performing voice recognition processing on the audio information of the program information. FIG. 19 is a block diagram showing the configuration of the server 1 in that case. 19, the server 1 includes a program information storage unit 11, a caption information receiving unit 12, a term extracting unit 13, a term string changing unit 14, a transmitting unit 15, a voice information receiving unit 16, and a voice recognizing unit. 17. Configurations and operations other than the voice information reception unit 16 and the voice recognition unit 17 are the same as those described above, and a description thereof will be omitted. The audio information receiving unit 16 receives audio information of a program. The voice recognition unit 17 acquires subtitle information by performing voice recognition processing on the voice information received by the voice information reception unit 16. The voice recognition process is already known and will not be described in detail. Although FIG. 19 shows the case where the audio information receiving unit 16 reads the audio information from the program information storage unit 11, this need not be the case. For example, the voice information receiving unit 16 may receive voice information input from the outside. Note that the voice information receiving unit 16 may or may not include a device (for example, a modem or a network card) for receiving. The audio information receiving unit 16 may be realized by hardware, or may be realized by software such as a driver that drives a predetermined device.

  In the present embodiment, the case where the server 1 transmits a term string included in program information has been described, but this need not be the case. The term string and the program information may be transmitted separately. Further, only the term string may be transmitted without transmitting the program information. In that case, the server 1 may not include the program information storage unit 11. When the server 1 does not include the program information storage unit 11, the caption information receiving unit 12 may receive, for example, caption information input from the outside. Further, when the terminal device 2 does not receive program information from the server 1, the terminal device 2 may receive program information from another server or the like, or may output only sign language video information without receiving program information. Good.

  Moreover, although this Embodiment demonstrated the case where the terminal device 2 was provided with the cache memory | storage part 24, it may not be so. The terminal device 2 may generate sign language video information without using the cache storage unit 24.

  In the present embodiment, the case where the terminal device 2 generates sign language video information using program-related information has been described, but this need not be the case. The sign language video information may be generated without using the program related information. In this case, the terminal device 2 may not include the program related information receiving unit 22.

  In the above embodiment, each process or each function may be realized by centralized processing by a single device or a single system, or may be distributedly processed by a plurality of devices or a plurality of systems. It may be realized by doing.

  In the above embodiment, information related to processing executed by each component, for example, information received, acquired, selected, generated, transmitted, or received by each component In addition, information such as threshold values, mathematical formulas, addresses, etc. used by each component in processing is retained temporarily or over a long period of time on a recording medium (not shown) even when not explicitly stated in the above description. It may be. Further, the storage of information in the recording medium (not shown) may be performed by each component or a storage unit (not shown). Further, reading of information from the recording medium (not shown) may be performed by each component or a reading unit (not shown).

  In the above embodiment, when information used by each component, for example, information such as a threshold value, an address, and various setting values used by each component may be changed by the user Even if it is not specified in the above description, the user may be able to change the information as appropriate, or it may not be. If the information can be changed by the user, the change is realized by, for example, a not-shown receiving unit that receives a change instruction from the user and a changing unit (not shown) that changes the information in accordance with the change instruction. May be. The change instruction received by the receiving unit (not shown) may be received from an input device, information received via a communication line, or information read from a predetermined recording medium, for example. .

  In the above embodiment, when two or more components of each device included in the sign language video generation system have a communication device, an input device, etc., the two or more components have a physically single device. Or you may have separate devices.

  In the above embodiment, each component may be configured by dedicated hardware, or a component that can be realized by software may be realized by executing a program. For example, each component can be realized by a program execution unit such as a CPU reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory. In addition, the software which implement | achieves the server 1 in the said embodiment is the following programs. That is, the program includes a subtitle information receiving unit that receives subtitle information that is information indicating a subtitle of a program, a term extracting unit that extracts a term that is an independent word from the subtitle information received by the subtitle information receiving unit, and The program for functioning as a term string changing unit that changes a term string that is a sequence of terms extracted by the term extracting unit and a transmitter that transmits the term string changed by the term string changing unit.

  Moreover, the following programs may be sufficient as the software which implement | achieves the terminal device 2 in the said embodiment. That is, the program uses a receiving unit that receives a term sequence that is a sequence of terms that are independent words extracted from subtitle information that is information indicating subtitles of a program, and a term sequence received by the receiving unit. For each term included, the sign language term correspondence information storage unit stores sign language term correspondence information, which is information for associating the term with sign language motion information that is information related to the sign language motion corresponding to the term. Using sign language action information associated with the sign language term correspondence information, a sign language image information generating unit that generates sign language image information that is information indicating a sign language image corresponding to the term string, and the sign language image information generating unit The term string is added with attribute information which is information indicating the attribute of the term string, and the sign language action information is attribute information. At least it includes information for each attribute indicated by the sign language video information generating unit generates the sign language video information using sign language operation information corresponding to the attribute information included in the term column, a program.

  Moreover, the following programs may be sufficient as the software which implement | achieves the terminal device 2 in the said embodiment. That is, the program uses a receiving unit that receives a term sequence that is a sequence of terms that are independent words extracted from subtitle information that is information indicating subtitles of a program, and a term sequence received by the receiving unit. A program-related information receiving unit that receives program-related information, which is information related to a corresponding program, and each term included in the term string received by the receiving unit includes a term and information on the sign language operation corresponding to the term. Sign language action information associated with the sign language term correspondence information stored in the sign language term correspondence information storage unit in which sign language term correspondence information, which is information for associating certain sign language action information, and the program related information receiving unit, A sign language video information generating unit that generates sign language video information, which is information indicating a sign language video corresponding to the term sequence, using the received program related information; Is a program for generating part to function as an output unit for outputting the generated sign language video information.

  In the program, the functions realized by the program do not include functions that can be realized only by hardware. For example, a function that can be realized only by hardware such as a modem and an interface card in a receiving unit that receives information, an output unit that outputs information, and the like is not included at least in the functions realized by the program.

  Further, this program may be executed by being downloaded from a server or the like, and a program recorded on a predetermined recording medium (for example, an optical disk such as a CD-ROM, a magnetic disk, a semiconductor memory, or the like) is read out. May be executed by Further, this program may be used as a program constituting a program product.

  Further, the computer that executes this program may be singular or plural. That is, centralized processing may be performed, or distributed processing may be performed.

  FIG. 20 is a schematic diagram illustrating an example of an external appearance of a computer that executes the program and realizes the server 1 and the terminal device 2 according to the embodiment. The above-described embodiment can be realized by computer hardware and a computer program executed on the computer hardware.

  20, a computer system 900 includes a computer 901 including a CD-ROM (Compact Disk Read Only Memory) drive 905 and an FD (Floppy (registered trademark) Disk) drive 906, a keyboard 902, a mouse 903, a monitor 904, and the like. Is provided.

  FIG. 21 is a diagram showing an internal configuration of the computer system 900. In FIG. 21, in addition to the CD-ROM drive 905 and the FD drive 906, a computer 901 is connected to an MPU (Micro Processing Unit) 911, a ROM 912 for storing a program such as a bootup program, and the MPU 911. A RAM (Random Access Memory) 913 that temporarily stores program instructions and provides a temporary storage space, a hard disk 914 that stores application programs, system programs, and data, and an MPU 911 and a ROM 912 are interconnected. And a bus 915. The computer 901 may include a network card (not shown) that provides connection to the LAN.

  A program that causes the computer system 900 to execute the functions of the server 1 and the terminal device 2 according to the above-described embodiment is stored in the CD-ROM 921 or the FD 922, inserted into the CD-ROM drive 905 or the FD drive 906, and the hard disk 914 may be forwarded. Instead, the program may be transmitted to the computer 901 via a network (not shown) and stored in the hard disk 914. The program is loaded into the RAM 913 when executed. The program may be loaded directly from the CD-ROM 921, the FD 922, or the network.

  The program does not necessarily include an operating system (OS) or a third-party program that causes the computer 901 to execute the functions of the server 1 and the terminal device 2 according to the above-described embodiment. The program may include only a part of an instruction that calls an appropriate function (module) in a controlled manner and obtains a desired result. How the computer system 900 operates is well known and will not be described in detail.

  Further, the present invention is not limited to the above-described embodiment, and various modifications are possible, and it goes without saying that these are also included in the scope of the present invention.

  As described above, according to the sign language video generation system and the like according to the present invention, an effect that appropriate load distribution can be performed between the server and the terminal device is obtained, which is useful as a system for generating a sign language video.

The block diagram which shows the structure of the sign language video generation system by Embodiment 1 of this invention. The block diagram which shows the structure of the server by the embodiment The block diagram which shows the structure of the term string change part by the embodiment The block diagram which shows the structure of the term string change part by the embodiment The block diagram which shows the structure of the term string change part by the embodiment The block diagram which shows the structure of the term string change part by the embodiment The block diagram which shows the structure of the terminal device by the embodiment The flowchart which shows operation | movement of the server by the embodiment The flowchart which shows operation | movement of the server by the embodiment The flowchart which shows operation | movement of the server by the embodiment The flowchart which shows operation | movement of the server by the embodiment The flowchart which shows operation | movement of the server by the embodiment The flowchart which shows operation | movement of the terminal device by the embodiment The figure which shows an example of the term string etc. in the embodiment The figure which shows an example of the program relevant information in the embodiment The figure which shows an example of the sign language term corresponding information in the embodiment The figure which shows an example of the program information etc. which were output in the embodiment The figure which shows an example of the program information etc. which were output in the embodiment The block diagram which shows the structure of the server by other embodiment. The schematic diagram which shows an example of the external appearance of the computer system in the said embodiment The figure which shows an example of a structure of the computer system in the said embodiment

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Server 2 Terminal device 11 Program information storage part 12 Subtitle information reception part 13 Term extraction part 14 Term string change part 15 Transmission part 16 Voice information reception part 17 Voice recognition part 21 Reception part 22 Program related information reception part 23 Sign language term corresponding information Storage unit 24 Cache storage unit 25 Sign language video information generation unit 26 Output unit 31 Term correspondence information storage unit 32 Term conversion unit 33 Term order information storage unit 34 Term rearrangement unit 35 Detection unit 36 Term supplementation unit 37 Attribute acquisition unit 38 Attribute information Additional means

Claims (17)

A sign language video generation system comprising a server and a terminal device,
The server
A subtitle information receiving unit that receives subtitle information that is information indicating the subtitles of the program;
A term extraction unit for extracting a term that is an independent word from the subtitle information received by the subtitle information reception unit;
A term string changing unit for changing a term string that is a sequence of terms extracted by the term extracting unit;
A transmission unit that transmits the term string changed by the term string changing unit,
The terminal device
A receiving unit for receiving the term string;
A sign language term correspondence information storage unit that stores sign language term correspondence information that is information that associates a term with sign language motion information that is information related to the sign language motion corresponding to the term;
Using the sign language action information associated with each term included in the term string received by the receiving unit in the sign language term correspondence information, sign language video information that is information indicating a sign language video corresponding to the term string is generated. A sign language video information generation unit,
An output unit for outputting the sign language video information generated by the sign language video information generation unit ,
The term string changing unit
A term correspondence information storage means for storing term correspondence information that is information for associating a conversion target term that is a term to be converted with a converted term that is a term after conversion;
A term conversion means for converting the conversion target term into a post-conversion term corresponding to the conversion target term when the conversion target term is included in the term string;
The term correspondence information is information that associates a conversion target term that is not associated with sign language action information in the sign language term correspondence information and a converted term that is associated with sign language action information in the sign language term correspondence information. , Sign language video generation system. A sign language video generation system comprising a server and a terminal device,
The server
A subtitle information receiving unit that receives subtitle information that is information indicating the subtitles of the program;
A term extraction unit for extracting a term that is an independent word from the subtitle information received by the subtitle information reception unit;
A term string changing unit for changing a term string that is a sequence of terms extracted by the term extracting unit;
A transmission unit that transmits the term string changed by the term string changing unit,
The terminal device
A receiving unit for receiving the term string;
A program related information receiving unit that receives program related information that is information related to a program corresponding to the term sequence received by the receiving unit;
A sign language term correspondence information storage unit for storing sign language term correspondence information, which is information for associating a term , sign language behavior information that is information about sign language behavior corresponding to the term, and program related information ;
Using the sign language action information associated with each term included in the term sequence received by the receiving unit and the program related information received by the program related information receiving unit using the sign language term correspondence information, A sign language video information generating unit for generating sign language video information which is information indicating a corresponding sign language video;
A sign language video generation system comprising: an output unit that outputs sign language video information generated by the sign language video information generation unit . A sign language video generation system comprising a server and a terminal device,
The server
A subtitle information receiving unit that receives subtitle information that is information indicating the subtitles of the program;
A term extraction unit for extracting a term that is an independent word from the subtitle information received by the subtitle information reception unit;
A term string changing unit for changing a term string that is a sequence of terms extracted by the term extracting unit;
A transmission unit that transmits the term string changed by the term string changing unit,
The terminal device
A receiving unit for receiving the term string;
A sign language term correspondence information storage unit that stores sign language term correspondence information that is information that associates a term with sign language motion information that is information related to the sign language motion corresponding to the term;
Using the sign language action information associated with each term included in the term string received by the receiving unit in the sign language term correspondence information, sign language video information that is information indicating a sign language video corresponding to the term string is generated. A sign language video information generation unit,
An output unit for outputting sign language video information generated by the sign language video information generation unit;
A program related information receiving unit that receives program related information that is information related to a program corresponding to the term sequence received by the receiving unit;
And a accessible cache storage unit faster than the sign language term correspondence information storage unit,
The sign language video information generation unit stores in advance the sign language operation information in the cache storage unit using the program related information received by the program related information reception unit, and the sign language operation stored in the cache storage unit information to generate a sign language video information using also a sign language video generation system. The program-related information is information indicating the genre of the program, according to claim 2 or claim 3 Sign image generation system according. The program related information is information indicating terms relating to the contents of the program,
The sign language video generation system according to claim 3 , wherein the sign language video information generation unit stores, in the cache storage unit, sign language action information corresponding to a term indicated by the program related information. The server
An audio information receiving unit for receiving audio information of the program;
A voice recognition unit that obtains subtitle information by performing voice recognition processing on the voice information received by the voice information reception unit;
The subtitle information receiving unit, the voice recognition unit accepts the subtitle information that is the result of speech recognition process, the sign language video generation system according to any one of claims 1 to 5. The server
A program information storage unit for storing program information that is program information;
The transmission unit reads the program information corresponding to the term string together with the term string from the program information storage unit and transmits the program information,
The receiving unit also receives the program information transmitted by the transmitting unit together with the term string,
The sign language image generation system according to any one of claims 1 to 6 , wherein the output unit outputs the program information received by the receiving unit together with the sign language image information. The transmission unit is for broadcasting the information, sign language video generation system according to any one of claims 1 to 7. A subtitle information receiving unit that receives subtitle information that is information indicating the subtitles of the program;
A term extraction unit for extracting a term that is an independent word from the subtitle information received by the subtitle information reception unit;
A term string changing unit for changing a term string that is a sequence of terms extracted by the term extracting unit;
A transmission unit that transmits the term sequence changed by the term sequence change unit to the terminal device ,
The term string changing unit
A term correspondence information storage means for storing term correspondence information that is information for associating a conversion target term that is a term to be converted with a converted term that is a term after conversion;
A term conversion means for converting the conversion target term into a post-conversion term corresponding to the conversion target term when the conversion target term is included in the term string;
The term correspondence information is information used to generate sign language video information from a term string in the terminal device, and is information that associates a term with sign language action information that is information related to a sign language action corresponding to the term. A server that is information that associates a conversion target term that is not associated with sign language action information in correspondence information with a converted term that is associated with sign language action information in the sign language term correspondence information . A receiving unit that receives a term string that is a sequence of terms that are independent words extracted from subtitle information that is information indicating subtitles of a program;
A program related information receiving unit that receives program related information that is information related to a program corresponding to the term sequence received by the receiving unit;
A sign language term correspondence information storage unit for storing sign language term correspondence information, which is information for associating a term , sign language behavior information that is information about sign language behavior corresponding to the term, and program related information ;
With each term pre Symbol receiver is included in the term sequence received, the sign language operation information which is associated with the sign language terms corresponding information to the program-related information the program-related information accepting unit accepts, the term column A sign language video information generating unit for generating sign language video information which is information indicating a sign language video corresponding to
An output unit that outputs sign language video information generated by the sign language video information generation unit. A receiving unit that receives a term string that is a sequence of terms that are independent words extracted from subtitle information that is information indicating subtitles of a program;
A sign language term correspondence information storage unit that stores sign language term correspondence information that is information that associates a term with sign language motion information that is information related to the sign language motion corresponding to the term
A program related information receiving unit that receives program related information that is information related to a program corresponding to the term sequence received by the receiving unit;
Using the sign language action information associated with each term included in the term string received by the receiving unit using the sign language term correspondence information, sign language video information that is information indicating a sign language video corresponding to the term string is generated. A sign language video information generation unit;
An output unit for outputting sign language video information generated by the sign language video information generation unit;
A cache storage unit accessible at a higher speed than the sign language term information storage unit,
The sign language video information generation unit stores in advance the sign language operation information in the cache storage unit using the program related information received by the program related information reception unit, and the sign language operation stored in the cache storage unit A terminal device that generates sign language video information using information as well. A term correspondence information storage means for storing term correspondence information that is information for associating a conversion target term that is a term to be converted with a converted term that is a term after conversion, and a term string changing unit having term conversion means; An information processing method processed using a caption information reception unit, a term extraction unit, and a transmission unit,
The subtitle information receiving unit receives subtitle information, which is information indicating the subtitles of the program,
The term extracting unit extracts a term that is an independent word from the caption information received in the caption information receiving step , and
A term string changing step in which the term string changing unit changes a term string that is a sequence of terms extracted in the term extracting step;
The transmission unit includes a transmission step of transmitting the term sequence changed in the term sequence change step to the terminal device ,
The term string changing step includes:
When the term conversion means includes the conversion target term in the term string, the term conversion unit has a term conversion step of converting the conversion target term into a post-conversion term corresponding to the conversion target term,
The term correspondence information is information used to generate sign language video information from a term string in the terminal device, and is information that associates a term with sign language action information that is information related to a sign language action corresponding to the term. An information processing method , which is information for associating a conversion target term that is not associated with sign language action information in correspondence information and a converted term that is associated with sign language action information in the sign language term correspondence information . An information processing method processed using a receiving unit, a program related information receiving unit, a sign language video information generating unit, and an output unit,
A receiving step of the receiving unit receives the terms column is a sequence of terms is independent word extracted from the caption information is information indicating the subtitle of the program,
A program related information receiving step in which the program related information receiving unit receives program related information which is information related to a program corresponding to the term string received in the receiving step;
The sign language video information generating unit includes information on terms included in the term string received in the receiving step and program related information received in the program related information receiving step, and information on sign language operations corresponding to the terms. using the sign language motion information is, the sign language operation information which is associated with the sign language terms corresponding information sign language terms corresponding information which is information for associating the program related information are stored in the sign language terms corresponding information storage unit to be stored A sign language video information generating step for generating sign language video information that is information indicating a sign language video corresponding to the term string;
An information processing method the output portion has and an output step of outputting a sign language video information generated by the sign language video information generating step. An information processing method processed using a receiving unit, a program related information receiving unit, a sign language video information generating unit, and an output unit,
A receiving step in which the receiving unit receives a term string that is a sequence of terms that are independent words extracted from caption information that is information indicating captions of a program;
A program related information receiving step in which the program related information receiving unit receives program related information which is information related to a program corresponding to the term string received in the receiving step;
The sign language video information generation unit is information for associating each term included in the term string received in the reception step with a term and sign language motion information that is information related to the sign language motion corresponding to the term. Sign language video information, which is information indicating a sign language video corresponding to the term string, using sign language action information associated with the sign language term correspondence information stored in the sign language term correspondence information storage unit in which information is stored. A sign language video information generation step to generate;
The output unit includes an output step of outputting the sign language video information generated in the sign language video information generation step;
In the sign language video information generation step, using the program related information received in the program related information receiving step, sign language action information is stored in advance in a cache storage unit accessible at a higher speed than the sign language term correspondence information storage unit. An information processing method for generating sign language video information using sign language action information stored in the cache storage unit. A computer that can access term correspondence information storage means in which term correspondence information, which is information that associates a conversion target term that is a conversion target term with a converted term that is a converted term, is stored .
A subtitle information receiving unit that receives subtitle information that is information indicating the subtitles of the program;
A term extraction unit for extracting a term that is an independent word from the subtitle information received by the subtitle information reception unit;
A term string changing unit for changing a term string that is a sequence of terms extracted by the term extracting unit;
The term string changed by the term string changing unit is caused to function as a transmission unit that transmits to the terminal device ,
The term string changing unit
When the conversion target term is included in the term string, the conversion unit includes a term conversion unit that converts the conversion target term into a post-conversion term corresponding to the conversion target term,
The term correspondence information is information used to generate sign language video information from a term string in the terminal device, and is information that associates a term with sign language action information that is information related to a sign language action corresponding to the term. A program that is information that associates a conversion target term that is not associated with sign language action information in correspondence information and a converted term that is associated with sign language action information in the sign language term correspondence information . Computer
A receiving unit that receives a term string that is a sequence of terms that are independent words extracted from subtitle information that is information indicating subtitles of a program;
A program related information receiving unit that receives program related information that is information related to a program corresponding to the term sequence received by the receiving unit;
Each term included in the term string received by the receiving unit and the program-related information received by the program-related information receiving unit include a term, sign language action information that is information related to sign language action corresponding to the term, and a program using sign language operation information which is associated with the sign language terms corresponding information sign language terms corresponding information which is information for associating the related information are stored in the sign language terms corresponding information storage unit to be stored, corresponding to the term column A sign language video information generation unit for generating sign language video information which is information indicating a sign language video;
A program for functioning as an output unit that outputs sign language video information generated by the sign language video information generation unit. Computer
A receiving unit that receives a term string that is a sequence of terms that are independent words extracted from subtitle information that is information indicating subtitles of a program;
A program related information receiving unit that receives program related information that is information related to a program corresponding to the term sequence received by the receiving unit;
Sign language term correspondence information in which each term included in the term string received by the receiving unit is stored with sign language term correspondence information that is information that associates a term with sign language action information that is information related to sign language action corresponding to the term. A sign language video information generating unit that generates sign language video information that is information indicating a video of a sign language corresponding to the term sequence, using sign language action information associated with the sign language term correspondence information stored in the information storage unit; ,
Function as an output unit that outputs the sign language video information generated by the sign language video information generation unit,
The sign language video information generation unit stores in advance a sign language action information in a cache storage unit accessible at a higher speed than the sign language term correspondence information storage unit using the program related information received by the program related information reception unit. A program for generating sign language video information using the sign language action information stored in the cache storage unit.

Images