JP7072390B2 - Sign language translator and program - Google Patents

Description

The present invention relates to a sign language translator and a program.

In sign language CG (Computer Graphics) animation generation, a method of real-time translation from a voice language such as Japanese to a sign language language is used. As this method, there is a method using machine translation using a corpus, which is bilingual data of Japanese and sign language (see, for example, Patent Documents 1 and 2).

Currently, in order to generate a sign language CG animation, first, a sign language word string that is a translation result is output using the above-mentioned translation method. Next, a method of reading the sign language motion data corresponding to each word in the sign language word string, synthesizing the motion data for each sentence, and reproducing it with a CG model is common.

Japanese Unexamined Patent Publication No. 2013-186673 Japanese Unexamined Patent Publication No. 2014-21180

It is difficult to completely match the length of the sign language generated as an animation with the length of the original voice because the utterance lengths of words representing the same vocabulary are different between the voice language and the sign language language. Furthermore, sign language, which is a visual language and involves complicated physical movements, often has a longer utterance length than speech language, which is an auditory language. Therefore, when the textualized voice is translated into sign language in real time, there is a delay from the original voice to the presentation of the sign language, and the delay is accumulated as the utterances overlap. For example, when a sign language CG animation that is real-time translated from a program sound is given to a TV program, a delay occurs between the main TV program and the sign language that should be interpolating the sound information, and the user who watches the program Is a big burden.

The present invention has been made in consideration of such circumstances, and provides a sign language translation device and a program capable of performing sign language translation with reduced delay from speech.

One aspect of the present invention is a sign language translation unit that converts the text of the utterance content into sign language translation information in which motion data indicating the movement of the sign language representing the sign language word is arranged for each frame, and the length of the voice of the utterance content. A speech length acquisition unit that acquires a certain voice speech length and a sign language speech length that is the sum of the reproduction times of the motion data included in the sign language translation information, and a sign language speech length so as to be close to the voice speech length. The sign language translation device is characterized by comprising a data editing unit for deleting a part of frames of the motion data included in the sign language translation information.

One aspect of the present invention is the sign language translation device described above, and the data editing unit deletes the frame from each of the plurality of motion data included in the sign language translation information.

One aspect of the present invention is the sign language translation device described above, wherein the data editing unit acquires the importance of the sign language word represented by each of the plurality of motion data included in the sign language translation information, and the plurality of the above. The frame of the ratio based on the importance of the corresponding sign language word is deleted from each of the motion data.

One aspect of the present invention is the sign language translation device described above, and the data editing unit acquires and acquires the importance of the sign language word represented by each of the plurality of motion data included in the sign language translation information. The frame is deleted in units of the sign language words from the plurality of motion data included in the sign language translation information based on the degree.

One aspect of the present invention is a program for making a computer function as any of the above-mentioned sign language translators.

According to the present invention, sign language translation with reduced delay from speech can be performed.

It is a functional block diagram which shows the structural example of the sign language translation apparatus by one Embodiment of this invention. It is a flowchart which shows an example of the processing procedure of the sign language translation apparatus by the same embodiment. It is a figure which shows the specific example of the sign language translation by the same embodiment.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
The real-time sign language translation technique used in computer graphics (CG) animation is a technique that has the potential to be widely used in various fields. In general sign language translation for CG animation, all voice languages such as Japanese, which is textualized as input information, are simply translated into sign language. In the present embodiment, the sign language language is data representing the movement (motion) of sign language. The sign language language may be, for example, motion data representing the position of each part of the body in each video frame in three-dimensional coordinates or the like, or video data in which a person actually moves the sign language. In the following, a case where motion data that can be converted into sign language CG is used as the sign language language will be described as an example. In the following, each frame constituting the motion data will be referred to as a “motion frame”.

When a sign language language is generated by simply arranging motion data as materials, the length of the sign language language is the sum of all the playback time lengths of each of the motion data. In conventional sign language translation, there is a difference between the utterance time of the spoken language and the utterance time of the sign language language (expression of motion by video). Therefore, assuming actual use, a delay occurs from the spoken language in the translation result into sign language each time the utterance is made, and the delay accumulates as time elapses.

Therefore, in order to improve the above problem, the sign language translation device of the present embodiment realizes flexible translation from the speech language to the sign language by considering the temporal utterance length of the speech language. In the first method, the playback speed is increased by thinning out motion frames from the motion data string representing the sign language translation. In the second method, the motion data itself of the sign language word of low importance is deleted from the motion data string, so that the sign language translation is a free translation without details. In other words, the motion frame is deleted in units of sign language words. It is also possible to combine the first method and the second method. By these methods, the sign language translator of the present embodiment automatically edits the motion data, which is the translation result from the Japanese text to the sign language, in the optimum form according to the spoken length of the input voice language, and from the voice. Minimize the delay. With the sign language translation device of the present embodiment, it is possible to suppress the delay of the sign language video corresponding to the sound of the program and present it, and it is possible to eliminate the burden on the viewing user.

FIG. 1 is a functional block diagram showing a configuration example of a sign language translation device 100 according to an embodiment of the present invention, and only functional blocks related to the present embodiment are extracted and shown. The sign language translation device 100 includes a voice recognition result effective unit 20, a sign language translation unit 1, an utterance length comparison unit 2, and a data editing unit 3.

The voice recognition result effective unit 20 acquires a voice recognition result from a voice recognition device (not shown) provided inside or outside the sign language translation device 100. When the use of the voice recognition result is enabled, the voice recognition result effective unit 20 inputs the Japanese text obtained as the voice recognition result into the handwriting translation unit 1, and the voice utterance calculated at the time of voice recognition is performed. The voice speech length, which is the time, is input to the speech length comparison unit 2.

The sign language translation unit 1 includes a Japanese-sign language translation unit 11 and a sign language motion data string generation unit 12. The sign language translation unit 1 translates the input Japanese text into sign language. Sign language translation is the generation of a sign language motion data string that represents the utterance content of a text. The sign language motion data string is a sequence of one or more sign language motion data representing one sign language word. The motion data used in the CG animation is motion-captured of actual human movements including fingers and facial expressions, and saved as motion data in a format such as BVH (Biovision Hierarchy). Each sign language motion data is motion data in which the movement of sign language is recorded for each sign language word. By arranging sign language motion data in chronological order and connecting sign language words, a motion data string of sign language video representing a sign language sentence is generated.

The Japanese-sign language translation unit 11 translates the input Japanese text into a sign language word string. The sign language word string of the translation result is data in which the data numbers of the sign language motion data are arranged in chronological order. For example, the Japanese-Sign Language Translation Department 11 puts the utterance content "Japanese Yamada player ..." indicated by the Japanese text into a sign language word string of "Japan", "mountain", "ta", "player", ... Convert. The translation result is obtained by arranging the data numbers that specify the sign language motion data representing each of these sign language words in the order of appearance of the sign language words. The Japanese-sign language translation unit 11 converts a Japanese text into a sign language word string by any conventional technique. As a conventional technique, for example, a machine translation technique such as Patent Documents 1 and 2 can be used, but the present invention is not limited to this. The sign language motion data string generation unit 12 is generated by associating the sign language word string output from the Japanese-sign language translation unit 11 with the sign language motion data specified by each of the data numbers constituting the sign language word string. Is output to the speech length comparison unit 2. The sign language motion data string generation unit 12 reads out the sign language motion data corresponding to the data number from the storage unit (not shown) included in the sign language translation unit 1 or an external storage device.

The utterance length comparison unit 2 includes a voice utterance length calculation unit 21, a sign language utterance length calculation unit 22, and a voice / sign language utterance length comparison unit 23. The utterance length comparison unit 2 operates as an utterance length acquisition unit that acquires the voice utterance length, which is the length of the voice of the utterance content, and the sign language utterance length, which is the sum of the reproduction times of the sign language motion data included in the sign language translation information. ..

The voice utterance length calculation unit 21 predicts and calculates the voice utterance length. The voice utterance length calculation unit 21 calculates the prediction of the voice utterance length, which is the total utterance time when the utterance content is voice-synthesized, from the input Japanese text. This calculation can be realized by using existing speech synthesis techniques and the like. The voice utterance length calculation unit 21 outputs the calculated predicted voice utterance length to the voice / sign language utterance length comparison unit 23.

The sign language utterance length calculation unit 22 inputs the sign language translation information generated by the sign language translation unit 1. The sign language utterance length calculation unit 22 calculates the sign language utterance length, which is the total utterance length of the prediction when the sign language word string of the sign language translation result is CG animated. Specifically, the sign language utterance length calculation unit 22 multiplies the result of adding the frame lengths of the sign language motion data of each sign language word included in the sign language translation information and the display time for one frame to make a prediction. Calculate the sign language utterance length.

The voice / sign language utterance length comparison unit 23 compares the voice utterance length input from the voice utterance length calculation unit 21 with the sign language utterance length input from the sign language utterance length calculation unit 22. When combined with the existing voice recognition technology, the total speech length when the voice recognition device performs voice recognition is retained, and it can be used instead of the voice speech length input from the voice speech length calculation unit 21. good. In that case, the voice / sign language utterance length comparison unit 23 inputs the voice utterance length from the voice recognition result effective unit 20. The voice / sign language utterance length comparison unit 23 calculates the utterance length difference, which is the difference between the voice utterance length and the sign language utterance length. When the voice / sign language utterance length comparison unit 23 is longer, the voice / sign language utterance length comparison unit 23 outputs the sign language motion data string of the sign language translation information as it is to the outside of the sign language translation device 100 without passing through the data editing unit 3. When the sign language utterance length is longer, the voice / sign language utterance length comparison unit 23 outputs the sign language translation information, the voice utterance length, and the utterance length difference to the data editing unit 3.

The data editing unit 3 includes a mode selection unit 31, a thinning mode processing unit 32, a summary mode processing unit 33, and a weighted thinning mode processing unit 34. The mode selection unit 31 thins out the sign language translation information based on the comparison between the utterance length difference input from the voice / sign language utterance length comparison unit 23 and the threshold value, or the user's preset setting, and the summary mode processing unit 32. It is output to either 33 or the weighted thinning mode processing unit 34. The threshold value may be set by the user or a predetermined value may be used.

The thinning mode processing unit 32 deletes all motion frames having a constant frame interval from each of the sign language motion data included in the sign language translation information according to the difference in utterance length input from the voice / sign language utterance length comparison unit 23. This shortens the motion length of each sign language word and inevitably shortens the overall length. The thinning mode processing unit 32 includes a frame length reduction rate calculation unit 321 and a frame length conversion processing unit 322. The frame length reduction rate calculation unit 321 calculates the reduction rate, which is the ratio of motion frames to be deleted from the sign language motion data string included in the sign language translation information. The frame length conversion processing unit 322 deletes motion frames from each of the sign language motion data included in the sign language translation information according to the reduction rate calculated by the frame length reduction rate calculation unit 321.

The summary mode processing unit 33 deletes the sign language motion data from the sign language motion data string included in the sign language translation information in units of the sign language words according to the importance of the sign language words. This shortens the overall length of the sign language translation result. The summary mode processing unit 33 includes a sign language word importance calculation unit 331 and a sign language word omission processing unit 332. The sign language word importance calculation unit 331 calculates the importance of each sign language word represented by each sign language motion data included in the sign language translation information. For example, the sign language word importance calculation unit 331 morphologically analyzes a sign language word string and determines the importance of each sign language word based on part of speech. The sign language word omission processing unit 332 deletes a part of the sign language motion data from the sign language motion data string included in the sign language translation information based on the importance of the sign language word calculated by the sign language word importance calculation unit 331.

The weighted thinning mode processing unit 34 deletes motion frames from the sign language motion data according to the importance of the sign language word. The weighted thinning mode processing unit 34 includes a sign language word importance calculation unit 341, a frame length reduction rate calculation unit 342, and a frame length conversion processing unit 343. The sign language word importance calculation unit 341 calculates the importance of each sign language word represented by each sign language motion data included in the sign language translation information, similarly to the sign language word importance calculation unit 331. The frame length reduction rate calculation unit 342 calculates the reduction rate of each sign language motion data according to the importance of the sign language word. The lowest reduction rate may be zero. The frame length conversion processing unit 343 deletes motion frames from each sign language motion data included in the sign language translation information according to the reduction rate calculated by the frame length reduction rate calculation unit 321.

FIG. 2 is a flowchart showing an example of the processing procedure of the sign language translator 100 of the present embodiment. When there is no voice recognition (step S105: NO), the sign language translator 100 reads the Japanese text (step S110). The Japanese-sign language translation unit 11 translates the read Japanese text into a sign language word string in which the data numbers of the sign language motion data are arranged in chronological order (step S115). The voice utterance length calculation unit 21 of the utterance length comparison unit 2 calculates the prediction of the voice utterance length when the utterance content is voice-synthesized from the Japanese text read in step S110 (step S120).

The sign language motion data string generation unit 12 associates the sign language motion data specified by the data number of each sign language word with the sign language word string output from the Japanese-sign language translation unit 11, and forms a standard such as a text file or array data. It is output as sign language translation information in the converted format (step S125). The sign language utterance length calculation unit 22 calculates the prediction of the sign language utterance length when the sign language word string is CG animated based on the total frame length of each sign language motion data included in the sign language translation information (step S130). ..

On the other hand, when there is voice recognition (step S105: YES), the voice recognition device inside or outside the sign language translation device 100 performs voice recognition (step S135). When the voice recognition result valid unit 20 is set to be valid (step S140: YES), the voice recognition result effective unit 20 acquires the voice recognition result by the voice recognition device and outputs it to the sign language translation unit 1. Further, the voice recognition result effective unit 20 acquires the total speech length of the voice recognized by the voice recognition device and outputs it to the voice / handwriting speech length comparison unit 23.

The Japanese-sign language translation unit 11 of the sign language translation unit 1 reads the Japanese text indicated by the voice recognition result from the voice recognition result valid unit 20 (step S145). The Japanese-sign language translation unit 11 performs the same processing as in step S115, and translates the Japanese text of the voice recognition result into a sign language word string (step S150). After the process of step S150, the sign language translator 100 performs the processes of steps S125 and S130. If the voice recognition result valid unit 20 is not set to have the voice recognition result valid (step S140: NO), the sign language translator 100 performs the process from step S110.

After the process of step S130, the voice / sign language utterance length comparison unit 23 compares the voice utterance length with the sign language utterance length (step S155). This voice utterance length is the voice utterance length calculated by the voice utterance length calculation unit 21 when the voice recognition is not effective, and the voice acquired by the voice recognition result effective unit 20 from the voice recognition device when the voice recognition is enabled. It is the total speech length of the voice to be recognized. When the voice / sign language utterance length comparison unit 23 determines that the sign language utterance length is less than or equal to the voice utterance length (step S155: voice ≥ sign language), the sign language motion data string of the sign language translation information is directly sent to the outside of the sign language translation device 100. Output.

When the voice / handwriting utterance length comparison unit 23 determines that the voice utterance length is longer than the voice utterance length, the utterance translation information, the utterance length difference obtained by subtracting the voice utterance length from the voice utterance length, and the voice utterance length are obtained. Output to the mode selection unit 31. The mode selection unit 31 determines the mode (step S160). When the difference in utterance length is smaller than the threshold value, the mode selection unit 31 determines that the mode is thinned out (step S160: thinned out mode). The mode selection unit 31 outputs sign language translation information, utterance length difference, and voice utterance length to the thinning mode processing unit 32. The frame length reduction rate calculation unit 321 of the thinning mode processing unit 32 calculates the reduction rate, which is the ratio of the motion frames to be deleted from the sign language motion data string so that the sign language utterance length is close to the voice utterance length (step S165). .. The frame length reduction rate calculation unit 321 adjusts so that the larger the difference in utterance length, the shorter the interval between the motion frames to be deleted, and the higher the reduction rate. The following is an example of determining the frame interval to be deleted, but this is not the case.

It is assumed that the result of the sign language translation is composed of the sign language motion data A and the sign language motion data B, and the sign language motion data A and B are each composed of 150 motion frames. Further, it is assumed that the sign language motion data is 60 fps (frames per second) and the utterance length difference is +2.0 seconds. In this case, the number of frames corresponding to the difference in utterance length is (+2.0 seconds) × (60 fps) = 120 frames. That is, by deleting 120 frames of motion frames from the entire sign language motion data string included in the sign language translation information, the voice utterance length and the sign language utterance length become equal.

Since the result of sign language translation is composed of two sign language motion data, 120/2 = 60 frames may be deleted from each sign language motion data. Here, if the number of frames to be deleted 60 is simply divided by the number of frames of sign language motion data 150, 60/150 = 1 / 2.5, and the reduction rate is 1/2 (1 frame out of 2 frames is deleted). It is between 1/3 (1 frame out of 3 frames is deleted). Here, the higher reduction rate 1/2 is selected, and the frame deletion interval is determined to be "1". If a frame is deleted from 150 frames at a frame deletion interval "1", 150 × (reduction rate 1/2) = 75 frames will be deleted. In this case, strictly speaking, 60 frames cannot be deleted, but the sign language utterance length can be brought closer to the voice utterance length.

The frame length conversion processing unit 322 deletes motion frames from each of the sign language motion data A and B included in the sign language translation information according to the frame deletion interval calculated by the frame length reduction rate calculation unit 321. Output to the outside (step S170). The frame length conversion processing unit 322 deletes motion frames at equal intervals from the entire motion frame when the sign language motion data included in the sign language translation information is arranged so that the sign language utterance length is closer to the voice utterance length. You may.

When the difference in utterance length is equal to or greater than the threshold value, the mode selection unit 31 determines that the mode is a summary mode (step S160: summary mode). The mode selection unit 31 outputs the sign language translation information and the voice utterance length to the summary mode processing unit 33. The sign language word importance calculation unit 331 of the summary mode processing unit 33 calculates the importance of each word represented by each sign language motion data included in the sign language translation information (step S175). For the calculation of importance, for example, there is a method using a general morphological analysis technique as follows, but the present invention is not limited to this.

The sign language word importance calculation unit 331 applies a general morphological analysis technique for each sentence to the word string represented by each sign language motion data included in the sign language translation information, divides it into word units, and divides each word into part speech. Is given. The sign language word importance calculation unit 331 obtains the importance of each word based on the preset data representing the correspondence between the part of speech and the importance and the data representing the relationship between the word and the importance. Regarding the importance of each part of speech, for example, nouns, proper nouns, verbs, etc. are of high importance, and particles, adjectives, and adverbs are of low importance. The importance of a word may be determined by the words before and after the word and the part of speech of the words before and after the word. If the importance of a word differs from the importance of the part of speech to which the word is classified, the importance of the word is prioritized. This makes it possible to tune important words according to the translation target. For example, when translating the audio of a sports program into sign language, even if the general rule is that the noun has a high priority, if the noun word "player" is preceded by a proper noun that represents the name, the priority is given. The degree can be lowered. The sign language word importance calculation unit 331 may perform morphological analysis on Japanese text. In this case, the importance of the word included in the Japanese text is used as the importance of the sign language word corresponding to the word.

The sign language word omission processing unit 332 has one or more sign language in order from the sign language motion data string indicated by the sign language translation information, in order from the sign language word with the lowest priority calculated by the sign language word importance calculation unit 331 so as to be closer to the voice speech length. The motion data is deleted (step S180). Similar to the processing in the thinning mode processing unit 32, the sign language word omission processing unit 332 deletes the number of sign language motion data as the difference in the utterance length output by the voice / sign language utterance length comparison unit 23 increases. Determine the number of sign language motion data to delete. At that time, the sign language word omission processing unit 332 prevents the sign language motion data string remaining as a result of the deletion from becoming less than the preset minimum number of words. By setting a threshold value for the minimum number of words that cannot be deleted in advance, it is possible to prevent the output of meaningless translation results. The sign language word omission processing unit 332 outputs the result of deleting the sign language motion data having a low priority from the sign language motion data string indicated by the sign language translation information to the outside of the sign language translation device 100.

Further, the mode selection unit 31 may determine the weighted thinning mode instead of the thinning mode according to the setting input in advance (step S160: weighted thinning mode). The mode selection unit 31 outputs the sign language translation information and the voice utterance length to the weighted thinning mode processing unit 34. The sign language word importance calculation unit 341 of the weighted thinning mode processing unit 34 calculates the importance of each word represented by each sign language motion data included in the sign language translation information by the same processing as the sign language word importance calculation unit 331 (. Step S185). The frame length reduction rate calculation unit 342 calculates a different number of reductions for each sign language motion data unit according to the importance of the sign language word calculated by the sign language word importance calculation unit 341 (step S190). The frame length reduction rate calculation unit 342 lowers the frame length reduction rate as the sign language motion data representing a sign language word of higher importance is represented. This makes it possible to change the utterance length more flexibly. The frame length conversion processing unit 343 deletes the motion frame of the sign language motion data string according to the reduction rate of each sign language motion data calculated by the frame length reduction rate calculation unit 342, and outputs the motion frame to the outside of the sign language translation device 100 (step S195). ).

The mode selection unit 31 can also determine that it is a hybrid mode according to the value of the difference in utterance length. For example, the mode selection unit 31 uses the first threshold value and the second threshold value larger than the first threshold value, and when the difference in utterance length is less than the first threshold value, it determines that it is the summary mode or the weighted thinning mode, and the first When it is equal to or more than one threshold value and less than the second threshold value, it is determined to be in summary mode. The mode selection unit 31 determines that the hybrid mode is used when the difference in utterance length is equal to or greater than the second threshold value. Alternatively, the mode selection unit 31 may change whether to perform the summary mode or the hybrid mode according to the setting input in advance.

When the mode selection unit 31 determines that the mode is a hybrid mode (step S160: hybrid mode), first, the summary mode processing unit 33 performs the same processing as in steps S175 to S180, and roughly speaks sign language in units of sign language motion data. Reduce the length (steps S200, S205). The summary mode processing unit 33 inputs the updated sign language translation information into the reduced sign language motion data string to the thinning mode processing unit 32. The thinning mode processing unit 32 performs the same processing as in steps S165 to S170 using the updated sign language translation information (steps S210 and S215). However, the thinning mode processing unit 32 uses the difference between the voice utterance length and the sign language utterance length of the sign language translation information output by the summary mode processing unit 33 as the utterance length difference. This makes it possible to finely adjust the utterance length on a frame-by-frame basis.

The sign language translation unit 1 determines whether or not the next input has been detected after the processing of step S170, step S180, step S195 or step S215 (step S220). When the sign language translation unit 1 detects the next input (step S220: YES), the sign language translation device 100 repeats the process from step S105. When the sign language translation unit 1 does not detect the next input (step S220: NO), the sign language translation device 100 ends the process of FIG. The sign language motion data string output from the sign language translator 100 is converted into a CG animation that visualizes the sign language motion by CG graphic processing. The sign language translation unit 1 may perform the processes from step S105 to step S215 for each utterance content of one sentence, or may perform a plurality of sentences at once.

FIG. 3 is a diagram showing a specific example of sign language translation. FIG. 3A shows an example of a conventional sign language translation technique. There is a tendency for a delay to occur in the utterance time of sign language, which is a translation of the same content as it is, with respect to the utterance time of the voice of the program or the synthetic voice synthesized from the text. For example, the voice spoken in an early tone, "Yamada of Japan ..." is translated into sign language in real time, and the sign language language is presented using CG animation. In this case, even if the delay in sign language translation and CG animation composition processing is not taken into consideration, the sign language translation results are in the order of "Japan", "mountain", "field", "player", ... It is a connection of sign language motion data that speaks sign language lightly at the average speed at the time of untouched motion capture. Therefore, the delay from the voice targeted for sign language translation gradually increases. In the present embodiment, the utterance length of the sign language translation is shortened by the thinning mode shown in FIG. 3 (b), the summary mode shown in FIG. 3 (c), and the like, and the problem is solved.

When the Japanese text "Japanese Yamada player ..." is input to the sign language translation device 100, the sign language translation unit 1 has the sign language word strings "Japan", "mountain", and "field". Generates a sign language motion data string representing "," player ", ...". Next, the utterance length comparison unit 2 calculates the predictions of the voice utterance length when the Japanese text is spoken and the sign language utterance length of the sign language word string, respectively. The utterance length comparison unit 2 compares those utterance lengths, calculates the utterance length difference representing the delay of sign language from the voice, and outputs it to the data editing unit 3. For example, when the threshold value of the utterance length difference is set to 10 seconds, the mode selection unit 31 of the data editing unit 3 selects the thinning mode when the utterance length difference is within 10 seconds, and summarizes when the utterance length difference is 10 seconds or more. Select a mode. It is also possible for the user to select a fixed mode by setting the mode in the mode selection unit 31 in advance.

When the thinning mode is selected, the data editing unit 3 corresponds to each sign language word "Japan", "mountain", "field", "player", ..., As shown in FIG. 3 (b). The motion frames of the sign language motion data to be used are deleted at equal intervals based on the value of the utterance length difference. By deleting the motion frame and increasing the playback speed of the motion of each sign language word, it is possible to shorten the utterance length of the sign language.

When the summarization mode is selected, the data editing unit 3 calculates the importance of each sign language word included in one utterance as shown in FIG. 3 (c). In the figure, the importance of "Japan" is 4, the importance of "mountain" is 5, the importance of "field" is 5, and the importance of "player" is 2. The data editing unit 3 deletes the sign language motion data of the sign language words "Japan" and "player", which are less important in the sign language word string, first so that the difference in speech length is minimized, and the sign language word "high importance" is used. Sign language motion data strings of only "mountain" and "field" are used as sign language translation information. By omitting the details and using a sign language translation as a free translation, the length of the sign language utterance is shortened.

The data editing unit 3 has the lowest importance in the sign language word strings "Japan", "mountain", "field", "player", ..., When the hybrid mode is a combination of the above two modes. Delete only "players" in summary mode. The data editing unit 3 shortens the frame length of the remaining sign language word strings "Japan", "mountain", and "field" in the thinning mode.

According to the embodiment described above, the sign language translation device includes a sign language translation unit, an utterance length acquisition unit, and a data editing unit. The utterance length acquisition unit corresponds to the utterance length comparison unit 2 of the embodiment. The sign language translation unit converts the text of the utterance content into sign language translation information in which motion data indicating the movement of the sign language representing the sign language word is arranged for each frame. The utterance length acquisition unit acquires the voice utterance length, which is the length of the voice of the utterance content, and the sign language utterance length, which is the sum of the reproduction times of the motion data included in the sign language translation information. The data editorial department deletes some frames of the motion data included in the sign language translation information so that the sign language utterance length is close to the voice utterance length. For example, the data editing unit evenly deletes frames from each of the plurality of motion data included in the sign language translation information. Alternatively, the data editorial department acquires the importance of the sign language word represented by each of the plurality of motion data included in the sign language translation information, and obtains a frame of a ratio based on the importance of the corresponding sign language word from each of the plurality of motion data. delete. Alternatively, the data editorial department acquires the importance of the sign language word represented by each of the plurality of motion data included in the sign language translation information, and the sign language word is obtained from the plurality of motion data included in the sign language translation information based on the acquired importance. Delete frames in units of.

The sign language translation device of the present embodiment minimizes the delay from the voice during translation by automatically editing the sign language word string and each sign language motion data in the optimum form according to the spoken length of the input voice language. , It is possible to realize automatic translation in a form closer to that of a sign language interpreter.

The sign language translator 100 described above has a computer system inside. The operation process of the sign language translator 100 is stored in a computer-readable recording medium in the form of a program, and the above processing is performed by the computer system reading and executing this program. The computer system referred to here includes hardware such as a CPU, various memories, an OS, and peripheral devices.

Further, the "computer system" includes the homepage providing environment (or display environment) if the WWW system is used.
Further, the "computer-readable recording medium" refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, or a CD-ROM, and a storage device such as a hard disk built in a computer system. Further, a "computer-readable recording medium" is a communication line for transmitting a program via a network such as the Internet or a communication line such as a telephone line, and dynamically holds the program for a short period of time. In that case, it also includes those that hold the program for a certain period of time, such as the volatile memory inside the computer system that is the server or client. Further, the above-mentioned program may be for realizing a part of the above-mentioned functions, and may be further realized by combining the above-mentioned functions with a program already recorded in the computer system.

1 ... Sign language translation unit 2 ... Speaking length comparison unit 3 ... Data editing unit 11 ... Japanese-Sign language translation unit 12 ... Sign language motion data string generation unit 20 ... Voice recognition result effective unit 21 ... Voice speech length calculation unit 22 ... Sign language speech Length calculation unit 23 ... Voice / sign language speech length comparison unit 31 ... Mode selection unit 32 ... Thinning mode processing unit 33 ... Summary mode processing unit 34 ... Weighted thinning mode processing unit 100 ... Sign language translation device 321, 342 ... Frame length reduction rate calculation Units 322, 343 ... Frame length conversion processing units 331, 341 ... Sign language word importance calculation unit 332 ... Sign language word omission processing unit

Claims (4)

A sign language translation unit that converts the text of the utterance content into sign language translation information in which motion data indicating the movement of the sign language representing the sign language word is arranged for each frame.
An utterance length acquisition unit that acquires a voice utterance length that is the length of the voice of the utterance content and a sign language utterance length that is the sum of the reproduction times of the motion data included in the sign language translation information.
A data editing unit that deletes a part of the frame of the motion data included in the sign language translation information so that the sign language utterance length is close to the voice utterance length.
Equipped with
The data editing unit acquires the importance of the sign language word represented by each of the plurality of motion data included in the sign language translation information, and is based on the importance of the corresponding sign language word from each of the plurality of motion data. Delete the percentage of the frame,
A sign language translator characterized by that. A sign language translation unit that converts the text of the utterance content into sign language translation information in which motion data indicating the movement of the sign language representing the sign language word is arranged for each frame.
An utterance length acquisition unit that acquires a voice utterance length that is the length of the voice of the utterance content and a sign language utterance length that is the sum of the reproduction times of the motion data included in the sign language translation information.
A data editing unit that deletes a part of the frame of the motion data included in the sign language translation information so that the sign language utterance length is close to the voice utterance length.
Equipped with
The data editing unit acquires the importance of the sign language word represented by each of the plurality of motion data included in the sign language translation information, and the plurality of motion data included in the sign language translation information based on the acquired importance. Delete the frame in units of the sign language word from
A sign language translator characterized by that. The data editing unit deletes the frame from each of the plurality of motion data included in the sign language translation information.
The sign language translation device according to claim 2 , wherein the sign language translator is characterized by the above. A program for making a computer function as a sign language translator according to any one of claims 1 to 3 .

Images