JP2000089660A - Sign language study supporting device and recording medium with sign language study supporting program recorded therein - Google Patents

Abstract

PROBLEM TO BE SOLVED: To facilitate study of sign language. SOLUTION: When key words and hand form data are inputted from an input part 101 and a sign language word for a study object is designated, each part is operated under the control of an entire control part 102 as follows: a sign language retrieval part 103 acquires a sign language word data from a sign language dictionary part 104 and a sign language image retrieval part acquires a sign language image from a sign language image storage part 108, respectively, and also a background retrieval part 105 acquires background information for sign language gesture explanation from a background dictionary part 106, and an animation generation part 109 acquires a material data and a scenario for generating the animation from an animation data storage part 110, and a background animation is generated which visually expresses a meaning of a sign language word for study. And, based on the synthesis information in a synthesis information storage part 113, a background image synthesis part 112 synthesizes the sign language image and the background animation and displays them on a display connected with an output part 111 together with the sign language word data, etc.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

The present invention relates to an apparatus for supporting learning of sign language.

[0002]

2. Description of the Related Art At present, when a hearing-impaired person who does not know a sign language or a hearing-impaired person learns a sign language, he or she learns directly from a hearing-impaired person who knows a sign language, or uses a video or text. Learning. Recently, CD
-There is also a learning system using a ROM or the like.

[0003] Sign language learning systems include a system that displays a video of conversation in sign language and the operation of each sign language word, and explains the operation in characters using characters (for example, Japan I).
CD-ROM “NHK” sold by BM Co., Ltd.
There is a sign language for everyone. Japanese Patent Laid-Open Publication No. Hei 6-83264 discloses a "sign language learning apparatus" which describes information such as the background of a sign language operation using a character string.

[0004]

In sign language learning, the sign language movement and the image of the hand shape at that time are very important. However, there is a conventional sign language learning system in which information such as explanations of actions and backgrounds of sign language actions are described by characters, and in such a system, it is difficult to grasp an image meaning of actions and hand shapes. .
In addition, the operation of sign language used in actual conversation often changes from a standard operation. For this reason, even if only the standard operation is learned, daily conversation cannot be performed.

Accordingly, the present invention facilitates learning of sign language movements by making it easier to grasp the meaning of sign language movements and the meanings of hand shapes and to make it easier to understand the meaning of sign language word movements. It is an object of the present invention to provide a sign language learning support device capable of learning a deformation of a sign language operation when actually used.

[0006]

Means for Solving the Problems and Effects of the Invention A first invention is a sign language learning support device for supporting sign language learning, comprising means for acquiring input information for designating a sign language word to be learned. A means for displaying a sign language image representing a sign language action for the learning target sign language word based on the input information; and a means for displaying an animation for visually expressing the meaning of the learning target sign language word based on the input information. , Is provided. According to the first aspect, an image showing the sign language operation is displayed for the sign language word to be learned, and the meaning is visually represented by the background animation. It is easy to understand, and the sign language operation can be easily learned.

A second invention is a sign language learning support device for supporting sign language learning, comprising input means for acquiring input information for specifying a sign language word to be learned, and a character representing the meaning of each sign language word. Sign language dictionary section storing semantic information as information, sign language image storing means storing a sign language image representing a sign language operation for each sign language word, material data indicating a material used for animation, and operation of the material. Animation data storage means for storing a scenario for visually expressing the meaning of each sign language word; sign language search means for searching the sign language dictionary for semantic information; and sign language from the sign language image storage means. A material language and a scenario are retrieved from a sign language image retrieval unit for retrieving an image and the animation data storage unit, and an animation is performed using the material data based on the scenario. And animation generating means for forming an output means for displaying an image and a character based on the input information,
The meaning information of the learning target sign language word is acquired by the sign language searching unit, the sign language image corresponding to the learning target sign language word is acquired by the sign language image searching unit, and the meaning of the learning target sign language word is visually determined. And a control unit for causing the animation generation unit to generate an animation expressed as a background animation and displaying the semantic information, the sign language image, and the background animation on the output unit. According to the second aspect, for each sign language word to be learned, an image indicating a sign language operation is displayed, character information indicating the meaning is displayed, and an animation that visually expresses the meaning is displayed on the background. Displayed as animation. Thereby, the meaning of the operation of the sign language word can be easily understood, and the sign language operation can be easily learned.

In a third aspect based on the second aspect, the background dictionary section stores a background explanation sentence which is character information for explaining a sign language operation for each sign language word, and a background explanation sentence is stored from the background dictionary section. A background search means for searching, wherein the control means acquires a background description corresponding to the sign language word to be learned by the background search means based on the input information, and also outputs the background description. It is characterized by displaying on means. According to the third aspect of the present invention, since the background explanation explaining the sign language operation for the sign language word to be learned is also displayed, the meaning of the operation of the sign language word can be further easily understood, and the sign language operation can be easily performed. You can learn.

In a fourth aspect based on the second or third aspect, the background image for generating a composite image by synthesizing the sign language image obtained by the sign language image search means and the animation generated by the animation generation means. A synthesizing unit, wherein the control unit displays the background animation on the output unit as a synthesized image synthesized with the sign language image of the sign language word to be learned by the background image synthesizing unit based on the input information. Features. According to the fourth aspect, the background animation that visually represents the meaning of the sign language word to be learned is displayed as a composite image synthesized with the sign language image of the sign language word. The connection between the action and the background animation expressing the meaning of the sign language word can be easily understood, and the sign language action can be more easily learned.

In a fifth aspect based on the fourth aspect, the combining by the background image combining means involves a morphing process from a hand portion in the sign language image to a material portion corresponding to the hand in the animation. It is characterized by. According to the fifth aspect, when the background animation is synthesized with the sign language image of the sign language word to be learned, morphing processing is performed from the hand part in the sign language image to the material part corresponding to the hand in the background animation. In addition, the connection between the sign language operation for the sign language word and the background animation expressing the meaning of the sign language word can be more easily understood.

According to a sixth aspect of the present invention, in the second or fourth aspect, a sign language modification dictionary unit storing a modification explanation sentence which is character information for explaining a modification of a sign language operation for each sign language word; Further comprising a modification search means for searching the modification explanation sentence from the dictionary unit, wherein the control means acquires a modification explanation sentence corresponding to the sign language word to be learned by the modification search means based on the input information; The modified explanation is displayed on the output means. According to the sixth aspect of the present invention, since the modified explanatory sentence is displayed for the sign language word to be learned, it is possible to learn about the modification of the sign language operation when the sign language is actually used.

In a seventh aspect based on the second or fourth aspect, each of the various conversational sentences and the individual transformed explanatory sentence which is character information for explaining the deformation of the sign language operation in each of the various conversational sentences are associated with the sign language word. Further comprising: a conversation dictionary section that has been accumulated and stored; and conversation retrieval means for retrieving a conversation sentence and an individual deformation explanation from the conversation dictionary section. The control means, based on the input information, Acquiring the conversation sentence and the individual transformation explanation corresponding to the conversation search means,
The conversation sentence is displayed on the output means together with the individual transformation explanation sentence for the conversation sentence. According to the seventh aspect, since the individual sentence for the conversation sentence is displayed together with the conversation sentence corresponding to the sign language word to be learned, the deformation in the actual use is more easily understood.

In an eighth aspect based on the seventh aspect, an environment image storage means for storing a conversation environment image, which is an image representing the meaning of each conversation sentence stored in the conversation dictionary section, Environment image retrieval means for retrieving a conversation environment image from the environment image storage means, wherein the control means acquires a conversation environment image corresponding to a conversation sentence displayed on the output means by the environment image retrieval means. And displaying the conversation environment image on the output means.
According to the eighth aspect, a conversation environment image representing the meaning of the conversation sentence is displayed together with the conversation sentence corresponding to the sign language word to be learned and the individual deformation explanation for the conversation sentence. It becomes easy to visually understand the deformation of the sign language operation in actual use.

According to a ninth aspect, in the seventh aspect, an image of the sign language action obtained by transforming the sign language action for each sign language word according to each conversation sentence stored in the conversation dictionary unit is stored as a modified sign language image. A modified sign language image accumulating means, a modified sign language image retrieving means for retrieving a modified sign language image from the modified sign language image accumulating means, and an image representing the meaning of each conversation sentence stored in the conversation dictionary unit. Environmental material storage means for storing material data indicating a material, and in accordance with a given conversation sentence, the transformed sign language image retrieval means acquires a transformed sign language image and acquires material data from the environmental material accumulation means. Environment synthesizing means for generating an environment synthesized image obtained by synthesizing the material indicated by the obtained material data with the obtained deformed sign language image, wherein the control means comprises the output means Environmental composite image corresponding to the conversational sentence displayed is generated in said environment combining means, and wherein the displaying the environment combined image to the output unit. According to the ninth aspect, the conversational sentence corresponding to the sign language word to be learned and the individual transformation explanation for the conversational sentence, and the meaning of the conversational sentence are displayed in the modified sign language image transformed according to the conversational sentence. Since the environment synthesized image in which the material of the image representing the content to be performed is synthesized is displayed, it is possible to easily understand the reason why the deformation of the sign language operation has occurred.

A tenth invention is a recording medium storing a sign language learning support program for supporting learning of a sign language as a program executed in a computer device,
Obtaining input information specifying a sign language word to be learned; displaying a sign language image representing a sign language action for the sign language word to be learned based on the input information; and sign language of the learning target based on the input information Displaying an animation for visually expressing the meaning of a word, and a sign language learning support program for realizing an operating environment on the computer device.

An eleventh invention is a recording medium storing a sign language learning support program for supporting learning of a sign language as a program executed in a computer device,
Obtaining input information that specifies a sign language word to be learned; and sign language words to be learned based on the input information from a sign language dictionary unit that stores semantic information that is character information representing the meaning of each sign language word. Acquiring the sign language image corresponding to the learning target sign language word based on the input information from the sign language image storage unit storing the sign language image representing the sign language operation for each sign language word. And animation data storage means that stores material data indicating a material to be used for animation and a scenario for visually expressing the meaning of each sign language word by operating the material. Material data and a scenario corresponding to the sign language word to be learned are extracted, and the sign language unit to be learned is used by using the material data based on the scenario. Generating an animation that visually expresses the meaning of the background language as a background animation, and displaying the meaning information of the learning target sign language word, the sign language image of the learning target sign language word, and the background animation. A sign language learning support program for realizing an operating environment on the computer device is recorded.

According to a twelfth aspect, in the eleventh aspect,
The operating environment further includes a step of generating a synthesized image by synthesizing a sign language image corresponding to the learning target sign language word and the background animation, and displaying the semantic information, the sign language image, and the background animation. In the step, the background animation is displayed by displaying the composite image.

In a thirteenth aspect based on the eleventh or twelfth aspect, the operating environment is a sign-language transformation dictionary unit which stores a transformation explanation sentence which is character information for explaining a transformation of a sign language operation for each sign language word. Further comprising the step of: acquiring a deformed explanatory sentence corresponding to the learning target sign language word based on the input information, wherein the displaying the semantic information, the sign language image and the background animation comprises: A modified description corresponding to a word is displayed.

In a fourteenth aspect based on the eleventh or twelfth aspect, the operating environment includes a variety of conversational sentences and an individual transformation description sentence which is character information for explaining a modification of the sign language operation in the various conversational sentences. Acquiring, from the conversation dictionary unit stored in association with the sign language word, a conversation sentence and an individual deformation explanation sentence corresponding to the learning target sign language word based on the input information; And displaying the conversation sentence to be performed together with the individual modified explanation for the conversation sentence.

According to a fifteenth aspect, in the fourteenth aspect,
The operation environment is displayed together with the individual deformation explanation from an environment image storage unit that stores a conversation environment image that is an image representing the meaning of each conversation sentence stored in the conversation dictionary unit. The method further includes a step of acquiring a conversation environment image corresponding to the conversation sentence, and a step of displaying the conversation environment image acquired in the step.

According to a sixteenth aspect, in the fourteenth aspect,
The operating environment is displayed together with the individual deformation explanation from an environment material storage unit that stores material data indicating a material in an image representing the meaning of each conversation sentence stored in the conversation dictionary unit. Acquiring material data corresponding to the conversation sentence, and storing a sign language operation image obtained by transforming the sign language operation for each sign language word according to each conversation sentence stored in the conversation dictionary unit as a modified sign language image. Obtaining a modified sign language image deformed in accordance with the conversational sentence displayed together with the individual deformation explanation sentence from the modified sign language image storing means, and obtaining the material indicated by the obtained material data. The method further includes a step of generating an environment composite image synthesized with the transformed sign language image, and a step of displaying the environment composite image.

[0022]

Embodiments of the present invention will be described below with reference to the accompanying drawings.

<Configuration of Sign Language Learning Support Apparatus According to Embodiment> FIG. 1 is a block diagram showing a configuration of a sign language learning support apparatus according to an embodiment of the present invention. The sign language learning support device according to the present embodiment includes an input unit 101, an overall control unit 102, a sign language search unit 103, a sign language dictionary unit 104, and a background search unit 105.
, A background dictionary unit 106, a sign language image search unit 107, a sign language image storage unit 108, an animation generation unit 109, an animation data storage unit 110, an output unit 111, a background image synthesis unit 112, and synthesis information storage. Unit 113 and transformation search unit 11
4, a transformation dictionary unit 115, a conversation search unit 116, a conversation dictionary unit 117, an environment generation unit 118, an environment image storage unit 119, an environment synthesis unit 120, and an environment material storage unit 121.
And

In the sign language learning support device having the above configuration, learning conditions such as keywords and hand shape data are input from the input unit 101 and sent to the overall control unit 102. Overall control unit 1
In step 02, based on the input information, each unit is controlled to generate data necessary for learning, and sends the data to the output unit 111. The sign language search unit 103 extracts sign language word data from the sign language dictionary unit 104 based on the input information. The background search unit 105 extracts background data from the background dictionary unit 106 based on the sign language word data. The sign language image search unit 107 extracts a sign language image from the sign language image storage unit 108 based on the sign language word data. The animation generation unit 109 generates a background animation using the data stored in the animation data storage unit 110. The background image combining unit 112 generates a combined image of the sign language image and the background animation based on the combined information stored in the combined information storage unit 113.
The transformation search unit 114 extracts transformation data from the sign language transformation dictionary unit 115. The conversation search unit 116 is a conversation dictionary unit 11
7, the conversation data and the deformation relation data are extracted. The environment generation unit 118 extracts an environment image from the environment image storage unit 119. The environment synthesis unit 120 generates an environment synthesis image obtained by synthesizing the material indicated by the environment material data stored in the environment material storage unit 121 with the environment image. Overall control unit 102
Is the sign language data, background data,
The output unit 111 sends the sign language image, the background animation, the composite image, the deformation data, the conversation data, the conversation deformation correspondence data, the environment image, and the environment composite image. The output unit 111
Any or all of these sign language data, background data, sign language image, background animation, composite image, transformation data, conversation data, conversation transformation correspondence data, environment image, environment composition image are output to an output device such as a connected display. Output.

Input unit 101, overall control unit 102, sign language search unit 103, background search unit 105, sign language image search unit 10
7. The animation generation unit 109, the output unit 111, the background image synthesis unit 112, the transformation search unit 114, the conversation search unit 116, the environment generation unit 118, and the environment synthesis unit 120 can be realized by one or a plurality of computers. By executing a predetermined program (hereinafter referred to as a "sign language learning support program") by the CPU in the computer, the functions of these units can be realized. Here, the sign language learning support program executed by the CPU is typically provided by a storage medium (flexible disk, CD-ROM, DVD, or the like) storing the program. That is, the user sets the purchased storage medium in the computer, reads the sign language learning support program stored therein, and installs the program in a storage device such as a hard disk device in the computer. Further, a sign language learning support program transmitted online via a communication line may be installed in the storage device. Furthermore, before the manufacturer ships the computer as a sign language learning support device, a sign language learning support program may be installed in a storage device in advance. Sign language dictionary 104, background dictionary 106, sign language image storage 108, animation data storage 110, sign language transformation dictionary 115, conversation dictionary 117, environment image storage 11
9. The environmental material storage unit 121 can be realized by a recording device such as a hard disk, a CD-ROM, and a DVD connected to a computer.

The input unit 101 is connected to an input device for inputting character information and the like. In the present embodiment, it is assumed that a keyboard and a mouse are connected as input devices. An output device for outputting images, characters, and sounds can be connected to the output unit 111. In this embodiment,
It is assumed that a display is connected as an output device.

The sign language learning support apparatus according to the present embodiment having the above-described configuration has a plurality of operation modes,
An operation mode is selected based on the instruction information input from step 1, and learning of sign language is supported in various modes according to each selected operation mode. Hereinafter, processing in each operation mode of the sign language learning support device of the present embodiment will be described.

<First Operation Mode> FIG. 2 is a flowchart showing processing in the first operation mode of the sign language learning support device of the present embodiment. Hereinafter, a procedure of setting the keyword "meet" as a learning condition and a hand shape related thereto and performing learning by the sign language learning support device will be described along each step in the flowchart of FIG.

(1) Processing in Step S001 The learning condition is input as input information using a keyboard and a mouse connected to the input unit 101 (hereinafter, this input information is indicated by a symbol "Iinf"). At this time, a learning condition setting screen is displayed on the display. In the present embodiment, a keyword and hand shape data are input as input information Iinf (hereinafter, a keyword input as input information Iinf is denoted by a code “Kwd” and hand shape data is denoted by a code “Hdat”). FIG. 3 shows an example of the learning condition setting screen 18. This learning condition setting screen 1
8, a character input unit 301, a hand shape list 302,
A reset button 303 and a learning start button 304 are displayed. As learning conditions, one or more necessary items of the character input unit 301 and the hand shape list 302 are set.

When learning a sign language related to a certain keyword such as a certain meaning or a word, a character string of the keyword is input from a keyboard. The input character string is set as a keyword Kwd constituting the input information Iinf.
When learning a sign language of a certain hand shape, the hand shape list 30
Select the hand shape you want to learn from step 2 using the mouse. The selected hand shape is displayed with a black background (shaded in the drawing to indicate that the background is displayed in black; the same applies to other drawings), and a hand shape that is not selected. Is displayed with a white background. The selected hand shape is
The input information Iinf is added to the hand shape data Hdat constituting the inf. A scroll bar is provided below the hand shape list 302. By using the scroll bar, the hand shapes displayed on the hand shape list 302 can be switched. By moving the cursor over a hand shape with a white background among the displayed hand shapes and pressing a mouse button, the hand shape to be learned is selected. When the cursor is moved over a hand shape having a black background and the mouse button is pressed, the background changes to white, and the hand shape is deleted from the hand shape data Hdat of the input information Iinf. A plurality of hand shapes can be set in the hand shape data Hdat.

To reset the settings from the beginning, a reset button 303 is pressed. When the reset button 303 is pressed, the contents of the character input section 301 and the hand shape list 302 are initialized, and the contents of the keyword Kwd and the hand shape data Hdat of the input information Iinf are cleared.

Character input unit 301 and hand shape list 30
After the setting of 2, the learning start button 304 is pressed. When the learning start button 304 is pressed, input information Iinf is sent from the input unit 101 to the overall control unit 102.

The hand shape data Hdat is represented by a hand shape number corresponding to the hand shape, and the hand shape number is sent to the overall control unit 102. FIG. 4 shows an example of a hand shape number corresponding to the hand shape. In FIG. 4, one number is assigned to one hand shape, but it is also possible to divide the hand shapes into groups and assign one number to one group. Hand shape numbers are similarly defined for hand shapes not included in FIG.

For example, when learning a sign language word with a hand shape number “6” related to the action of “meeting”, enter “meeting” in the character input section 301, and enter the hand shape number “6” from the hand shape list. Select a hand shape. When the learning start button 304 is pressed after the setting, "meet" is set as the keyword Kwd, and "6" is set as the hand shape data Hdat as the input information Iinf.
02. When the learning conditions are set as described above, the process proceeds to step S002.

(2) Processing of Step SS002 The sign language search unit 103 extracts, from the sign language dictionary unit 104, sign language word data of a sign language word to be learned based on the contents of the input information Iinf. The sign language dictionary unit 104 includes the sign language information 21.
Has been accumulated. FIG. 5 shows an example of the sign language information 21 stored in the sign language dictionary unit 104. This sign language information 21
Is sign language number 501, hand shape information 502, and semantic data 5
03 and a keyword 504. Sign language number 501
Is assigned to each sign language word stored in the sign language dictionary unit 104 one by one. Only one hand shape is set in the case of sign language performed with one hand. In the case of sign language performed with both hands, the hand shape of the hand on the dominant arm side (hereinafter referred to as “strong hand”) and the hand shape of the hand opposite to the dominant arm (hereinafter referred to as “weak hand”) are set. As the meaning data 503, a Japanese meaning corresponding to the sign language word is set. If there is more than one meaning, the meaning is set repeatedly after the separator ";". As the keywords 504, classification of sign language words, related keywords, and the like are set. When a plurality of keywords are set, similar to the semantic data 503, a repeated keyword is set after the separator ";".

When the keyword Kwd of the input information Iinf contains a character string, the keyword Kwd of the input information Iinf
Is compared with the character string set as the semantic data 503, and data including the matching character string in the sign language information 21 of the sign language dictionary unit 104 is extracted as key match data. Next, the keyword Kwd of the input information Iinf is compared with the character string set in the keyword 504, and data including a matching character string from the sign language information 21 of the sign language dictionary unit 104 is added to the key match data.

If the hand shape data Hdat of the input information Iinf includes a hand shape, the hand shape data Hd of the input information Iinf
The hand shape number of at is compared with the hand shape number of the hand shape information 502, and data including the matching hand shape in the sign language information 21 of the sign language dictionary unit 104 is extracted as hand shape matching data. When a plurality of hand shape numbers are set as learning conditions, all data including at least one of the set hand shapes is extracted.

The extracted key match data and the hand shape match data are compared, and sign language information included in common is returned to the overall control unit 102 as extracted sign language word data.

For example, the keyword Kwd of the input information Iinf
The case where “6” is set in the hand shape data Hdat will be described. As a sign language word set to "meet" in the meaning data 503, a sign language word <aU
> (In the present embodiment, the representative meaning of Japanese for the sign language word is described by an alphabet in which only the first character is lowercase and expressed by a label enclosed by <>) is key matching data. Is extracted as Next, as a sign language word set to “meet” in the keyword 504, for example, <bASYO>, <eKI>, <
oTOKO>, <oNNNA>, <hAHA>, <cH
ICHI> and <iKU> are added to the key match data.

Next, from the sign language information 21 of the sign language dictionary unit 104, as a sign language word in which the sign shape information 502 is set to “6”, for example, the sign language labels are <aU>, <iKU
>, <AKA> are extracted as hand shape matching data.

｛<AU>, <bASY of key match data
O>, <eKI>, <oTOKO>, <oNNNA>,
<HAHA>, <cHICHI>, <iKU>} and hand shape matching data {<aU>, <iKU>, <aKA
Of <>, <aU> and <iKU> are sent to the overall control unit 102 as extracted sign language word data. When the sign language word data is extracted in this way, the process proceeds to step S003.

(3) Step S 003 The overall control unit 102 outputs the extracted sign language word data to the output unit 11.
Send to 1. The output unit 111 displays the extracted sign language display screen 25 on a display connected to the output unit 111. FIG. 6 shows an example of the extracted sign language display screen 25. The extracted sign language display screen 25 displays the number of sign language information 601, an extracted sign language information list 602, a learning start button 603, and an initial screen return button 604. Sign language information 60
As 1, the number of the extracted sign language words is displayed. The extracted sign language information list 602 displays the sign language numbers and the meaning data of all the extracted sign language word data.

When the number of data included in the extracted sign language word data is 0, the fact that there is no sign language information corresponding to the learning condition is displayed on a display connected to the output unit 111. FIG. 7 shows an example of a condition mismatch display screen showing that there is no data matching the input information Iinf. On this screen, an initial screen return button 701 is displayed. When the initial screen return button 701 is pressed, the process returns to step S001.

From the sign language information list 602, sign language information of a sign language word to be learned is selected by a mouse. The selected sign language information changes its background to black and is set as learning sign language word data. After that, sign language information list 6
When the unselected sign language information in 02 is further selected, the background of the sign language information also changes to black and is added to the learning sign language word data. A plurality of learning sign language word data can be selected. When the selected sign language information is selected again with the mouse, the background changes to white and is deleted from the learning sign language word data.

When the learning start button 603 is pressed, the sign language number of the learning sign language word data is set in the learning sign language list. When the initial input screen button 604 is pressed, a step S0
Return to 01.

For example, the extracted sign language information list 602 in FIG.
From <aU>, <iKU>}, select <aU>
When the learning start button 603 is pressed, the sign language number "1" is set in the learning sign language list, and the process proceeds to step S004.

(4) Processing of Step S004 The sign language number of the head data of the learning sign language list is set as the learning sign language number. For example, when the head data of the learning sign language list is <aU>, “1” is set as the learning sign language number. When the learning sign language number is set in this way,
Proceed to step S005.

(5) Processing of Step S005 The background search unit 105 acquires background data relating to the sign language to be displayed from the background information 29 stored in the background dictionary unit 106 based on the learning sign language number. FIG. 8 shows the background dictionary unit 10.
6 shows an example of the background information 29 stored in No. 6. The background information 29 includes a background number 801, a sign language number 802, a strong hand correspondence 803, a weak hand correspondence 804, and a background description 805. The background search unit 105 sets data in which the learning sign language number and the sign language number 802 match each other as background data. If there is no matching data, the background number included in the background data is set to 0.

For example, when the learning sign language number is “1”, the data of the background number “1” whose sign language word number 802 is “1” is set as the background data. When the background data is set in this way, the process proceeds to step S006.

(6) Processing in Step S 006 The sign language image search unit 107 extracts the sign language image information stored in the sign language image storage unit 108 based on the learning sign language number.
Acquire a sign language image of the sign language word to be displayed. The sign language image storage unit 108 stores image data of a sign language image, which is an image of the action of the sign language word, and a sign language image table 33 indicating the correspondence between the sign language word and the sign language image. FIG. 9 shows an example of the sign language image table 33. The sign language image table 33 includes a sign language number 901 and a sign language image name 902.
Consisting of The image data is stored in the sign language image storage unit 108 under the name of the sign language image name 902. This storage format is a standardized image format, and in the present embodiment, the AVI format is adopted. The sign language image search unit 107 extracts the sign language image name 902 of the data whose sign language number 901 matches the learning sign language number from the sign language image table 33 and sets it as the output sign language image name. Then, an image having the output sign language image name is extracted from the image data stored in the sign language image storage unit 108 and set as an output sign language image.

For example, when the learning sign language number is “1”, the data whose sign language number 901 is “1” is searched from the sign language image table 33, and “Meet.av” is output as the sign language image name.
i ”is set, and“ Meet. avi "is taken out and set as an output sign language image.

In the above description, the image data of the sign language image obtained by capturing the operation of the sign language word is stored in the sign language image storage unit 108. Instead of capturing the operation of the sign language word, the sign language image corresponding to each sign language word is stored. The image data of the generated sign language image may be stored in the sign language image storage unit 108 in advance by using computer graphics.
When the output sign language image is set in this way, the process proceeds to step S007.

(7) Step S007 The animation generation unit 109 checks the background number of the background data.
If the background number is not 0, the animation data storage 11
A background animation is generated by using the data stored in “0”. The animation data storage unit 110 stores a scene image, an animation material, a background description scenario, and a background image table. FIG. 10 shows an example of the background image table 38. The background image table 38 includes a sign language number 1001, a scene image name 1002, and a background description scenario name 1003. The scene image is a scene image name 100
2 and stored in an image format such as JPEG. The background description scenario is stored in a text format under the name of the background description scenario name 1003. The animation generation unit 109 searches the background image table 38 for data in which the sign language number 1001 matches the learning sign language number (hereinafter, referred to as “background image data”).

If background image data exists, the scene image having the scene image name 1002 of the background image data is extracted from the animation data storage unit 110. Next, the background description scenario of the background description scenario name 1003 of the background image data is extracted from the animation data storage unit 110. The background description scenario describes a data name of an animation material and an operation instruction of the animation material necessary for creating a background animation that visually expresses the meaning of each sign language word.

An example of a description format of the background description scenario will be described. In the background description scenario, the operation of the animation material is described between “START” and “END” (hereinafter, a line describing such an operation is referred to as an “operation description line”). The line before “START” and “EN”
The line after D "is ignored as a comment line. The operation description line is described in one line for each operation, and includes a data name,
An operation instruction is described. For example, an action description line that describes an action of walking to the left with the data name of the animation material “MAN” is MAN WALKTOLEFT 550, 240 35
0,200. “WALKTOLEFT” indicates a command to walk to the left, followed by a start point and an end point.
The required number of operation description lines are "START" and "END"
Describe between. This description format is an example, and any other format may be used as long as the operation of the animation material for creating the background animation can be described.

The animation generation unit 109 synthesizes the scene image with the animation material operated according to the background description scenario,
Set the composite image as the output background animation.

For example, when the learning sign language number is “1”, data in which the sign language number 1001 is “1” in the background image table 38 is searched, and the scene image “Meet_Bk.jpg” is retrieved.
From the animation data storage unit 110. Next, the background description scenario “Meet.sce” is extracted from the animation data storage unit 110. Background description scenario "Mee
t. For example, the following is described in “sce”: START MAN WALKTOLEFT 550, 240 35
0,240 MAN WALKTORIGHT 100,240 30
0,240 END

FIG. 11 is a schematic diagram showing a procedure for creating an output background animation of the sign language word <aU>. In FIG. 11, 1101 indicates a scene image, 1102 indicates an animation material “MAN”, and 1103 indicates a background animation of the created sign language word <aU>. FIG.
In the example shown in FIG. 7, the animation generation unit 109 stores the scene image 1101 and the data name “M” from the animation data storage unit 110.
AN "is extracted from the male material 1102. In the extracted scene image 1101, (550, 2
Background animation 1103 combining a man walking left from 40) to (350, 240) and a man walking right from (100, 240) to (300, 240).
And set this as the output background animation. When the background animation is generated and set in this way, the process proceeds to step S008.

(8) Processing in Step S008 The output unit 111 displays the sign language learning information display screen 44 on the display. FIG. 12 shows a sign language learning information display screen 44.
An example is shown below. The sign language learning information display screen 44 includes a sign language number display unit 1201, a meaning display unit 1202, a background description display unit 1203, an image display unit 1204, an operation image output button 1205, a background image output button 1206, a next word display button 1207, A learning end button 1208 is provided. However, when the number of data in the learning sign language list is one, the next word display button 1207 is not displayed. The sign language number display section 1201 displays the learning sign language number.
The meaning display section 1202 displays a character string of the meaning data set in the sign language information of the sign language word of the learning sign language number. The background description display section 1203 displays a character string of the background description set in the background information of the sign language of the learning sign language number.

For example, when the learning sign language number is “1”, “1” is displayed on the sign language number display unit 1201, “meet and greet” on the meaning display unit 1202, and “people are approaching” on the background description display unit 1203. Represents the state where they met. " When the sign language learning information is displayed on the screen in this way, the process proceeds to step S009.

(9) Processing in Step S009 The type of button pressed on the sign language learning information display screen 44 is determined, and the following operation is performed according to the determination result. When the operation image output button 1205 is pressed, the process proceeds to step S010. When the background image output button 1206 is pressed, the process proceeds to step S011. Next word display button 1
When 207 is pressed, the process proceeds to step S012. When the learning end button 1208 is pressed, all are returned to the initial state, and the process proceeds to step S001.

(10) Processing in Step S010 The overall control unit 102 sends the output sign language image to the output unit 111 and displays it on the image display unit 1204 of the sign language learning information display screen 44. When the output sign language image is displayed to the end, the process proceeds to step S009.

For example, when the learning sign language number is “1”, the image “Meet.av” set as the output sign language image
“i” is displayed on the image display unit 1204.

(11) Processing of Step S011 The overall control unit 102 sends the output background animation to the output unit 111 and displays it on the image display unit 1204 of the sign language learning information display screen 44. When the output background animation is displayed to the end, the process proceeds to step S009.

For example, when the learning sign language number is “1”, the background animation 1103 set as the output background animation is displayed.

(12) Processing in Step S012 The sign language number of the next data in the learning sign language list is set as the next learning sign language number. However, if the current learning sign language number corresponds to the last data of the learning sign language list, the sign language number of the first data in the learning sign language list is set as the next learning sign language number. When the next learning sign language number is set in this way, the process proceeds to step S005.

In step S003, in addition to <aU>, <iU
If KU> is also selected, “1” is displayed in the learning sign language list.
Then, the sign language number “4” of <iKU> is set. In this case, when the sign language number “1” is set as the learning sign language number and the next word display button 1207 is pressed, “4” is set as the learning sign language number in step S012, and the process proceeds to step S005.

According to the procedure of steps S001 to S012 described above, sign language words can be learned.

According to the first operation mode in this embodiment, the sign language operation for the sign language word is explained by the background explanation, and the meaning of the sign language word is visually expressed by the background animation. The meaning of the operation of the sign language word can be easily understood, and the sign language operation can be easily learned.

<Second Operation Mode> FIG. 13 is a flowchart showing processing in the second operation mode of the sign language learning support device of the present embodiment. The processing in the second operation mode includes steps S008 to S012 of the flowchart showing the processing in the first operation mode.
Is realized by replacing a part of the procedure with the steps described below. Hereinafter, the processing procedure in the second operation mode will be described along the steps in FIG.

In the second operation mode, first, the first
As in the case of the operation mode described above, the processing is performed according to the procedure of steps S001 to S007. Then, after the end of step S007, the process proceeds to step S101.

(1) Process of Step S101 The background image synthesizing section 112 searches the synthesizing information stored in the synthesizing information accumulating section 113 for synthesizing data relating to the image to be synthesized. FIG. 14 shows an example of the synthesis information 6. The synthesis information 6 includes a sign language number 1401, a sign language head frame number 1402, a background head frame number 1403, and a background end frame number 1404. The background image synthesizing unit 112 searches for data in which the sign language number 1401 matches the learning sign language number. Then, the background image synthesizing unit 112
Generates a composite image based on the composite data obtained by the search.

Two methods of generating a composite image will be described using the sign language word <aU> as an example. FIG. 15 is a schematic diagram showing a procedure for synthesizing a composite image related to the sign language word <aU>. In FIG. 15, reference numeral 1501 denotes a sign language image, 1502 denotes a background animation image (hereinafter referred to as “background animation image”), 1503 denotes a sign language top frame image, 150
Reference numeral 4 denotes a composite intermediate image, 1505 denotes a background top frame image, 1506 denotes a composite image generated by the method of pattern 1 (hereinafter referred to as “pattern 1 composite image”), and 1507 denotes a composite image generated by the method of pattern 2 A composite image (hereinafter, referred to as “pattern 2 composite image”) is shown.

First, generation of a composite image by the method of pattern 1 will be described. The background image synthesizing unit 112 sets the sign language head frame number 1402 of the synthesized data obtained by the search for the synthesis information 6 to S0, sets the background head frame number 1403 to B0, and converts the sign language image frame S0 to the background animation image frame. A composite image is generated by morphing the image of B0. That is, first,
The morphing process is performed on the animation material “MAN” corresponding to the strong hand and the weak hand in the background description scenario extracted in step S007 and the hand portion in the sign language head frame image 1503, and from the sign language head frame image 1503 to the synthesized intermediate image 1504 Generate an image sequence. Next, the scene image “Meet_Bk.jp” extracted in step S007
g ”and a portion of the composite intermediate image 1504 other than the animation material“ MAN ”are subjected to morphing processing to obtain a composite intermediate image 1
An image sequence from 504 to the background top frame image 1505 is generated. Synthetic intermediate image 1 from sign language top image 1503
The image sequence up to 504 and the image sequence from the composite intermediate image 1504 to the background top frame image 1505 are set in order to frames I0 to Ik of the pattern 1 composite image 1506. The number of frames of the morphed image is k + 1. The images from the frame following the frame B0 of the background animation image to the frame Bn with the number of the background end frame number 1404 of the composite data are converted to the pattern 1 composite image 15
It is set after the frame I (k + 1) 06. The pattern 1 composite image 1506 thus obtained is set as an output composite image.

Next, generation of a composite image by the method of pattern 2 will be described. First, sign language top frame image 1
An image sequence from 503 to the combined intermediate image 1504 is created by the above-described method. The created image sequence is set in frames I0 to Ik of the pattern 2 composite image 1507. Next, an image is created in which the hand portion in the image after the frame number “S0 + 1” of the sign language image 1501 is replaced with the animation material “MAN”, and the pattern 2 composite image 1507 is sequentially created.
Is set after frame I (k + 1). The pattern 2 composite image 1507 thus obtained is set as an output composite image. By generating the output composite image by the method of the pattern 2, the image of the hand in the sign language operation can be easily understood.

An output composite image is generated by either the pattern 1 or the pattern 2 described above, and thereafter, the flow advances to step S008.

In step S008, sign language learning information is displayed on the screen as in the case of the first operation mode (FIG. 12).
Then, the process proceeds to step S009.

The processing content of step S009 is the same as that in the first operation mode. Depending on which button on the sign language learning information display screen 44 is pressed, any one of steps S010, S102 and S012 is performed. Proceed to that step. Here, the background image output button 120
If No. 6 is pressed, step S0 is performed in the first operation mode.
Then, in the second operation mode, the process proceeds to step S102 instead. Steps to proceed in other cases are first
Is the same as in the case of the operation mode. Therefore, step S
Only 102 will be described below.

(2) Processing in Step S 102 The overall control unit 102 sends the output composite image to the output unit 111 and displays it on the image display unit 1204 of the sign language learning information display screen 44. When the output composite image has been displayed to the end, the process proceeds to step S009.

The processing in the second operation mode is realized by the above-described procedure, whereby learning of a sign language word can be performed. According to such a second operation mode, by synthesizing the sign language image and the background animation image, the connection between the sign language operation for the sign language word and the background animation expressing the meaning of the sign language word can be more easily understood. And the sign language operation can be easily learned.

<Third Operation Mode> FIG. 16 is a flowchart showing processing in the third operation mode of the sign language learning support device of the present embodiment. In the processing in the third operation mode, the following step S201 is added between steps S005 and S006 of the flowchart showing the processing in the second operation mode, and step S008 is performed.
Is changed to step S202.

First, the additional step S201 will be described. (1) Processing in Step S201 The deformation search unit 114 searches for the deformation information stored in the sign language deformation dictionary unit 115. FIG. 17 shows an example of the deformation information 46. This deformation information 46 has a deformation data number 1701
And a sign language number 1702 and a modified description 1703.
The transformation search unit 114 obtains the sign language number 17 from the transformation information 46.
02 retrieves data that matches the learning sign language number and sets it as learning transformation data. If the sign language number 1702 that matches the learning sign language number does not exist in the deformation information 46, the deformation data number included in the learning deformation data is set to 0.

For example, when the learning sign language number is “1”, the transformation search unit 114 extracts data in which the sign language number 1702 is set to “1” from the sign language transformation dictionary unit 115, that is, the transformation data number 1701. Is extracted and data having a modified data number 1701 of "2" is extracted and set as learning modified data.

After setting the learning deformation data in this way, the process proceeds to step S006. And step S00
6, S007 and S101 are executed in order, and the processing contents of these steps are the same as those in the case of the second operation mode.

Next, step S202, which replaces step S008, will be described. (2) Processing in Step S202 The output unit 111 displays the sign language learning information display screen 47 on the display. FIG. 18 shows an example of the sign language learning information display screen 47 in the third operation mode. The sign language learning information display screen 47 includes a sign language number display section 1801, a meaning display section 1802, a background description display section 1803, a modified description display section 1804, an image display section 1805, an operation image output button 1806, and a background image output button 1807. And a next word display button 1808 and a learning end button 1809. However, when the number of data in the learning sign language list is one, the next word display button 1808 is not displayed. The sign language number display section 1801 displays the learning sign language number.
The meaning display section 1802 displays a character string of the meaning data set in the sign language information of the sign language word of the learning sign language number. The background description display section 1803 displays a character string of the background description set in the background information of the sign language of the learning sign language number. The modified explanatory note display unit 1804 displays a character string of the modified explanatory note set in the sign language modification information of the learning sign language number.

For example, when the learning sign language number is “1”, “1” is displayed on the sign language number display section 1801, “meet and say hello” on the meaning display section 1802, and “People are approaching” on the background description display section 1803. "The position of the hand changes according to the location of the person to meet. The hand shape changes according to the sex of the person to meet."

When the sign language learning information is displayed on the screen in this way, the process proceeds to step S009. Subsequent processing is the second
Is the same as in the case of the operation mode.

The processing in the third operation mode is realized by the above-described procedure, whereby the sign language word can be learned. According to such a third operation mode, since a modified explanatory sentence is displayed for each sign language word to be learned, it is possible to learn about a modification of the sign language operation when the sign language is actually used.

<Fourth Operation Mode> FIG. 19 is a flowchart showing processing in the fourth operation mode of the sign language learning support device of the present embodiment. In the processing in the fourth operation mode, step S3 is performed after step S201 in the flowchart showing the processing in the third operation mode.
01 and step S302 are added. Step S202 is replaced with step S303, and step S303 is performed.
009 is replaced with step S304, and step S305 is added.

Hereinafter, these additional steps and replacement steps will be described. (1) Processing of Step S301 The conversation search unit 116 searches the conversation dictionary unit 117 for deformation-related data. The conversation dictionary unit 117 stores deformation related information. FIG. 20 shows an example of the deformation relationship information 49. The transformation relation information 49 includes a transformation relation data number 2001, a sign language number 2002, and a transformation data number 2003.
And a conversation data number 2004 and an individual deformation explanation 2005. The conversation search unit 116 extracts all data in which the sign language number 2002 of the transformation relation information 49 matches the learning sign language number as transformation relation data. Then, the number extracted as the deformation relation data is set as the deformation number.

For example, when the learning sign language number is “1”, the conversation search unit 116 extracts data in which the sign language number 2002 is set to “1”, that is, the transformation relation data number 2001 is “1”. Data and deformation relation data number 20
01 is "2" and is extracted and set as deformation-related data, and "2" is set as the number of deformations. When the deformation relation data is extracted in this manner, step S3
Go to 02.

(2) Processing in Step S302 The conversation search unit 116 extracts conversation data from the conversation dictionary unit 117. The conversation dictionary 117 stores conversation information. FIG. 21 shows an example of the conversation information 51. The conversation information 51 includes a conversation data number 2101 and a conversation sentence 210
2 Deformation-related information 49 among words in a conversation sentence
Are enclosed in <>, and immediately after the first parenthesis <, the modification relation data number 2 of the corresponding modification relation information
001 is described. Data of all conversation data numbers set as deformation relation data is extracted as learning conversation data. The number of learning conversation data is set as the number of conversations.

For example, when the learning sign language number is “1”, “1” and “2” are added to the conversation data number 2004 of the transformation relation data.
Is set, the data having the conversation data number 2101 in the conversation information 51 of "1" and the data of "2" are set as learning conversation data, and "2" is set as the number of conversations.

When the learning conversation data is set as described above, the process proceeds to step S006. Thereafter, as in the case of the third operation mode, steps S006, S007, S10
1 and then execute step S303. This step S303 replaces step S202 in the third operation mode.

(3) Processing in Step S303 The output unit 111 displays the sign language learning information display screen 44b on the display. FIG. 22 shows an example of the sign language learning information display screen 44b in the fourth operation mode. The sign language learning information display screen 44b includes a sign language number display unit 2201, a meaning display unit 2202, a background description display unit 2203, a modified description display unit 2204, an image display unit 2205, an operation image output button 2206, a background image output button 2207, A conversation example output button 2208, a next word display button 2209, and a learning end button 2210 are provided. However, when the number of data in the learning sign language list is one, the next word display button 2209 is not displayed. The sign language number display section 2201 displays the learning sign language number. The meaning display unit 2202 displays a character string of the meaning data set in the sign language information of the sign language word of the learning sign language number. The background description display unit 2203 displays the character string of the background description set in the background information of the sign language of the learning sign language number. The modified explanatory note display unit 2204 displays the character string of the modified explanatory note set in the sign language modification information of the learning sign language number.

For example, when the learning sign language number is “1”, “1” is displayed on the sign language number display section 2201, “meet and greet” on the meaning display section 2202, and “People are approaching” on the background description display section 2203. The modified description display unit 2204 displays "The hand position changes depending on the location of the meeting person. The hand shape changes depending on the sex of the meeting person."

When the sign language learning information is displayed on the screen in this way, the process proceeds to step S304. This step S3
Step 04 replaces step S009 in the third operation mode.

(4) Processing in Step S304 The type of button pressed on the sign language learning information display screen 44b is determined, and the following operation is performed according to the determination result. When the operation image output button 2206 is pressed, the process proceeds to step S010. When the background image output button 2207 is pressed, the process proceeds to step S102. Conversation example output button 220
When 8 is pressed, the process proceeds to step S305. When the next word display button 2209 is pressed, the process proceeds to step S012. When the learning end button 2210 is pressed, everything is returned to the initial state, and the process proceeds to step S001.

Here, steps S010, S102, S
The processing contents of 012 are the same as those in the third operation mode. Since step S305 is a step added in the fourth operation mode, the processing of this step will be described below.

(5) Processing in Step S305 The output unit 111 displays the conversation example display screen 53 on the display. FIG. 23 shows an example of the conversation example display screen 53. The conversation example display screen 53 includes a sign language number display section 230.
1, a meaning display unit 2302, a conversation display unit 2303, a modified explanation display unit 2304, a conversation example selection button 2305, and a screen deletion button 2306. Sign language number display 2
At 301, a learning sign language number is displayed. Meaning display part 2
In 302, the meaning of the sign language word of the learning sign language number in the sign language number display unit 2301 is displayed. When a conversation example sentence is selected by the conversation example sentence selection button 2305, the conversation sentence of the learning conversation data corresponding to the selected conversation example sentence is displayed on the conversation sentence display section 2303. As the conversation example sentence selection button 2305, buttons are displayed by the number of learning conversation data (the number of conversations) extracted in step S302. If the number of conversations is 6 or more, that is, if there are 6 or more conversation example sentences, the conversation When the downward triangle button at the lower end of the example sentence selection button 2305 is pressed, buttons for selecting the next five conversation example sentences are displayed. In the selected conversation example sentence, the part enclosed in <> in the corresponding conversation sentence is displayed with an underline,
The number after <is circled below the underline. The modified sentence display unit 2304 includes a conversation sentence display unit 23
The character string of the individual deformation explanation 2005 of the deformation relation data corresponding to the underlined part in the conversation example sentence displayed in 03 is displayed. When the screen deletion button 2306 is pressed, the conversation example display screen 53 is deleted, and the process returns to step S303.

For example, when the learning sign language number is “1”, “1” is displayed on the sign language number display section 2301 and “meet and say hello.” Is displayed on the meaning display section 2302. When example 1 is selected with the conversation example sentence selection button 2305, the conversation sentence display section 2303
, A conversation sentence with a conversation data number 2101 of “1” is displayed as a conversation example sentence. In this conversation example sentence, in the conversation sentence with conversation data number 2101 of “1”, “meet” surrounded by <> is displayed with an underline, and number 1 set after <is surrounded by a circle. Displayed under the underline. In the modification explanation display portion 2304, "Depending on the position indicating the father" is displayed following the display of "1" surrounded by a circle.

The processing in the fourth operation mode is realized by the above-described procedure, whereby the sign language word can be learned. According to such a fourth operation mode, a conversation sentence and an individual deformation explanation are linked and displayed as shown in a conversation example display screen 53 in FIG. 23, thereby making it easier to understand the deformation in actual use. Can be.

<Fifth Operation Mode> FIG. 24 is a flowchart showing processing in the fifth operation mode of the sign language learning support device of the present embodiment. In the processing in the fifth operation mode, step S4 is performed after step S007 in the flowchart showing the processing in the fourth operation mode.
01 has been added. First, step S401 will be described.

(1) Processing in Step S401 The environment generation unit 118 extracts a conversation environment image from the environment image storage unit 119. The environment image storage unit 119 stores an environment image table and an environment image file. FIG.
An example of the environment image table 56 is shown in FIG. The environment image table 56 includes a conversation data number 2501 and an environment image file name 2502. Conversation data number 2501
Is the same as the conversation data number 2101 stored in the conversation information storage unit 117. The environment image file is stored in a standard image file format as an image file having the environment file name 2502. In the present embodiment, it is assumed that the data is stored in the AVI format. The image may be an actual video or a two-dimensional or three-dimensional computer graphic created by a computer. The environment generating unit 118 searches the environment image table 56 for a conversation data number 2501 that matches the conversation data number of the learning conversation data extracted in step S302, and finds an environment image file name 2502 corresponding to the matching conversation data number 2501. Obtained from the environment image table 56. Then, the environment image file having the environment image file name 2502 is extracted from the environment image storage unit 119 and set as a conversation environment image.

For example, when the learning sign language number is “1”, the image “E” of the conversation data numbers “1” and “2” of the learning conversation data
XP001. av "and" EXP002. avi "is taken out and set as a conversation environment image.

When the conversation environment image is set as described above, the flow advances to step S101. Thereafter, steps S101 and S303 are executed as in the case of the fourth operation mode. The operation in the next step S304 is the same as that in the fourth operation mode, and the learning information display screen 44b in FIG.
The process branches to each step according to the type of the button to be pressed in the step. However, when the conversation example output button 2208 is pressed,
In the fifth operation mode, the process proceeds to step S402 instead of step S305. Hereinafter, the processing after step S402 will be described.

(2) Processing in Step S402 The output unit 111 displays the conversation example display screen 58 on the display. FIG. 26 shows an example of the conversation example display screen 58 in the fifth operation mode. This conversation example display screen 58
Is a sign language number display section 2601, a meaning display section 2602, a conversation sentence display section 2603, a modified explanation display section 2604, a conversation environment image display section 2605, and a conversation example sentence selection button 2606.
And conversation image playback button 2607 and screen erase button 260
8 is provided. The sign language number display section 2601 displays a learning sign language number. The meaning display section 2602 displays a character string of the meaning data set in the sign language information of the sign language word of the learning sign language number.

When a conversation example sentence is selected by the conversation example sentence selection button 2606, the conversation sentence of the corresponding learning conversation data is displayed on the conversation sentence display section 2603 as the selected conversation example sentence. The number of conversation example sentence selection buttons 2606 is displayed as many as the number of conversation data (the number of conversations) extracted in step S302.
If there is more than one conversation example sentence, pressing the downward triangle button at the lower end of the conversation example sentence selection button 2606 displays a button for selecting the next five conversation example sentences. In the selected conversation example sentence, a portion surrounded by <> in the corresponding conversation sentence is displayed with an underline, and under the underline, a number after <is circled and displayed. Descriptive explanation display part 2
Reference numeral 604 denotes an individual deformation description sentence 2005 set in the deformation relation data corresponding to the underlined portion in the conversation example sentence.
Is displayed.

For example, when the learning sign language number is “1”, “1” is displayed on the sign language number display section 2601 and “meet and say hello.” Is displayed on the meaning display section 2602. When example sentence 1 is selected with conversation example sentence selection button 2606, conversation example sentence display section 26
In 03, a conversation sentence with a conversation data number “1” is displayed as a conversation example sentence. This conversation example sentence is conversation data number 2
In the conversation sentence where 101 is “1”, “meet” surrounded by <> is displayed with an underline, and the number “1” set after <is circled and displayed under the underline. Is done. In the modification explanation display unit 2604, "Depending on the position indicating the father" is displayed following the display of "1" surrounded by a circle. When such a conversation example display screen 58 is displayed, step S4 is performed.
Go to 03.

(3) Processing in Step S403 The type of button pressed on the conversation example display screen 58 is determined, and the following operation is performed according to the determination result. When the conversation image playback button 2607 is pressed, step S40 is performed.
Proceed to 4. When the screen deletion button 2608 is pressed, the conversation example display screen 58 is deleted, and the process returns to step S303.

(4) Processing in Step S404 A conversation example sentence selection button 2 is displayed on the conversation environment image display unit 2605.
A conversation environment image of the conversation example sentence selected in 606 is displayed. For example, when the learning sign language number is “1” and the example sentence 1 is selected, an image of “EXP001.avi” is displayed on the conversation environment image display unit 2605. That is, FIG.
As a conversation environment image for the sign language image shown in FIG.
A moving image that changes from FIG. 27 (b) to (d) is displayed. This conversation environment image is a conversation example sentence selection button 2606
Is displayed in order to explain how the sign language action corresponding to the learning sign language number is changed depending on the context of the conversation sentence of the example sentence 1. The sign language of "meet" is usually expressed by moving the hand forward, but the sign language action is changed according to the actual conversation content. In other words, if you express your father in the middle of a conversation, during that conversation your father will be at the position where you expressed your father, and when you say "go to see my father", "meet" will be in the direction of moving your hand. Move not toward the front but toward a position expressing the father (in the example shown in FIG. 27, to the front right as viewed from the speaker). The conversation environment images in this case are shown in FIGS.
As shown in (d), an image showing that the father is at the right front position expressing the father in sign language, that he is at the initial position at the back left, and that he walks and moves to the right front where his father is located, Become. In the conversation environment images shown in FIGS. 27B to 27D, the material around the father's house is arranged behind the father, but this may be omitted. When the conversation environment image is displayed to the end,
It returns to step S402.

The processing in the fifth operation mode is realized by the above-described procedure, whereby learning of the sign language word can be performed. According to the fifth operation mode, the meaning of the actual conversation accompanied by the deformation of the sign language operation is displayed as the conversation environment image, so that it is easy to visually understand the deformation of the sign language operation in actual use. can do.

<Sixth Operation Mode> FIG. 28 is a flowchart showing processing in the sixth operation mode of the sign language learning support device of the present embodiment. In the sixth operation mode,
Steps S404 and S501 of the flowchart showing the processing in the fifth operation mode are described in steps S501 and S501.
Replace with 502. Hereinafter, these steps S501
And S502 will be described.

(1) Processing in Step S501 The environment synthesizing unit 120 extracts the conversational environment synthetic material from the environmental material accumulating unit 121, and synthesizes this with the transformed sign language image extracted from the sign language image accumulating unit 108, thereby forming a conversation. Generate an environment composite image.

The environment material storage unit 121 stores a conversation environment table and a conversation environment synthesis material. In FIG.
6 shows an example of a conversation environment table 61. The conversation environment table 61 includes a conversation environment data number 2801, a conversation data number 2802, a material file name 2803, and
04, a combination parameter 2805, a combination start frame number 2806, and a combination end frame number 2807.
In the conversation data number 2802, a conversation number 2501 (see the environment image table in FIG. 25) of the conversation for combining the materials is set. In the material file name 2803, the name of a file storing material data is set. In the combining command 2804, a material combining command that defines the movement of the conversation environment combining material when combining the conversation environment combining material is set. In the combination parameter 2805, parameters necessary for executing the material combination instruction are set. In the combination start frame number 2806, the first frame number of the combination of the materials is set. In the combination end frame number 2807, a frame number for ending the combination of the materials is set. To combine up to the last frame of the conversation environment image, “FF” is set. Even if it is not “FF”,
Other character strings or empty may be used as long as the last frame can be specified.

The sign language image storage unit 108 stores image data of a sign language image obtained by capturing the sign language operation for each sign language word as described above, and further stores these sign language operations in the conversation dictionary unit 117. The image data of the transformed sign language image, which is the image of the sign language operation transformed according to each conversation sentence,
It is stored in association with the conversation data number of each conversation sentence. The AVI format is also used as the format of the image data of the modified sign language image.

Note that the image data of the transformed sign language image stored in the sign language image storage unit 108 is created by photographing a sign language operation in which the sign language operation for each sign language word is transformed according to each conversational sentence. However, instead of photographing the transformed sign language motion, a transformed sign language image corresponding to each conversation sentence is generated by computer graphics, and the image data of the generated transformed sign language image is stored in the sign language image storage unit 108. It may be.

The conversation environment synthesis material is for explaining the deformation of the sign language operation, and corresponds to the material of the conversation environment image in the fifth operation mode. The conversation environment composite material is used in the conversation environment table 61 shown in FIG. It is associated with the data. From the conversation environment table 61, the environment synthesis unit 120
All data whose conversation data number 2802 matches the conversation data number of the conversation example sentence selected in step S402 is extracted and set as conversation environment data. Then, a conversation data number, a material file name, a composition command, a composition parameter, a composition start frame number, and a composition end frame number of one piece of conversation environment data are extracted, and a conversation environment composite image is generated based on these information. That is, first, the environment synthesizing unit 120 extracts the file with the material file name from the environmental material storage unit 121, and
The sign language image retrieval unit 107 converts the sign language image transformed according to the conversation sentence of the conversation data number into the sign language image storage unit 108.
Extract from Then, the environment composition unit 120 composes the material with the images of all the frames from the composition start frame number to the composition end frame number in the modified sign language image according to the composition command. In this way, by combining the materials indicated by all the material data of the conversation environment data with the corresponding transformed sign language images, a conversation environment combined image is generated.

A description will be given of an example of creating a conversation environment composite image when the learning sign language number is “1” and the example sentence 1 is selected. In this case, all the data having the conversation data number 2802 of “1” is extracted from the conversation environment table 61 as conversation environment data, and a conversation environment composite image is generated based on the extracted conversation environment data. That is, first, the file name “Man1.da” set in the conversation environment table 61 is set.
The material data of "t" is extracted from the environmental material storage unit 121. Next, the sign language word <aU> is transformed into a modified sign language image in accordance with the conversation sentence of the example sentence 1, and the sign language image storage unit 10
Remove from 8. And 1 in this transformed sign language image
"Man" in the image from the 0th frame to the last frame
1. The material indicated by the material data of “dat” is synthesized at this time, in accordance with the instruction “LineMove” set as the synthesis instruction of the conversation environment data.
“ove” is a linear motion command, which causes a linear motion between the start position and the end position set in the synthesis parameter of the conversation environment data. In the example of FIG. 29, the start position is (500,
240), the end position is (100, 350), so (5
The material indicated by “Man1.dat” is synthesized so as to linearly move from (00, 240) to (100, 350). Similarly, “Father.dat” and “H
use. At this time, the “Static” of the combining command in the example of FIG. 29 is a still command, and the material is stopped at the position set in the combining parameter. da
The material indicated by “t” is “Hou” at the position (50, 350).
se. The material indicated by "dat" is synthesized from the 10th frame to the last frame at the position of (50, 250). When the conversation environment synthesized image is generated in this manner, the process proceeds to step S502.

(2) Processing in Step S502 The conversation environment composite image generated in step S501 is
6 is displayed on the conversation environment image display unit 2605 of the conversation example sentence display screen 58 shown in FIG.

For example, if the learning sign language number is “1” and example sentence 1 is selected, the operation of the sign language word <aU> is
"Ma" in the transformed sign language image transformed according to the conversation sentence
n1. dat "," Father. dat ”and“ H
use. 30 (a) is displayed as a conversation environment composite image for the sign language image shown in FIG. 30 (a). The moving image that changes to (d) is synthesized, and the conversation environment synthesized image is displayed as an image that describes how the sign language action is deformed depending on the context of the conversation sentence of Example Sentence 1.

After displaying all the frames of the conversation environment composite image, the process returns to step S402.

The processing in the sixth operation mode is realized by the above procedure, whereby the learning of the sign language word can be performed. According to the sixth operation mode, the explanatory material is synthesized in the sign language image of the actual conversation with the deformation of the sign language operation, so that the reason why the deformation has occurred can be more easily understood. .

<Modification> In the embodiment described above, various types of image data are stored in a predetermined storage unit in a predetermined format in advance, and the image data is required according to the sign language word to be learned and the selected conversation example sentence. In this case, a predetermined image is generated by computer graphics in accordance with the sign language word to be learned, the selected conversation example sentence, and the like. Is also good. For example, in the above-described embodiment, the sign language image obtained by capturing the action of the sign language word and the image data of the modified sign language image that is the image of the sign language action obtained by transforming the action according to each conversational sentence are stored in the sign language image storage unit 108. However, instead of accumulating image data in this way, computer graphics generates a sign language image according to the sign language word to be learned, and generates a modified sign language image according to the selected conversation example sentence, These generated images may be displayed on the sign language learning information display screen 44 or the conversation example display screen 58.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of a sign language learning support device according to an embodiment of the present invention.

FIG. 2 is a flowchart showing processing in a first operation mode of the embodiment.

FIG. 3 is a view showing an example of a learning condition setting screen in the embodiment.

FIG. 4 is a diagram showing an example of a hand shape number corresponding to a hand shape.

FIG. 5 is a view showing an example of sign language information stored in a sign language dictionary unit in the embodiment.

FIG. 6 is a view showing an example of an extracted sign language display screen in the embodiment.

FIG. 7 is a view showing an example of a condition mismatch display screen indicating that there is no data matching input information in the embodiment.

FIG. 8 is a view showing an example of background information stored in a background dictionary unit in the embodiment.

FIG. 9 is a view showing an example of a sign language image table in the embodiment.

FIG. 10 is a view showing an example of a background image table in the embodiment.

FIG. 11 is an exemplary diagram showing an example of a procedure for creating a background animation in the embodiment.

FIG. 12 is a view showing an example of a sign language learning information display screen in the first and second operation modes of the embodiment.

FIG. 13 is a flowchart showing processing in a second operation mode of the embodiment.

FIG. 14 is a view showing an example of composite information stored in a composite information storage unit in the embodiment.

FIG. 15 is an exemplary diagram showing an example of a combining procedure by a background image combining unit in the embodiment.

FIG. 16 is a flowchart showing processing in a third operation mode of the embodiment.

FIG. 17 is a diagram showing an example of deformation information stored in a deformation information storage unit in the embodiment.

FIG. 18 is a view showing an example of a sign language learning information display screen in a third operation mode of the embodiment.

FIG. 19 is a flowchart showing processing in a fourth operation mode of the embodiment.

FIG. 20 is a view showing an example of deformation relation information stored in a conversation dictionary unit in the embodiment.

FIG. 21 is a diagram showing an example of conversation information stored in the conversation dictionary unit in the embodiment.

FIG. 22 is a view showing an example of a sign language learning information display screen in a fourth operation mode of the embodiment.

FIG. 23 is a view showing an example of a conversation example display screen in a fourth operation mode of the embodiment.

FIG. 24 is a flowchart showing processing in a fifth operation mode of the embodiment.

FIG. 25 is a diagram showing an example of an environment image table stored in an environment image storage unit in the embodiment.

FIG. 26 is a view showing an example of a conversation example display screen in a fifth operation mode of the embodiment.

FIG. 27 is a view showing an example of a conversation environment image in a fifth operation mode of the embodiment.

FIG. 28 is a flowchart showing processing in a sixth operation mode of the embodiment.

FIG. 29 is a view showing an example of a conversation environment table stored in an environmental material storage unit in the embodiment.

FIG. 30 is a view showing an example of a conversation environment composite image in a sixth operation mode of the embodiment.

[Explanation of symbols]

 18 ... learning condition setting screen 21 ... sign language information 25 ... extracted sign language display screen 29 ... background information 44, 44b ... sign language learning information display screen 46 ... deformation information 49 ... deformation related information 51 ... conversation information 58, 53 ... conversation example display screen 101 input unit 102 overall control unit 103 sign language search unit 104 sign language dictionary unit 105 background search unit 106 background dictionary unit 107 sign language image search unit 108 sign language image storage unit 109 animation generation unit 110 animation data Storage unit 111 Output unit 112 Background image synthesis unit 113 Synthetic information storage unit 114 Deformation search unit 115 Sign language deformation dictionary unit 116 Conversation search unit 117 Conversation dictionary unit 118 Environment generation unit 119 Environmental image storage unit 120… Environmental synthesis unit 121… Environmental material storage unit 503… Semantic data 805… Background explanation 1 01 ... Scene image 1102 ... Animation material data 1202 ... Meaning display unit 1203 ... Background description display unit 1204 ... Image display unit 1501 ... Sign language image 1502 ... Background animation image 1703 ... Deformation description 1804 ... Deformation description display unit 2005 ... Individual Modified explanatory sentence 2303 ... Conversation sentence display unit 2304 ... Modified explanatory sentence display unit 2605 ... Conversation environment image display unit

 ────────────────────────────────────────────────── ─── Continued on the front page (72) Inventor Yuji Nagashima 4-16-19 Sakai, Musashino-shi, Tokyo M2 Heights 102 F-term (reference) 2C028 AA12 BA01 BB04 BC01 BD03 CA12

Claims (16)

[Claims] 1. A sign language learning support device for supporting learning of a sign language, comprising: means for acquiring input information for specifying a sign language word to be learned; and sign language for the sign language word to be learned based on the input information. A sign language learning support device comprising: means for displaying a sign language image representing an action; and means for displaying an animation for visually expressing the meaning of the sign language word to be learned based on the input information. 2. A sign language learning support device for supporting sign language learning, comprising: input means for acquiring input information for designating a sign language word to be learned; and a meaning being character information representing the meaning of each sign language word. Sign language dictionary section storing information, sign language image storage means storing sign language images representing the sign language action for each sign language word, material data indicating a material used for animation, and each sign language by operating the material Anime data storage means for storing a scenario for visually expressing the meaning of a word; sign language search means for searching semantic information from the sign language dictionary unit; and searching for a sign language image from the sign language image storage means. Material data and a scenario are extracted from the sign language image search means and the animation data storage means, and an animation is generated using the material data based on the scenario. A name generating means; an output means for displaying images and characters; and a sign language image corresponding to the sign language word to be learned, the semantic information of the sign language word to be learned is acquired by the sign language search means based on the input information. Is acquired by the sign language image search means, and the animation generation means generates an animation that visually represents the meaning of the sign language word to be learned as a background animation, and the semantic information, the sign language image, and the background animation are generated. A sign language learning support device, comprising: control means for displaying on an output means. 3. A background dictionary unit that stores a background description sentence, which is character information describing a sign language operation for each sign language word, and a background search unit that searches the background dictionary for a background description sentence, The control means obtains a background description corresponding to the sign language word to be learned by the background search means based on the input information, and causes the output means to display the background description. Item 3. The sign language learning support device according to Item 2. 4. A background image synthesizing means for synthesizing a sign language image obtained by the sign language image searching means and an animation generated by the animation generating means to generate a synthesized image, wherein the control means comprises: The sign language learning according to claim 2 or 3, wherein the background animation is displayed on the output unit as a synthesized image obtained by synthesizing the background animation with the sign language image of the sign language word to be learned by the background image synthesizing unit based on the information. Support device. 5. The composition according to claim 4, wherein the composition by the background image composition means includes a morphing process from a hand portion in the sign language image to a material portion corresponding to the hand in the animation. Sign language learning support device. 6. A sign language transformation dictionary storing character transformation explanations as character information for explaining transformation of a sign language operation for each sign language word, and a transformation search means for retrieving the transformation explanation from the sign language transformation dictionary. The control unit acquires a modified explanatory sentence corresponding to the learning target sign language word by the modified search unit based on the input information, and displays the modified explanatory sentence on the output unit. The sign language learning support device according to claim 2 or 4, wherein: 7. A conversation dictionary unit that stores various conversational sentences and individual transformation explanations, which are character information for explaining transformations of sign language operations in various conversational sentences, in association with sign language words, and the conversation dictionary. A conversation search unit for searching a conversation sentence and an individual deformation explanation sentence from the unit, wherein the control unit converts the conversation sentence and the individual deformation explanation corresponding to the sign language word to be learned based on the input information into the conversation. 5. The sign language learning support device according to claim 2, wherein said sign is acquired by a retrieval unit, and said conversation sentence is displayed on said output unit together with said individual transformation explanation for said conversation sentence. 8. An environment image storage means for storing a conversation environment image which is an image representing the meaning of each conversation sentence stored in the conversation dictionary unit; and a conversation environment image from the environment image storage means. Environment image search means for searching, wherein the control means acquires a conversation environment image corresponding to a conversation sentence displayed on the output means by the environment image search means, and outputs the conversation environment image to the output means. The sign language learning support device according to claim 7, wherein the device is displayed. 9. A modified sign language image storing means for storing, as a modified sign language image, an image of the sign language operation obtained by transforming the sign language operation for each sign language word according to each conversation sentence stored in the conversation dictionary unit, A modified sign language image retrieval unit for retrieving a modified sign language image from the modified sign language image storage unit; and material data indicating a material in an image representing the meaning of each conversation sentence stored in the conversation dictionary unit. Environmental material accumulating means and, in accordance with a given conversation sentence, acquiring a modified sign language image by the modified sign language image retrieving means and acquiring material data from the environmental material accumulating means, acquiring the acquired modified sign language image. Environment synthesizing means for generating an environment synthesized image obtained by synthesizing the material indicated by the material data; The formed image is generated on the environment combining means, sign language learning support system according to the environment combined image to claim 7, characterized in that to be displayed on the output unit. 10. A recording medium on which a sign language learning support program for supporting learning of a sign language is recorded as a program executed in a computer device, wherein input information for specifying a sign language word to be learned is obtained; A step of displaying a sign language image representing a sign language action for the sign language word of the learning target based on the input information; and a step of displaying an animation visually expressing the meaning of the sign language word of the learning target based on the input information. A recording medium recording a sign language learning support program for realizing an operating environment including the above on a computer device. 11. A recording medium on which a sign language learning support program for supporting sign language learning is recorded as a program to be executed in a computer device, wherein input information for specifying a sign language word to be learned is obtained. Acquiring the meaning information of the sign language word to be learned based on the input information from a sign language dictionary unit that stores semantic information that is character information representing the meaning of the sign language word; and representing the sign language operation for each sign language word. Acquiring a sign language image corresponding to the sign language word to be learned based on the input information from a sign language image accumulating unit that accumulates the sign language image; and operating the material data indicating the material used for the animation and the material. From the animation data storage means storing a scenario for visually expressing the meaning of each sign language word. Based on the information, material data and a scenario corresponding to the learning target sign language word are extracted, and an animation for visually expressing the meaning of the learning target sign language word is generated as a background animation using the material data based on the scenario. Displaying the sign language word of the learning target, the sign language image of the learning target sign language word, and the background animation. A recording medium on which a support program is recorded. 12. The operating environment further includes a step of synthesizing a sign language image corresponding to the learning target sign language word and the background animation to generate a synthesized image, wherein the semantic information, the sign language image, and the background are generated. 12. The recording medium according to claim 11, wherein the background animation is displayed by displaying the composite image in the step of displaying an animation. 13. The learning environment for a sign language word to be learned based on the input information from a sign language transformation dictionary unit storing a transformation explanation sentence which is character information for explaining a transformation of a sign language operation for each sign language word. Further comprising a step of acquiring a modified explanatory sentence corresponding to the above, wherein in the displaying the semantic information, the sign language image and the background animation, a modified explanatory sentence corresponding to the sign language word to be learned is displayed. A recording medium on which the sign language learning support program according to claim 11 or 12 is recorded. 14. A conversation dictionary unit which stores various conversational sentences and individual transformation explanations, which are character information for explaining transformations of the sign language operation in the various conversational sentences, in association with sign language words. Acquiring a conversation sentence and an individual deformation explanation corresponding to the learning target sign language word based on the input information; and a conversation sentence corresponding to the learning target sign language word together with the individual deformation explanation for the conversation sentence. 13. The recording medium according to claim 11, further comprising a step of displaying the sign language learning support program. 15. The individual deformation description sentence is stored in an environment image storage unit that stores a conversation environment image that is an image representing the meaning of each conversation sentence stored in the conversation dictionary unit. 15. The sign language learning according to claim 14, further comprising: obtaining a conversation environment image corresponding to the conversation sentence displayed together with the conversation sentence; and displaying the conversation environment image acquired in the step. A recording medium on which a support program is recorded. 16. The individual deformation explanation from an environment material storage unit that stores material data indicating a material in an image representing the meaning of each conversation sentence stored in the conversation dictionary unit. Acquiring material data corresponding to the conversation sentence displayed together with the sentence; and transforming an image of the sign language operation in which the sign language operation for each sign language word is transformed according to each conversation sentence stored in the conversation dictionary unit. Acquiring, from the modified sign language image storage means stored as a sign language image, a transformed sign language image transformed in accordance with the conversation sentence displayed together with the individual transformation explanatory note; and a material indicated by the acquired material data. Generating an environment composite image synthesized with the acquired transformed sign language image, and displaying the environment composite image. Recording medium for recording a sign language learning support program according to claim 14 that.