JPH08328460A - Sign language translating device - Google Patents

Abstract

PURPOSE: To provide a sign language translating device in which the differences among sign language talkers are compensated for and misrecognition is reduced without disturbing the talkers and without raising unpleasantness by using a still picture. CONSTITUTION: The conditions of the sign language talking being conducted are recorded in a still picture recording section 2 through an input section 1. The positional conditions of both shoulders, both elbows and hands are detected by a detecting section 3 and a hand picture is generated. A recognition section 5 retrieves the recognition elements of a similar standard hand picture from a standard hand picture storage section 6 and corresponding word information is retrieved from the recognition element columns and outputted to an output section 8. In order to compensate for the differences among the talkers, the hand picture is standardized by the standardization correction values for a standard hand picture which are beforehand computed for specific sign language elements by a standardization section 4 and then, the hand picture is supplied to the section 5 after the standardization.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a sign language translation apparatus for converting words expressed in sign language into output signals such as characters and voices.

[0002]

2. Description of the Related Art Conventionally, people with language disabilities convey their will by using a sign communication method called sign language.

Sign language is defined by gestures and hand movements.
In addition, it is a word expressed by the expression of the face, and the word is expressed by imitating the shape and movement, or by expressing the characteristic of an object.

However, a person with normal hearing (a person who does not know sign language) could not convey his will, which hindered communication.

Therefore, as a countermeasure against this, Japanese Patent Laid-Open No. 63-1
72297, JP-A-2-144675, JP-A-3-28
Techniques such as 8276 and JP-A-4-134515 have been proposed.

Among the above, in Japanese Patent Laid-Open No. 172297/1988, a sensor unit mounted on a finger or an arm to detect movements thereof, an encoder for coding a detection signal from the sensor unit, and a coding unit. Consists of a coded dictionary that is referred to when doing this, and a voice synthesizer that decodes the code from the encoder and converts this to voice, making it easy to convey your intentions by voice even to people who do not know the sign language rules. It is supposed to be possible.

In Japanese Patent Laid-Open No. 2-144675, a glove colored with a plurality of colors is used, and a hand movement recognition device having means for extracting color information corresponding to each color and a sign language for recognizing the movement of a finger are used. It is equipped with a device that compares it with the finger character pattern in to convert it into a sign language, and can be used for reading and reading finger characters.

In Japanese Patent Laid-Open No. 3-288576, an input device for inputting sign language as character data stores standard sign language motion data in advance, and the sign language motion is image-input and compared with each other. A method of identifying the sign language content and outputting the corresponding character is disclosed.

Further, in Japanese Patent Laid-Open No. 4-134515, a detector for detecting bending of joints of fingers, hands and arms and twisting and raising and lowering of arms necessary for sign language, and information obtained from these detectors are used. The sign language recognition unit for recognizing the meaning of sign language and the output conversion unit for converting the sign signal into an output signal according to the recognition result enable the hearing-impaired person to understand what the speech-impaired person wants to convey.

Next, FIG. 3 shows a block diagram of an apparatus according to the above-mentioned Japanese Patent Laid-Open No. 4-134515 as an example of a conventional sign language translation apparatus.

In FIG. 4, a sign language translation device 100, which is the device, includes an optical fiber cable 110 and a detection unit 10.
1, a recognition unit 102, an initial registration unit 103, a conversion table 104, an output unit 105, a voice synthesis unit 106, a display processing unit 107, a speaker 108, and a display 109.

Before using the sign language, the user must first
The optical fiber cable 110 is attached with all the fingers, hands, and arm joints extended, and the initial values are registered in the initial registration unit 103 via the detection unit 101.

Next, when sign language is performed, the light returning through the optical fiber cable 110 attached so that the bending of the joints of the fingers, hands, and arms can be detected passes through the detection unit 101, which is an optical sensor. Through this, information about which optical fiber signal is and how much is input to the recognition unit 102.

By comparing this information with the initial value, how the joints of the fingers, hands, and arms are bent is recognized by the state of each joint, and a judgment is made.

The result is converted into word information from the conversion table 104 and the information is sent to the output unit 105.

The output unit 105 outputs the received word information to the outside.

The voice synthesizer 106 and the speaker 108 convert word information into voice information and generate it. The display processing unit 107 and the display 109 convert the word information into character information and display it.

[0018]

DISCLOSURE OF THE INVENTION The above-described conventional sign language translation apparatus is provided with a finger of a person who uses a sensor, special gloves, or the like.
Since it is directly attached to the hand and arm, it has a drawback that it is troublesome and causes discomfort.

Further, the data corresponding to the sign language state is recognized by comparing the data of the sign language state of the sign language person with the data stored in advance, but the body shape, the hand, the finger, etc. are considerably different depending on the sign language person. However, there is a drawback that erroneous recognition based on this is unavoidable.

It is an object of the present invention to provide a sign language translation apparatus which can perform sign language translation with less misrecognition by compensating for differences between sign speakers without causing trouble and discomfort to the user. is there.

[0021]

A sign language translation device according to a first aspect of the present invention is a standard form which is a standard form of a hand image including points indicating positions of both shoulders and elbows of a sign language element and positions and forms of both hands in advance. Standard hand image storage means for storing a hand image together with an identifier for each sign language element, and a conversion table for storing word information corresponding to the identifier of the standard hand image and word information corresponding to a combination of the identifiers of the plurality of standard hand images And input means for inputting a still image of a sign language element of the sign language of the signer, and a point indicating the positions of both shoulders and elbows and the position and form of both hands are detected from the still image to create a hand image. Detecting means and a standardized correction value for approximating a hand image of the signer for a specific sign language element to a standard hand image corresponding to the specific sign language element stored in the standard hand image storage means are obtained in advance. Every above The standard hand image supplied from the output means, the standardized means for creating a standardized hand image by the standardized correction value, and the identifier of the standardized hand image approximate to the standardized hand image supplied from the standardization means are searched in order. Generating a sequence of identifiers in response to a conversion instruction supplied from the outside, and recognizing means for retrieving word information corresponding to the sequence of identifiers from the conversion table, and outputting corresponding voice based on the word information. It is configured to include output means for displaying.

Further, the sign language translation device of the second invention signs a standard sign image which is a standard form of a sign image including points indicating positions of both shoulders and elbows of the sign language element and positions and forms of both hands in advance. A standard hand image storage means for storing each element together with an identifier, a conversion table storing word information corresponding to the identifier of the standard hand image and word information corresponding to a combination of a plurality of the standard hand image identifiers, and a signer. An input means for inputting a still image of a sign language element in the sign language state, and a detection means for detecting a point indicating the positions of both shoulders and elbows and the positions and forms of both hands from the still image to create a hand image; For a specific sign language element, a correction value for approximating the standard sign image corresponding to the specific sign language element stored in the standard sign image storage means to the sign image of the signer is obtained in advance, and the standard image is stored. means A conversion means for creating a standard hand image in which the standard hand image read from the image is corrected by the correction value, and an identifier of the corrected standard hand image supplied from the conversion means that approximates the hand image supplied from the detection means in order. Retrieval means for searching and creating an identifier string and, in response to a conversion instruction supplied from the outside, searching word information corresponding to this identifier string from the conversion table, and outputting a corresponding voice based on the word information. And output means for performing corresponding display.

[0023]

Embodiments of the present invention will now be described with reference to the drawings.

FIG. 1 is a block diagram showing a first embodiment of the sign language translation apparatus of the present invention.

As shown in FIG. 1, the sign language translation apparatus of the first embodiment has a standard hand image storage section 6 for storing standard hand image data of a plurality of sign language elements used for sign language in advance together with its identifiers. A conversion table 7 that stores a combination of standard hand image identifiers and corresponding word information, an input unit 1 that inputs an image in the sign language state of a signer, and a still image that stores the image input from the input unit 1. The recording unit 2, the detection unit 3 that performs image processing of the image stored in the still image recording unit 2 to detect a feature and create a hand image, the standardization unit 4 that standardizes the feature, and the standardized The standard hand image storage unit 6 is searched for an identifier of the standard hand image that approximately corresponds to the input hand image, and the word information corresponding to the input hand image is converted into a conversion table 7.
Recognizer 5 that recognizes what sign language means by searching from
An output unit 8 for outputting word information as sound or display, a voice synthesizing unit 9 for converting the word information into voice information, a speaker 10 for uttering it, and a display processing unit 11 for converting the word information into character information. And a display 12 for displaying it.

There are various hand images used for sign language depending on gestures, hand forms, positions, etc. As a basic form of them, for example, a signer turns his / her hands up to the chest with his palms up. Adopt a posture that opens and faces the front.

The hand image is characterized by the position and shape of both hands and the positions of both shoulders and elbows (this corresponds to the body shape of the signer).

The standard hand image storage unit 6 stores hand images of a plurality of sign language elements used for sign language and their identifiers (the hand images stored in the standard hand image storage unit 6 are used as standard hand images). I will call it an image).

The conversion table 7 stores the identifiers of standard hand images, an identifier string consisting of combinations of these identifiers, and word information corresponding to them.

The input unit 1 photographs and inputs the sign language state of a signer corresponding to the elements constituting the sign language for each element. For example,
A still image obtained by normal optical photography may be used, or a still image of the temperature distribution of the subject may be obtained by providing a temperature sensor capable of recognizing high and low temperatures.
FIG. 2A shows an example A of a still image in the basic form of sign language.

The detection unit 3 recognizes both shoulders and elbows as feature points from the still image and creates a hand image in which the shapes of both hand parts including the coordinate points are extracted. Figure 2 (b)
The coordinate positions B1 and B2 of both shoulders and the coordinate position B3 of both elbows.
, B4 and an example B of the hand image corresponding to FIG. 2A in which the position forms of both hand parts are extracted.

The standardization unit 4 standardizes the hand image created by the detection unit 3.

Since there is a considerable difference in the height of the sign language, the difference in body shape such as the degree of obesity, the size of the hand, the thickness of the fingers, the length, etc. Since there is a possibility of misjudgment in judging the sign language content by comparing with the standard sign image having the same sign language content stored in the standard sign image storage unit 6 as it is, it is compared with the standard sign image. In order to make the determination easier and reduce the number of erroneous determinations, the standardization of the sign images of the actual sign language is performed.

That is, prior to sign language, a basic form of a hand image of a signer (that is, a hand image in which the signer faces the front, raises both hands to the chest, opens the palms of the hands, and faces the front) This is compared with the basic form of the standard hand image stored in the standard hand image storage unit 6. Then, the expansion / contraction ratios of the shoulder and elbow coordinate positions (for example, B1 to B2 in FIG. 2B) are used as the standardized correction values for the body shape, and the hand size, the finger length, and the thickness are used as the standardized correction values for the hands and fingers. The respective expansion / contraction ratios are calculated. That is, the standardized correction value is a correction value for approximating the hand image of the signer to the standard hand image in the basic form of sign language.

Then, the standardization unit 4 corrects the hand image of the signer created by the detection unit 3 by these standardized correction values to perform standardization.

In the recognition unit 5, the standardized hand image sent from the standardization unit 4 is sequentially subjected to pattern matching with various standard hand images in the standard hand image storage unit 6, and the identifiers of the corresponding standardized hand images are approximated. In response to the search instruction and the conversion instruction, the conversion table 7 is searched for the word information corresponding to the identifier string thus obtained and is supplied to the output unit 8. The conversion instruction is supplied to the recognition unit 5 when the signer finishes the sign language corresponding to one word.

Next, the operation of the sign language translation apparatus of this actual example will be described.

First, in order to obtain the standardized correction value, the signer is made to perform the sign language motion of the basic form of the hand image.

That is, the signer faces the front, raises both hands to the chest, opens the palms of both hands, and takes the posture to face the front. In response to the supply of the initial setting signal, the input unit 1 captures this. Then, the still image is stored in the still image recording unit 2.

The detection unit 3 creates a basic hand image of the signer (an image including the coordinate positions of both shoulders and elbows, the position and shape of the hand) from the still image, and supplies it to the standardization unit 4.

In the standardization unit 4, the basic form of the standard hand image is read from the standard hand image storage unit 6, and from this and the supplied hand image of the basic form of the signer, the body shape and the size of the hand of the signer,
The expansion / contraction rate for each finger length and thickness is calculated and stored as a standardized correction value. This completes the initial setting operation.

Next, the signer enters into the sign language action, and the image is input by the input unit 1 for each element constituting the sign language and stored in the still image recording unit 2.

Then, the detection unit 3 generates a hand image including the coordinate positions of both shoulders and elbows, the position and shape of the hand from the still image supplied from the still image recording unit 2, and the standardization unit 4 uses this hand image. Images are provided.

The standardization unit 4 corrects the positions and hands of both shoulders and elbows of the hand image supplied by each standardized correction value obtained and stored in the initial setting operation, and the supplied hand. The image is standardized and supplied to the recognition unit 5.

The recognition unit 5 searches the standard hand image storage unit 6 for the identifier of the standard hand image that is the closest to the supplied standardized hand image of the signer, and continues this operation until there is a conversion instruction. To create. When there is a conversion instruction, the conversion table 7 is searched for word information corresponding to the identifier string created in response to the conversion instruction, and the word information is supplied to the output unit 8.

The output unit 8 converts this word information into a voice synthesis unit 9
To the display processing unit 11 and output as voice through the speaker 10 to the display unit 1.
Display words via 2.

In the first embodiment, the standardized correction value for approximating the hand image of the signer to the standard hand image is used. However, the present invention is not limited to this, and the standard hand image of the signer is used. It can also be implemented using a correction value to approximate the hand image.

That is, as shown in FIG. 3, in the second embodiment, a conversion unit 13 is newly provided in addition to the standardization unit 4 in the first embodiment.

At the time of initial setting, the basic hand image of the signer is supplied from the detection unit 3, and the standard hand image of the basic form is supplied from the standard hand image storage unit 6 to approximate the standard hand image to the hand image of the signer in advance. A correction value for is calculated.

At the time of sign language operation, the standard hand image from the standard hand image storage unit 6 is corrected with this correction value and supplied to the recognition unit 5,
The recognition unit 5 searches for the identifier of the corrected standard hand image supplied from the conversion unit 13 that approximately corresponds to the hand image supplied from the detection unit 3. The subsequent operation is similar to that of the first embodiment.

As described above, the sign language translation apparatuses according to the first and second embodiments use the still image so that the sign language is not bothered and uncomfortable, and the sign language between the signers can be reduced. This has the effect of correcting the difference between the two and reducing misrecognition to enable sign language translation.

[0052]

As described above, the sign language translation apparatus of the present invention uses a still image so as not to bother the signer without causing discomfort and to correct the difference between the signers. This has the effect that sign language translation can be performed with less misrecognition.

[Brief description of drawings]

FIG. 1 is a block diagram showing a first embodiment of a sign language translation apparatus of the present invention.

FIG. 2A is an image diagram showing an example of a basic still image of sign language according to the present embodiment, and FIG. 2B is an example of a hand image of FIG. 2A which is a basic still image of sign language according to the present embodiment. It is an image figure to show.

FIG. 3 is a block diagram showing a second embodiment of the sign language translation apparatus of the present invention.

FIG. 4 is a block diagram showing an example of a conventional sign language translation apparatus.

[Explanation of symbols]

 1 Input Unit 2 Still Image Recording Unit 3, 101 Detecting Unit 4 Standardizing Unit 5, 102 Recognition Unit 6 Standard Hand Image Storage Unit 7, 104 Conversion Table 8, 105 Output Unit 9, 106 Voice Synthesis Unit 10, 108 Speaker 11, 107 Display processing unit 12, 109 Display 13 Conversion unit 100 Sign language translation device 103 Initial registration unit 110 Optical fiber cable

Claims (2)

[Claims] 1. A standard hand image in which a standard hand image, which is a standard form of a hand image including points indicating positions of both shoulders and elbows of the sign language element and positions and forms of both hands, is stored together with an identifier for each sign language element. A storage unit, a conversion table that stores word information corresponding to the identifier of the standard hand image and word information corresponding to a combination of the identifiers of the plurality of standard hand images, and a still image of a sign language element in the sign language state of a signer. Input means for inputting, detection means for detecting a position indicating the positions of both shoulders and elbows and the positions and forms of both hands from the still image to create a hand image, and a hand of the signer for a specific sign language element. A standardized correction value for approximating the image to the standard hand image corresponding to the specific sign language element stored in the standard hand image storage means is obtained in advance, and the hand image supplied from the detection means is standardized and corrected. A standardization means for creating a hand image standardized by a value, and an identifier string created by sequentially searching the identifiers of the standard hand image approximate to the standardized hand image supplied from the standardization means to create an identifier string and externally supplied conversion A recognition means for searching the conversion table for word information corresponding to the identifier string in response to an instruction; and an output means for outputting a corresponding voice based on the word information and performing a corresponding display. Sign language translation device. 2. A standard hand image in which a standard hand image, which is a standard form of a hand image including points indicating positions of both shoulders and elbows of the sign language element and positions and forms of both hands, is stored together with an identifier for each sign language element. A storage unit, a conversion table that stores word information corresponding to the identifier of the standard hand image and word information corresponding to a combination of the identifiers of the plurality of standard hand images, and a still image of a sign language element in the sign language state of a signer. Input means for inputting, detection means for detecting a position indicating the positions of both shoulders and elbows and the positions and forms of both hands from the still image to create a hand image, and a hand of the signer for a specific sign language element. The standard hand image stored in the standard hand image storage means is approximated to the standard hand image corresponding to the specific sign language element, and the standard hand image read out from the standard image storage means is obtained as the correction value. And a conversion means for creating a modified standard hand image, and an identifier of the modified standard hand image supplied from the conversion means, which is similar to the hand image supplied from the detection means, are sequentially searched to create an identifier string and externally A recognition means for searching the conversion table for word information corresponding to the identifier string in response to a conversion instruction supplied from the output means, and an output means for outputting a corresponding voice based on the word information and performing a corresponding display. A sign language translation device characterized by including.