CN111090998A - Sign language conversion method and device and sign language conversion device - Google Patents
Sign language conversion method and device and sign language conversion device Download PDFInfo
- Publication number
- CN111090998A CN111090998A CN201811217343.8A CN201811217343A CN111090998A CN 111090998 A CN111090998 A CN 111090998A CN 201811217343 A CN201811217343 A CN 201811217343A CN 111090998 A CN111090998 A CN 111090998A
- Authority
- CN
- China
- Prior art keywords
- vocabulary
- sign language
- natural
- sequence
- action
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
- G09B21/00—Teaching, or communicating with, the blind, deaf or mute
- G09B21/009—Teaching or communicating with deaf persons
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- General Health & Medical Sciences (AREA)
- Business, Economics & Management (AREA)
- Physics & Mathematics (AREA)
- Educational Administration (AREA)
- Educational Technology (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Machine Translation (AREA)
Abstract
The embodiment of the invention provides a sign language conversion method, a sign language conversion device and a sign language conversion device, wherein the method specifically comprises the following steps: determining a substitute vocabulary corresponding to the natural vocabulary according to the word vector corresponding to the natural vocabulary; taking the sign language vocabulary corresponding to the alternative vocabulary as the sign language vocabulary corresponding to the natural vocabulary; and converting the natural vocabulary into a sign language action according to the sign language vocabulary corresponding to the natural vocabulary. The embodiment of the invention can convert the natural vocabulary into the sign language action with accurate meaning, thereby avoiding the situation that the natural vocabulary cannot be converted or the conversion is wrong, and improving the accuracy of sign language conversion.
Description
Technical Field
The invention relates to the technical field of computers, in particular to a sign language conversion method and device and a device for converting a sign language.
Background
Sign language is one of the ways in which a particular user, who is hearing impaired or unable to speak, communicates, but a normal user does not usually know sign language, and thus, communication between the normal user and the particular user is difficult.
At present, normal users and special users can communicate by means of sign language translation tools. Taking Chinese as an example, the sign language translation tool can recognize characters input by a normal user and convert the characters into sign language actions corresponding to sign language vocabularies. Alternatively, the sign language translation tool may also convert sign language actions made by a particular user into corresponding text.
However, in practical applications, the sign language vocabulary corresponding to sign language actions is usually limited. Taking Chinese as an example, the sign language action only corresponds to 5586 sign language words, and the number of Chinese words is far greater than that of the sign language words; therefore, in the process of converting characters into sign language action, some Chinese words do not have corresponding sign language words. For example, the meaning of the Chinese word "monkey thunder" is very luck, but the sign language word "monkey thunder" does not exist, which may result in failure of sign language conversion or occurrence of an incorrect sign language conversion result, so that the accuracy of sign language conversion is low.
Disclosure of Invention
The embodiment of the invention provides a sign language conversion method and device and a sign language conversion device, which can solve the problem of sign language conversion errors caused by the fact that Chinese words do not have corresponding sign language words, and further can improve the accuracy of sign language conversion.
In order to solve the above problem, an embodiment of the present invention discloses a sign language conversion method, including:
determining a substitute vocabulary corresponding to the natural vocabulary according to the word vector corresponding to the natural vocabulary;
taking the sign language vocabulary corresponding to the alternative vocabulary as the sign language vocabulary corresponding to the natural vocabulary;
and converting the natural vocabulary into a sign language action according to the sign language vocabulary corresponding to the natural vocabulary.
On the other hand, the embodiment of the invention discloses a sign language conversion device, which comprises:
the first determining module is used for determining a substitute vocabulary corresponding to the natural vocabulary according to the word vector corresponding to the natural vocabulary;
the replacing module is used for taking the sign language vocabulary corresponding to the replacing vocabulary as the sign language vocabulary corresponding to the natural vocabulary;
and the first conversion module is used for converting the natural vocabulary into the sign language action according to the sign language vocabulary corresponding to the natural vocabulary.
In yet another aspect, an embodiment of the present invention discloses an apparatus for sign language conversion, comprising a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors comprise instructions for:
determining a substitute vocabulary corresponding to the natural vocabulary according to the word vector corresponding to the natural vocabulary;
taking the sign language vocabulary corresponding to the alternative vocabulary as the sign language vocabulary corresponding to the natural vocabulary;
and converting the natural vocabulary into a sign language action according to the sign language vocabulary corresponding to the natural vocabulary.
In yet another aspect, embodiments of the invention disclose a machine-readable medium having instructions stored thereon, which when executed by one or more processors, cause an apparatus to perform a sign language conversion method as described in one or more of the preceding.
The embodiment of the invention has the following advantages:
in the process of converting the sign language, if the natural vocabulary does not have the corresponding sign language vocabulary, the embodiment of the invention determines the alternative vocabulary corresponding to the natural vocabulary according to the word vector corresponding to the natural vocabulary, and uses the sign language vocabulary corresponding to the alternative vocabulary as the sign language vocabulary corresponding to the natural vocabulary, so that the natural vocabulary can be converted into the sign language action according to the sign language vocabulary corresponding to the alternative vocabulary. Because the words with similar meanings and the corresponding word vectors are also similar, the substitute vocabulary determined according to the word vectors has the same or similar meaning with the natural vocabulary, and the substitute vocabulary with the same or similar meaning is used, so that the natural vocabulary can be converted into sign language action with accurate meaning, the situation that the natural vocabulary cannot be converted or conversion is wrong is avoided, and the accuracy of sign language conversion is improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments of the present invention will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to these drawings without inventive labor.
FIG. 1 is a flow chart of the steps of an embodiment of a sign language conversion method of the present invention;
FIG. 2 is a schematic representation of a sign language action picture sequence of the present invention;
FIG. 3 is a flow chart of steps in another embodiment of a sign language conversion method of the present invention;
FIG. 4 is a block diagram of an embodiment of a sign language conversion apparatus according to the present invention;
FIG. 5 is a block diagram of an apparatus 800 for sign language conversion in accordance with the present invention; and
fig. 6 is a schematic diagram of a server in some embodiments of the invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Method embodiment
Referring to fig. 1, a flowchart illustrating steps of an embodiment of a sign language conversion method according to the present invention is shown, which may specifically include the following steps:
102, taking the sign language vocabulary corresponding to the alternative vocabulary as the sign language vocabulary corresponding to the natural vocabulary;
and 103, converting the natural vocabulary into a sign language action according to the sign language vocabulary corresponding to the natural vocabulary.
The embodiment of the invention can be operated on a terminal, and the terminal specifically comprises but is not limited to: smart phones, tablet computers, electronic book readers, MP3 (Moving Picture experts Group Audio Layer III) players, MP4 (Moving Picture experts Group Audio Layer IV) players, laptop portable computers, car-mounted computers, desktop computers, set-top boxes, smart televisions, wearable devices, and the like.
The embodiment of the present invention does not limit the language type corresponding to sign language conversion, and for example, the input chinese, english, japanese, korean, etc. can be converted into sign language actions of the corresponding country. For convenience of description, the embodiment of the present invention uses chinese as an example, and application scenarios of other language types may be referred to each other.
The natural vocabulary specifically refers to a vocabulary commonly used by a user with normal hearing, for example, the natural vocabulary may be a vocabulary included in a modern chinese dictionary, a "new chinese dictionary" or the like. The sign language vocabulary specifically refers to vocabularies commonly used by users with abnormal hearing, and each sign language vocabulary has a corresponding sign language action, for example, the sign language vocabulary specifically may be a vocabulary included in "Chinese sign language dictionary" or the like.
In practical application, the number of sign language words in the Chinese sign language dictionary is limited, and the situation that natural words do not have corresponding sign language words often occurs, so that sign language conversion errors are caused. In order to solve the problem, in the process of converting a natural vocabulary into a sign language vocabulary, if a certain natural vocabulary does not have a corresponding sign language vocabulary, the embodiment of the invention can determine a substitute vocabulary corresponding to the natural vocabulary according to a word vector corresponding to the natural vocabulary, and use the sign language vocabulary corresponding to the substitute vocabulary as the sign language vocabulary corresponding to the natural vocabulary, so that the natural vocabulary has a corresponding sign language vocabulary, and further can convert the natural vocabulary into a sign language action according to the sign language vocabulary corresponding to the natural vocabulary, so as to solve the problem that the natural vocabulary cannot be converted or conversion errors occur.
In an optional embodiment of the present invention, the determining, according to the word vector corresponding to the natural vocabulary, the alternative vocabulary corresponding to the natural vocabulary may specifically include:
step S11, determining a first word vector corresponding to the natural vocabulary;
step S12, determining a second word vector corresponding to the first word vector according to the distance between the word vectors; wherein, the natural vocabulary corresponding to the second word vector has a corresponding sign language vocabulary;
and step S13, taking the natural vocabulary corresponding to the second word vector as a substitute vocabulary.
The word vector has good semantic characteristics, and is a common way for representing word features. The value of each dimension of the word vector represents a feature with a certain semantic and grammatical interpretation. Therefore, each dimension of a word vector can be referred to as a word feature, and for words with similar meaning, the corresponding word vectors are similar.
In the embodiment of the present invention, a word vector table may be established in advance, where the word vector table includes a corresponding relationship between natural vocabularies and word vectors, and in a sign language conversion process, if a certain natural vocabulary does not have a corresponding sign language vocabulary, a first word vector corresponding to the natural vocabulary may be found in the word vector table, and according to a distance between word vectors, a second word vector may be found in the word vector table, where the distance between the second word vector and the first word vector is closest, and the natural vocabulary corresponding to the second word vector has a corresponding sign language vocabulary, and the natural vocabulary corresponding to the second word vector is used as a substitute vocabulary. Because the distance between the second word vector and the first word vector is relatively close, the meaning of the natural vocabulary corresponding to the second word vector is similar to that of the natural vocabulary corresponding to the first word vector, and therefore the natural vocabulary corresponding to the second word vector can be used as a substitute vocabulary.
In one application example of the present invention, since the natural word "monkey thunk" does not have a corresponding sign language word, a substitute word having a meaning closest to that of the natural word and having a corresponding sign language word can be specified from a word vector corresponding to the natural word "monkey thunk". Specifically, a first word vector such as V1 corresponding to the natural word "monkey thunder" may be found in the word vector table, and a second word vector such as V2 closest to V1 and having a corresponding sign language word may be found, and if the natural word corresponding to V2 is "good rhinocery", the "good rhinocery" may be used as a substitute word for "monkey thunder".
In another application example of the present invention, as for the natural word "666", the meaning is very excellent according to the context, but there is no sign language word "666" indicating very excellent, and therefore, in the conversion process, the "666" may be converted into a sign language action indicating the number six hundred sixty six, resulting in a conversion error. By the embodiment of the invention, the natural vocabulary closest to the word vector of 666 can be searched in the word vector table and is named as the bar, and the sign language vocabulary corresponding to the bar exists. Thus, a "wand" may be used as an alternative to "666".
In specific applications, there are many methods for generating Word vectors, including statistical-based methods or language model-based methods, for example, Word vectors can be generated according to the Word2vec model. Word2vec may project words into a K-dimensional vector space, and each Word may be represented by a K-dimensional vector. It is to be understood that the embodiment of the present invention does not impose a limitation on the specific method of generating the word vector.
In an optional embodiment of the invention, the method may further comprise:
step S21, performing word segmentation on the text sequence to be converted to obtain a natural vocabulary sequence corresponding to the text sequence to be converted;
step S22, determining sign language vocabularies corresponding to the natural vocabularies in the natural vocabulary sequence;
step S23, the sign language vocabularies corresponding to the natural vocabularies are sequenced to generate sign language vocabulary sequences conforming to the grammar rules of the sign language;
and step S24, converting the sign language vocabulary sequence into a sign language action sequence according to the sign language action corresponding to the sign language vocabulary.
Firstly, the embodiment of the invention can receive a text sequence input by a user with normal hearing, and perform word segmentation on the text sequence according to natural words to obtain a natural word sequence corresponding to the text sequence; the text sequence may be a sequence of characters, words, sentences and the like input by a user.
Then, determining sign language vocabularies corresponding to the natural vocabularies in the natural vocabulary sequence; if a certain natural vocabulary in the natural vocabulary sequence does not have a corresponding sign language vocabulary, determining a substitute vocabulary corresponding to the natural vocabulary according to a word vector corresponding to the natural vocabulary; and taking the sign language vocabulary corresponding to the alternative vocabulary as the sign language vocabulary corresponding to the natural vocabulary.
And then, sequencing the sign language vocabularies corresponding to the respective natural vocabularies to generate a sign language vocabulary sequence conforming to the grammar rule of the sign language.
In practical applications, sign language usually has unique grammatical rules, such as flip-flop of moving object, flip-flop of title, flip-flop of modifier, post-negation, and flip-flop of some fixed vocabulary and fixed expressions. Therefore, after the sign language vocabulary corresponding to each natural vocabulary in the natural vocabulary sequence is determined, the embodiments of the present invention also need to sequence each sign language vocabulary to generate the sign language vocabulary sequence conforming to the grammar rule of the sign language.
In an application example of the present invention, it is assumed that the natural vocabulary sequence is: today/,/nationality/water conservancy/science and technology/congress/in/beijing/recalling/, according to the grammar rule of the sign language, the sign language word sequence corresponding to the natural word sequence can be obtained as follows: today/,/major/nationwide/water conservancy/science/technology/recalling/Beijing/.
And finally, converting the sign language vocabulary sequence into a sign language action sequence according to the sign language action corresponding to the sign language vocabulary.
Specifically, each sign language vocabulary has a corresponding sign language action, and each sign language vocabulary in the sign language vocabulary sequence is replaced by the corresponding sign language action, so that the sign language vocabulary sequence can be converted into a sign language action sequence, that is, a text sequence input by a user is converted into the corresponding sign language action sequence.
In an optional embodiment of the present invention, the converting the natural vocabulary into a sign language action according to the sign language vocabulary corresponding to the natural vocabulary specifically may include:
step S31, according to the sign language vocabulary corresponding to the natural vocabulary, inquiring a sign language vocabulary library to obtain a sign language action picture sequence corresponding to the sign language vocabulary; the sign language vocabulary library comprises corresponding relations between sign language vocabularies and sign language action picture sequences;
and step S32, generating the sign language action corresponding to the natural vocabulary according to the sign language action picture sequence.
Referring to fig. 2, a schematic of a sign language action picture sequence of the present invention is shown, in which the sign language vocabulary corresponding to the sign language action in the picture sequence is "hello". Therefore, under the condition that the natural vocabulary does not have the corresponding sign language vocabulary, the embodiment of the invention can use the alternative vocabulary with the same or similar meaning to convert the natural vocabulary into the sign language action with accurate meaning so as to improve the accuracy of sign language conversion.
In an optional embodiment of the present invention, the generating, according to the sign language motion picture sequence, a sign language motion corresponding to the natural vocabulary may specifically include: and splicing the sign language action picture sequences in sequence to obtain a sign language action video corresponding to the natural vocabulary.
In order to enable the hearing-abnormal user to more intuitively watch the converted sign language action, the embodiment of the invention can also splice the sign language action picture sequence to generate a sign language action video, thereby further providing convenience for the communication between the hearing-abnormal user and the hearing-normal user. It can be understood that the specific manner of generating the sign language motion video is not limited by the embodiment of the present invention, for example, the sign language motion picture sequence may be spliced by using opencv (open source computer vision library) to generate the sign language motion video.
Referring to fig. 3, a flowchart illustrating steps of another embodiment of a sign language conversion method according to the present invention is shown, which may specifically include the following steps:
and 305, converting the natural vocabulary into a sign language action according to the sign language vocabulary corresponding to the natural vocabulary.
The embodiment of the invention can receive the text content or the non-text content input by the user with normal hearing through the terminal, and convert the text content or the non-text content into the corresponding sign language action, if the received non-text content is received, the non-text content is firstly converted into the corresponding text sequence, and then the obtained text sequence is converted into the corresponding sign language action, thereby simplifying the input process of the user and further improving the efficiency of sign language conversion.
In an optional embodiment of the present invention, the non-text content may specifically include: voice or pictures. Specifically, the embodiment of the present invention may perform text conversion on the received voice to obtain a text sequence corresponding to the voice, perform word segmentation on the text sequence to obtain a natural vocabulary corresponding to the voice, and further convert the natural vocabulary into a corresponding sign language action.
Or, the embodiment of the present invention may perform image recognition and character conversion on the received picture to obtain a text sequence corresponding to the picture, perform word segmentation on the text sequence to obtain a natural vocabulary corresponding to the picture, and further convert the natural vocabulary into a corresponding sign language action. It can be understood that the specific content of the picture is not limited in the embodiment of the present invention, for example, the picture may include content such as text, graphics, or expressions.
To sum up, in the process of converting sign language, if there is no corresponding sign language vocabulary in the natural vocabulary, the embodiment of the present invention determines the alternative vocabulary corresponding to the natural vocabulary according to the word vector corresponding to the natural vocabulary, and uses the sign language vocabulary corresponding to the alternative vocabulary as the sign language vocabulary corresponding to the natural vocabulary, so as to convert the natural vocabulary into sign language action according to the sign language vocabulary corresponding to the alternative vocabulary. Because the words with similar meanings and the corresponding word vectors are also similar, the substitute vocabulary determined according to the word vectors has the same or similar meaning with the natural vocabulary, and the substitute vocabulary with the same or similar meaning is used, so that the natural vocabulary can be converted into sign language action with accurate meaning, the situation that the natural vocabulary cannot be converted or conversion is wrong is avoided, and the accuracy of sign language conversion is improved.
It should be noted that, for simplicity of description, the method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present invention is not limited by the illustrated order of acts, as some steps may occur in other orders or concurrently in accordance with the embodiments of the present invention. Further, those skilled in the art will appreciate that the embodiments described in the specification are presently preferred and that no particular act is required to implement the invention.
Device embodiment
Referring to fig. 4, a block diagram of a sign language conversion apparatus according to an embodiment of the present invention is shown, which may specifically include: a first determination module 401, a substitution module 402 and a first conversion module 403.
The first determining module 401 is configured to determine, according to a word vector corresponding to a natural vocabulary, a substitute vocabulary corresponding to the natural vocabulary;
a replacing module 402, configured to use the sign language vocabulary corresponding to the replacing vocabulary as the sign language vocabulary corresponding to the natural vocabulary;
a first conversion module 403, configured to convert the natural vocabulary into a sign language action according to the sign language vocabulary corresponding to the natural vocabulary.
Optionally, the first determining module 401 may specifically include:
the first determining submodule is used for determining a first word vector corresponding to a natural word;
the second determining submodule is used for determining a second word vector corresponding to the first word vector according to the distance between the word vectors; wherein, the natural vocabulary corresponding to the second word vector has a corresponding sign language vocabulary;
and the third determining submodule is used for taking the natural vocabulary corresponding to the second word vector as a substitute vocabulary.
Optionally, the first conversion module 403 may specifically include:
the query submodule is used for querying a sign language vocabulary library according to the sign language vocabulary corresponding to the natural vocabulary so as to obtain a sign language action picture sequence corresponding to the sign language vocabulary; the sign language vocabulary library comprises corresponding relations between sign language vocabularies and sign language action picture sequences;
and the generation submodule is used for generating the sign language action corresponding to the natural vocabulary according to the sign language action picture sequence.
Optionally, the generating sub-module may specifically include:
and the splicing unit is used for splicing the sign language action picture sequences in sequence to obtain the sign language action video corresponding to the natural vocabulary.
Optionally, the apparatus may further include:
the second conversion module is used for performing character conversion on the received non-text content to obtain a text sequence corresponding to the non-text content;
and the first word segmentation module is used for performing word segmentation on the text sequence to obtain a natural word corresponding to the non-text content.
Optionally, the non-text content may specifically include: voice or pictures.
Optionally, the apparatus may further include:
the second word segmentation module is used for performing word segmentation on the text sequence to be converted to obtain a natural vocabulary sequence corresponding to the text sequence to be converted;
the second determining module is used for determining sign language vocabularies corresponding to the natural vocabularies in the natural vocabulary sequence;
the sequencing module is used for sequencing the sign language vocabularies corresponding to the respective natural vocabularies so as to generate a sign language vocabulary sequence conforming to the sign language grammar rule;
and the third conversion module is used for converting the sign language vocabulary sequence into a sign language action sequence according to the sign language action corresponding to the sign language vocabulary.
For the device embodiment, since it is basically similar to the method embodiment, the description is simple, and for the relevant points, refer to the partial description of the method embodiment.
The embodiments in the present specification are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other.
With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.
An embodiment of the present invention provides an apparatus for sign language conversion, comprising a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by one or more processors, the one or more programs including instructions for: determining a substitute vocabulary corresponding to the natural vocabulary according to the word vector corresponding to the natural vocabulary; taking the sign language vocabulary corresponding to the alternative vocabulary as the sign language vocabulary corresponding to the natural vocabulary; and converting the natural vocabulary into a sign language action according to the sign language vocabulary corresponding to the natural vocabulary.
Fig. 5 is a block diagram illustrating an apparatus 800 for sign language conversion in accordance with an exemplary embodiment. For example, the apparatus 800 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, an exercise device, a personal digital assistant, and the like.
Referring to fig. 5, the apparatus 800 may include one or more of the following components: processing component 802, memory 804, power component 806, multimedia component 808, audio component 810, input/output (I/O) interface 812, sensor component 814, and communication component 816.
The processing component 802 generally controls overall operation of the device 800, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. The processing elements 802 may include one or more processors 820 to execute instructions to perform all or a portion of the steps of the methods described above. Further, the processing component 802 can include one or more modules that facilitate interaction between the processing component 802 and other components. For example, the processing component 802 can include a multimedia module to facilitate interaction between the multimedia component 808 and the processing component 802.
The memory 804 is configured to store various types of data to support operation at the device 800. Examples of such data include instructions for any application or method operating on device 800, contact data, phonebook data, messages, pictures, videos, and so forth. The memory 804 may be implemented by any type or combination of volatile or non-volatile memory devices such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks.
The multimedia component 808 includes a screen that provides an output interface between the device 800 and a user. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front facing camera and/or a rear facing camera. The front-facing camera and/or the rear-facing camera may receive external multimedia data when the device 800 is in an operating mode, such as a shooting mode or a video mode. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.
The audio component 810 is configured to output and/or input audio signals. For example, the audio component 810 includes a Microphone (MIC) configured to receive external audio signals when the apparatus 800 is in an operational mode, such as a call mode, a recording mode, and a voice data processing mode. The received audio signals may further be stored in the memory 804 or transmitted via the communication component 816. In some embodiments, audio component 810 also includes a speaker for outputting audio signals.
The I/O interface 812 provides an interface between the processing component 802 and peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to: a home button, a volume button, a start button, and a lock button.
The sensor assembly 814 includes one or more sensors for providing various aspects of state assessment for the device 800. For example, the sensor assembly 814 may detect the open/closed state of the device 800, the relative positioning of the components, such as a display and keypad of the apparatus 800, the sensor assembly 814 may also detect a change in position of the apparatus 800 or a component of the apparatus 800, the presence or absence of user contact with the apparatus 800, orientation or acceleration/deceleration of the apparatus 800, and a change in temperature of the apparatus 800. Sensor assembly 814 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.
The communication component 816 is configured to facilitate communications between the apparatus 800 and other devices in a wired or wireless manner. The device 800 may access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 further includes a Near Field Communication (NFC) module to facilitate short-range communications. For example, the NFC module may be implemented based on radio frequency data processing (RFID) technology, infrared data association (IrDA) technology, Ultra Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.
In an exemplary embodiment, the apparatus 800 may be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, micro-controllers, microprocessors or other electronic components for performing the above-described methods.
In an exemplary embodiment, a non-transitory computer-readable storage medium comprising instructions, such as the memory 804 comprising instructions, executable by the processor 820 of the device 800 to perform the above-described method is also provided. For example, the non-transitory computer readable storage medium may be a ROM, a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.
Fig. 6 is a schematic diagram of a server in some embodiments of the invention. The server 1900, which may vary widely in configuration or performance, may include one or more Central Processing Units (CPUs) 1922 (e.g., one or more processors) and memory 1932, one or more storage media 1930 (e.g., one or more mass storage devices) storing applications 1942 or data 1944. Memory 1932 and storage medium 1930 can be, among other things, transient or persistent storage. The program stored in the storage medium 1930 may include one or more modules (not shown), each of which may include a series of instructions operating on a server. Still further, a central processor 1922 may be provided in communication with the storage medium 1930 to execute a series of instruction operations in the storage medium 1930 on the server 1900.
The server 1900 may also include one or more power supplies 1926, one or more wired or wireless network interfaces 1950, one or more input-output interfaces 1958, one or more keyboards 1956, and/or one or more operating systems 1941, such as Windows Server, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.
A non-transitory computer-readable storage medium in which instructions, when executed by a processor of an apparatus (server or terminal), enable the apparatus to perform a sign language conversion method shown in fig. 1 or 3.
A non-transitory computer-readable storage medium in which instructions, when executed by a processor of an apparatus (server or terminal), enable the apparatus to perform a sign language conversion method, the method comprising: determining a substitute vocabulary corresponding to the natural vocabulary according to the word vector corresponding to the natural vocabulary; taking the sign language vocabulary corresponding to the alternative vocabulary as the sign language vocabulary corresponding to the natural vocabulary; and converting the natural vocabulary into a sign language action according to the sign language vocabulary corresponding to the natural vocabulary.
The embodiment of the invention discloses A1 and a sign language conversion method, which comprises the following steps:
determining a substitute vocabulary corresponding to the natural vocabulary according to the word vector corresponding to the natural vocabulary;
taking the sign language vocabulary corresponding to the alternative vocabulary as the sign language vocabulary corresponding to the natural vocabulary;
and converting the natural vocabulary into a sign language action according to the sign language vocabulary corresponding to the natural vocabulary.
A2, according to the method in A1, the determining an alternative vocabulary corresponding to the natural vocabulary according to the word vector corresponding to the natural vocabulary includes:
determining a first word vector corresponding to a natural word;
determining a second word vector corresponding to the first word vector according to the distance between the word vectors; wherein, the natural vocabulary corresponding to the second word vector has a corresponding sign language vocabulary;
and taking the natural vocabulary corresponding to the second word vector as a substitute vocabulary.
A3, the method according to A1, wherein the converting the natural vocabulary into the sign language action according to the sign language vocabulary corresponding to the natural vocabulary comprises:
inquiring a sign language vocabulary library according to sign language vocabularies corresponding to natural vocabularies to obtain a sign language action picture sequence corresponding to the sign language vocabularies; the sign language vocabulary library comprises corresponding relations between sign language vocabularies and sign language action picture sequences;
and generating the sign language action corresponding to the natural vocabulary according to the sign language action picture sequence.
A4, according to the method in A3, the generating the sign language action corresponding to the natural vocabulary according to the sign language action picture sequence includes:
and splicing the sign language action picture sequences in sequence to obtain a sign language action video corresponding to the natural vocabulary.
A5, the method of any one of A1 to A4, the method further comprising:
performing character conversion on received non-text content to obtain a text sequence corresponding to the non-text content;
and performing word segmentation on the text sequence to obtain a natural vocabulary corresponding to the non-text content.
A6, the method of A5, the non-textual content comprising: voice or pictures.
A7, the method of any one of A1 to A4, the method further comprising:
performing word segmentation on a text sequence to be converted to obtain a natural vocabulary sequence corresponding to the text sequence to be converted;
determining sign language vocabularies corresponding to the natural vocabularies in the natural vocabulary sequence;
sequencing the sign language vocabularies corresponding to the respective natural vocabularies to generate a sign language vocabulary sequence conforming to the sign language grammar rule;
and converting the sign language vocabulary sequence into a sign language action sequence according to the sign language action corresponding to the sign language vocabulary.
The embodiment of the invention discloses B8, a sign language conversion device, comprising:
the first determining module is used for determining a substitute vocabulary corresponding to the natural vocabulary according to the word vector corresponding to the natural vocabulary;
the replacing module is used for taking the sign language vocabulary corresponding to the replacing vocabulary as the sign language vocabulary corresponding to the natural vocabulary;
and the first conversion module is used for converting the natural vocabulary into the sign language action according to the sign language vocabulary corresponding to the natural vocabulary.
B9, the apparatus of B8, the first determining module comprising:
the first determining submodule is used for determining a first word vector corresponding to a natural word;
the second determining submodule is used for determining a second word vector corresponding to the first word vector according to the distance between the word vectors; wherein, the natural vocabulary corresponding to the second word vector has a corresponding sign language vocabulary;
and the third determining submodule is used for taking the natural vocabulary corresponding to the second word vector as a substitute vocabulary.
B10, the apparatus of B8, the first conversion module comprising:
the query submodule is used for querying a sign language vocabulary library according to the sign language vocabulary corresponding to the natural vocabulary so as to obtain a sign language action picture sequence corresponding to the sign language vocabulary; the sign language vocabulary library comprises corresponding relations between sign language vocabularies and sign language action picture sequences;
and the generation submodule is used for generating the sign language action corresponding to the natural vocabulary according to the sign language action picture sequence.
B11, the apparatus according to B10, the generating submodule includes:
and the splicing unit is used for splicing the sign language action picture sequences in sequence to obtain the sign language action video corresponding to the natural vocabulary.
B12, the apparatus according to any one of B8 to B11, further comprising:
the second conversion module is used for performing character conversion on the received non-text content to obtain a text sequence corresponding to the non-text content;
and the first word segmentation module is used for performing word segmentation on the text sequence to obtain a natural word corresponding to the non-text content.
B13, the apparatus of B12, the non-textual content comprising: voice or pictures.
B14, the apparatus of any one of claims B8 to B11, further comprising:
the second word segmentation module is used for performing word segmentation on the text sequence to be converted to obtain a natural vocabulary sequence corresponding to the text sequence to be converted;
the second determining module is used for determining sign language vocabularies corresponding to the natural vocabularies in the natural vocabulary sequence;
the sequencing module is used for sequencing the sign language vocabularies corresponding to the respective natural vocabularies so as to generate a sign language vocabulary sequence conforming to the sign language grammar rule;
and the third conversion module is used for converting the sign language vocabulary sequence into a sign language action sequence according to the sign language action corresponding to the sign language vocabulary.
The embodiment of the invention discloses C15, an apparatus for sign language conversion, comprising a memory, and one or more programs, wherein the one or more programs are stored in the memory, and the one or more programs configured to be executed by the one or more processors comprise instructions for:
determining a substitute vocabulary corresponding to the natural vocabulary according to the word vector corresponding to the natural vocabulary;
taking the sign language vocabulary corresponding to the alternative vocabulary as the sign language vocabulary corresponding to the natural vocabulary;
and converting the natural vocabulary into a sign language action according to the sign language vocabulary corresponding to the natural vocabulary.
C16, the apparatus according to C15, the determining an alternative vocabulary corresponding to the natural vocabulary according to the word vector corresponding to the natural vocabulary includes:
determining a first word vector corresponding to a natural word;
determining a second word vector corresponding to the first word vector according to the distance between the word vectors; wherein, the natural vocabulary corresponding to the second word vector has a corresponding sign language vocabulary;
and taking the natural vocabulary corresponding to the second word vector as a substitute vocabulary.
C17, the apparatus according to C15, the method for converting natural vocabulary into sign language action according to sign language vocabulary corresponding to the natural vocabulary comprises:
inquiring a sign language vocabulary library according to sign language vocabularies corresponding to natural vocabularies to obtain a sign language action picture sequence corresponding to the sign language vocabularies; the sign language vocabulary library comprises corresponding relations between sign language vocabularies and sign language action picture sequences;
and generating the sign language action corresponding to the natural vocabulary according to the sign language action picture sequence.
C18, the device according to C17, wherein the generating of the sign language action corresponding to the natural vocabulary according to the sign language action picture sequence comprises:
and splicing the sign language action picture sequences in sequence to obtain a sign language action video corresponding to the natural vocabulary.
C19, the device of any of C15-C18, the device configured to execute the one or more programs by one or more processors including instructions for:
performing character conversion on received non-text content to obtain a text sequence corresponding to the non-text content;
and performing word segmentation on the text sequence to obtain a natural vocabulary corresponding to the non-text content.
C20, the apparatus of C19, the non-textual content comprising: voice or pictures.
C21, the device of any of C15-C18, the device configured to execute the one or more programs by one or more processors including instructions for:
performing word segmentation on a text sequence to be converted to obtain a natural vocabulary sequence corresponding to the text sequence to be converted;
determining sign language vocabularies corresponding to the natural vocabularies in the natural vocabulary sequence;
sequencing the sign language vocabularies corresponding to the respective natural vocabularies to generate a sign language vocabulary sequence conforming to the sign language grammar rule;
and converting the sign language vocabulary sequence into a sign language action sequence according to the sign language action corresponding to the sign language vocabulary.
Embodiments of the present invention disclose D22, a machine-readable medium having instructions stored thereon, which when executed by one or more processors, cause an apparatus to perform a sign language conversion method as described in one or more of a 1-a 7.
Other embodiments of the invention will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the invention and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the invention being indicated by the following claims.
It will be understood that the invention is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the invention is limited only by the appended claims.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.
The present invention provides a sign language conversion method, a sign language conversion device and a device for sign language conversion, which are introduced in detail above, and the principle and the implementation of the present invention are explained by applying specific examples, and the descriptions of the above examples are only used to help understanding the method and the core idea of the present invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.
Claims (10)
1. A sign language conversion method, characterized in that the method comprises:
determining a substitute vocabulary corresponding to the natural vocabulary according to the word vector corresponding to the natural vocabulary;
taking the sign language vocabulary corresponding to the alternative vocabulary as the sign language vocabulary corresponding to the natural vocabulary;
and converting the natural vocabulary into a sign language action according to the sign language vocabulary corresponding to the natural vocabulary.
2. The method of claim 1, wherein determining an alternative vocabulary corresponding to the natural vocabulary from the word vector corresponding to the natural vocabulary comprises:
determining a first word vector corresponding to a natural word;
determining a second word vector corresponding to the first word vector according to the distance between the word vectors; wherein, the natural vocabulary corresponding to the second word vector has a corresponding sign language vocabulary;
and taking the natural vocabulary corresponding to the second word vector as a substitute vocabulary.
3. The method of claim 1, wherein converting the natural vocabulary into a sign language action according to a sign language vocabulary corresponding to the natural vocabulary comprises:
inquiring a sign language vocabulary library according to sign language vocabularies corresponding to natural vocabularies to obtain a sign language action picture sequence corresponding to the sign language vocabularies; the sign language vocabulary library comprises corresponding relations between sign language vocabularies and sign language action picture sequences;
and generating the sign language action corresponding to the natural vocabulary according to the sign language action picture sequence.
4. The method according to claim 3, wherein the generating of the sign language action corresponding to the natural vocabulary according to the sign language action picture sequence comprises:
and splicing the sign language action picture sequences in sequence to obtain a sign language action video corresponding to the natural vocabulary.
5. The method according to any one of claims 1 to 4, further comprising:
performing character conversion on received non-text content to obtain a text sequence corresponding to the non-text content;
and performing word segmentation on the text sequence to obtain a natural vocabulary corresponding to the non-text content.
6. The method of claim 5, wherein the non-textual content comprises: voice or pictures.
7. The method according to any one of claims 1 to 4, further comprising:
performing word segmentation on a text sequence to be converted to obtain a natural vocabulary sequence corresponding to the text sequence to be converted;
determining sign language vocabularies corresponding to the natural vocabularies in the natural vocabulary sequence;
sequencing the sign language vocabularies corresponding to the respective natural vocabularies to generate a sign language vocabulary sequence conforming to the sign language grammar rule;
and converting the sign language vocabulary sequence into a sign language action sequence according to the sign language action corresponding to the sign language vocabulary.
8. A sign language conversion apparatus, comprising:
the first determining module is used for determining a substitute vocabulary corresponding to the natural vocabulary according to the word vector corresponding to the natural vocabulary;
the replacing module is used for taking the sign language vocabulary corresponding to the replacing vocabulary as the sign language vocabulary corresponding to the natural vocabulary;
and the first conversion module is used for converting the natural vocabulary into the sign language action according to the sign language vocabulary corresponding to the natural vocabulary.
9. An apparatus for sign language conversion, comprising a memory, and one or more programs, wherein the one or more programs are stored in the memory, and wherein execution of the one or more programs by one or more processors comprises instructions for:
determining a substitute vocabulary corresponding to the natural vocabulary according to the word vector corresponding to the natural vocabulary;
taking the sign language vocabulary corresponding to the alternative vocabulary as the sign language vocabulary corresponding to the natural vocabulary;
and converting the natural vocabulary into a sign language action according to the sign language vocabulary corresponding to the natural vocabulary.
10. A machine-readable medium having stored thereon instructions, which when executed by one or more processors, cause an apparatus to perform a sign language conversion method as recited in one or more of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811217343.8A CN111090998A (en) | 2018-10-18 | 2018-10-18 | Sign language conversion method and device and sign language conversion device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811217343.8A CN111090998A (en) | 2018-10-18 | 2018-10-18 | Sign language conversion method and device and sign language conversion device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111090998A true CN111090998A (en) | 2020-05-01 |
Family
ID=70391257
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811217343.8A Pending CN111090998A (en) | 2018-10-18 | 2018-10-18 | Sign language conversion method and device and sign language conversion device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111090998A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112073749A (en) * | 2020-08-07 | 2020-12-11 | 中国科学院计算技术研究所 | Sign language video synthesis method, sign language translation system, medium and electronic equipment |
| CN113657101A (en) * | 2021-07-20 | 2021-11-16 | 北京搜狗科技发展有限公司 | Data processing method and device and data processing device |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101079268A (en) * | 2006-12-28 | 2007-11-28 | 腾讯科技（深圳）有限公司 | System and method for sign language synthesis and display |
| CN103956167A (en) * | 2014-05-06 | 2014-07-30 | 北京邮电大学 | Visual sign language interpretation method and device based on Web |
-
2018
- 2018-10-18 CN CN201811217343.8A patent/CN111090998A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101079268A (en) * | 2006-12-28 | 2007-11-28 | 腾讯科技（深圳）有限公司 | System and method for sign language synthesis and display |
| CN103956167A (en) * | 2014-05-06 | 2014-07-30 | 北京邮电大学 | Visual sign language interpretation method and device based on Web |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112073749A (en) * | 2020-08-07 | 2020-12-11 | 中国科学院计算技术研究所 | Sign language video synthesis method, sign language translation system, medium and electronic equipment |
| CN113657101A (en) * | 2021-07-20 | 2021-11-16 | 北京搜狗科技发展有限公司 | Data processing method and device and data processing device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110210310B (en) | Video processing method and device for video processing | |
| CN107564526B (en) | Processing method, apparatus and machine-readable medium | |
| CN111128183B (en) | Speech recognition method, apparatus and medium | |
| CN111369978B (en) | Data processing method and device for data processing | |
| CN108628819B (en) | Processing method and device for processing | |
| CN107665046B (en) | Input method and device and input device | |
| CN111160047A (en) | Data processing method and device and data processing device | |
| CN113539233A (en) | Voice processing method and device and electronic equipment | |
| CN111090998A (en) | Sign language conversion method and device and sign language conversion device | |
| CN109992754B (en) | Document processing method and device | |
| CN113987128A (en) | Related article searching method and device, electronic equipment and storage medium | |
| CN113657101A (en) | Data processing method and device and data processing device | |
| CN112002313B (en) | Interaction method and device, sound box, electronic equipment and storage medium | |
| CN111640452B (en) | Data processing method and device for data processing | |
| CN111414766B (en) | Translation method and device | |
| CN110780749B (en) | Character string error correction method and device | |
| CN110795014B (en) | Data processing method and device and data processing device | |
| CN109992790B (en) | Data processing method and device for data processing | |
| CN114154395A (en) | Model processing method and device for model processing | |
| CN112837668B (en) | Voice processing method and device for processing voice | |
| CN109388328B (en) | Input method, device and medium | |
| CN109992121B (en) | Input method, input device and input device | |
| CN107102747B (en) | Information input method and device for inputting information | |
| CN112905023A (en) | Input error correction method and device for input error correction | |
| CN108073566B (en) | Word segmentation method and device and word segmentation device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |