WO2017124582A1

WO2017124582A1 - Image selection-based sign language input method, device, and mobile phone

Info

Publication number: WO2017124582A1
Application number: PCT/CN2016/072774
Authority: WO
Inventors: 钟世雄
Original assignee: 钟世雄
Priority date: 2016-01-19
Filing date: 2016-01-29
Publication date: 2017-07-27
Also published as: CN106981237A

Abstract

An image selection-based sign language input method comprises: S1, receiving login information of a sign language user; S2, receiving a sign language type selected by the sign language user, and importing all sign language images of the type; S3, determining sign language images selected by the user, and storing the sign language images as communication information; S4, determining whether the user has completed input of the information, and if not, returning to step S2, otherwise proceeding to step S5; and S5, sending the communication information. Also disclosed are an image selection-based sign language input device and mobile phone. Employing the image selection-based sign language input method, device, or mobile phone enables deaf-mute people, especially deaf-mute people having a lower level of literacy, to communicate in sign languages by using mobile phones.

Description

Method, device and mobile phone for voice input based on picture selection

This patent application is filed on January 19, 2016, the application number is 201610035839.8, and the applicant is Zhong Shixiong, the priority of the Chinese patent application entitled "A method, device and mobile phone for voice input based on picture selection", The entire content of this application is hereby incorporated by reference.

Technical field

The present invention relates to the field of dummy input, and in particular to a method, device and mobile phone for voice input based on picture selection.

Background technique

Sign language is the language used by deaf people. It is a relatively stable expression system composed of hand-shaped movements and expression gestures. It is a special language that communicates by action/visual. Interpersonal interactions are often eloquent. In addition to natural language (spoken, written language), human language (expressions, body, gestures) is one of the basic ways of interpersonal interaction. Sign language is a very important part of human language. It is a human language with the most information. It has the same expression ability as natural language such as spoken and written language. Therefore, in terms of human-computer interaction, sign language can be used as a kind of Means, and has a strong visual effect, it is vivid, vivid and intuitive.

With the rapid development of technology, people from different regions are more and more convenient to communicate through mobile phones, but there are still problems in communication with some special people. With the increase of the public's attention to the disadvantaged groups in recent years, the communication problems between the disabled have become more and more concerned, and the deaf-mute people are eager to communicate via mobile phones.

Summary of the invention

In order to solve the above problems, the embodiments of the present invention provide a method, a device, and a mobile phone for enabling dumb input based on picture selection for a deaf person to realize normal communication through a mobile phone.

The technical solution of the embodiment of the present invention can be implemented in the following manner:

The embodiment of the invention discloses a method for inputting a dummy word based on picture selection, comprising:

S1. Accept the login information of the dummy user;

S2. accepting the selection of the dummy word classification by the dummy user, and importing all the dummy pictures under the selected category;

S3. confirming the dummy picture selected by the user, and storing the dummy picture as the communication information;

S4. Determine whether the user completes the input of information, and if not, returns to step S1; if so, proceeds to step S5;

S5. Send the communication information.

As a further improvement of the method for selecting a dummy word based on picture selection according to the embodiment of the present invention, the following steps may be performed before the step S1:

A1. Accepting registration information of the user; the registration information includes: user name information, password information, and user type information; the user type information includes: a user who can communicate by sign language is the first type, and can use voice information. The user of the communication is of the second type;

A2. Determine which type of user type is selected, if it is the first type, go to step S1; if it is the second type, go to step A3;

A3. Grab the information entered by the user for communication.

The method for inputting a dummy word based on picture selection according to the embodiment of the present invention is further improved, wherein the dummy picture is stored as communication information, and specifically: the chat content expressed by the dummy picture is converted into a correspondence table by using a correspondence table. The text information is stored as communication information, wherein the correspondence table is stored in the background of the local communication device or the communication server.

The method for the dummy input based on the picture selection according to the embodiment of the present invention is further improved, and the step S5 may further include the following steps:

B1. Accept the communication information described;

B2. Determine the user type, if it is the first type, convert the text information into a dummy picture for display, and if it is the second type, directly prompt the communication information.

The embodiment of the invention discloses a device for inputting a dummy word based on picture selection, comprising:

a dummy user login information unit for accepting login information of the dummy user;

a classification selection unit, configured to accept the selection of the dummy category by the dummy user, and import all the dummy pictures under the selected category;

a dummy picture confirmation unit, configured to confirm a dummy picture selected by the user, and store the dummy picture as communication information;

The input completion determining unit is configured to determine whether the user completes the input of the information, and if not, returns to the step classification selection unit; if yes, enters the sending unit;

a sending unit, configured to send the communication information.

As a further improvement of the device for selecting a dummy word based on picture selection according to the present invention, the dummy user login information unit may further have the following units:

a user registration unit, configured to accept registration information of the user; the registration information includes: user name information, password information, and user type information; and the user type information includes: a user who can communicate through sign language is the first type. The second type of users who can communicate using voice information;

a user type determining unit, configured to determine the selected user type, if the first type, enter the dummy user login information unit; if it is the second type, enter the crawling unit;

The crawling unit is used to capture information input by the user for communication.

According to a further improvement of the device for selecting a dummy word based on picture selection according to the embodiment of the present invention, the dummy picture is stored as communication information, and specifically: the chat content expressed by the dummy picture is converted into a correspondence table by using a correspondence table. The text information is stored as communication information, wherein the correspondence table is stored in the background of the local communication device and/or the communication server.

As a further improvement of the apparatus for muting input based on picture selection according to the embodiment of the present invention, the transmitting unit further has the following units:

An information accepting unit, configured to accept the communication information;

The information display unit is configured to determine the user type. If the first type is, the text information is converted into a dummy picture for display, and if the second type is, the communication information is directly prompted.

The embodiment of the invention discloses a mobile phone based on a dummy input of picture selection, the mobile phone has a database composed of a plurality of dummy picture information and a controller connected to the database, and the controller has the above-mentioned A device for dummy input based on picture selection.

A further improvement of the mobile phone based on the picture selection-based dummy input according to the embodiment of the present invention, wherein the dummy picture information is a coding unit in a sign language or a gesture picture, and each coding unit corresponds to one a combination of text or text.

A method, device and mobile phone for implementing dummy input based on picture selection according to embodiments of the present invention have the following beneficial technical effects:

The embodiments of the present invention can enable deaf-mute people, especially deaf-mute people with relatively low file level, to use mobile phones to communicate through sign language, improve and improve the life learning and working conditions of deaf people, and provide them with better services. .

DRAWINGS

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the embodiments or the description of the prior art will be briefly described below. Obviously, the drawings in the following description are only It is a certain embodiment of the present invention, and other drawings can be obtained from those skilled in the art without any inventive labor.

1 is a flow chart of a method for dummy input based on picture selection according to an embodiment of the present invention;

FIG. 1a is a diagram showing an example of correspondence between a picture and a text according to an embodiment of the present invention; FIG.

2 is a block diagram of an apparatus for dummy input based on picture selection in an embodiment of the present invention;

3 is a diagram of a mobile phone module based on a dummy input of a picture selection according to an embodiment of the present invention.

detailed description

The technical solutions in the embodiments of the present invention are clearly and completely described in the following with reference to the accompanying drawings in the embodiments of the present invention. It is obvious that the described embodiments are only a part of the embodiments of the present invention, but not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts are within the scope of the present invention.

Referring to FIG. 1 , an embodiment of the present invention provides a method for muting input based on picture selection, including:

S1. Accept the login information of the dummy user;

In the embodiment of the present invention, a button can be set on the terminal, and the button is logged in, wherein the login action can be an action of directly turning on the deaf person.

S2. accepting the selection of the dummy category by the dummy user and importing all the dummy pictures under the selected category;

In the embodiment of the present invention, the dummy picture may be thousands of standard dummy pictures stored in a database.

In the embodiment of the present invention, the picture selection input method is a sign language and a gesture picture as a coding unit, and is characterized in that each sign language and gesture picture corresponds to a text group: as shown in FIG. 1a.

For example, when the picture is: the middle finger of the finger points to the user, the text information corresponding to the picture is “I”, and when the picture is: the middle finger of the finger points away from the user, the text information corresponding to the picture is “you "and many more.

In this way, the sign language gesture pictures are classified and encoded, so that a voice text can be quickly input on a computer, a mobile phone, and the like.

In the embodiment of the present invention, the mute word classification can be divided into a plurality of large-scale pictures, each of which has a plurality of sub-categories, and each sub-category has a plurality of related pictures, and the user selects the pictures under the sub-categories. You can also make a Lenovo input to increase the input speed.

For example, the classification of the mute pictures may be: first divided into fourteen categories of pictures, each of which has multiple sub-categories.

First sort the first way: in the order, sorted as follows:

One, 1-15, a total of 15 sub-categories; (standard dummy words corresponding to the characters)

Second, 16-29, a total of 14 sub-categories; (standard dumb images of life standards corresponding to the text)

Third, 30-42, a total of 13 sub-categories; (standard mute pictures of clothing corresponding to the text)

Four, 43-57, a total of 15 sub-categories; (standard food-speech pictures corresponding to the text)

5, 58-73, a total of 16 sub-categories; (common goods standard mute pictures corresponding to the text)

Six, 74-89, a total of 16 sub-categories; (standard mute pictures corresponding to the adjective class corresponding to the text)

Seven, 90-101, a total of 12 sub-categories; (standard mute pictures corresponding to the verb class corresponding to the text)

Eight, 102-116, a total of 15 sub-categories; (standard mute pictures corresponding to the activity class corresponding to the text)

Nine, 117-132, a total of 16 sub-categories; (standard mute corresponding to the earning class corresponding to the text) image)

Ten, 133-147, a total of 15 sub-categories; (standard mute pictures corresponding to the cultural category corresponding to the text)

Eleven, 148-163, a total of 16 sub-categories; (standard mute pictures corresponding to the political and military categories corresponding to the text)

Twelve, 164-178, a total of 15 sub-categories; (standard mute pictures corresponding to the science and education class corresponding to the text)

Thirteen, 179-191, a total of 13 sub-categories; (standard mute pictures corresponding to the industrial category corresponding to the text)

Fourteen, 192-203, a total of 12 sub-categories; (standard mute pictures corresponding to other classes corresponding to the text)

The advantage of this method is that the coding of each sign language is in order.

The above sequence can be set on the terminal by means of a button, and the deaf-mute person needs to learn the corresponding picture combination of each major class, just like the five-stroke key Chinese input principle.

Or, carry out the second classification, classification 2:

For example, the above 13 items (179-191), 14 items (192-203), and 12 items (164-178) are grouped together, which are counted as a large group, a total of 40 sub-categories, all of which belong to geo-geological minerals. image.

For example, in the above 11 items (148-163), there are 16 sub-categories of "characteristics of things", "personality and talents", and "nature performance", which are classified as adjectives.

In this way, the standard picture of the classification method 1 can be divided into a total of 14 major categories: each major class is as follows: large class name, small class name, small class number, small class number.

Take the above-mentioned standard dummy word picture corresponding to the character as an example to illustrate:

Large class name: character; small class name: title; small class number: occupation; small class number: position.

Correspondence is as follows: People: 16 pictures, title: 2 to 6 pictures, occupation: 7 to 9 pictures, positions 10 to 11 pictures.

For example, when deaf-mute people choose "I miss you", they select two pictures "I" and "you" from the character classification, and select the verb category from the seventh category to select the "think" picture to form three picture information. Used for communication information and converted.

For example, when the system grabs a dummy picture pointing to the user, the picture is considered to be equivalent to the input of the text "I".

S4. Determine whether the user completes the input of information, and if not, returns to step S2; if yes, proceeds to step S5;

When the system captures the corresponding mute image of the text message, after the user confirms, the display interface is refreshed, and the user is selected to select the mute action picture again.

S5. Send the communication information.

Select the send button on the terminal and the corresponding communication information will be sent out.

Further, in another embodiment of the embodiment of the present invention, the technical effect is that the communication between the deaf-mute person and the ordinary person can be realized, that is, the following steps are performed before the step S1:

A1. Accepting the registration information of the user; the registration information includes: user name information, password information, and user type information; the user type information includes: the user who can communicate by sign language is the first type, and the user who can use the voice information exchange is The second type, for example, a dumb-hearing person who can communicate by sign language as the first type of user, and a healthy person who can use voice information exchange as the second type of user;

A2. According to the user type information in the registration information, determine which type of user is selected, if it is the first type, proceeds to step S1; if it is the second type, proceeds to step A3;

A3. Grab the information entered by the user for communication.

That is, step A3 is a textual and/or voice communication performed by a normal healthy person in the prior art.

Further, in order to realize the manner in which the deaf-mute person and the ordinary person use the communication received or called, that is, after the step S5, the following steps are also performed:

B1. Accept the communication information;

B2. Determine which type of user is, if it is the first type, convert the text information into a dummy picture for display, and if it is the second type, directly prompt the communication information.

The normal person button and the deaf-mute button can be set on the terminal to facilitate the deaf-mute person to log in to the terminal, and the terminal can be a mobile phone or a computer.

The dummy picture is stored as communication information. Specifically, the chat content expressed by the dummy picture is converted into text information through a correspondence table and stored as communication information, wherein the correspondence table is stored in the background of the local communication device and/or the communication server.

Specifically, the technical solutions of the embodiments of the present invention may include:

(1) User registration, registration information includes: user name, password, user type and other information. Among them, the user types are divided into two types: the first user type is a user who can communicate through sign language, for example, a deaf-hearing person can only communicate through sign language and gestures; the second type is a user who can use text communication. For example, ordinary people use text information to communicate;

(2) User login: After the user registers, you can log in to the platform.

(3) After the user logs in, select the chat object and enable the chat function. For example, the client software implements input of chat content in different ways according to the type of the user. For example, an ordinary person selects text information, and the deaf-mute hearing person selects a sign language gesture picture according to a software setting manner, thereby implementing input chat content.

(4) The chat content input by the deaf-hearing hearing-impaired person is converted into text information through the app software through a corresponding relationship table, and the correspondence table between the text information and the sign language gesture is saved in the local, and can also be saved in the background of the server.

(5) The input information is sent to the background server through the network, and the server sends it to the recipient. The chat message sent by the sender is displayed on the recipient. The recipient decides whether to display the text message or the sign language gesture image according to the user type: the ordinary person displays the text information, and the deaf and dumb person is displayed as the sign language gesture picture.

The apparatus for implementing the method of the embodiment of the present invention is described below. For the part not described in detail in the present apparatus, reference may be made to the above method.

Referring to FIG. 2, a device 1 for voice input based on picture selection includes:

The dummy user login information unit 10 is configured to accept login information of the dummy user;

a classification selection unit 20, configured to accept the selection of the dummy category by the dummy user, and import all the dummy pictures under the selected category;

The dummy picture confirming unit 30 is configured to confirm the dummy picture selected by the user, and use the dummy picture as Communication information is stored;

The input completion determining unit 40 is configured to determine whether the user completes the input of the information, and if not, returns to the step dummy category selection unit 20; if yes, proceeds to the step sending unit 50;

The sending unit 50 is configured to send communication information.

Further, the dummy user has the following units before the login information unit 10, so as to facilitate the communication between the deaf and the ordinary person, namely:

The user registration unit 5 is configured to accept registration information of the user; the registration information includes: user name information, password information, and user type information; the user type information includes: the user who can communicate through the sign language is the first type, and can use the voice. The user of information exchange is of the second type. For example, a deaf-hearing person who can communicate through sign language is the first type, and a healthy person who can use voice information exchange is regarded as the second type;

The user type judging unit 6 is configured to determine which type of user is selected, if it is the first type, enter the mute user login information unit 10; if it is the second type, enter the fetch unit 7;

The capture unit 7 is configured to capture information input by the user for communication.

The dummy picture is stored as communication information. Specifically, the chat content expressed by the dummy picture is converted into text information by a correspondence table, and stored as communication information, wherein the correspondence table is stored in the background of the local communication device or the communication server.

Further, the transmitting unit 50 further has the following units after:

The information receiving unit 60 is configured to receive communication information;

The information display unit 70 is configured to determine the user type. If it is the first type, the text information is converted into a dummy picture for display, and if it is the second type, the communication information is directly prompted.

The information captured by the crawling unit 7 is directly obtained by the information receiving unit 60, that is, the communication mode used by the normal healthy person in the prior art.

Referring to FIG. 3, a mobile phone 100 based on picture selection and dummy input, the mobile phone 100 has a database 300 composed of a plurality of dummy picture information and a controller 400 connected to the database, and the controller 400 has the above-mentioned mute based on picture selection. Input device 1.

The mute picture information is a coding unit in sign language or gesture picture, and each coding unit corresponds to a text or a combination of words.

The method, device and system for voice input based on picture selection in the embodiment of the present invention have the following beneficial technical effects:

The above is only the preferred embodiment of the present invention, and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., which are included in the spirit and scope of the present invention, should be included in the present invention. Within the scope of protection.

Claims

A method for inputting a dummy word based on picture selection, comprising:

S1. Accept the login information of the dummy user;

S2. accepting the selection of the dummy word classification by the dummy user, and importing all the dummy pictures under the selected category;

S3. confirming the dummy picture selected by the user, and storing the dummy picture as the communication information;

S4. Determine whether the user completes the input of information, and if not, returns to step S2; if yes, proceeds to step S5;

S5. Send the communication information.
The method for inputting a dummy word based on picture selection according to claim 1, wherein the step S1 further comprises the following steps:

A1. Accepting registration information of the user; the registration information includes: user name information, password information, and user type information; the user type information includes: a user who can communicate by sign language is the first type, and can use voice information. The user of the communication is of the second type;

A2. Determine which type of user type is selected, if it is the first type, go to step S1; if it is the second type, go to step A3;

A3. Grab the information entered by the user for communication.
The method for inputting a dummy word based on picture selection according to any one of claims 1 to 2, wherein the dummy picture is stored as communication information, specifically: the chat content expressed by the dummy picture is passed A correspondence table is converted into text information and stored as communication information, wherein the correspondence table is stored in the background of the local communication device and/or the communication server.
The method for inputting a dummy word based on picture selection according to claim 2, wherein the step S5 further comprises the following steps:

B1. Accept the communication information described;

B2. Determine which type of user type, if it is the first type, convert the text information into a dummy picture for display, and if it is the second type, directly prompt communication information.
A device for inputting a dummy word based on picture selection, comprising:

a dummy user login information unit for accepting login information of the dummy user;

a classification selection unit, configured to accept the selection of the dummy category by the dummy user, and import all the dummy pictures under the selected category;

a dummy picture confirmation unit, configured to confirm a dummy picture selected by the user, and store the dummy picture as communication information;

The input completion determining unit is configured to determine whether the user completes the input of the information, and if not, returns to the classification selecting unit; if yes, enters the sending unit;

a sending unit, configured to send the communication information.
The apparatus for inputting a dummy word based on picture selection according to claim 5, wherein the dummy user login information unit further has the following unit:

a user registration unit, configured to accept registration information of the user; the registration information includes: user name information, password information, and user type information; and the user type information includes: a user who can communicate through sign language is the first type. The second type of users who can communicate using voice information;

a user type determining unit, configured to determine a type of the selected user type, and if the first type is the first type, enter the dummy user login information unit; if the second type is, enter the crawling unit ;

The crawling unit is used to capture information input by the user for communication.
The apparatus for inputting a dummy word based on picture selection according to any one of claims 5 to 6, wherein the dummy picture is stored as communication information, specifically: the chat content expressed by the dummy picture is passed A correspondence table is converted into text information and stored as communication information, wherein the correspondence table is stored in the background of the local communication device and/or the communication server.
The device for muting input based on picture selection according to claim 6, wherein the transmitting unit further has the following units:

An information accepting unit, configured to accept the communication information;

The information display unit is configured to determine the type of the user type, and if the first type is, convert the text information into a dummy picture for display, and if the second type is, directly prompt the communication information. .
A mobile phone for voice input based on picture selection, characterized in that the mobile phone has a database composed of a plurality of dummy picture information and a controller connected to the database, and the controller has claim 5 to A device for mute input based on picture selection according to any of the preceding claims.
The mobile phone for voice input based on picture selection according to claim 9, wherein The dummy picture information is a coding unit in a sign language or a gesture picture, and each coding unit corresponds to a text or a combination of characters.