WO2020119496A1 - 基于人工智能的通信方法、装置、设备及可读存储介质 - Google Patents

基于人工智能的通信方法、装置、设备及可读存储介质 Download PDF

Info

Publication number
WO2020119496A1
WO2020119496A1 PCT/CN2019/122329 CN2019122329W WO2020119496A1 WO 2020119496 A1 WO2020119496 A1 WO 2020119496A1 CN 2019122329 W CN2019122329 W CN 2019122329W WO 2020119496 A1 WO2020119496 A1 WO 2020119496A1
Authority
WO
WIPO (PCT)
Prior art keywords
word
sequence
preset
sentence
words
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2019/122329
Other languages
English (en)
French (fr)
Inventor
温桂龙
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
OneConnect Smart Technology Co Ltd
Original Assignee
OneConnect Smart Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by OneConnect Smart Technology Co Ltd filed Critical OneConnect Smart Technology Co Ltd
Publication of WO2020119496A1 publication Critical patent/WO2020119496A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/26—Speech to text systems

Definitions

  • the present application relates to the field of artificial intelligence, and in particular to a communication method, device, device and readable storage medium based on artificial intelligence.
  • deaf-mute people have certain deficiencies in speech, they usually communicate with the outside world in sign language, so the voice communication function of current social applications is not conducive to the communication and use between deaf-mute people and normal people.
  • the meaning of the sign language of deaf-mute people cannot be obtained without systematic learning, which brings inconvenience to both parties.
  • the main purpose of the present application is to provide a communication method, device and readable storage medium based on artificial intelligence, aiming to solve the technical problem that deaf-mute people and normal people cannot perform social communication through voice functions.
  • the present application provides an artificial intelligence-based communication method.
  • the artificial intelligence-based communication method includes:
  • the present application also provides an artificial intelligence-based communication method.
  • the artificial intelligence-based communication method includes:
  • the present application also provides an artificial intelligence-based communication device, the artificial intelligence-based communication device includes:
  • the voice recognition module is configured to perform voice recognition on the first voice information when receiving the first voice information, to obtain the corresponding first voice text;
  • Text correction module including sentence division unit, target word replacement unit, score calculation unit and text obtaining unit,
  • the sentence dividing unit is configured to divide the original sentence of the first speech text based on a preset Chinese word segmentation rule to obtain a corresponding sentence word sequence
  • the target word replacement unit is configured to obtain confusion words corresponding to the target words in the sentence word sequence, and replace the target words in the sentence word sequence with the confusion words to obtain a candidate sequence;
  • the score calculation unit is configured to separately calculate a grammatical score of the sentence word sequence and the candidate sequence based on a preset grammatical language model, and determine a correction sequence in the sentence word sequence and candidate sequence according to the grammar score;
  • the text obtaining unit is used for obtaining the corresponding corrected text according to the correction sequence
  • a video packaging module configured to query a corresponding sign language sample frame in a preset sign language library according to the corrected text, and package the sign language sample frame into a corresponding sign language video;
  • the video processing module is used to send the sign language video to the corresponding smart terminal or play the sign language video.
  • the present application also provides an artificial intelligence-based communication device, the artificial intelligence-based communication device includes:
  • Feature extraction module including preprocessing unit, image segmentation unit and feature extraction unit,
  • the pre-processing unit is used to pre-process each video frame to obtain an image to be extracted
  • the image segmentation unit is configured to perform instance area recognition on the image to be extracted, and perform image segmentation on the image to be extracted according to the recognition result to obtain a gesture area image;
  • the feature extraction unit is used to perform feature extraction on the gesture area image to obtain corresponding sign language features
  • the semantic determination module is used to query a predetermined sign language library for sign language sample frames corresponding to each video frame according to the sign language characteristics, and determine the video frame semantics of each video frame according to the sign language sample frame;
  • An information generation module configured to obtain the sign language video semantics of the sign language video according to the video frame semantics of each video frame, and generate corresponding second speech/text information according to the sign language video semantics;
  • the information processing module is used to send the second voice/text information to the corresponding smart terminal, or play/display the second voice/text information.
  • the present application also provides an artificial intelligence-based communication device
  • the artificial intelligence-based communication device includes a processor, a memory, and a memory that can be executed by the processor and stored on the memory Computer readable instructions, where the computer readable instructions, when executed by the processor, implement the steps of the artificial intelligence-based communication method as described above.
  • the present application also provides a readable storage medium on which computer-readable instructions are stored, wherein when the computer-readable instructions are executed by a processor, the Steps of artificial intelligence communication method.
  • FIG. 1 is a schematic diagram of a hardware structure of a communication device based on artificial intelligence involved in an embodiment of the present application
  • FIG. 2 is a schematic flowchart of a first embodiment of a communication method based on artificial intelligence of the present application.
  • the communication method based on artificial intelligence involved in the embodiments of the present application is mainly applied to a communication device based on artificial intelligence.
  • the communication device may be a personal computer (PC), a notebook computer, a mobile terminal (such as a tablet computer, a palmtop computer, a mobile phone) ) And other devices with data processing functions.
  • FIG. 1 is a schematic diagram of a hardware structure of an artificial intelligence-based communication device involved in an embodiment of the present application.
  • the communication device may include a processor 1001 (for example, a central processing unit, CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005.
  • a processor 1001 for example, a central processing unit, CPU
  • a communication bus 1002 for example, a central processing unit, CPU
  • user interface 1003 for example, a user interface 1003, a network interface 1004, and a memory 1005.
  • the communication bus 1002 is used to realize the connection and communication between these components;
  • the user interface 1003 may include a display (Display), an input unit such as a keyboard (Keyboard);
  • the network interface 1004 may optionally include a standard wired interface, a wireless interface (Such as wireless fidelity WIreless-FIdelity, WI-FI interface);
  • the memory 1005 can be a high-speed random access memory (random access memory, RAM), or a stable memory (non-volatile memory), such as disk memory, memory 1005 may optionally be a storage device independent of the foregoing processor 1001.
  • RAM random access memory
  • non-volatile memory such as disk memory
  • memory 1005 may optionally be a storage device independent of the foregoing processor 1001.
  • the memory 1005 as a computer-readable storage medium may include an operating system, a network communication module, and computer-readable instructions; the network communication module may be used to connect to a server (or base station) and perform data communication with the server (or base station); and the processor 1001 can call computer-readable instructions stored in the memory 1005, and execute the artificial intelligence-based communication method provided by the embodiments of the present application.
  • the embodiments of the present application provide a communication method based on artificial intelligence.
  • FIG. 2 is a schematic flowchart of a first embodiment of a communication method based on artificial intelligence of the present application.
  • the communication method based on artificial intelligence includes the following steps:
  • Step S10 when receiving the first voice information, perform voice recognition on the first voice information to obtain a first voice text
  • the communication method based on artificial intelligence in this embodiment is implemented by a communication device based on artificial intelligence.
  • the communication device may be a personal computer (PC), a notebook computer, a mobile terminal (such as a tablet computer, a palmtop computer, a mobile phone) )Wait.
  • the communication device is described by taking a mobile phone as an example; the mobile phone is pre-installed with a number of social applications (such as instant messaging applications), and the user of the mobile phone is a normal person, which can be called a normal user;
  • the application communicates with a deaf-mute user, it can start the social application in the mobile phone and record its first voice information by voice.
  • the first voice message is sent to a smart terminal (such as a mobile phone) used by the deaf and dumb user, due to the hearing impairment of the deaf and dumb user, it may appear that the truth of the first voice information cannot be obtained Meaning; in this regard, when a normal user's mobile phone receives the voice message, it can convert the voice message into a corresponding sign language video, and then send the sign language video to the deaf-mute user's smart terminal.
  • a smart terminal such as a mobile phone
  • the mobile phone when the mobile phone receives the first voice information, it first analyzes and recognizes the first voice information, extracts the voice features in the voice information, and performs feature search in a preset voice library according to the voice features And matching, the corresponding voice text is obtained, and the corresponding first text information is obtained according to the voice text.
  • the process of recognizing the first speech information and generating the first text information may be achieved through a pre-established speech recognition model, that is, first collect a certain number of speech samples and text samples corresponding to these speech samples, and establish an initial The speech recognition model, and then iteratively train the initial speech recognition model through these speech samples and text samples, and continuously adjust the model parameters until the optimal model parameters (or the number of iteration trainings that meet the preset number of times or the recognition accuracy rate are reached Certain threshold), so as to obtain a trained speech recognition model according to the optimal model parameters.
  • the conversion process of the voice information to text information may also be through an API interface (Application Programming Interface) provided by a third party, an SDK (Software Development Kit, software development kit), etc.
  • the mobile phone when it obtains the first voice information, it can call an API interface or SDK provided by a third party, and call the corresponding application or toolkit through the API interface or SDK to perform voice recognition on the first voice information to obtain the corresponding First speech text.
  • an API interface or SDK provided by a third party
  • Step S20 Perform text correction on the first speech text through a preset text correction model to obtain corrected text
  • the mobile phone of this embodiment when it obtains the first speech text, it will perform text correction processing on the first speech text through a preset text correction model, so as to obtain a text that conforms to certain grammatical rules (or general language habits) Correct the text, and then perform sign language conversion processing on the corrected text.
  • the text correction is performed on the first speech text through the preset text correction model, and the steps of obtaining the corrected text include:
  • the original sentence in the first phonetic text needs to be divided into words according to the preset Chinese word segmentation rule, and a continuous sentence is also divided into several individual words.
  • the order is sorted, that is, after division is a sequence of sentence words.
  • the words that need to be corrected in the sentence word sequence may be called target words.
  • target words Of course, in reality, there may be multiple target words.
  • the preset Chinese word segmentation rule can be set based on the string matching method, that is, a dictionary is stored in advance, and the dictionary includes a number of standard character strings, and then the word segmentation is performed through the standard string matching method; the preset Chinese
  • the word segmentation rules can also be set based on statistical word segmentation, that is, on the basis of collecting a large number of word segmentation samples, the rule of word segmentation (that is, training) is learned through a statistical machine learning model to obtain a word segmentation model, and then use the word segmentation model to perform Participle.
  • a third-party word segmentation tool for word segmentation such as SnowNLP, THULAC, jieba word segmentation, etc.
  • the confusion word may include some Chinese characters with similar pronunciation, such as Chinese homophones ("word” and “self”), nasal sounds ("Tibetan”) and non-nasal sounds (“Like”), curly tongue (" ⁇ ” and non-Chinese) Tongue “zao” etc.; can also include similar Chinese characters, such as the same root of Chinese characters (“solid” and “hui”) and similar Wubi input methods (" ⁇ ” and “ ⁇ ”); in addition, it can also include Other types (such as statistics-based) of Chinese characters (such as “wife” and “west”, “sister-in-law” and “sou”, etc.) It is worth noting that for a target word confusion, there may be more than two (this Where "above” includes the number, the same below), that is, the confusion set is obtained.
  • the target word in the sentence word sequence is replaced by the confusion word to obtain the candidate sequence; for example, the original sentence
  • the sequence of sentence words is (Li Mou/ ⁇ / ⁇ / ⁇ / ⁇ / ⁇ / ⁇ )
  • the target word is the first "yes” from the left
  • the corresponding confusion word is "thing”
  • the corresponding candidate sequence is ( (Li Mou/ ⁇ / ⁇ / ⁇ / ⁇ / ⁇ / ⁇ ).
  • the corresponding candidate sequence is also more than two.
  • the grammatical scores of the sentence word sequence and the candidate sequence can be calculated separately according to the preset grammatical language model; the grammatical score can be regarded as the occurrence probability of the sentence corresponding to the sequence in the entire Chinese language , The higher the grammar score, the greater the probability of its occurrence.
  • the preset grammatical language model based on the idea of the N-Gram Chinese language model, it can be binary (assuming that the probability of occurrence of a word depends only on the preceding word) or ternary (assuming the probability of a word depends on its The first two words) a grammatical language model, which combines Markov assumptions, maximum likelihood estimation, and conditional probability to set.
  • the model includes a preset corpus;
  • the preset corpus includes a number of sample sentences collected in advance, and also includes a table of the number of occurrences of the word counted according to these sample sentences, and a table of the number of adjacent times of multiple words;
  • the word occurrence count table records the number of occurrences of each word of each sample sentence in the preset corpus in the entire preset corpus, as shown in Table 1 below:
  • the multiple word adjacent frequency table records the number of occurrences of multiple words in the entire preset library. For example, for the binary word adjacent frequency table, it records the number of times that two words appear adjacent in a certain order. As shown in Table 2 below:
  • each value indicates that the words in the (row, column) order appear next to each other.
  • the value "786" in the third column of the second row it means “I” and " The two words "" in the form of "me/" appear in the preset anticipation database for 786 times.
  • the number of times that three words appear adjacent in a certain order is recorded, and so on.
  • the number of adjacent words in the corpus of each word in the sentence word sequence and the previous historical word in the corpus, and the number of adjacent words in the candidate corpus of each word in the candidate sequence in the preset corpus will also be counted; it is worthwhile Note that for different grammatical and semantic models, the number of adjacent words that need to be counted may be different; for example, for a sequence of sentence words (Li Mou/ ⁇ / ⁇ / ⁇ / ⁇ / ⁇ / ⁇ / ⁇ ), when the preset
  • the statistics are the adjacent times of binary words, that is, "Li//”, “ ⁇ / ⁇ ”, “ ⁇ / ⁇ ”, “ ⁇ / ⁇ ”, “Yes/one”, "one/question” sentence word adjacent times; and when the preset grammatical language model is a ternary grammatical language model, the statistics include the number of adjacent binary words and ternary adjacent words At that time, you need to count "Li//", "Li////”,
  • the grammatical score of each sequence can be calculated separately; when calculating, the number of occurrences of the word corresponding to the sentence word sequence and the adjacent number of sentence words are substituted
  • the preset score formula calculates the grammatical score of the sentence word sequence, and substitutes the number of occurrences of the word corresponding to the candidate sequence and the adjacent times of the candidate words into the preset score formula to calculate the grammatical score of the candidate sequence.
  • the preset score formula is:
  • P is the grammatical score of the sentence sequence or candidate sequence
  • k1 is the first preset scoring coefficient, k1>0;
  • n is the number of words in the sentence word sequence or candidate sequence, m ⁇ 2;
  • C 0 is the total number of words in the preset corpus, C 0 >0;
  • C(w 1 ) is the number of occurrences of the first word in the sentence word sequence or candidate sequence in the preset corpus
  • C(w i-1 ) is the number of occurrences of the i-1th word in the sentence word sequence or candidate sequence in the preset corpus, 2 ⁇ i ⁇ m;
  • C(w i-1 ,w i ) is the number of times the i-th word in the sentence word sequence and the previous historical word are adjacent to the sentence word in the preset corpus, or the i-th word in the candidate sequence and the previous one The number of times that historical words are adjacent to candidate words in a preset corpus.
  • the preset score formula is:
  • P is the grammatical score of the sentence sequence or candidate sequence
  • k2 is the second preset scoring coefficient, k2>0;
  • n is the number of words in the sentence word sequence or candidate sequence, m ⁇ 3;
  • C 0 is the total number of words in the preset corpus, C 0 >0;
  • C(w 1 ) is the number of occurrences of the first word in the sentence word sequence or candidate sequence in the preset corpus
  • C(w 1 ,w 2 ) is the number of times the first word and the second word in the sentence word sequence are adjacent to the sentence word in the preset corpus, or the first word and the second word in the candidate sequence are in the pre Set the number of adjacent words in the corpus;
  • C(w i-2 ,w i-1 ) is the number of times the previous historical word and the first two historical words of the i-th word in the sentence word sequence are adjacent to the sentence word in the preset corpus, or in the candidate sequence The number of times the previous historical word of the i-th word is adjacent to the candidate words of the first two historical words in the preset corpus;
  • C(w i-2 ,w i-1 ,w i ) is the number of times that the i-th word in the sentence word sequence is adjacent to the sentence word in the preset corpus of the previous historical word and the first two historical words, or is The number of times the i-th word in the candidate sequence is adjacent to the candidate word in the preset corpus of the previous historical word and the first two historical words.
  • the grammatical score can be regarded as the occurrence probability of the sentence corresponding to the sequence in the entire Chinese language.
  • the sequence corresponding to the sequence with the highest score is regarded as the correction sequence.
  • the sentence word sequence may be the correction sequence, or the candidate sequence may be the correction sequence.
  • the correction sequence is determined, the corresponding correction text can be obtained according to the correction sequence.
  • Step S30 query the corresponding sign language sample frame in the preset sign language library according to the corrected text, and encapsulate the sign language sample frame into the corresponding sign language video;
  • the sign language video can be converted according to the corrected text.
  • the mobile phone may perform semantic recognition on the corrected text to determine the text semantics of the corrected text; of course, the text semantics may be recorded in multiple paragraphs.
  • the server will search for the corresponding sign language sample frames in the preset sign language library according to the text semantics, and then stitch and encapsulate the various sign language sample frames to obtain the corresponding sign language video;
  • a preset sign language library pre-stores a number of semantically corresponding standard sign language sample videos. These sign language sample videos may be pre-recorded and stored.
  • the preset sign language library it can be a server or database stored in the social application, and the mobile phone can query the server according to the correction text to save the storage space of the phone; of course, the preset sign language library can also be stored locally on the phone.
  • the preset sign language library can also be stored locally on the phone.
  • Step S40 Send the sign language video to the corresponding smart terminal, or play the sign language video.
  • the sign language video when the mobile phone obtains the corresponding sign language video according to the corrected text, the sign language video can be sent to the smart terminal (such as a mobile phone) used by the deaf user, so that the deaf user can see the sign language video through the smart terminal And learn the meanings expressed by normal users.
  • the sign language video may be directly played for normal users to view, and the sign language video may be sent, stored, or deleted according to the operation of the normal user.
  • the conversion of voice information into sign language video is described by taking “normal users convert before sending voice messages to deaf and dumb users” as an example; in practice, normal users can also directly send
  • the smart terminal of the deaf user sends voice information.
  • the voice information can be converted into a sign language video according to the operation of the deaf user.
  • the process is similar to the steps of the above embodiments. It will not be repeated here.
  • the smart terminal of the deaf-mute user can also be regarded as the communication device based on artificial intelligence in this embodiment.
  • the voice information is first converted into text information; and if the normal user directly enters the text information by text entry, the mobile phone can also directly The text is corrected, and sign language video conversion is further performed on the corrected text.
  • the process is similar to the above, and will not be repeated here.
  • the function of converting voice information into sign language video in this embodiment may be a function in a social application, or may be integrated in a terminal such as a mobile phone in the form of an API interface or SDK to be used in the terminal.
  • the application makes the call.
  • the first voice information when the first voice information is received, perform voice recognition on the first voice information to obtain the corresponding first voice text; perform text correction on the first voice text through a preset text correction model, Obtain the corrected text; query the corresponding sign language sample frame in the preset sign language library according to the corrected text, and encapsulate the sign language sample frame into the corresponding sign language video; send the sign language video to the corresponding smart terminal, or play The sign language video.
  • this embodiment implements the conversion of voice to sign language through artificial intelligence, so that deaf users can also learn the information sent by normal users in social applications by voice, which provides convenience for deaf users and improves deaf users Experience;
  • voice text corresponding to the voice information will be corrected to obtain the corrected text that meets certain grammatical rules (or general language habits), and then the corrected text will be converted into sign language
  • grammatical rules or general language habits
  • the communication method based on artificial intelligence includes the following steps:
  • Step S50 When receiving the sign language video, perform feature extraction on each video frame of the sign language video based on a preset feature extraction algorithm to obtain the sign language characteristics of each video frame;
  • this embodiment proposes a communication method based on artificial intelligence, which implements the conversion of sign language video to voice/text through artificial intelligence, so that normal people can also use this method to communicate with deaf and dumb people through social applications. Provided convenience for both parties.
  • the communication method based on artificial intelligence in this embodiment is implemented by a communication device based on artificial intelligence.
  • the communication device may be a personal computer (PC), a notebook computer, a mobile terminal (such as a tablet computer, a palmtop computer, a mobile phone) )Wait.
  • the communication device is described by taking a mobile phone as an example; the mobile phone is pre-installed with several social applications (such as instant messaging applications), and the user of the mobile phone is a deaf-mute person, which can be called a deaf-mute user; when deaf-mute
  • users need to communicate with a normal user through the social application, they can start the social application in the mobile phone and record their own sign language video through the mobile phone.
  • the mobile phone When the mobile phone receives the sign language video, it first extracts the features of each video frame of the sign language video based on a preset feature extraction algorithm to obtain the sign language features of each video frame, such as the shape of the gesture, the pointing of each finger, and the movement of the hand movement Trajectory, etc.
  • the preset feature extraction algorithm used for the feature extraction process can be obtained by machine learning; that is, a certain number of sign language image samples are collected first, and an initial feature extraction model is established, and then the initial features Extract the model for iterative training, and continuously adjust the model parameters until the optimal model parameters are obtained (or the number of iterative trainings that meets the preset number of times or the recognition accuracy rate reaches a certain threshold), so that the trained model is obtained according to the optimal model parameters
  • the model logic in the feature extraction model is the preset feature extraction algorithm.
  • the preset feature extraction algorithm can also be set according to static rules.
  • the meaning is mainly expressed by gestures and hand movement trajectories, etc. Therefore, when feature extraction is performed, it is also mainly performed on the hand part image in the sign language video frame.
  • a certain segmentation process may be performed on each video frame first, and then feature extraction is performed on the segmented image.
  • the step of extracting the features of each video frame of the sign language video based on a preset feature extraction algorithm to obtain the sign language features of each video frame includes:
  • the mobile phone When the mobile phone obtains the sign language video, it can first preprocess each video frame of the sign language video to obtain the image to be extracted, thereby providing convenience for subsequent image segmentation processing, such as image binarization and noise reduction.
  • binarization refers to the conversion of video frames into binary images containing only two black and white colors. This is because the amount of information contained in color images is huge, and the amount of calculation will increase when subsequent processing is performed. The time will also be long, so in order to improve the efficiency of processing and recognition, the color image can be binarized first; denoising is to solve the digital image due to noise interference caused by the image quality degradation through filters or other methods problem.
  • the content of preprocessing can also be added or replaced with other processing operations according to the actual situation.
  • an instance area recognition is performed on the image to be extracted, and the gesture area is identified; for the instance area recognition process, the pixel value of each pixel in the image of the object to be measured can be performed ,
  • the pixel mutation point is determined by the pixel value of each pixel point and the pixel value of a similar point, and then the contour of each area is obtained, and then the object contour is identified on the area contour, and the meaning of each contour is determined to identify the contour of the gesture area ; Then, the image can be segmented according to the contour to obtain the gesture area image.
  • gesture area image When the gesture area image is obtained, feature extraction operation can be performed on the gesture area image to obtain the corresponding sign language features, thereby eliminating the need for related detection and feature extraction of the non-gesture area image in the video frame, reducing the amount of calculation and improving Feature extraction efficiency.
  • Step S60 query a preset sign language library for a sign language sample frame corresponding to each video frame according to the sign language characteristics, and determine the video frame semantics of each video frame according to the sign language sample frame;
  • the mobile phone when extracting the sign language feature, can query the sign language sample frame corresponding to each video frame in the preset sign language library according to the sign language feature; the preset sign language library includes standard sign language sample frames, The sample semantics of the sign language sample frame and the sample features of the sign language sample frame; and the sample features of the sign language sample frame are also extracted from the sign language sample frame by the preset feature extraction algorithm in step S50.
  • the sign feature of the video frame is compared with the sample feature of each sign language sample frame.
  • the sign language sample frame to which the sample feature belongs corresponds to the video frame; and for the similarity, it can be characterized by distance (such as Euclidean distance, etc.). It is worth noting that if the similarity between more than two sample features and a sign language feature reaches a certain threshold, the sign language sample frame to which the sample feature with the highest similarity belongs can be used as the sign language sample frame corresponding to the video frame .
  • the mobile phone When querying the sign language sample frame corresponding to the video frame, acquires the sample meaning of the sign language sample frame, and determines the video frame voice of the video frame according to the sample meaning.
  • Step S70 Obtain the sign language video semantics of the sign language video according to the video frame semantics of each video frame, and generate corresponding second voice/text information according to the sign language video semantics;
  • the semantics of each video frame can be spliced to obtain the sign language video semantics of the entire sign language video, and the corresponding second voice/ Text information.
  • whether to generate voice information or text information can be determined according to the selection of the deaf user, or pre-set in the mobile phone system, or both can be generated).
  • Step S80 Send the second voice/text information to the corresponding smart terminal, or play/display the second voice/text information.
  • the second voice/text information when the mobile phone obtains the corresponding second voice/text information, the second voice/text information can be sent to a smart terminal (such as a mobile phone) used by a normal user, so that the normal user can see through the smart terminal
  • the second voice/text information and learn the meaning expressed by the deaf-mute user.
  • the mobile phone when it obtains the second voice/text information, it may also play the second voice information/display the second text information, and perform transmission, storage, or the second voice/text information according to the operation of the deaf-mute user. Delete operation.
  • the conversion of sign language video into voice/text information in this embodiment is described by taking “deaf and dumb users to convert before sign language video to normal users” as an example; in practice, deaf and dumb users can also pass
  • the mobile phone directly sends the sign language video to the normal user's smart terminal.
  • the normal user's smart terminal receives the sign language video
  • the sign language video can be converted into voice information/text information according to the normal user's operation.
  • the smart terminal of the normal user may also be regarded as the communication device based on artificial intelligence in this embodiment.
  • the sign language video to voice/text information conversion function in this embodiment may be a function in a social application, or may be integrated in a terminal such as a mobile phone in the form of an API interface or SDK to be used in the terminal.
  • the application makes the call.
  • a sign language video when a sign language video is received, feature extraction is performed on each video frame of the sign language video based on a preset feature extraction algorithm to obtain the sign language characteristics of each video frame;
  • the library queries the sign language sample frames corresponding to the video frames, and determines the video frame semantics of the video frames according to the sign language sample frames; obtaining the sign language video of the sign language video according to the video frames semantics of the video frames Semantic, and generate corresponding second voice/text information according to the semantics of the sign language video; send the second voice/text information to the corresponding smart terminal, or play/display the second voice/text information.
  • this embodiment implements the conversion of sign language video to speech/text by means of artificial intelligence, so that normal users can easily learn the meaning of deaf users’ sign language, and deaf users can also communicate with normal users in the way of sign language expression
  • Social communication provides convenience for both parties and improves the user experience of both parties.
  • the embodiments of the present application also provide an artificial intelligence-based communication device.
  • the artificial intelligence-based communication device includes:
  • the voice recognition module is configured to perform voice recognition on the first voice information when receiving the first voice information, to obtain the corresponding first voice text;
  • a text correction module configured to perform text correction on the first speech text through a preset text correction model to obtain corrected text
  • a video packaging module configured to query a corresponding sign language sample frame in a preset sign language library according to the corrected text, and package the sign language sample frame into a corresponding sign language video;
  • the video processing module is used to send the sign language video to the corresponding smart terminal or play the sign language video.
  • each virtual function module of the above artificial intelligence-based communication device is stored in the memory 1005 of the artificial intelligence-based communication device shown in FIG. 1 and is used to implement all functions of computer-readable instructions; when each module is executed by the processor 1001 , Can realize the function of conversion and transmission between voice and sign language.
  • the text correction module includes:
  • the sentence dividing unit is used to divide the original sentence of the first speech text based on a preset Chinese word segmentation rule to obtain a corresponding sentence word sequence;
  • the target word replacement unit is used to obtain confusion words corresponding to the target words in the sentence word sequence, and replace the target words in the sentence word sequence with the confusion words to obtain a candidate sequence;
  • a score calculation unit configured to calculate a grammar score of the sentence word sequence and the candidate sequence based on a preset grammatical language model, respectively, and determine a correction sequence in the sentence word sequence and candidate sequence according to the grammar score;
  • the text obtaining unit is used for the text to obtain the corresponding corrected text according to the correction sequence.
  • the score calculation unit includes:
  • the first statistical subunit is used to count the number of occurrences of the words in the sentence word sequence and the candidate sequence in the preset corpus;
  • the second statistical subunit is used to count the number of times that each word in the sentence word sequence and the previous historical word are adjacent to the sentence word in the preset corpus, and to count each word in the candidate sequence and the previous historical word in the Describe the number of adjacent words in the preset corpus;
  • the score calculation subunit is used to substitute the occurrence times of the word and the adjacent times of the sentence word into a preset score formula to calculate the grammatical score of the sentence word sequence, and compare the occurrence times of the word with the candidate word phase The number of neighbors is substituted into a preset score formula to calculate the grammatical score of the candidate sequence.
  • the preset grammar language model is a binary grammar language model
  • the preset score formula is:
  • P is the grammatical score of the sentence word sequence or the candidate sequence
  • k1 is the first preset scoring coefficient, k1>0;
  • n is the number of words in the sentence word sequence or the candidate sequence, m ⁇ 2;
  • C 0 is the total number of words in the preset corpus, C 0 >0;
  • C(w 1 ) is the number of occurrences of the first word in the sentence word sequence or the candidate sequence in the preset corpus
  • C(wi -1 ) is the number of occurrences of the i-1th word in the sentence word sequence or the candidate sequence in the preset corpus, 2 ⁇ i ⁇ m;
  • C(w i-1 ,w i ) is the number of times the i-th word in the sentence word sequence is adjacent to the sentence word of the previous historical word in the preset corpus, or the i-th word in the candidate sequence The number of times that a word is adjacent to the candidate word of the previous historical word in the preset corpus.
  • the preset grammar language model is a ternary grammar language model
  • the preset score formula is:
  • P is the grammatical score of the sentence word sequence or the candidate sequence
  • k2 is the second preset scoring coefficient, k2>0;
  • n is the number of words in the sentence word sequence or the candidate sequence, m ⁇ 3;
  • C 0 is the total number of words in the preset corpus, C 0 >0;
  • C(w 1 ) is the number of occurrences of the first word in the sentence word sequence or the candidate sequence in the preset corpus
  • C(w 1 ,w 2 ) is the number of times the first word and the second word in the sentence word sequence are adjacent to the sentence word in the preset corpus, or the first word in the candidate sequence and The number of times the second word is adjacent to the candidate word in the preset corpus;
  • C(w i-2 , w i-1 ) is the number of times the previous historical word and the first two historical words of the i-th word in the sentence word sequence are adjacent to the sentence word in the preset corpus, or The number of times the previous historical word of the i-th word in the candidate sequence is adjacent to the candidate words of the first two historical words in the preset corpus;
  • C(w i-2 , w i-1 , w i ) is the i- th word in the sentence word sequence adjacent to the sentence words in the preset corpus of the previous historical word and the first two historical words
  • the number of times, or the number of times the i-th word in the candidate sequence is adjacent to the candidate word in the preset corpus with the previous historical word and the first two historical words.
  • each module in the above-mentioned artificial intelligence-based communication device corresponds to the steps in the first embodiment of the above-mentioned artificial intelligence-based communication method, and the functions and implementation processes thereof will not be repeated here one by one.
  • the embodiments of the present application also provide an artificial intelligence-based communication device.
  • the artificial intelligence-based communication device includes:
  • the feature extraction module when receiving the sign language video, performs feature extraction on each video frame of the sign language video based on a preset feature extraction algorithm to obtain the sign language feature of each video frame;
  • the semantic determination module is used to query a predetermined sign language library for sign language sample frames corresponding to each video frame according to the sign language characteristics, and determine the video frame semantics of each video frame according to the sign language sample frame;
  • An information generation module configured to obtain the sign language video semantics of the sign language video according to the video frame semantics of each video frame, and generate corresponding second speech/text information according to the sign language video semantics;
  • the information processing module is used to send the second voice/text information to the corresponding smart terminal, or play/display the second voice/text information.
  • each virtual function module of the above-mentioned artificial intelligence-based communication device can also be stored in the memory 1005 of the artificial intelligence-based communication device shown in FIG. 1 for implementing all functions of computer-readable instructions; each module is used by the processor 1001 During execution, the function of conversion and transmission between sign language and voice/text can be realized.
  • the feature extraction module includes:
  • a pre-processing unit which is used to pre-process each video frame to obtain an image to be extracted
  • An image segmentation unit configured to perform instance area recognition on the image to be extracted, and perform image segmentation on the image to be extracted according to the recognition result to obtain a gesture area image
  • the feature extraction unit is used to perform feature extraction on the gesture area image to obtain corresponding sign language features.
  • each module in the above-mentioned artificial intelligence-based communication device corresponds to the steps in the second embodiment of the above-mentioned artificial intelligence-based communication method, and its function and implementation process will not be repeated here one by one.
  • an embodiment of the present application further provides a readable storage medium, and the computer-readable storage medium may be a non-volatile readable storage medium.
  • Computer readable instructions are stored on the readable storage medium of the present application, wherein when the computer readable instructions are executed by a processor, the steps of the first embodiment or the second embodiment of the above-mentioned artificial intelligence-based communication method are implemented.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computational Linguistics (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Multimedia (AREA)
  • Machine Translation (AREA)

Abstract

一种基于人工智能的通信方法、设备及可读存储介质，通过人工智能的方式实现语音与手语之间的转换，使得聋哑用户也可获知正常用户以语音方式在社交应用发送的信息，为聋哑用户提供了方便，提高聋哑用户的使用体验，而对正常用户而言，也可方便地获知聋哑用户的手语含义，为双方提供了方便；其次，在语音向手语的转化过程中，还将对语音信息对应的语音文本进行校正处理，以得到符合一定语法规则或通用语言习惯的校正文本，再对该校正文本进行手语转换处理，从而避免语句不通顺、错别字等情况对手语视频的转换准确度造成不利的影响，有利于聋哑用户获知语音信息的正确含义。

Description

基于人工智能的通信方法、装置、设备及可读存储介质
本申请要求于2018年12月14日提交中国专利局、申请号为201811539391.9、发明名称为“基于人工智能的通信方法、设备及可读存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中.
技术领域
本申请涉及人工智能领域，尤其涉及一种基于人工智能的通信方法、装置、设备及可读存储介质。
背景技术
随着终端技术的不断发展，社交应用在人们日常生活中的使用越来越频繁，人们可以在智能终端的社交应用中通过发送语音信息的方式来表达自己想要描述的内容，极大的便利了人们之间的沟通。
但是，由于聋哑人语音方面具有一定的缺陷，其通常是以手语与外界进行沟通，因此目前社交应用的语音沟通功能并不利于聋哑人与正常人之间的交流和使用，正常人在不经过系统学习的情况下也无法获取聋哑人的手语所要表达的含义，这就为双方带来了不便。
发明内容
本申请的主要目的在于提供一种基于人工智能的通信方法、设备及可读存储介质，旨在解决聋哑人和正常人无法通过语音功能进行社交通信的技术问题。
为实现上述目的，本申请提供一种基于人工智能的通信方法，所述基于人工智能的通信方法包括：
在接收到第一语音信息时，对所述第一语音信息进行语音识别，获得对应的第一语音文本；
基于预设中文分词规则对所述第一语音文本的原始语句进行词划分，得到对应的语句词序列；
获取所述语句词序列中的目标词对应的混淆词，并通过所述混淆词替换所述语句词序列中的目标词，得到候选序列；
基于预设文法语言模型分别计算所述语句词序列和所述候选序列的文法得分，并根据 所述文法得分在所述语句词序列和候选序列中确定校正序列；
根据所述校正序列获得对应的校正文本；
根据所述校正文本在预设手语库中查询对应的手语样本帧，并将所述手语样本帧封装为对应的手语视频；
将所述手语视频发送至对应的智能终端，或播放所述手语视频。
此外，为实现上述目的，本申请还提供一种基于人工智能的通信方法，所述基于人工智能的通信方法包括：
在接收到手语视频时，对所述各视频帧进行预处理，得到待提取图像；
对所述待提取图像进行实例区域识别，并根据识别结果对所述待提取图像进行图像分割，得到手势区域图像；
对所述手势区域图像进行特征提取，获得对应的手语特征；
根据所述手语特征在预设手语库查询与所述各视频帧对应的手语样本帧，并根据所述手语样本帧确定所述各视频帧的视频帧语义；
根据所述各视频帧的视频帧语义获取所述手语视频的手语视频语义，并根据所述手语视频语义生成对应的第二语音/文本信息；
将所述第二语音/文本信息发送至对应的智能终端，或播放/显示所述第二语音/文本信息。
此外，为实现上述目的，本申请还提供一种基于人工智能的通信装置，所述基于人工智能的通信装置包括：
语音识别模块，用于在接收到第一语音信息时，对所述第一语音信息进行语音识别，获得对应的第一语音文本；
文本校正模块，包括语句划分单元、目标词替换单元、得分计算单元和文本获得单元，
其中，所述语句划分单元，用于基于预设中文分词规则对所述第一语音文本的原始语句进行词划分，得到对应的语句词序列；
所述目标词替换单元，用于获取所述语句词序列中的目标词对应的混淆词，并通过所述混淆词替换所述语句词序列中的目标词，得到候选序列；
所述得分计算单元，用于基于预设文法语言模型分别计算所述语句词序列和所述候选序列的文法得分，并根据所述文法得分在所述语句词序列和候选序列中确定校正序列；
所述文本获得单元，用于文本根据所述校正序列获得对应的校正文本；
视频封装模块，用于根据所述校正文本在预设手语库中查询对应的手语样本帧，并将所述手语样本帧封装为对应的手语视频；
视频处理模块，用于将所述手语视频发送至对应的智能终端，或播放所述手语视频。
此外，为实现上述目的，本申请还提供一种基于人工智能的通信装置，所述基于人工智能的通信装置包括：
特征提取模块，包括预处理单元、图像分割单元和特征提取单元，
其中，所述预处理单元，用于对所述各视频帧进行预处理，得到待提取图像；
所述图像分割单元，用于对所述待提取图像进行实例区域识别，并根据识别结果对所述待提取图像进行图像分割，得到手势区域图像；
所述特征提取单元，用于对所述手势区域图像进行特征提取，获得对应的手语特征；
语义确定模块，用于根据所述手语特征在预设手语库查询与所述各视频帧对应的手语样本帧，并根据所述手语样本帧确定所述各视频帧的视频帧语义；
信息生成模块，用于根据所述各视频帧的视频帧语义获取所述手语视频的手语视频语义，并根据所述手语视频语义生成对应的第二语音/文本信息；
信息处理模块，用于将所述第二语音/文本信息发送至对应的智能终端，或播放/显示所述第二语音/文本信息。
此外，为实现上述目的，本申请还提供一种基于人工智能的通信设备，所述基于人工智能的通信设备包括处理器、存储器、以及存储在所述存储器上并可被所述处理器执行的计算机可读指令，其中所述计算机可读指令被所述处理器执行时，实现如上述的基于人工智能的通信方法的步骤。
此外，为实现上述目的，本申请还提供一种可读存储介质，所述可读存储介质上存储有计算机可读指令，其中所述计算机可读指令被处理器执行时，实现如上述的基于人工智能的通信方法的步骤。
本申请的一个或多个实施例的细节在下面的附图和描述中提出。本申请的其他特征和优点将从说明书、附图以及权利要求书变得明显。
附图说明
图1为本申请实施例方案中涉及的基于基于人工智能的通信设备的硬件结构示意图；
图2为本申请基于人工智能的通信方法第一实施例的流程示意图。
本申请目的的实现、功能特点及优点将结合实施例，参照附图做进一步说明。
具体实施方式
应当理解，此处所描述的具体实施例仅仅用以解释本申请，并不用于限定本申请。
本申请实施例涉及的基于人工智能的通信方法主要应用于基于人工智能的通信设备，该通信设备可以是个人计算机(personal computer，PC)、笔记本电脑、移动终端(如平板电脑、掌上电脑、手机)等具有数据处理功能的设备。
参照图1，图1为本申请实施例方案中涉及的基于人工智能的通信设备的硬件结构示意图。本申请实施例中，该通信设备可以包括处理器1001(例如中央处理器Central Processing Unit，CPU)，通信总线1002，用户接口1003，网络接口1004，存储器1005。其中，通信总线1002用于实现这些组件之间的连接通信；用户接口1003可以包括显示屏(Display)、输入单元比如键盘(Keyboard)；网络接口1004可选的可以包括标准的有线接口、无线接口(如无线保真WIreless-FIdelity，WI-FI接口)；存储器1005可以是高速随机存取存储器(random access memory，RAM)，也可以是稳定的存储器(non-volatile memory)，例如磁盘存储器，存储器1005可选的还可以是独立于前述处理器1001的存储装置。本领域技术人员可以理解，图1中示出的硬件结构并不构成对本申请的限定，可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件布置。作为一种计算机可读存储介质的存储器1005可以包括操作系统、网络通信模块以及计算机可读指令；网络通信模块可用于连接服务器(或基站)，与服务器(或基站)进行数据通信；而处理器1001可以调用存储器1005中存储的计算机可读指令，并执行本申请实施例提供的基于人工智能的通信方法。
本申请实施例提供了一种基于人工智能的通信方法。
参照图2，图2为本申请基于人工智能的通信方法第一实施例的流程示意图。
本实施例中，所述基于人工智能的通信方法包括以下步骤：
步骤S10，在接收到第一语音信息时，对所述第一语音信息进行语音识别，获得第一语音文本；
本实施例中的基于人工智能的通信方法是由基于人工智能的通信设备实现的，该通信设备可以是个人计算机(personal computer，PC)、笔记本电脑、移动终端(如平板电脑、掌上电脑、手机)等。本实施例中该通信设备以手机为例进行说明；该手机中预先安装有若干社交应用(如即时通讯应用)，该手机的用户为一正常人，可称为正常用户；当需要通过该社交应用与某一聋哑用户进行交流时，可以启动该手机中的社交应用，并通过语音的方式进行录入自己的第一语音信息。而对与聋哑用户而言，若将该第一语音信息发送至聋哑用户所用的智能终端(如手机)，由于聋哑用户的听力缺陷，可能会出现无法获知该第一语音信息的真实含义；对此，正常用户的手机在接收到该语音信息时，可对该语音信息进行转换，将其转换为对应的手语视频，再将该手语视频发送至聋哑用户的智能终端。 具体的，手机在接收到该第一语音信息时，首先会对该第一语音信息进行分析和识别，提取该语音信息中的语音特征，并根据该语音特征在预设语音库中进行特征查找和匹配，获取到对应的语音文本，并根据该语音文本获取对应的第一文本信息。其中，对第一语音信息的识别和第一文本信息的生成过程，可以是通过预先建立的语音识别模型实现，也即先收集一定数量的语音样本和这些语音样本对应的文本样本，并建立初始语音识别模型，然后通过这些语音样本和文本样本对初始语音识别模型进行迭代训练，不断调整模型参数，直至得到最优模型参数(或是满足预设次数的迭代训练次数、或是识别正确率达到一定阈值)，从而根据该最优模型参数得到训练好的语音识别模型。当然，在实际中，对于该语音信息向文本信息的转换过程，也可以是通过第三方提供的API接口(Application Programming Interface，应用程序编程接口)、SDK(Software Development Kit，软件开发工具包)等实现的，例如手机在获取到第一语音信息时，可以调用第三方提供的API接口或SDK，并通过该API接口或SDK调用对应应用程序或工具包对第一语音信息进行语音识别，获得对应的第一语音文本。
步骤S20，通过预设文本校正模型对所述第一语音文本进行文本校正，得到校正文本；
本实施例中，考虑到部分用户由于口语习惯的原因，可能会出现第一语音信息的语句构成与标准语句不完全匹配的情况(句子不通顺)，而在语音信息转化为文本信息的过程中，也可能会出现误匹配、误转化，进而导致出现错别字的情况；这些情况会对手语视频的转换准确度造成不利的影响，甚至可能会出现无法识别和转换的情况。对此，本实施例的手机得到在对第一语音文本时，将通过一预设的文本校正模型对该第一语音文本进行文本校正处理，以得到符合一定语法规则(或通用语言习惯)的校正文本，再对该校正文本进行手语转换处理。
具体的，通过预设文本校正模型对第一语音文本进行文本校正，得到校正文本的的步骤包括：
基于预设中文分词规则对第一语音文本的原始语句进行词划分，得到对应的语句词序列；
在文本校正过程中，首先需要根据预设中文分词规则对第一语音文本中的原始语句进行词划分，也将一个连续的句子划分成若干个单独的词，这些若干个单独词由于是按照一定的顺序排序，也即划分后得到的是一个语句词序列。在该语句词序列中需要纠正处理的词可称为目标词，当然在实际中该目标词可能具有多个。而对于该预设中文分词规则，可以基于字符串匹配的方式设置，也即预先存储一词典，该词典中包括有若干标准字符串，然后通过标准字符串匹配的方式进行分词；该预设中文分词规则还可以是基于统计分词的 方式设置，也即在收集大量分词样本的基础上，通过统计类机器学习模型学习词语切分的规律(即训练)，得到分词模型，再用该分词模型进行分词。当然，还可以是接入第三方的分词工具进行分词(如SnowNLP、THULAC、jieba分词等)。
获取所述语句词序列中的目标词对应的混淆词，并通过所述混淆词替换所述语句词序列中的目标词，得到候选序列；
在得到语句词序列时，将确定该语句词序列中的目标词，并获取目标词的混淆词。其中，该混淆词可包括一些发音相似的汉字，如汉语同音字(“字”和“自”)、鼻音(“藏”)和非鼻音(“赞”)、卷舌(“找”和非卷舌“早”等；还可包括形似的汉字，例如汉字的同一字根(“固”和“回”)和类似的五笔输入法(“丏”和“丐”)；此外，还可包括其它类型(如基于统计)的汉字(如“妻”和“西”、“嫂”和“搜”等)。值得说明的是，对于某个目标词的混淆词，可能具有两个以上(此处“以上”包括本数，下同)，也即得到可以混淆集。在得到该目标词的混淆词时，会通过混淆词替换语句词序列中的目标词，得到候选序列；例如，原始语句的语句词序列为(李某/的/是/的确/是/一个/问题)，目标词为左起的第一个“是”，对应的混淆词为“事”，则对应的候选序列为(李某/的/事/的确/是/一个/问题)。当然，若混淆词有两个以上，则对应的候选序列也为两个以上。
基于预设文法语言模型分别计算所述语句词序列和所述候选序列的文法得分，并根据所述文法得分在所述语句词序列和候选序列中确定校正序列；
在得到语句词序列和候选序列时，即可根据预设文法语言模型分别计算语句词序列和候选序列的文法得分；该文法得分可认为是该序列所对应的语句在整个中文语言中的出现概率，文法得分越高，其出现概率越大。对于该预设文法语言模型，基于N-Gram汉语语言模型的思想，可以为二元(假设一个词的出现概率只取决于其前面的一个单词)或三元(假设一个词的概率取决于其前面的两个词)文法语言模型，该模型结合了马尔科夫假设、最大似然估计和条件概率进行设置。
具体的，该模型中包括有预设语料库；该预设预料库中包括有预先收集的若干样本语句，还包括有根据这些样本语句所统计的单词出现次数表、以及多元词相邻次数表；其中单词出现次数表记录有预设语料库中各样本语句的每个词在整个预设预料库中的出现次数，如下表1所示：
表1 单词出现次数表
词 单词出现次数
我 1215
的 3437
事 938
... ...
总计 13738
而多元词相邻次数表，则记录有多个词在整个预设预料库中的出现次数，例如对于二元词相邻次数表，则记录有两个词以一定顺序相邻出现的次数，如下表2所示：
表2 二元词出现的次数
  我 的 事 ...
我 8 786 3 ...
的 20 0 988 ...
事 121 56 228 ...
... ... ... ... ...
对于上述表2，各数值所表示的是(行，列)顺序的单词相邻出现的此时，例如对于上述第二行第三列的数值“786”，所表示的是“我”和“的”两个词以“我/的”形式在预设预料库中出现次数为786次。对于三元词相邻次数表，则记录有三个词以一定顺序相邻出现的次数，以此类推。
在计算语句词序列和候选序列的文法得分时，首先分别需要统计语句词序列和候选序列中各单词在预设语料库中的单词出现次数；对于该单词出现次数的统计，可根据各序列所包括的单词在预设预料库中的单词出现次数表查询得到。其次，还将统计语句词序列中各单词与前位历史词在语料库中的语句词相邻次数，以及候选序列中各单词与前位历史词在预设语料库中的候选词相邻次数；值得说明的是，对于不同的文法语义模型，所需要统计的相邻词的词数可能不同；例如，对于语句词序列(李某/的/是/的确/是/一个/问题)，当预设文法语言模型为三元文法语言模型时，所统计的是二元词相邻次数，即需要统计“李某/的”、“的/是”、“是/的确”、“的确/是”、“是/一个”、“一个/问题”的语句词相邻次数；而当预设文法语言模型为三元文法语言模型时，所统计的包括二元词相邻次数和三元词相邻此时，即需要统计“李某/的”、“李某/的/是”、“的/是/的确”、“是/的确/是”、“的确/是/一个”、“是/一个/问题”的语句词相邻次数；以此类推。在得到各单词出现次数、语句词相邻次数和候选词相邻次数时，即可分别计算各序列的文法得分；在计算时，即将语句词序列对应的单词出现次数和语句词相邻次数代入至预设得分公式计算语句词序列的文法得分，并将候选 序列对应的单词出现次数和候选词相邻次数代入预设得分公式计算所述候选序列的文法得分。
可选地，当预设文法语言模型为二元文法语言模型，该预设得分公式为：
Figure PCTCN2019122329-appb-000001
其中，P为语句词序列或候选序列的文法得分；
k1为第一预设得分系数，k1＞0；
m为语句词序列或候选序列的单词数，m≥2；
C 0为预设语料库中的单词总数，C 0＞0；
C(w 1)为语句词序列或候选序列中第1位单词在预设语料库中的单词出现次数；
C(w i-1)为语句词序列或候选序列中第i-1位单词在预设语料库中的单词出现次数，2≤i≤m；
C(w i-1,w i)为语句词序列中第i位单词与前一位历史词在预设语料库中的语句词相邻次数，或为候选序列中第i位单词与前一位历史词在预设语料库中的候选词相邻次数。
可选地，当预设文法语言模型为三元文法语言模型，该预设得分公式为：
Figure PCTCN2019122329-appb-000002
其中，P为语句词序列或候选序列的文法得分；
k2为第二预设得分系数，k2＞0；
m为语句词序列或候选序列的单词数，m≥3；
C 0为预设语料库中的单词总数，C 0＞0；
C(w 1)为语句词序列或候选序列中第1位单词在预设语料库中的单词出现次数；
C(w 1,w 2)为语句词序列中第1位单词与第2位单词在预设语料库中的语句词相邻次数，或为候选序列中第1位单词与第2位单词在预设语料库中的候选词相邻次数；
C(w i-2,w i-1)为语句词序列中第i位单词的前一位历史词与前二位历史词在预设语料库中的语句词相邻次数，或为候选序列中第i位单词的前一位历史词与前二位历史词在预设语料库中的候选词相邻次数；
C(w i-2,w i-1,w i)为语句词序列中第i位单词与前一位历史词、前二位历史词在预设语料库中的语句词相邻次数，或为候选序列中第i位单词与前一位历史词、前二位历史词在预设语料库中的候选词相邻次数。
在计算得到语句词序列和候选序列的文法得分时，该文法得分可认为是该序列所对应 的语句在整个中文语言中的出现概率，文法得分越高，其出现概率越大，因此可将文法得分最高的序列所对应的序列作为校正序列。当然，在实际中，可能是语句词序列即为校正序列，也可能是候选序列为校正序列。在确定校正序列时，即可根据该校正序列获得对应的校正文本。
步骤S30，根据所述校正文本在预设手语库中查询对应的手语样本帧，并将所述手语样本帧封装为对应的手语视频；
本实施例中，在得到校正文本时，即可根据该校正文本进行手语视频的转换。具体的，手机可对该校正文本进行语义识别，确定该校正文本的文本语义；当然该文本语义可能是以多段的方式记录的。在得到各段文本语义时，服务器将根据该文本语义在预设手语库中查找与之对应的手语样本帧，然后将各段手语样本帧进行拼接和封装，得到对应的手语视频；其中对于该预设手语库，预先存储有若干语义对应的标准手语样本视频，这些手语样本视频可以是预先录制并存储的。对于该预设手语库，可以是存储在社交应用的服务器或数据库，由手机根据校正文本在服务器中查询得到，从而节约手机的存储空间；当然该预设手语库也可以是存储在手机本地，以方便离线使用。而对于封装的手语视频，则可以根据实际情况(如手机播放器的支持格式或网络条件)选择不同的视频格式(如mp4、mkv、flv等)和清晰度(如480P、720P等)等。
步骤S40，将所述手语视频发送至对应的智能终端，或播放所述手语视频。
本实施例中，手机根据校正文本得到对应的手语视频时，即可将该手语视频发送至聋哑用户所用的智能终端(如手机)，以使得聋哑用户通过该智能终端看到该手语视频，并获知正常用户所表达的含义。当然，手机在得到该手语视频时，也可以是直接播放该手语视频，以供正常用户查看，并根据正常用户的操作对该手语视频执行发送、存储或是删除操作。
进一步的，本实施例中的语音信息转化手语视频，是以“正常用户要向聋哑用户发送语音消息前进行转换”为例进行说明；而在实际中，还可以是正常用户通过手机直接向聋哑用户的智能终端发送语音信息，当聋哑用户的智能终端接收到该语音信息时，可根据聋哑用户的操作将该语音信息转化为手语视频，其过程与上述实施例的各步骤类似，此处不再赘述，此时，聋哑用户的智能终端也可认为是本实施例中的基于人工智能的通信设备。
再进一步的，本实施例中的语音信息转化为手语视频的过程中，是先将语音信息转化为文本信息；而若正常用户是直接以文本录入的方式录入文本信息，则手机也可直接对该文本进行校正处理，并进一步对校正后的文本执行手语视频的转化，其过程与上述类似，此处不再赘述。
值得说明的是，本实施例中的语音信息向手语视频转化功能，可以是社交应用中的某个功能，也可以是以API接口或SDK的方式集成在手机等终端中，以供终端中的应用进行调用。
本实施例中，在接收到第一语音信息时，对所述第一语音信息进行语音识别，获得对应的第一语音文本；通过预设文本校正模型对所述第一语音文本进行文本校正，得到校正文本；根据所述校正文本在预设手语库中查询对应的手语样本帧，并将所述手语样本帧封装为对应的手语视频；将所述手语视频发送至对应的智能终端，或播放所述手语视频。通过以上方式，本实施例通过人工智能的方式实现语音向手语的转换，使得聋哑用户也可获知正常用户以语音方式在社交应用发送的信息，为聋哑用户提供了方便，提高聋哑用户的使用体验；其次，在语音转化过程中，还将对语音信息对应的语音文本进行校正处理，以得到符合一定语法规则(或通用语言习惯)的校正文本，再对该校正文本进行手语转换处理，从而避免语句不通顺、错别字等情况对手语视频的转换准确度造成不利的影响，有利于聋哑用户获知语音信息的正确含义。
基于图2所示实施例，提出本申请基于人工智能的通信方法第二实施例。本实施例中，所述基于人工智能的通信方法包括以下步骤：
步骤S50，在接收到手语视频时，基于预设特征提取算法对所述手语视频的各视频帧进行特征提取，获得所述各视频帧的手语特征；
目前，由于聋哑人语音方面具有一定的缺陷，因此通常是以手语与外界进行沟通，在这种情况下聋哑人往往无法直接使用社交应用的语音功能，但对正常人而言在不经过系统学习的情况下也无法获取聋哑人的手语所要表达的含义，这就为双方带来了不便。对此，本实施例中提出一种基于人工智能的通信方法，通过人工智能的方式实现手语视频向语音/文本进行转换，使得正常人利用该方法也可通过社交应用与聋哑人进行沟通，为双方提供了方便。
本实施例中的基于人工智能的通信方法是由基于人工智能的通信设备实现的，该通信设备可以是个人计算机(personal computer，PC)、笔记本电脑、移动终端(如平板电脑、掌上电脑、手机)等。本实施例中该通信设备以手机为例进行说明；该手机中预先安装有若干社交应用(如即时通讯应用)，该手机的用户为一聋哑人，可称为聋哑用户；当聋哑用户需要通过该社交应用与某一正常用户进行交流时，可以启动该手机中的社交应用，并通过手机录像的方式录入自己的手语视频。手机在接收到该手语视频时，首先基于预设特征提取算法对手语视频的各视频帧进行特征提取，获取到各视频帧的手语特征，例如手势形状、各手指的指向、手部动作的运动轨迹等。对于该特征提取的过程所使用的预设特征 提取算法，可以是依靠机器学习得到的；也即先收集一定数量的手语图像样本，并建立初始特征提取模型，然后通过这些手语图像样本对初始特征提取模型进行迭代训练，不断调整模型参数，直至得到最优模型参数(或是满足预设次数的迭代训练次数、或是识别正确率达到一定阈值)，从而根据该最优模型参数得到训练好的特征提取模型，该特征提取模型中的模型逻辑即为预设特征提取算法。当然，在实际中，该预设特征提取算法也可以根据静态规则设置的。
进一步，对于手语而言，其含义主要是靠手势以及手部动作运动轨迹等进行表达，因此在进行特征提取时，也主要是针对手语视频帧中的手部分图像进行。对此，为了提高特征提取的效率，可先对各视频帧进行一定的分割处理，再对分割后的图像进行特征提取。具体的，基于预设特征提取算法对所述手语视频的各视频帧进行特征提取，获得所述各视频帧的手语特征的步骤包括：
对所述各视频帧进行预处理，得到待提取图像；
手机在得到手语视频时，首先可对手语视频的各视频帧进行预处理，得到待提取图像，从而为后续的图像分割处理提供方便，例如图像二值化、降噪等操作。其中，二值化是指将视频帧转化为只含两种黑白颜色的二值图像，这是由于彩色图像所含信息量巨大，对其进行后续处理时计算量也会随之增大，花费的时间也会边长，因此为提高处理和识别的效率，可先对彩色图进行二值化处理；去噪则是通过滤波器或是其它方式解决数字图像由于噪声干扰而导致图像质量下降的问题。当然预处理的内容还可以根据实际情况添加或替换为其它的处理操作。
对所述待提取图像进行实例区域识别，并根据识别结果对所述待提取图像进行图像分割，得到手势区域图像；
在得到待提取图像时，将对该待提取图像进行实例区域识别，识别出其中的手势区域；对于该实例区域识别的过程，可以通过基于待测物的图像中每个像素点的像素值进行，例如通过各像素点的像素值以及临近相似点的像素值确定出像素突变点，进而得出各区域轮廓，然后对区域轮廓进行对象识别，确定各区域轮廓的含义，从而识别出手势区域轮廓；然后即可根据该轮廓进行图像分割，得到手势区域图像。
对所述手势区域图像进行特征提取，获得对应的手语特征。
在得到手势区域图像时，即可对该手势区域图像进行特征提取操作，获得对应的手语特征，从而无需对视频帧中的非手势区域图像进行相关检测和特征提取，降低了运算量，提高了特征提取的效率。
步骤S60，根据所述手语特征在预设手语库查询与所述各视频帧对应的手语样本帧， 并根据所述手语样本帧确定所述各视频帧的视频帧语义；
本实施例中，手机在提取得到手语特征时，即可根据该手语特征在预设手语库中查询与各视频帧对应的手语样本帧；其中该预设手语库中包括有标准手语样本帧、该手语样本帧的样本语义以及该手语样本帧的样本特征；而该手语样本帧的样本特征，也是通过上述步骤S50的预设特征提取算法对手语样本帧提取得到。在查询与各视频帧对应的手语样本帧时，即将视频帧的手语特征与各手语样本帧的样本特征进行比较，当某一样本特征与该手语特征的相似度达到一定阈值时，即可认为该样本特征所属的手语样本帧与该视频帧对应；而对于该相似度，则可以是通过距离的方式进行表征(如欧式距离等)。值得说明的是，若存在两个以上的样本特征与某一手语特征的相似度均达到一定阈值，则可将相似度最高的样本特征所属的手语样本帧作为与该视频帧对应的手语样本帧。在查询到与视频帧对应的手语样本帧时，手机获取该手语样本帧的样本含义，并根据该样本含义确定该视频帧的视频帧语音。
步骤S70，根据所述各视频帧的视频帧语义获取所述手语视频的手语视频语义，并根据所述手语视频语义生成对应的第二语音/文本信息；
本实施例中，在确定各视频帧的视频帧语义时，即可将各视频帧语义进行拼接，从而获得整段手语视频的手语视频语义，并根据该手语视频语义生成对应的第二语音/文本信息。当然，对于是生成语音信息还是文本信息，可以是根据聋哑用户的选择进行确定，又或是在手机系统中进行预先设置，又或者是两者均进行生成)。
步骤S80，将所述第二语音/文本信息发送至对应的智能终端，或播放/显示所述第二语音/文本信息。
本实施例中，手机得到对应的第二语音/文本信息时，即可将该第二语音/文本信息发送至正常用户所用的智能终端(如手机)，以使得正常用户通过该智能终端看到该第二语音/文本信息，并获知聋哑用户所表达的含义。当然，手机在得到该第二语音/文本信息时，也可以是播放第二语音信息/显示第二文本信息，并根据聋哑用户的操作对该第二语音/文本信息执行发送、存储或是删除操作。
进一步的，本实施例中的手语视频转化语音/文本信息，是以“聋哑用户要向正常用户发送手语视频前进行转换”为例进行说明；而在实际中，还可以是聋哑用户通过手机直接向正常用户的智能终端发送手语视频，当正常用户的智能终端接收到该手语视频时，可根据正常用户的操作将该手语视频转化为语音信息/文本信息，其过程与上述实施例的各步骤类似，此处不再赘述，此时，正常用户的智能终端也可认为是本实施例中的基于人工智能的通信设备。再进一步，本实施例中的手语视频向语音/文本信息转化功能，可以是社交应用 中的某个功能，也可以是以API接口或SDK的方式集成在手机等终端中，以供终端中的应用进行调用。
本实施例中，在接收到手语视频时，基于预设特征提取算法对所述手语视频的各视频帧进行特征提取，获得所述各视频帧的手语特征；根据所述手语特征在预设手语库查询与所述各视频帧对应的手语样本帧，并根据所述手语样本帧确定所述各视频帧的视频帧语义；根据所述各视频帧的视频帧语义获取所述手语视频的手语视频语义，并根据所述手语视频语义生成对应的第二语音/文本信息；将所述第二语音/文本信息发送至对应的智能终端，或播放/显示所述第二语音/文本信息。通过以上方式，本实施例通过人工智能的方式实现手语视频向语音/文本的转换，使得正常用户可以方便地获知聋哑用户的手语含义，而聋哑用户也可以以手语表达的方式与正常用户进行社交通信，为双方提供了方便，提高了双方的使用体验。
值得说明的是，对于上述第一实施例的语音向手语的转换功能和第二实施例的手语向语音/文字的转换功能，可以是同时集成在同一基于人工智能的通信设备中，根据用户的选择和设置确定是否启动其中的某一功能，从而为用户的使用提供方便。
此外，本申请实施例还提供一种基于人工智能的通信装置，所述基于人工智能的通信装置包括：
语音识别模块，用于在接收到第一语音信息时，对所述第一语音信息进行语音识别，获得对应的第一语音文本；
文本校正模块，用于通过预设文本校正模型对所述第一语音文本进行文本校正，得到校正文本；
视频封装模块，用于根据所述校正文本在预设手语库中查询对应的手语样本帧，并将所述手语样本帧封装为对应的手语视频；
视频处理模块，用于将所述手语视频发送至对应的智能终端，或播放所述手语视频。
其中，上述基于人工智能的通信装置的各虚拟功能模块存储于图1所示基于人工智能的通信设备的存储器1005中，用于实现计算机可读指令的所有功能；各模块被处理器1001执行时，可实现语音向手语之间的转换和发送的功能。
进一步的，所述文本校正模块包括：
语句划分单元，用于基于预设中文分词规则对所述第一语音文本的原始语句进行词划分，得到对应的语句词序列；
目标词替换单元，用于获取所述语句词序列中的目标词对应的混淆词，并通过所述混淆词替换所述语句词序列中的目标词，得到候选序列；
得分计算单元，用于基于预设文法语言模型分别计算所述语句词序列和所述候选序列的文法得分，并根据所述文法得分在所述语句词序列和候选序列中确定校正序列；
文本获得单元，用于文本根据所述校正序列获得对应的校正文本。
进一步的，所述得分计算单元包括：
第一统计子单元，用于统计所述语句词序列和所述候选序列中各单词在预设语料库中的单词出现次数；
第二统计子单元，用于统计所述语句词序列中各单词与前位历史词在预设语料库中的语句词相邻次数，并统计所述候选序列中各单词与前位历史词在所述预设语料库中的候选词相邻次数；
得分计算子单元，用于将所述单词出现次数和所述语句词相邻次数代入至预设得分公式计算所述语句词序列的文法得分，并将所述单词出现次数和所述候选词相邻次数代入至预设得分公式计算所述候选序列的文法得分。
进一步的，所述预设文法语言模型为二元文法语言模型，所述预设得分公式为：
Figure PCTCN2019122329-appb-000003
其中，P为所述语句词序列或所述候选序列的文法得分；
k1为第一预设得分系数，k1＞0；
m为所述语句词序列或所述候选序列的单词数，m≥2；
C 0为所述预设语料库中的单词总数，C 0＞0；
C(w 1)为所述语句词序列或所述候选序列中第1位单词在所述预设语料库中的单词出现次数；
C(w i-1)为所述语句词序列或所述候选序列中第i-1位单词在所述预设语料库中的单词出现次数，2≤i≤m；
C(w i-1,w i)为所述语句词序列中第i位单词与前一位历史词在所述预设语料库中的语句词相邻次数，或为所述候选序列中第i位单词与前一位历史词在所述预设语料库中的候选词相邻次数。
进一步的，所述预设文法语言模型为三元文法语言模型，所述预设得分公式为：
Figure PCTCN2019122329-appb-000004
其中，P为所述语句词序列或所述候选序列的文法得分；
k2为第二预设得分系数，k2＞0；
m为所述语句词序列或所述候选序列的单词数，m≥3；
C 0为所述预设语料库中的单词总数，C 0＞0；
C(w 1)为所述语句词序列或所述候选序列中第1位单词在所述预设语料库中的单词出现次数；
C(w 1,w 2)为所述语句词序列中第1位单词与第2位单词在所述预设语料库中的语句词相邻次数，或为所述候选序列中第1位单词与第2位单词在所述预设语料库中的候选词相邻次数；
C(w i-2,w i-1)为所述语句词序列中第i位单词的前一位历史词与前二位历史词在所述预设语料库中的语句词相邻次数，或为所述候选序列中第i位单词的前一位历史词与前二位历史词在所述预设语料库中的候选词相邻次数；
C(w i-2,w i-1,w i)为所述语句词序列中第i位单词与前一位历史词、前二位历史词在所述预设语料库中的语句词相邻次数，或为所述候选序列中第i位单词与前一位历史词、前二位历史词在所述预设语料库中的候选词相邻次数。
其中，上述基于人工智能的通信装置中各个模块的功能实现与上述基于人工智能的通信方法第一实施例中各步骤相对应，其功能和实现过程在此处不再一一赘述。
此外，本申请实施例还提供一种基于人工智能的通信装置，所述基于人工智能的通信装置包括：
特征提取模块，在接收到手语视频时，基于预设特征提取算法对所述手语视频的各视频帧进行特征提取，获得所述各视频帧的手语特征；
语义确定模块，用于根据所述手语特征在预设手语库查询与所述各视频帧对应的手语样本帧，并根据所述手语样本帧确定所述各视频帧的视频帧语义；
信息生成模块，用于根据所述各视频帧的视频帧语义获取所述手语视频的手语视频语义，并根据所述手语视频语义生成对应的第二语音/文本信息；
信息处理模块，用于将所述第二语音/文本信息发送至对应的智能终端，或播放/显示所述第二语音/文本信息。
其中，上述基于人工智能的通信装置的各虚拟功能模块也可存储于图1所示基于人工智能的通信设备的存储器1005中，用于实现计算机可读指令的所有功能；各模块被处理器1001执行时，可实现手语向语音/文本之间的转换和发送的功能。
进一步的，所述特征提取模块包括：
预处理单元，用于对所述各视频帧进行预处理，得到待提取图像；
图像分割单元，用于对所述待提取图像进行实例区域识别，并根据识别结果对所述待 提取图像进行图像分割，得到手势区域图像；
特征提取单元，用于对所述手势区域图像进行特征提取，获得对应的手语特征。
其中，上述基于人工智能的通信装置中各个模块的功能实现与上述基于人工智能的通信方法第二实施例中各步骤相对应，其功能和实现过程在此处不再一一赘述。
此外，本申请实施例还提供一种可读存储介质，所述计算机可读存储介质可以为非易失性可读存储介质。本申请可读存储介质上存储有计算机可读指令，其中所述计算机可读指令被处理器执行时，实现如上述的基于人工智能的通信方法第一实施例或第二实施例的的步骤。
其中，计算机可读指令被执行时所实现的方法可参照上述基于人工智能的通信方法的各实施例，此处不再赘述。
以上仅为本申请的优选实施例，并非因此限制本申请的专利范围，凡是利用本申请说明书及附图内容所作的等效结构或等效流程变换，或直接或间接运用在其他相关的技术领域，均同理包括在本申请的专利保护范围内。

Claims (20)

  1. 一种基于人工智能的通信方法，其中，所述基于人工智能的通信方法包括：
    在接收到第一语音信息时，对所述第一语音信息进行语音识别，获得对应的第一语音文本；
    基于预设中文分词规则对所述第一语音文本的原始语句进行词划分，得到对应的语句词序列；
    获取所述语句词序列中的目标词对应的混淆词，并通过所述混淆词替换所述语句词序列中的目标词，得到候选序列；
    基于预设文法语言模型分别计算所述语句词序列和所述候选序列的文法得分，并根据所述文法得分在所述语句词序列和候选序列中确定校正序列；
    根据所述校正序列获得对应的校正文本；
    根据所述校正文本在预设手语库中查询对应的手语样本帧，并将所述手语样本帧封装为对应的手语视频；
    将所述手语视频发送至对应的智能终端，或播放所述手语视频。
  2. 如权利要求1所述的基于人工智能的通信方法，其中，所述基于预设文法语言模型分别计算所述语句词序列和所述候选序列的文法得分的步骤包括：
    统计所述语句词序列和所述候选序列中各单词在预设语料库中的单词出现次数；
    统计所述语句词序列中各单词与前位历史词在预设语料库中的语句词相邻次数，并统计所述候选序列中各单词与前位历史词在所述预设语料库中的候选词相邻次数；
    将所述单词出现次数和所述语句词相邻次数代入至预设得分公式计算所述语句词序列的文法得分，并将所述单词出现次数和所述候选词相邻次数代入至预设得分公式计算所述候选序列的文法得分。
  3. 如权利要求2所述的基于人工智能的通信方法，其中，所述预设文法语言模型为二元文法语言模型，所述预设得分公式为：
    Figure PCTCN2019122329-appb-100001
    其中，P为所述语句词序列或所述候选序列的文法得分；
    k1为第一预设得分系数，k1＞0；
    m为所述语句词序列或所述候选序列的单词数，m≥2；
    C 0为所述预设语料库中的单词总数，C 0＞0；
    C(w 1)为所述语句词序列或所述候选序列中第1位单词在所述预设语料库中的单词出现次数；
    C(w i-1)为所述语句词序列或所述候选序列中第i-1位单词在所述预设语料库中的单词出现次数，2≤i≤m；
    C(w i-1,w i)为所述语句词序列中第i位单词与前一位历史词在所述预设语料库中的语句词相邻次数，或为所述候选序列中第i位单词与前一位历史词在所述预设语料库中的候选词相邻次数。
  4. 如权利要求2所述的基于人工智能的通信方法，其中，所述预设文法语言模型为三元文法语言模型，所述预设得分公式为：
    Figure PCTCN2019122329-appb-100002
    其中，P为所述语句词序列或所述候选序列的文法得分；
    k2为第二预设得分系数，k2＞0；
    m为所述语句词序列或所述候选序列的单词数，m≥3；
    C 0为所述预设语料库中的单词总数，C 0＞0；
    C(w 1)为所述语句词序列或所述候选序列中第1位单词在所述预设语料库中的单词出现次数；
    C(w 1,w 2)为所述语句词序列中第1位单词与第2位单词在所述预设语料库中的语句词相邻次数，或为所述候选序列中第1位单词与第2位单词在所述预设语料库中的候选词相邻次数；
    C(w i-2,w i-1)为所述语句词序列中第i位单词的前一位历史词与前二位历史词在所述预设语料库中的语句词相邻次数，或为所述候选序列中第i位单词的前一位历史词与前二位历史词在所述预设语料库中的候选词相邻次数；
    C(w i-2,w i-1,w i)为所述语句词序列中第i位单词与前一位历史词、前二位历史词在所述预设语料库中的语句词相邻次数，或为所述候选序列中第i位单词与前一位历史词、前二位历史词在所述预设语料库中的候选词相邻次数。
  5. 一种基于人工智能的通信方法，其中，所述基于人工智能的通信方法包括：
    在接收到手语视频时，对所述各视频帧进行预处理，得到待提取图像；
    对所述待提取图像进行实例区域识别，并根据识别结果对所述待提取图像进行图像分割，得到手势区域图像；
    对所述手势区域图像进行特征提取，获得对应的手语特征；
    根据所述手语特征在预设手语库查询与所述各视频帧对应的手语样本帧，并根据所述手语样本帧确定所述各视频帧的视频帧语义；
    根据所述各视频帧的视频帧语义获取所述手语视频的手语视频语义，并根据所述手语视频语义生成对应的第二语音/文本信息；
    将所述第二语音/文本信息发送至对应的智能终端，或播放/显示所述第二语音/文本信息。
  6. 一种基于人工智能的通信装置，其中，所述基于人工智能的通信装置包括：
    语音识别模块，用于在接收到第一语音信息时，对所述第一语音信息进行语音识别，获得对应的第一语音文本；
    文本校正模块，包括语句划分单元、目标词替换单元、得分计算单元和文本获得单元，
    其中，所述语句划分单元，用于基于预设中文分词规则对所述第一语音文本的原始语句进行词划分，得到对应的语句词序列；
    所述目标词替换单元，用于获取所述语句词序列中的目标词对应的混淆词，并通过所述混淆词替换所述语句词序列中的目标词，得到候选序列；
    所述得分计算单元，用于基于预设文法语言模型分别计算所述语句词序列和所述候选序列的文法得分，并根据所述文法得分在所述语句词序列和候选序列中确定校正序列；
    所述文本获得单元，用于文本根据所述校正序列获得对应的校正文本；
    视频封装模块，用于根据所述校正文本在预设手语库中查询对应的手语样本帧，并将所述手语样本帧封装为对应的手语视频；
    视频处理模块，用于将所述手语视频发送至对应的智能终端，或播放所述手语视频。
  7. 如权利要求6所述的基于人工智能的通信装置，其中，所述得分计算单元包括：
    第一统计子单元，用于统计所述语句词序列和所述候选序列中各单词在预设语料库中的单词出现次数；
    第二统计子单元，用于统计所述语句词序列中各单词与前位历史词在预设语料库中的语句词相邻次数，并统计所述候选序列中各单词与前位历史词在所述预设语料库中的候选词相邻次数；
    得分计算子单元，用于将所述单词出现次数和所述语句词相邻次数代入至预设得分公式计算所述语句词序列的文法得分，并将所述单词出现次数和所述候选词相邻次数代入至预设得分公式计算所述候选序列的文法得分。
  8. 如权利要求7所述的基于人工智能的通信装置，其中，所述预设文法语言模型为二元文法语言模型，所述预设得分公式为：
    Figure PCTCN2019122329-appb-100003
    其中，P为所述语句词序列或所述候选序列的文法得分；
    k1为第一预设得分系数，k1＞0；
    m为所述语句词序列或所述候选序列的单词数，m≥2；
    C 0为所述预设语料库中的单词总数，C 0＞0；
    C(w 1)为所述语句词序列或所述候选序列中第1位单词在所述预设语料库中的单词出现次数；
    C(w i-1)为所述语句词序列或所述候选序列中第i-1位单词在所述预设语料库中的单词出现次数，2≤i≤m；
    C(w i-1,w i)为所述语句词序列中第i位单词与前一位历史词在所述预设语料库中的语句词相邻次数，或为所述候选序列中第i位单词与前一位历史词在所述预设语料库中的候选词相邻次数。
  9. 如权利要求7所述的基于人工智能的通信装置，其中，所述预设文法语言模型为三元文法语言模型，所述预设得分公式为：
    Figure PCTCN2019122329-appb-100004
    其中，P为所述语句词序列或所述候选序列的文法得分；
    k2为第二预设得分系数，k2＞0；
    m为所述语句词序列或所述候选序列的单词数，m≥3；
    C 0为所述预设语料库中的单词总数，C 0＞0；
    C(w 1)为所述语句词序列或所述候选序列中第1位单词在所述预设语料库中的单词出现次数；
    C(w 1,w 2)为所述语句词序列中第1位单词与第2位单词在所述预设语料库中的语句词相邻次数，或为所述候选序列中第1位单词与第2位单词在所述预设语料库中的候选词相邻次数；
    C(w i-2,w i-1)为所述语句词序列中第i位单词的前一位历史词与前二位历史词在所述预设语料库中的语句词相邻次数，或为所述候选序列中第i位单词的前一位历史词与前二位历史词在所述预设语料库中的候选词相邻次数；
    C(w i-2,w i-1,w i)为所述语句词序列中第i位单词与前一位历史词、前二位历史词在所述预设语料库中的语句词相邻次数，或为所述候选序列中第i位单词与前一位历史词、 前二位历史词在所述预设语料库中的候选词相邻次数。
  10. 一种基于人工智能的通信装置，其中，所述基于人工智能的通信装置包括：
    特征提取模块，包括预处理单元、图像分割单元和特征提取单元，
    其中，所述预处理单元，用于对所述各视频帧进行预处理，得到待提取图像；
    所述图像分割单元，用于对所述待提取图像进行实例区域识别，并根据识别结果对所述待提取图像进行图像分割，得到手势区域图像；
    所述特征提取单元，用于对所述手势区域图像进行特征提取，获得对应的手语特征；
    语义确定模块，用于根据所述手语特征在预设手语库查询与所述各视频帧对应的手语样本帧，并根据所述手语样本帧确定所述各视频帧的视频帧语义；
    信息生成模块，用于根据所述各视频帧的视频帧语义获取所述手语视频的手语视频语义，并根据所述手语视频语义生成对应的第二语音/文本信息；
    信息处理模块，用于将所述第二语音/文本信息发送至对应的智能终端，或播放/显示所述第二语音/文本信息。
  11. 一种基于人工智能的通信设备，其中，所述基于人工智能的通信设备包括处理器、存储器、以及存储在所述存储器上并可被所述处理器执行的计算机可读指令，其中所述计算机可读指令被所述处理器执行时，实现以下步骤：
    在接收到第一语音信息时，对所述第一语音信息进行语音识别，获得对应的第一语音文本；
    基于预设中文分词规则对所述第一语音文本的原始语句进行词划分，得到对应的语句词序列；
    获取所述语句词序列中的目标词对应的混淆词，并通过所述混淆词替换所述语句词序列中的目标词，得到候选序列；
    基于预设文法语言模型分别计算所述语句词序列和所述候选序列的文法得分，并根据所述文法得分在所述语句词序列和候选序列中确定校正序列；
    根据所述校正序列获得对应的校正文本；
    根据所述校正文本在预设手语库中查询对应的手语样本帧，并将所述手语样本帧封装为对应的手语视频；
    将所述手语视频发送至对应的智能终端，或播放所述手语视频。
  12. 如权利要求11所述的基于人工智能的通信设备，其中，所述基于预设文法语言模型分别计算所述语句词序列和所述候选序列的文法得分的步骤包括：
    统计所述语句词序列和所述候选序列中各单词在预设语料库中的单词出现次数；
    统计所述语句词序列中各单词与前位历史词在预设语料库中的语句词相邻次数，并统计所述候选序列中各单词与前位历史词在所述预设语料库中的候选词相邻次数；
    将所述单词出现次数和所述语句词相邻次数代入至预设得分公式计算所述语句词序列的文法得分，并将所述单词出现次数和所述候选词相邻次数代入至预设得分公式计算所述候选序列的文法得分。
  13. 如权利要求12所述的基于人工智能的通信设备，其中，所述预设文法语言模型为二元文法语言模型，所述预设得分公式为：
    Figure PCTCN2019122329-appb-100005
    其中，P为所述语句词序列或所述候选序列的文法得分；
    k1为第一预设得分系数，k1＞0；
    m为所述语句词序列或所述候选序列的单词数，m≥2；
    C 0为所述预设语料库中的单词总数，C 0＞0；
    C(w 1)为所述语句词序列或所述候选序列中第1位单词在所述预设语料库中的单词出现次数；
    C(w i-1)为所述语句词序列或所述候选序列中第i-1位单词在所述预设语料库中的单词出现次数，2≤i≤m；
    C(w i-1,w i)为所述语句词序列中第i位单词与前一位历史词在所述预设语料库中的语句词相邻次数，或为所述候选序列中第i位单词与前一位历史词在所述预设语料库中的候选词相邻次数。
  14. 如权利要求12所述的基于人工智能的通信设备，其中，所述预设文法语言模型为三元文法语言模型，所述预设得分公式为：
    Figure PCTCN2019122329-appb-100006
    其中，P为所述语句词序列或所述候选序列的文法得分；
    k2为第二预设得分系数，k2＞0；
    m为所述语句词序列或所述候选序列的单词数，m≥3；
    C 0为所述预设语料库中的单词总数，C 0＞0；
    C(w 1)为所述语句词序列或所述候选序列中第1位单词在所述预设语料库中的单词出现次数；
    C(w 1,w 2)为所述语句词序列中第1位单词与第2位单词在所述预设语料库中的语 句词相邻次数，或为所述候选序列中第1位单词与第2位单词在所述预设语料库中的候选词相邻次数；
    C(w i-2,w i-1)为所述语句词序列中第i位单词的前一位历史词与前二位历史词在所述预设语料库中的语句词相邻次数，或为所述候选序列中第i位单词的前一位历史词与前二位历史词在所述预设语料库中的候选词相邻次数；
    C(w i-2,w i-1,w i)为所述语句词序列中第i位单词与前一位历史词、前二位历史词在所述预设语料库中的语句词相邻次数，或为所述候选序列中第i位单词与前一位历史词、前二位历史词在所述预设语料库中的候选词相邻次数。
  15. 一种基于人工智能的通信设备，其中，所述基于人工智能的通信设备包括处理器、存储器、以及存储在所述存储器上并可被所述处理器执行的计算机可读指令，其中所述计算机可读指令被所述处理器执行时，实现以下步骤：
    在接收到手语视频时，对所述各视频帧进行预处理，得到待提取图像；
    对所述待提取图像进行实例区域识别，并根据识别结果对所述待提取图像进行图像分割，得到手势区域图像；
    对所述手势区域图像进行特征提取，获得对应的手语特征；
    根据所述手语特征在预设手语库查询与所述各视频帧对应的手语样本帧，并根据所述手语样本帧确定所述各视频帧的视频帧语义；
    根据所述各视频帧的视频帧语义获取所述手语视频的手语视频语义，并根据所述手语视频语义生成对应的第二语音/文本信息；
    将所述第二语音/文本信息发送至对应的智能终端，或播放/显示所述第二语音/文本信息。
  16. 一种可读存储介质，其中，所述可读存储介质上存储有计算机可读指令，其中所述计算机可读指令被处理器执行时，实现以下步骤：
    在接收到第一语音信息时，对所述第一语音信息进行语音识别，获得对应的第一语音文本；
    基于预设中文分词规则对所述第一语音文本的原始语句进行词划分，得到对应的语句词序列；
    获取所述语句词序列中的目标词对应的混淆词，并通过所述混淆词替换所述语句词序列中的目标词，得到候选序列；
    基于预设文法语言模型分别计算所述语句词序列和所述候选序列的文法得分，并根据所述文法得分在所述语句词序列和候选序列中确定校正序列；
    根据所述校正序列获得对应的校正文本；
    根据所述校正文本在预设手语库中查询对应的手语样本帧，并将所述手语样本帧封装为对应的手语视频；
    将所述手语视频发送至对应的智能终端，或播放所述手语视频。
  17. 如权利要求16所述的可读存储介质，其中，所述基于预设文法语言模型分别计算所述语句词序列和所述候选序列的文法得分的步骤包括：
    统计所述语句词序列和所述候选序列中各单词在预设语料库中的单词出现次数；
    统计所述语句词序列中各单词与前位历史词在预设语料库中的语句词相邻次数，并统计所述候选序列中各单词与前位历史词在所述预设语料库中的候选词相邻次数；
    将所述单词出现次数和所述语句词相邻次数代入至预设得分公式计算所述语句词序列的文法得分，并将所述单词出现次数和所述候选词相邻次数代入至预设得分公式计算所述候选序列的文法得分。
  18. 如权利要求17所述的可读存储介质，其中，所述预设文法语言模型为二元文法语言模型，所述预设得分公式为：
    Figure PCTCN2019122329-appb-100007
    其中，P为所述语句词序列或所述候选序列的文法得分；
    k1为第一预设得分系数，k1＞0；
    m为所述语句词序列或所述候选序列的单词数，m≥2；
    C 0为所述预设语料库中的单词总数，C 0＞0；
    C(w 1)为所述语句词序列或所述候选序列中第1位单词在所述预设语料库中的单词出现次数；
    C(w i-1)为所述语句词序列或所述候选序列中第i-1位单词在所述预设语料库中的单词出现次数，2≤i≤m；
    C(w i-1,w i)为所述语句词序列中第i位单词与前一位历史词在所述预设语料库中的语句词相邻次数，或为所述候选序列中第i位单词与前一位历史词在所述预设语料库中的候选词相邻次数。
  19. 如权利要求17所述的可读存储介质，其中，所述预设文法语言模型为三元文法语言模型，所述预设得分公式为：
    Figure PCTCN2019122329-appb-100008
    其中，P为所述语句词序列或所述候选序列的文法得分；
    k2为第二预设得分系数，k2＞0；
    m为所述语句词序列或所述候选序列的单词数，m≥3；
    C 0为所述预设语料库中的单词总数，C 0＞0；
    C(w 1)为所述语句词序列或所述候选序列中第1位单词在所述预设语料库中的单词出现次数；
    C(w 1,w 2)为所述语句词序列中第1位单词与第2位单词在所述预设语料库中的语句词相邻次数，或为所述候选序列中第1位单词与第2位单词在所述预设语料库中的候选词相邻次数；
    C(w i-2,w i-1)为所述语句词序列中第i位单词的前一位历史词与前二位历史词在所述预设语料库中的语句词相邻次数，或为所述候选序列中第i位单词的前一位历史词与前二位历史词在所述预设语料库中的候选词相邻次数；
    C(w i-2,w i-1,w i)为所述语句词序列中第i位单词与前一位历史词、前二位历史词在所述预设语料库中的语句词相邻次数，或为所述候选序列中第i位单词与前一位历史词、前二位历史词在所述预设语料库中的候选词相邻次数。
  20. 一种可读存储介质，其中，所述可读存储介质上存储有计算机可读指令，其中所述计算机可读指令被处理器执行时，实现以下步骤：
    在接收到手语视频时，对所述各视频帧进行预处理，得到待提取图像；
    对所述待提取图像进行实例区域识别，并根据识别结果对所述待提取图像进行图像分割，得到手势区域图像；
    对所述手势区域图像进行特征提取，获得对应的手语特征；
    根据所述手语特征在预设手语库查询与所述各视频帧对应的手语样本帧，并根据所述手语样本帧确定所述各视频帧的视频帧语义；
    根据所述各视频帧的视频帧语义获取所述手语视频的手语视频语义，并根据所述手语视频语义生成对应的第二语音/文本信息；
    将所述第二语音/文本信息发送至对应的智能终端，或播放/显示所述第二语音/文本信息。
PCT/CN2019/122329 2018-12-14 2019-12-02 基于人工智能的通信方法、装置、设备及可读存储介质 Ceased WO2020119496A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201811539391.9A CN109740447A (zh) 2018-12-14 2018-12-14 基于人工智能的通信方法、设备及可读存储介质
CN201811539391.9 2018-12-14

Publications (1)

Publication Number Publication Date
WO2020119496A1 true WO2020119496A1 (zh) 2020-06-18

Family

ID=66360410

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/122329 Ceased WO2020119496A1 (zh) 2018-12-14 2019-12-02 基于人工智能的通信方法、装置、设备及可读存储介质

Country Status (2)

Country Link
CN (1) CN109740447A (zh)
WO (1) WO2020119496A1 (zh)

Cited By (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112183216A (zh) * 2020-09-02 2021-01-05 温州大学 一种用于残障人士沟通的辅助系统
CN112989819A (zh) * 2021-03-22 2021-06-18 北京小米移动软件有限公司 中文文本分词方法、装置及存储介质
CN113191157A (zh) * 2021-05-18 2021-07-30 清华大学 一种处理文本单元的方法及系统
CN113395578A (zh) * 2020-11-27 2021-09-14 腾讯科技（深圳）有限公司 一种提取视频主题文本的方法、装置、设备及存储介质
CN113590772A (zh) * 2021-02-25 2021-11-02 腾讯科技（深圳）有限公司 异常评分的检测方法、装置、设备及计算机可读存储介质
CN113641854A (zh) * 2021-07-28 2021-11-12 上海影谱科技有限公司 一种将文字转化为视频的方法及系统
CN114157920A (zh) * 2021-12-10 2022-03-08 深圳Tcl新技术有限公司 一种展示手语的播放方法、装置、智能电视及存储介质
CN114299525A (zh) * 2021-12-10 2022-04-08 维沃移动通信有限公司 图片处理方法、装置及电子设备
CN115410575A (zh) * 2022-08-10 2022-11-29 湖北星纪时代科技有限公司 手语信息显示方法、介质及可穿戴设备
CN115953837A (zh) * 2022-12-21 2023-04-11 科大讯飞华南人工智能研究院（广州）有限公司 一种手语连续句的视频合成方法、装置以及设备
CN116805272A (zh) * 2022-10-29 2023-09-26 武汉行已学教育咨询有限公司 一种可视化教育教学分析方法、系统及存储介质
CN118053426A (zh) * 2024-04-16 2024-05-17 深圳市轻生活科技有限公司 一种互联互控的智能无线开关及其离线语音控制系统

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109740447A (zh) * 2018-12-14 2019-05-10 深圳壹账通智能科技有限公司 基于人工智能的通信方法、设备及可读存储介质
CN110265019B (zh) * 2019-07-03 2021-04-06 中通智新(武汉)技术研发有限公司 一种语音识别的方法及语音机器人系统
CN110533750A (zh) * 2019-07-10 2019-12-03 浙江工业大学 一种将音频转换为带有自定义3d角色的手语动画的方法
CN110570877B (zh) * 2019-07-25 2022-03-22 咪咕文化科技有限公司 手语视频生成方法、电子设备及计算机可读存储介质
CN110736223A (zh) * 2019-10-29 2020-01-31 珠海格力电器股份有限公司 空调控制方法及装置
CN110931042B (zh) * 2019-11-14 2022-08-16 北京欧珀通信有限公司 同声传译方法、装置、电子设备以及存储介质
CN113035199B (zh) * 2021-02-01 2024-05-07 深圳创维－Rgb电子有限公司 音频处理方法、装置、设备及可读存储介质
US11817126B2 (en) 2021-04-20 2023-11-14 Micron Technology, Inc. Converting sign language
CN113326746A (zh) * 2021-05-13 2021-08-31 中国工商银行股份有限公司 用于人体模型的手语播报方法及装置
CN115484493A (zh) * 2022-09-09 2022-12-16 深圳市小溪流科技有限公司 一种iptv音视频实时转虚拟手语视频的实时智能流媒体系统

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050033578A1 (en) * 2003-08-07 2005-02-10 Mara Zuckerman Text-to-video sign language translator
CN102708866A (zh) * 2012-06-01 2012-10-03 武汉大学 基于语义计算的听力或语言障碍人士交流系统及其方法
CN108615009A (zh) * 2018-04-24 2018-10-02 山东师范大学 一种基于动态手势识别的手语翻译交流系统
CN109740447A (zh) * 2018-12-14 2019-05-10 深圳壹账通智能科技有限公司 基于人工智能的通信方法、设备及可读存储介质

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050033578A1 (en) * 2003-08-07 2005-02-10 Mara Zuckerman Text-to-video sign language translator
CN102708866A (zh) * 2012-06-01 2012-10-03 武汉大学 基于语义计算的听力或语言障碍人士交流系统及其方法
CN108615009A (zh) * 2018-04-24 2018-10-02 山东师范大学 一种基于动态手势识别的手语翻译交流系统
CN109740447A (zh) * 2018-12-14 2019-05-10 深圳壹账通智能科技有限公司 基于人工智能的通信方法、设备及可读存储介质

Cited By (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112183216A (zh) * 2020-09-02 2021-01-05 温州大学 一种用于残障人士沟通的辅助系统
CN113395578B (zh) * 2020-11-27 2023-06-30 腾讯科技（深圳）有限公司 一种提取视频主题文本的方法、装置、设备及存储介质
CN113395578A (zh) * 2020-11-27 2021-09-14 腾讯科技（深圳）有限公司 一种提取视频主题文本的方法、装置、设备及存储介质
CN113590772A (zh) * 2021-02-25 2021-11-02 腾讯科技（深圳）有限公司 异常评分的检测方法、装置、设备及计算机可读存储介质
CN112989819A (zh) * 2021-03-22 2021-06-18 北京小米移动软件有限公司 中文文本分词方法、装置及存储介质
CN113191157A (zh) * 2021-05-18 2021-07-30 清华大学 一种处理文本单元的方法及系统
CN113641854A (zh) * 2021-07-28 2021-11-12 上海影谱科技有限公司 一种将文字转化为视频的方法及系统
CN113641854B (zh) * 2021-07-28 2023-09-26 上海影谱科技有限公司 一种将文字转化为视频的方法及系统
CN114157920A (zh) * 2021-12-10 2022-03-08 深圳Tcl新技术有限公司 一种展示手语的播放方法、装置、智能电视及存储介质
CN114157920B (zh) * 2021-12-10 2023-07-25 深圳Tcl新技术有限公司 一种展示手语的播放方法、装置、智能电视及存储介质
CN114299525A (zh) * 2021-12-10 2022-04-08 维沃移动通信有限公司 图片处理方法、装置及电子设备
CN115410575A (zh) * 2022-08-10 2022-11-29 湖北星纪时代科技有限公司 手语信息显示方法、介质及可穿戴设备
CN116805272A (zh) * 2022-10-29 2023-09-26 武汉行已学教育咨询有限公司 一种可视化教育教学分析方法、系统及存储介质
CN115953837A (zh) * 2022-12-21 2023-04-11 科大讯飞华南人工智能研究院（广州）有限公司 一种手语连续句的视频合成方法、装置以及设备
CN118053426A (zh) * 2024-04-16 2024-05-17 深圳市轻生活科技有限公司 一种互联互控的智能无线开关及其离线语音控制系统

Also Published As

Publication number Publication date
CN109740447A (zh) 2019-05-10

Similar Documents

Publication Publication Date Title
WO2020119496A1 (zh) 基于人工智能的通信方法、装置、设备及可读存储介质
CN111968649B (zh) 一种字幕纠正方法、字幕显示方法、装置、设备及介质
US20240202446A1 (en) Method for training keyword extraction model, keyword extraction method, and computer device
CN112784696B (zh) 基于图像识别的唇语识别方法、装置、设备及存储介质
US10176804B2 (en) Analyzing textual data
US10114809B2 (en) Method and apparatus for phonetically annotating text
CN111435595B (zh) 文本规整方法及装置
US20170270086A1 (en) Apparatus, method, and computer program product for correcting speech recognition error
WO2018157703A1 (zh) 自然语言的语义提取方法及装置和计算机存储介质
WO2018076450A1 (zh) 一种输入方法和装置、一种用于输入的装置
CN111159377B (zh) 属性召回模型训练方法、装置、电子设备以及存储介质
CN103678271B (zh) 一种文本校正方法及用户设备
CN111445898B (zh) 语种识别方法、装置、电子设备和存储介质
CN107564526B (zh) 处理方法、装置和机器可读介质
CN111192586B (zh) 语音识别方法及装置、电子设备、存储介质
CN109117477B (zh) 面向中文领域的非分类关系抽取方法、装置、设备及介质
CN113035199B (zh) 音频处理方法、装置、设备及可读存储介质
CN112002311A (zh) 文本纠错方法、装置、计算机可读存储介质及终端设备
CN111539199A (zh) 文本的纠错方法、装置、终端、及存储介质
KR102345625B1 (ko) 자막 생성 방법 및 이를 수행하는 장치
CN107967250A (zh) 一种信息处理方法及装置
CN105843797A (zh) 归一化方法及装置
CN112632956A (zh) 文本匹配方法、装置、终端和存储介质
US20170061957A1 (en) Method and apparatus for improving a language model, and speech recognition method and apparatus
CN112395863A (zh) 文本的处理方法和装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19896533

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS (EPO FORM 1205A DATED 30.09.021)

122 Ep: pct application non-entry in european phase

Ref document number: 19896533

Country of ref document: EP

Kind code of ref document: A1