TW202205135A - Intelligent audiovisual combination system and implementation method thereof - Google Patents

Abstract

An intelligent audiovisual combination system and implementation method thereof are disclosed, mainly comprising a message processing server and at least one user communication device. The message processing server can be in information connection with the communication device of each user through the Internet. After the message processing server receives a first electronic message sent by the user communication device and a situation setting parameter set by the user, the message processing server produces an automatic text modified second electronic message and an associated audiovisual customer service information based on the situation setting parameter. Another communication device can play the second electronic message and the audiovisual customer service information. Furthermore, based on the situation selected by the user, the audiovisual customer service information can play an associated customer service video and provide a vivid electronic message.

Description

Intelligent audio-visual fusion system and its implementation method

The present invention is applied to the instant communication technology. The present invention mainly retrieves an electronic message input by the user through the message processing server, re-edits the original electronic message according to the context selected by the user, and sends it to the user of the communication party. The communication device is displayed as a virtual customer service animation.

Press, instant messaging technology has become very popular. People can install instant messaging software on their smartphones to achieve instant messaging through smartphones. The current instant messaging software can send text messages and voice messages, and users can also use instant messaging However, the actual use of the instant messaging software shows that the user must enter the message in person when the two communicating parties communicate. However, in some usage states, the user cannot concentrate on entering the message. When the user needs to focus on driving, when operating the communication software, he may only reply to the message with simple words, for example, the user is on his way home, and he may need to delay the time to arrive home due to traffic jams , however, the user is limited by the driving operation and can only reply "coming home soon". The short message cannot fully express the real situation of the user, and the recipient cannot understand the real meaning of the sender. This is an existing real-time message. The communication software needs to be improved. In addition, some users think that when communicating with voice, the too bland speaking style may make the communication object feel boring. Therefore, some users have the need to change the voice. The method of sound can give the communication object a refreshing feeling, so as to activate the whole communication process, which is the demand that the existing instant communication software needs to provide.

In view of the above-mentioned problems and needs, the present inventor has conducted research and improvement on instant messaging systems and communication methods based on years of experience in related industries; for this reason, the main purpose of the present invention is to provide a kind of message content that can be actively modified, An intelligent audio-visual fusion system and an implementation method thereof, which integrate information into audio-visual content for replying.

In order to achieve the above-mentioned purpose, the intelligent video-audio fusion system and its implementation method of the present invention mainly have a message processing server and at least one user communication device, and the message processing server can complete the information connection with each user communication device through the Internet , the message processing server can receive a first electronic message to be sent by the user's communication device, and after a context setting parameter set by the user, the message processing server generates a second electronic message with automatic text modification according to the context setting parameter The message and an audio-visual customer service information corresponding to the situation, the user communication device of the other communication party can play the second electronic message and the audio-visual customer service information. Furthermore, the audio-visual customer service information can be played according to the situation selected by the user. The situational customer service animation provides vivid electronic messages, and if the user cannot input a complete sentence, the message processing server can re-edit the message so that the receiver can receive a relatively complete sentence to assist The receiver understands the meaning of the sender, and the user can change the tone of the voice and display it as an audio-visual animation through the selection of the dialogue situation, so as to increase the interest of the dialogue.

In order to enable your examiners to clearly understand the purpose, technical features and effects of the present invention, the following descriptions are combined with the diagrams for illustration, please refer to.

Please refer to "FIG. 1", which is a schematic diagram of the composition of the present invention (1). As shown in the figure, the intelligent video-audio fusion system 10 of the present invention mainly has a message processing server 101, and the message processing server 101 Complete the information connection with at least one user communication device 102; the message processing server 101 can complete the information connection with each user communication device 102 through the Internet, and the message processing server 101 can exchange each user communication device 102 An electronic message to be transmitted between the two, and the message processing server 101 can capture the electronic message received and sent by the user communication device 102, and can further compare and analyze the electronic message, and re-analyze the electronic message Editing and audio-visualization are performed; the user communication device 102 can be used to input, display and play electronic information, wherein the user communication device 102 can be a smart phone or a computer, and these devices can be installed with application programs.

Please refer to "FIG. 2", which is a schematic diagram (2) of the present invention. As shown in the figure, the message processing server 101 mainly has a central processing module 1011, a message management module 1012, and a message analysis module. The module 1013, a context setting module 1014 and a storage module 1015 complete the information connection with the central processing module 1011, wherein: (1) The central processing module 1011 is used to run the information processing server 101 and drive the above modules, and has functions such as logical operation, temporary storage of operation results, and storage of execution command positions, and it can be a central a processor (Central Processing Unit, CPU) or a microcontroller (Microcontroller Unit, MCU); (2) The message management module 1012 can manage the electronic messages received and transmitted by each user communication device 102, and can transmit an audio-visual customer service information edited by the message processing server 101 to the user communication of both parties. device 102 (sender or receiver); (3) The message analysis module 1013 can capture electronic messages of both parties in communication, and can analyze the electronic messages to capture a communication relationship parameter associated with the first electronic message, the communication The relationship parameter can indicate the relationship between the two communication parties, wherein the message analysis module 1013 can further perform a word segmentation process on the first electronic message, analyze the sentence structure of the first electronic message to generate a message structure information, and then based on the set Then, edit and modify the message structure information to generate a second electronic message, wherein the message analysis module 1013 is based on natural language processing (NLP) technology, such as CKIP Chinese word segmentation system and Jieba word segmentation system, etc. , but not limited to this, according to the natural language used by the user, add a natural language processing (NLP) processing type; (4) The context setting module 1014 can receive a context setting parameter set by the user on the user communication device 102, and set an associated audio-visual customer service information according to the context setting parameter; (5) The storage module 1015 includes a message database 10151, a relationship database 10152 and a context database 10153, wherein the message database 10151 can be stored and communicated by each user communication device to receive and the electronic message sent, the relationship database 10152 stores at least one piece of communication account information, and each piece of communication account information includes a communication identification code and a communication relationship parameter, the communication identification code can identify the communication Account information, the communication relationship parameter records other user relationship data associated with the communication account information, the communication relationship parameter includes a title word information and a greeting word information, the context database 10153 stores at least A set of situational comparison parameters, and a piece of audio-visual customer service information associated with the situational comparison parameters, wherein the situational comparison parameters at least include a family situational comparison parameter, a love situational comparison parameter, a friend situational comparison parameter, etc. The situational comparison parameter can be It is added and constructed according to needs, and the audio-visual customer service information includes at least one audio-visual customer service animation with different context styles, and each audio-visual customer service information corresponds to different context comparison parameters. Furthermore, the audio-visual customer service animation system includes an audio-visual customer service animation. Video and an audio-visual customer service audio.

Please refer to "FIG. 3", which is a schematic diagram (3) of the composition of the present invention. As shown in the figure, the user communication device 102 of the present invention mainly has a microprocessor module 1021, and a message input module 1022, An instant messaging module 1023 and a message playing module 1024 complete the information connection with the microprocessor module 1021, wherein: (1) The micro-processing module 1021 can drive the above-mentioned modules, and has functions such as logical operation, temporary storage of operation results, and storage of execution instruction positions, and it can be a central processing unit (CPU). ) or a microcontroller (Microcontroller Unit, MCU); (2) The message input module 1022 can be used for the user to input an electronic message, the electronic message can be a text message or a voice message, and the message input module 1022 can be one of a virtual keyboard and a microphone or its combination; (3) The instant communication module 1023 can execute a communication application program, so that the communication device 102 and other communication devices can send messages to each other. The unit 10231 can be used by the user to set a context setting parameter and input electronic information, wherein the context setting parameter is associated with the audio-visual customer service information, and each context setting parameter can respectively correspond to the audio-visual customer service of the specified context; (4) The message playing module 1024 can be used to play the second electronic message and the corresponding audio-visual customer service information sent by the message processing server 101, so that the virtual customer service unit 10231 can pass the message playing module 1024 at the same time. Play the audio-visual customer service information and the second electronic message, wherein, the message playing module 1024 is a combination of a speaker and a display screen.

Please refer to “FIG. 4”, which shows the flow chart of the implementation of the present invention. Please refer to “FIG. 1” to “FIG. 3” in conjunction. The electronic messages sent and received by the user communication device 102 can be detected and captured, and the electronic messages can be further edited and converted into audio and video. The steps of the intelligent message reply method of the present invention are as follows: (1) Confirm the relationship between the two communication parties. Step S1: Please refer to "Fig. 5". The figure shows a schematic diagram (1) of the implementation of the present invention. As shown in the figure, when the message processing server 101 is running 101 can maintain an information connection state with the user communication device 102 and another user communication device 102', and a message management module 1012 of the message processing server 101 can continuously detect and receive the user communication device 102, as shown in the figure As shown, after another user communication device 102' serving as the sender sends a sender message D1 to the user communication device 102, the message management module 1012 can retrieve the sender message D1, and The sender message D1 is stored in a message database 10151, and after the message management module 1012 retrieves the sender message D1, the message management module 1012 will first confirm a communication identifier of the sender message D1, and Search for a communication relationship parameter associated with the communication ID from a relationship database (not shown in this figure); (2) Step S2 of Setting the Communication Scenario: Please refer to FIG. 6 , which is a schematic diagram (2) of the implementation of the present invention. As shown in the figure, when the user receives the sender message D1 through the user communication device 102 After that, the user can further drive an audio-visual customer service unit 10231 through the user communication device 102, and the user can set a context setting parameter through the audio-visual customer service unit 10231, wherein the user can select an appropriate context as the setting according to the communication object, for example , when the communication object is the wife, the user can select the family context, and after the context setting parameters are set, the user can input a first electronic message through the audio-visual customer service unit 10231, so that the user communication device 102 can send the first electronic message The message and the context setting parameters are sent to the message processing server 101 together; (3) Analyzing the first electronic message Step S3: Please refer to "Fig. 7", which is a schematic diagram of the implementation of the present invention (3), as shown in the figure, the message processing server 101 receives the first electronic message and the context After the parameters are set, a message analysis module 1013 of the message processing server 101 can perform a word segmentation process on the first electronic message to obtain at least one piece of word segmentation information, for example, the first electronic message is "coming home, traffic jam On the road, the message processing server 101 generates multiple pieces of word segmentation information according to the word segmentation operation results as “coming soon”, “coming home”, “traffic jam”, and “on the road”. After completing the word segmentation process, the message processing server 101 can generate at least one message structure information; (4) Step S4 of generating the second electronic message: After the message analysis module 1013 obtains the communication relationship parameters and the message structure information, the message analysis module 1013 can edit the message structure information based on the communication relationship parameters, wherein the message analysis module 1013 is the appellation word information and greeting word information that is added to the communication relationship parameter in the message structure information, for example, "Mrs. Wu (title) hello (honor), Mr. Wu (title) is going home soon, the traffic jam is on the road, please wait with confidence (Greeting)", after editing the message analysis module 1013, a second electronic message can be generated; (5) Audio and video customer service information setting step S5: after the message analysis module 1013 generates the second electronic message, the context setting module 1014 can query the context database 10153 for a corresponding context comparison parameter based on the context setting parameters, and retrieve An audio-visual customer service information of the matched context comparison parameter, for example, the work situation comparison parameter can be associated with the work audio-visual customer service information, wherein the work audio-visual customer service information provides a serious-style audio-visual customer service animation; (6) Publishing audio and video customer service information Step S6: Please refer to "Fig. 8", which is a schematic diagram of the implementation of the present invention (4), as shown in the figure, when the message processing server 101 completes the second electronic message and audio and video customer service Information setting, as shown in the figure, the message processing server 101 can send back the second electronic message D2 and the audio-visual customer service information D3 to another user communication device 102', and further, another user communication device 102' After receiving the second electronic message D2 and the audio-visual customer service information D3, the message playing module (not shown in the figure) of the other user communication device 102' can simultaneously play the second electronic message D2 and the audio-visual customer service information D3.

Please refer to FIG. 9, another embodiment (1) of the present invention is shown in the figure. As shown in the figure, the user communication device 102 further includes a speech recognition module 1025. The speech recognition module 1025 is connected to The microprocessor module 1021 completes the information connection, and the voice recognition module 1025 can convert the voice message into a text message. During implementation, when the first electronic message input by the user is a voice message, the voice recognition module 1025 can convert the voice message. Message conversion can be achieved through speech-to-text recognition technology (Speech To Text, STT) and semantic analysis (semantic analysis) technology. According to this, when the speech recognition module 1025 captures the first electronic message (voice), the voice The identification module 1025 can convert the first electronic message into a text message, and transmit the first electronic message and the context setting parameters to the message processing server 101 through the user communication device 102 (not shown in this figure).

Please refer to "FIG. 10", which shows another embodiment (2) of the present invention. As shown in the figure, the message processing server 101 further includes a voice setting module 1016. The voice setting module 1016 and The central processing module 1011 completes the information connection, and the voice setting module 1016 stores at least one voice setting information. The voice setting module 1016 can select the matching voice setting information based on the audio and video customer service information in different situations. S5", after the context setting module 1014 sets the audio and video customer service information, the voice setting module 1016 can retrieve the corresponding voice setting information according to the set audio and video customer service information, and, in the "release audio and video customer service information step S6", The message processing server 101 can transmit the second electronic message together with the audio and video customer service information and the voice setting information to another user communication device 102', so that the message playing module of the other user communication device 102' can receive the above information Afterwards, it can be played based on the playback timeline of the voice setting information and the audio-visual customer service information; wherein, the voice setting information can set a situation, and pre-record voice files with different tones to meet various usage situations.

Please refer to "FIG. 11", which shows another embodiment (3) of the present invention. As shown in the figure, the instant messaging module 1023 of the user communication device 102 further includes a voice synchronization unit 10232. The voice The synchronization unit 10232 can be used by the user to set the analog voice request. After the user has set the context setting parameters through the audio-visual customer service unit 10231 and completed the input of the first electronic message, the user communication device 102 can send the analog voice request to the user communication device 102 and the first electronic message are sent to the message processing server 101, in "analyzing the first electronic message step S3" ~ "generating the second electronic message step S4", the message analysis module 1013 can skip the "word segmentation process" and " Steps such as adding communication relationship parameters", the message analysis module 1013 maintains the integrity of the first electronic message based on the analog voice request, and further, after another user communication device 102' receives the first reply message and the audio-visual customer service information, The other user communication device 102' can receive the voice messages of the other party using different tones, and match the set video and audio customer service animations.

As can be seen from the above, the intelligent video-audio fusion system of the present invention and the implementation method thereof mainly have a message processing server and at least one user communication device, and the message processing server can receive the information sent by the user communication device. A first electronic message, and the message processing server can generate a second electronic message with automatic text modification and an audio-visual customer service information corresponding to the context based on a set context setting parameter, and finally combine the second electronic message with the context. The audio and video customer service information is transmitted to the user communication device of the other communication party for playback; therefore, after the implementation of the present invention, it is possible to provide an intelligent system that can actively modify the content of the message and integrate the message into the audio and video content for replying. The purpose of an audio-visual fusion system and its implementation method.

However, the above descriptions are only preferred embodiments of the present invention, and are not intended to limit the scope of the present invention; anyone familiar with the art can make equal changes and modifications without departing from the spirit and scope of the present invention. , all should be covered within the patent scope of the present invention.

To sum up, the effect of the present invention is that it has the patent requirements of "industrial applicability", "novelty" and "progressiveness" of the invention; the applicant should file an invention patent with the Jun Bureau in accordance with the provisions of the Patent Law Apply.

10: Intelligent audio-visual fusion system 101: message processing server 1011: Central Processing Module 1012: Message Management Module 1013: Message Analysis Module 1014: Situation setting module 1015: Storage Module 10151: Message Database 10152: Relational database 10153: Context Database 1016: Voice setting module 102: User Communication Device 1021: Microprocessor Module 1022: Message input module 1023: Instant Messaging Module 10231: Audio and video customer service unit 10232: Voice synchronization unit 1024: Message playback module 1025: Speech Recognition Module S1: Steps to confirm the relationship between the communication parties S2: Steps to set the communication situation S3: step of analyzing the first electronic message S4: Step of generating a second electronic message S5: Video customer service information setting steps S6: Steps for releasing audio and video customer service information

Figure 1 is a schematic diagram (1) of the composition of the present invention. Figure 2 is a schematic diagram (2) of the composition of the present invention. Fig. 3 is a schematic diagram (3) of the composition of the present invention. FIG. 4 is a flow chart of the implementation of the present invention. FIG. 5 is a schematic diagram (1) of the implementation of the present invention. FIG. 6 is a schematic diagram (2) of the implementation of the present invention. FIG. 7 is a schematic diagram (3) of the implementation of the present invention. Fig. 8 is a schematic diagram (4) of the implementation of the present invention. FIG. 9 is another embodiment (1) of the present invention. Fig. 10 is another embodiment (2) of the present invention. FIG. 11 is another embodiment (3) of the present invention.

S1: Steps to confirm the relationship between the communication parties

S2: Steps to set the communication situation

S3: step of analyzing the first electronic message

S4: Step of generating a second electronic message

S5: Video customer service information setting steps

S6: Steps for releasing audio and video customer service information

Claims (10)

An implementation method of an intelligent audio-visual fusion system, comprising: A step of confirming the relationship between the two communication parties: after a message processing server detects and captures a sender message received by a user communication device, the message processing server first retrieves a communication identifier of the sender message code, and confirm a communication relationship parameter of the communication identification code; A step of setting communication context: after the user communication device completes the setting of a context setting parameter and a first electronic message, the user communication device sends the context setting parameter and a first electronic message to the message processing server; A step of analyzing the first electronic message: the message processing server performs word segmentation processing on the first electronic message to obtain at least one keyword, and then compares each keyword with a keyword database, so that the message processing server Generate a segmented word information based on the comparison result; A step of generating a second electronic message: after the message processing server obtains the communication relationship parameter and the word segmentation information, the message processing server edits the word segmentation information based on the communication relationship parameter to generate a second electronic message message; An audio-visual customer service information setting step: the message processing server selects an audio-visual customer service information based on the situation setting parameter; and A step of publishing video and audio customer service information: the message processing server transmits the second electronic message and the audio and video customer service information to another user communication device for playback. The implementation method of the intelligent video-audio fusion system of claim 1, wherein the message processing server performs word segmentation processing on the first electronic message to generate a message structure information, so that the message processing server can use the communication relationship parameters, and The message structure information is edited into the second electronic message. The implementation method of the intelligent video-audio fusion system of claim 1, wherein the message processing server receives the context setting parameter, compares a context database with the context comparison parameter corresponding to the context setting parameter, and the message processing server Set the video customer service information according to the comparison result. The implementation method of the intelligent video-audio fusion system of claim 1, wherein, in the "audio-video customer service information setting step", after the message processing server completes the setting of the video-audio customer service information, the message processing server is based on the video-audio customer service information. Retrieve a corresponding voice setting information, and in the "publishing audio and video customer service information step", the message processing server transmits the second electronic message, the audio and video customer service information and the voice setting information to another user together Communication device playback. An intelligent audio-visual fusion system, which includes: a user communication device capable of driving an instant messaging module, the instant messaging module including an audio-visual customer service unit capable of inputting a first electronic message and setting a situation setting parameter; A message processing server completes an information connection with the user communication device, the message processing server captures the first electronic message, analyzes a communication relationship parameter of the first electronic message, and generates the first electronic message based on the communication relationship parameter. The electronic message is re-edited into a second electronic message, and the message processing server sets an audio-visual customer service information based on the situation setting parameter; and The message processing server transmits the processed second electronic message and the audio-visual customer service information to another user communication device that communicates with the user communication device, and then plays the second electronic message and the audio-visual customer service information . The intelligent video-audio fusion system of claim 5, wherein the user communication device includes a voice recognition module, and the voice recognition module and the processing module complete information connection. The intelligent video-audio fusion system according to claim 5, wherein the message processing server performs word segmentation on the first electronic message to generate message structure information, so that the message processing server can use the communication relationship parameters and the message The structural information is edited into the second electronic message. The intelligent video-audio fusion system of claim 5, wherein the message processing server includes a storage module, the storage module has a context database, the context database stores at least one context comparison parameter, each context The comparison parameter corresponds to the audio-visual customer service information of the relevant situation. The intelligent video-audio fusion system of claim 8, wherein the message processing server includes a context setting module, the context setting module receives the context setting parameter, and compares the context database with the context setting parameter The context comparison parameter is set, and the context setting module sets the audio-visual customer service information according to the comparison result. The intelligent video-audio fusion system of claim 5, wherein the message processing server captures a communication identification code to confirm the communication relationship parameter of the communication identification code.

Images