CN113727051A - Bidirectional video method, system, equipment and storage medium based on virtual agent - Google Patents
Bidirectional video method, system, equipment and storage medium based on virtual agent Download PDFInfo
- Publication number
- CN113727051A CN113727051A CN202111017617.0A CN202111017617A CN113727051A CN 113727051 A CN113727051 A CN 113727051A CN 202111017617 A CN202111017617 A CN 202111017617A CN 113727051 A CN113727051 A CN 113727051A
- Authority
- CN
- China
- Prior art keywords
- voice
- virtual
- video
- text
- response
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 65
- 230000002457 bidirectional effect Effects 0.000 title claims abstract description 44
- 230000004044 response Effects 0.000 claims abstract description 99
- 230000008569 process Effects 0.000 claims description 22
- 238000004590 computer program Methods 0.000 claims description 11
- 230000011664 signaling Effects 0.000 claims description 5
- 230000003993 interaction Effects 0.000 abstract description 20
- 238000010586 diagram Methods 0.000 description 7
- 238000004891 communication Methods 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 3
- 230000015572 biosynthetic process Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000003786 synthesis reaction Methods 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000001574 biopsy Methods 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 230000002194 synthesizing effect Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/14—Systems for two-way working
- H04N7/141—Systems for two-way working between two video terminals, e.g. videophone
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L13/00—Speech synthesis; Text to speech systems
- G10L13/02—Methods for producing synthetic speech; Speech synthesisers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/60—Network streaming of media packets
- H04L65/65—Network streaming protocols, e.g. real-time transport protocol [RTP] or real-time control protocol [RTCP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/02—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/76—Television signal recording
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/14—Systems for two-way working
- H04N7/141—Systems for two-way working between two video terminals, e.g. videophone
- H04N7/147—Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals
Abstract
The invention discloses a bidirectional video method, a system, equipment and a storage medium based on virtual seats, wherein the method comprises the following steps: the virtual seat responds to a video recording instruction sent by the H5 front end and obtains a phonetics text sent by the H5 front end, and synthesized voice is obtained according to the phonetics text; the virtual seat sends the synthesized voice to the front end of H5 through a bidirectional video channel, and plays the synthesized voice through the front end of H5; the virtual seat receives and identifies a voice response obtained according to the synthesized voice, and acquires a response text generated according to the voice response; and the virtual seat sends the response text to the H5 front end, responds to the video recording ending instruction sent by the H5 front end and acquires a recorded video. By the method, the audio and video of the virtual seat and the audio and video of the H5 front end for intelligent voice interaction are recorded by the H5 front end, so that the virtual seat obtains the audio and video recorded by the H5 front end, and the problem that the H5 front end cannot complete video recording and intelligent voice interaction for opening a witness at the front end is solved.
Description
Technical Field
The present invention relates to the field of communications technologies, and in particular, to a bidirectional video method, system, device, and storage medium based on a virtual agent.
Background
At present, with the release of the one-way video account opening policy, the two-way video witness problem which troubles a plurality of dealer for years is finally broken through.
Under the one-way video witnesses, the investors can record one-way videos in a self-service mode in the account opening process without waiting for manual witnesses. Especially in the time period with centralized account opening amount, queuing and waiting in the account opening process are not needed. For security dealer, the one-way video witness also saves the labor cost to a certain extent.
The intelligent voice interaction is a common interaction mode in account-opening witnesses; the system automatically broadcasts the question of needing to confirm the client's account opening intention at the account opening front end, the client answers (yes) or (no) according to the prompt, and the system performs voice recognition to confirm whether the answer is passed. However, limited by the processing power of the H5 front end, the H5 one-way video temporarily fails to complete video recording and intelligent voice interaction of the account-opening witnesses at the front end.
Disclosure of Invention
Therefore, the present invention is directed to solve the deficiencies in the prior art at least to some extent, and therefore to provide a bidirectional video method and system, device and storage medium based on virtual agents.
In a first aspect, the present invention provides a bidirectional video method based on virtual agents, the method comprising:
the virtual seat responds to a video recording instruction sent by the H5 front end, acquires a phonetics text sent by the H5 front end, and obtains a synthesized voice according to the phonetics text;
the virtual seat sends the synthesized voice to the H5 front end through a bidirectional video channel, and plays the synthesized voice through the H5 front end;
the virtual seat receives and identifies a voice response obtained according to the synthesized voice and acquires a response text generated according to the voice response;
and the virtual seat sends the response text to the H5 front end, responds to a video recording ending instruction sent by the H5 front end and acquires a recorded video.
In a second aspect, the present invention provides a virtual agent based two-way video system, the system comprising:
an acquisition module: the virtual agent obtains a phonetics text sent by the front end of H5, and obtains a synthesized voice according to the phonetics text;
a playing module: for the virtual agent to send the synthesized speech to the H5 front end over a bi-directional video channel and play the synthesized speech based on the H5 front end;
a generation module: the virtual seat is used for receiving and identifying a voice response obtained according to the synthesized voice and acquiring a response text generated according to the voice response;
a sending module: for the virtual agent to send the answer text to the H5 front end.
In a third aspect, the present invention further provides a virtual agent-based bidirectional video device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, the processor implements the steps in the virtual agent-based bidirectional video method according to the first aspect.
In a fourth aspect, the present invention also provides a storage medium having stored thereon a computer program which, when executed, performs the steps of the virtual agent-based bidirectional video method according to the first aspect.
The invention provides a bidirectional video method based on virtual seats, which comprises the following steps: the virtual seat responds to a video recording instruction sent by the H5 front end and obtains a phonetics text sent by the H5 front end, and synthesized voice is obtained according to the phonetics text; the virtual seat sends the synthesized voice to the front end of H5 through a bidirectional video channel, and plays the synthesized voice through the front end of H5; the virtual seat receives and identifies a voice response obtained according to the synthesized voice, and acquires a response text generated according to the voice response; and the virtual seat sends the response text to the H5 front end, responds to the video recording ending instruction sent by the H5 front end and acquires a recorded video. By the method, based on intelligent voice interaction between the virtual seat and the H5 front end, the audio and video of the intelligent voice interaction between the virtual seat and the H5 front end is recorded by the H5 front end, so that the virtual seat obtains the audio and video recorded by the H5 front end, and the problem that the H5 front end cannot complete video recording and intelligent voice interaction for evidence of opening an account at the front end is solved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the structures shown in the drawings without creative efforts.
FIG. 1 is a schematic flow chart of a virtual agent-based two-way video method according to the present invention;
FIG. 2 is a sub-flow diagram of the virtual agent based bi-directional video method of the present invention;
FIG. 3 is another sub-flow diagram of the virtual agent based bi-directional video method of the present invention;
FIG. 4 is a schematic view of another sub-flow of the virtual agent-based two-way video method of the present invention;
FIG. 5 is a block diagram of the present invention showing the program modules of the virtual agent based two-way video system.
Detailed Description
In order to make the objects, features and advantages of the present invention more obvious and understandable, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is apparent that the described embodiments are only a part of the embodiments of the present invention, and not all the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, fig. 1 is a schematic flowchart of a bidirectional video method based on a virtual agent according to an embodiment of the present application, where in the embodiment, the bidirectional video method based on a virtual agent includes:
In this embodiment, after the virtual agent is connected with the H5 front end, the H5 front end sends a video recording instruction to the virtual agent, the virtual agent starts recording after responding to the video recording instruction, the H5 front end sends a dialog text to the virtual agent, and the virtual agent receives the dialog text sent by the H5 front end, where the processing capability of the H5 front end is limited, and the video recording and intelligent voice interaction of the evidence of opening an account cannot be completed at the front end in the process of opening an account by an investor, so that the audio and video collected by the H5 front end is transmitted to the back end, and the virtual agent completes related functions, so the H5 front end sends the dialog text to the virtual agent. The front end of H5 is a technology set, which includes HTML5, CSS3, JS and other technologies; the virtual agent is a virtual role obtained through computer generation, animation and artificial intelligence, provides services for customers through the function of the chat robot, and can answer questions posed by the customers and provide information about products and services of companies.
In this embodiment, after the virtual agent receives the dialogistic text sent by the front end of H5, the virtual agent processes the dialogistic text to finally obtain the processed dialogistic text.
And 102, the virtual seat sends the synthesized voice to the front end of the H5 through a bidirectional video channel, and plays the synthesized voice through the front end of the H5.
In this embodiment, after the virtual seat receives the synthesized voice returned by the intelligent voice end, the virtual seat sends the synthesized voice to the H5 front end through the bidirectional video channel, and after the H5 front end receives the synthesized voice, the synthesized voice is played through the H5 front end, wherein the bidirectional video channel plays a role in mutual communication, so that the virtual seat and the H5 front end perform an interaction.
And 103, receiving and identifying the voice response obtained according to the synthesized voice by the virtual seat, and acquiring a response text generated according to the voice response.
In this embodiment, in step 102, after the H5 front end plays the synthesized voice, and after the user knows the synthesized voice played by the H5 front end during the process of interacting with the H5 front end, the user sends a voice response according to the synthesized voice, the H5 front end collects the voice response of the user and sends the voice response to the virtual seat, and the virtual seat receives the voice response, recognizes the voice response, and generates a response text according to the voice response.
And step 104, the virtual seat sends the response text to the H5 front end, responds to a video recording ending instruction sent by the H5 front end, and obtains a recorded video.
In this embodiment, the virtual agent sends the response text obtained in step 103 to the front end of H5, so that the virtual agent forms an interaction with the front end of H5, the user sends a voice response, the virtual agent obtains a response text according to the voice response and sends the response text to the front end of H5, and the front end of H5 delivers the information of the response text to the user.
In this embodiment, the H5 front end sends a video recording ending instruction to the virtual seat, the virtual seat responds to the video recording ending instruction sent by the H5 front end, and obtains a video recorded by the H5 front end, and by obtaining a video file of the intelligent voice interaction between the user and the virtual seat, it is effectively verified that the witness body of the video is a client, and the recorded video is retained.
The embodiment of the application provides a bidirectional video method based on a virtual agent, which comprises the following steps: the virtual seat responds to a video recording instruction sent by the H5 front end and obtains a phonetics text sent by the H5 front end, and synthesized voice is obtained according to the phonetics text; the virtual seat sends the synthesized voice to the front end of H5 through a bidirectional video channel, and plays the synthesized voice through the front end of H5; the virtual seat receives and identifies a voice response obtained according to the synthesized voice, and acquires a response text generated according to the voice response; and the virtual seat sends the response text to the H5 front end, responds to the video recording ending instruction sent by the H5 front end and acquires a recorded video. By the method, based on intelligent voice interaction between the virtual seat and the H5 front end, the audio and video of the intelligent voice interaction between the virtual seat and the H5 front end is recorded by the H5 front end, so that the virtual seat obtains the audio and video recorded by the H5 front end, and the problem that the H5 front end cannot complete video recording and intelligent voice interaction for evidence of opening an account at the front end is solved.
Further, after the virtual agent sends the response text to the H5 front end, the method further includes:
and logically processing the response text based on the H5 front end.
In this embodiment, after the virtual agent sends the response text to the front end of H5, the front end of H5 performs a logic process on the response text, and after the front end of H5 performs a logic process on the response cost, the response text after the logic process is completed is transmitted to the user, where the transmission mode may be voice, text, and the like, which is not limited.
Further, the virtual agent acquires the recorded video recorded at the front end of the H5 through a C + + audio/video export interface of the WebRTC.
In this embodiment, the video recording is to mix audio for intelligent voice interaction between the user and the virtual seat, and to merge the mixed audio and the user video into a video file, that is, a video file recorded at the front end of H5. The virtual seat realizes the acquisition of internal audio and video data by a method of registering audio and video callback; the virtual seat realizes the acquisition of external audio and video data by replacing the acquisition source corresponding to the audio and video track in the media stream.
Further, the virtual agent establishes the bidirectional video channel and the bidirectional signaling channel with the H5 front end based on the WebRTC, and the sending of the response text by the virtual agent to the H5 front end specifically includes:
and the virtual seat sends the response text to the H5 front end through the bidirectional signaling channel.
In this embodiment, the virtual agent implements a bidirectional video based on WebRTC (Web Real-Time Communication) and an H5 front end, and the virtual agent and the H5 front end implement an audio-video interaction process, where the H5 front end can be applied to a mobile end, such as a mobile end with an apple system and a mobile end with an android system, and the mobile ends all use browsers supporting the WebRTC, recommend that the mobile end with the apple system uses a Safari browser, and the mobile end with the android system uses QQ, wechat, and the like. The virtual seat realizes a set of PeerConnection (peer-to-peer connection) process based on the C + + version of the WebRTC source code. In this process, the virtual agent has the capability of performing two-way video communication with the H5 front end, where the WebRTC is an API (Application Programming Interface) that supports a web browser to perform real-time voice conversation or video conversation.
Further, the sending, by the virtual agent, the synthesized speech to the front end of the H5 through the bidirectional video channel specifically includes:
and the virtual seat sends the obtained synthesized voice to the H5 front end in real time.
In this embodiment, after the virtual agent processes the dialog text by calling the interface of the intelligent voice end, the virtual agent does not wait for the intelligent voice end to process all the dialog text, and then sends the processed synthesized voice to the front end of H5, but when the virtual agent receives the synthesized voice data sent by the intelligent voice end, the synthesized voice data is sent to the front end of H5 synchronously, so that the waiting problem in the process of synthesizing voice caused by long length of the dialog text is avoided.
Further, referring to fig. 2, fig. 2 is a schematic sub-flow diagram of a bidirectional video method based on a virtual agent in the embodiment of the present application, in this embodiment, a virtual agent responds to a video recording instruction sent by an H5 front end and obtains a phonetics text sent by the H5 front end, and obtaining a synthesized voice according to the phonetics text specifically includes:
In this embodiment, the H5 front end sends a dialogistic text to the virtual seat, after the virtual seat receives the dialogistic text, the interface of the intelligent voice end is called to process the dialogistic text, after the intelligent voice end obtains a synthesized voice, the obtained synthesized voice is returned to the virtual seat, that is, the intelligent voice end sends an answer to the dialogistic text to the virtual seat, and finally the virtual seat obtains the synthesized voice after the intelligent voice end processes.
Further, referring to fig. 3, fig. 3 is another schematic sub-flow diagram of a bidirectional video method based on a virtual agent in the embodiment of the present application, where in this embodiment, the receiving and recognizing, by the virtual agent, a voice response obtained according to the synthesized voice, and acquiring a response text generated according to the voice response specifically include:
In this embodiment, after the H5 front end plays the synthesized voice, the user forms a voice response for the played synthesized voice, the H5 front end collects the voice response of the user again and sends the voice response of the user to the virtual seat, after the virtual seat receives the voice response of the user, the interface of the intelligent voice end is called to recognize the voice response of the user, after the intelligent voice end recognizes the voice response, a response text is obtained, the response text is returned to the virtual seat, and finally the virtual seat obtains the response text.
Further, referring to fig. 4, fig. 4 is a schematic sub-flow diagram of a bidirectional video method based on a virtual agent in this embodiment of the present application, where in this embodiment, the receiving and recognizing, by the virtual agent, a voice response obtained according to the synthesized voice, and acquiring a response text generated according to the voice response further include:
and step 402, the virtual seat sends a preset recording to the front end of the H5 according to the fixed telephone operation.
In the embodiment, the virtual seat calls the interface of the intelligent voice end to synthesize the recording aiming at the fixed telephone operation proposed by the user in advance, the recording is sent to the virtual seat after the intelligent voice end synthesizes the recording, and when the virtual seat meets the fixed telephone operation proposed by the user, the recording preset in advance is directly sent, so that unnecessary synthesis requests can be reduced.
Further, the specific implementation steps of the embodiment of the application include:
1. the front end of H5 carries out video recording, the virtual seat processes the fixed telephone operation in advance, the fixed telephone operation is processed through the interface of the intelligent voice end, and after the intelligent voice end processes the fixed telephone operation, a recording is obtained and returned to the virtual seat;
2. the front end of H5 sends a speech text to the virtual seat, after the virtual seat receives the speech text, the interface of the intelligent voice end is called to process the speech text to obtain a synthetic voice, the intelligent voice end returns the synthetic voice obtained after processing to the virtual seat, and if the virtual seat receives a fixed speech, the virtual seat directly sends a recording to the front end of H5;
3. after receiving the synthesized voice returned by the intelligent voice end, the virtual seat is sent to the H5 front end in real time through a bidirectional video channel between the virtual seat and the H5 front end, and the synthesized voice is played through the H5 front end;
4. after the front end of H5 plays the synthesized voice, the user transmits a voice response to the front end of H5 according to the obtained synthesized voice, the front end of H5 collects the voice response transmitted by the user and then sends the voice response to the virtual seat, the virtual seat calls an interface of the intelligent voice end to perform voice recognition on the voice response after receiving the voice response, and the intelligent voice end processes the voice response to obtain a response text and returns the response text to the virtual seat;
5. after receiving the response text returned by the intelligent voice end, the virtual seat is sent to the H5 front end through a bidirectional signaling channel, the H5 front end performs logic processing on the response text and transmits the response text after the logic processing to a user, the H5 front end finishes recording videos, and the virtual seat obtains audios and videos recorded by the H5 front end.
Further, in this embodiment of the present application, before recording the video, the method further includes:
1. detecting a living body;
verifying that the main body of the video witness is a real person instead of a photo through biopsy action, and meanwhile, outputting a user positive photo and a living body key frame;
2. comparing human faces;
the main body of the witness in the video is verified to be the principal by comparing the positive photo of the user, the living key frame and the identity card photo.
Further, in the embodiment of the present application, the logic architecture of the unidirectional video includes:
1. user layer
The mobile terminal (front end of an android H5 and front end of an apple H5) uses a browser supporting WebRTC to access, and recommends the android to use QQ and WeChat; apple uses a Safari browser.
2. Application layer
Video basic capability (video communication, video recording), voice basic capability (voice synthesis, voice recognition), face basic capability (live body detection, face comparison) are provided.
3. Service layer
Providing corresponding service support for an application layer; the method comprises the following steps: video basic service, voice basic service, face basic service.
4. Storage layer
And the picture storage and video storage capacity is provided.
Further, an embodiment of the present application further provides a bidirectional video system 500 based on a virtual agent, and fig. 5 is a schematic diagram of program modules of the bidirectional video system based on a virtual agent in the embodiment of the present application, in which the bidirectional video system 500 based on a virtual agent includes:
the obtaining module 501: the virtual seat is used for responding to a video recording instruction sent by the front end of H5, acquiring a phonetics text sent by the front end of H5 and obtaining synthesized voice according to the phonetics text;
the playing module 502: for the virtual agent to send the synthesized speech to the H5 front end over a bi-directional video channel and play the synthesized speech based on the H5 front end;
the generation module 503: the virtual seat is used for receiving and identifying a voice response obtained according to the synthesized voice and acquiring a response text generated according to the voice response;
the sending module 504: and the virtual seat is used for sending the response text to the H5 front end, responding to a video recording ending instruction sent by the H5 front end and acquiring a recorded video.
The embodiment of the present application provides a bidirectional video system 500 based on virtual agents, which can implement: the virtual seat responds to a video recording instruction sent by the H5 front end and obtains a phonetics text sent by the H5 front end, and synthesized voice is obtained according to the phonetics text; the virtual seat sends the synthesized voice to the front end of H5 through a bidirectional video channel, and plays the synthesized voice through the front end of H5; the virtual seat receives and identifies a voice response obtained according to the synthesized voice, and acquires a response text generated according to the voice response; and the virtual seat sends the response text to the H5 front end, responds to the video recording ending instruction sent by the H5 front end and acquires a recorded video. By the method, based on intelligent voice interaction between the virtual seat and the H5 front end, the audio and video of the intelligent voice interaction between the virtual seat and the H5 front end is recorded by the H5 front end, so that the virtual seat obtains the audio and video recorded by the H5 front end, and the problem that the H5 front end cannot complete video recording and intelligent voice interaction for evidence of opening an account at the front end is solved.
Further, the present application also provides a virtual agent-based bidirectional video device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor implements the steps in the virtual agent-based bidirectional video method when executing the computer program.
Further, the present application also provides a storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the virtual agent based bidirectional video method as described above.
Each functional module in the embodiments of the present invention may be integrated into one processing module, or each module may exist alone physically, or two or more modules are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. The integrated module, if implemented in the form of a software functional module and sold or used as a separate product, may be stored in a computer readable storage medium.
Based on such understanding, the technical solution of the present invention, which is described in the specification or contributes to the prior art in essence, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
It should be noted that, for the sake of simplicity, the above-mentioned method embodiments are described as a series of acts or combinations, but those skilled in the art should understand that the present invention is not limited by the described order of acts, as some steps may be performed in other orders or simultaneously according to the present invention. Further, those skilled in the art will appreciate that the embodiments described in the specification are presently preferred and that no acts or modules are necessarily required of the invention. In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
For those skilled in the art, according to the idea of the embodiments of the present application, there may be variations in the specific implementation and application scope, and in summary, the content of the present description should not be construed as a limitation to the present invention.
Claims (10)
1. A bidirectional video method based on virtual agents, characterized in that the method comprises:
the virtual seat responds to a video recording instruction sent by the H5 front end, acquires a phonetics text sent by the H5 front end, and obtains a synthesized voice according to the phonetics text;
the virtual seat sends the synthesized voice to the H5 front end through a bidirectional video channel, and plays the synthesized voice through the H5 front end;
the virtual seat receives and identifies a voice response obtained according to the synthesized voice and acquires a response text generated according to the voice response;
and the virtual seat sends the response text to the H5 front end, responds to a video recording ending instruction sent by the H5 front end and acquires a recorded video.
2. The method according to claim 1, wherein the virtual agent obtains the recorded video recorded at the front end of the H5 through a C + + audio video export interface of WebRTC.
3. The method of claim 1, wherein the virtual agent establishes the bidirectional video channel and the bidirectional signaling channel with the H5 front end based on the WebRTC, and the virtual agent sending the answer text to the H5 front end specifically comprises:
and the virtual seat sends the response text to the H5 front end through the bidirectional signaling channel.
4. The method according to claim 1, wherein the virtual agent sending the synthesized speech to the H5 front end through a bidirectional video channel specifically comprises:
and the virtual seat sends the obtained synthesized voice to the H5 front end in real time.
5. The method according to claim 1, wherein the virtual agent responds to a video recording instruction sent by an H5 front end and obtains a dialect text sent by the H5 front end, and obtaining the synthesized speech according to the dialect text specifically comprises:
the virtual seat acquires the dialect text sent by the H5 front end;
and the virtual seat calls an interface of an intelligent voice end to process the dialect text to obtain the synthesized voice.
6. The method according to claim 5, wherein the virtual agent receives and recognizes a voice response obtained from the synthesized voice, and the obtaining of the response text generated from the voice response specifically comprises:
the virtual seat receives a voice response obtained by the H5 front end according to the synthesized voice;
and the virtual seat calls an interface of the intelligent voice end to identify the voice response to obtain the response text.
7. The method of claim 1, wherein the virtual agent receiving and recognizing a voice response from the synthesized voice and obtaining a response text generated from the voice response further comprises:
the virtual seat receives and identifies the fixed telephone operation sent by the H5 front end;
and the virtual seat sends a preset recording to the front end of the H5 according to the fixed telephone operation.
8. A virtual agent based two-way video system, the system comprising:
an acquisition module: the virtual seat is used for responding to a video recording instruction sent by the front end of H5, acquiring a phonetics text sent by the front end of H5 and obtaining synthesized voice according to the phonetics text;
a playing module: for the virtual agent to send the synthesized speech to the H5 front end over a bi-directional video channel and play the synthesized speech based on the H5 front end;
a generation module: the virtual seat is used for receiving and identifying a voice response obtained according to the synthesized voice and acquiring a response text generated according to the voice response;
a sending module: and the virtual seat is used for sending the response text to the H5 front end, responding to a video recording ending instruction sent by the H5 front end and acquiring a recorded video.
9. A virtual agent based bi-directional video device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor when executing the computer program performs the steps of the virtual agent based bi-directional video method of any of claims 1-7.
10. A storage medium having stored thereon a computer program, wherein the computer program, when executed by a processor, performs the steps of the virtual agent based bi-directional video method of any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111017617.0A CN113727051A (en) | 2021-08-31 | 2021-08-31 | Bidirectional video method, system, equipment and storage medium based on virtual agent |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111017617.0A CN113727051A (en) | 2021-08-31 | 2021-08-31 | Bidirectional video method, system, equipment and storage medium based on virtual agent |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113727051A true CN113727051A (en) | 2021-11-30 |
Family
ID=78680169
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111017617.0A Pending CN113727051A (en) | 2021-08-31 | 2021-08-31 | Bidirectional video method, system, equipment and storage medium based on virtual agent |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113727051A (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109618068A (en) * | 2018-11-08 | 2019-04-12 | 上海航动科技有限公司 | A kind of voice service method for pushing, device and system based on artificial intelligence |
| KR20200092272A (en) * | 2019-01-24 | 2020-08-03 | 주식회사 브리지텍 | Metho and apparstus for providing for call center service |
| CN111885273A (en) * | 2020-07-24 | 2020-11-03 | 南京易米云通网络科技有限公司 | Man-machine cooperation controllable intelligent voice outbound method and intelligent outbound robot platform |
| CN112017056A (en) * | 2020-10-26 | 2020-12-01 | 广州佰锐网络科技有限公司 | Intelligent double-recording method and system |
| WO2021135548A1 (en) * | 2020-06-05 | 2021-07-08 | 平安科技（深圳）有限公司 | Voice intent recognition method and device, computer equipment and storage medium |
-
2021
- 2021-08-31 CN CN202111017617.0A patent/CN113727051A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109618068A (en) * | 2018-11-08 | 2019-04-12 | 上海航动科技有限公司 | A kind of voice service method for pushing, device and system based on artificial intelligence |
| KR20200092272A (en) * | 2019-01-24 | 2020-08-03 | 주식회사 브리지텍 | Metho and apparstus for providing for call center service |
| WO2021135548A1 (en) * | 2020-06-05 | 2021-07-08 | 平安科技（深圳）有限公司 | Voice intent recognition method and device, computer equipment and storage medium |
| CN111885273A (en) * | 2020-07-24 | 2020-11-03 | 南京易米云通网络科技有限公司 | Man-machine cooperation controllable intelligent voice outbound method and intelligent outbound robot platform |
| CN112017056A (en) * | 2020-10-26 | 2020-12-01 | 广州佰锐网络科技有限公司 | Intelligent double-recording method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10262195B2 (en) | Predictive and responsive video analytics system and methods | |
| CN110661927B (en) | Voice interaction method and device, computer equipment and storage medium | |
| CN102017513B (en) | Method for real time network communication as well as method and system for real time multi-lingual communication | |
| JP4167057B2 (en) | Speech recognition method and system for determining the status of outgoing telephone calls | |
| US8718262B2 (en) | Method and system for automatically routing a telephonic communication base on analytic attributes associated with prior telephonic communication | |
| US8868657B2 (en) | Method and system for generating a collaboration timeline illustrating application artifacts in context | |
| US8005676B2 (en) | Speech analysis using statistical learning | |
| CN112233690B (en) | Double recording method, device, terminal and storage medium | |
| US8983055B1 (en) | Quality review of contacts between customers and customer service agents | |
| CN109417583A (en) | It is a kind of by audio signal real time transcription be text system and method | |
| CN112836037A (en) | Method and device for recommending dialect | |
| CN110536029B (en) | Interaction method, network side equipment, terminal equipment, storage medium and system | |
| WO2017172655A1 (en) | Analysis of a facial image to extract physical and emotional characteristics of a user | |
| CN111866544B (en) | Data processing method, device, equipment and computer readable storage medium | |
| US20060265088A1 (en) | Method and system for recording an electronic communication and extracting constituent audio data therefrom | |
| CN109150556A (en) | More people's teleconferences based on speech recognition record system | |
| CN113727051A (en) | Bidirectional video method, system, equipment and storage medium based on virtual agent | |
| CN113079387A (en) | Remote video recording method, device, equipment and storage medium | |
| US10580410B2 (en) | Transcription of communications | |
| US11830120B2 (en) | Speech image providing method and computing device for performing the same | |
| CN112995568B (en) | Customer service system based on video and construction method | |
| US20240046540A1 (en) | Speech image providing method and computing device for performing the same | |
| CN111968632B (en) | Call voice acquisition method, device, computer equipment and storage medium | |
| US20220300993A1 (en) | System and method for conducting a survey by a survey bot | |
| CN115914673A (en) | Compliance detection method and device based on streaming media service |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |