CN110418099B

CN110418099B - Audio and video processing method and device and storage medium

Info

Publication number: CN110418099B
Application number: CN201811000798.4A
Authority: CN
Inventors: 任旻
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2018-08-30
Filing date: 2018-08-30
Publication date: 2021-08-31
Anticipated expiration: 2038-08-30
Also published as: CN110418099A

Abstract

The embodiment of the invention discloses an audio and video processing method, an audio and video processing device and a storage medium, wherein the embodiment of the invention can receive a reservation request sent by a sender, determine meeting time selected by the sender according to the reservation request and generate a link address corresponding to an audio and video meeting; adding the sender to the audio and video conference within the conference time; receiving a joining request sent by a receiver through a third-party platform based on the link address in the conference time, and joining the receiver into the audio and video conference according to the joining request; and sending the audio and video data generated in the audio and video conference to the sender and the receiver through the third-party platform. According to the scheme, the sender and the receiver can join the same audio and video conference through different platforms to carry out audio and video data interaction, and the flexibility and the convenience of audio and video data transmission are improved.

Description

Audio and video processing method and device and storage medium

Technical Field

The invention relates to the technical field of internet, in particular to an audio and video processing method, an audio and video processing device and a storage medium.

Background

With the development of internet technology, users not only meet the communication and social contact of characters, but also have stronger and stronger requirements on real-time voice and video, so that the users are more and more popularized in information interaction through audio and video conferences, and can join different audio and video conferences through clients according to own requirements so as to perform information interaction in the audio and video conferences.

In the prior art, multiple users need to join an audio and video conference through the same client, so that audio and video data interaction is performed in the same audio and video conference. For example, a user a and a user B may install an Instant Messaging (IM) application equivalent to a wechat on a terminal, and then both the user a and the user B may use an account registered on a wechat platform through the wechat application, and a wechat friend needs to be added between the user a and the user B, and at this time, an audio and video conference may be established through the wechat account to perform a video call, and the like. Or, the user a and the user B may install the web conference application on the terminal, and then both the user a and the user B register the account on the web conference application, and establish the audio and video conference through the account to perform the video call at the specified time. Therefore, unified applications need to be installed among different users, unified account numbers need to be registered, and an audio and video conference can be established for video call.

In the research and practice process of the prior art, the inventor of the invention finds that when different users carry out audio and video data interaction, the users need to install uniform application and register uniform account numbers to join the same audio and video conference, so that the limitation of audio and video data interaction is large, the universality is low, and the use is very inconvenient.

Disclosure of Invention

The embodiment of the invention provides an audio and video processing method, an audio and video processing device and a storage medium, and aims to improve the flexibility and convenience of processing audio and video data.

In order to solve the above technical problems, embodiments of the present invention provide the following technical solutions:

an audio and video processing method, comprising:

receiving a reservation request sent by a sender, determining conference time selected by the sender according to the reservation request, and generating a link address corresponding to an audio and video conference;

adding the sender to the audio and video conference within the conference time;

receiving a joining request sent by a receiver through a third-party platform based on the link address in the conference time, and joining the receiver into the audio and video conference according to the joining request;

and sending the audio and video data generated in the audio and video conference to the sender and the receiver through the third-party platform.

An audio-video processing apparatus comprising:

the processing unit is used for receiving a reservation request sent by a sender, determining the conference time selected by the sender according to the reservation request and generating a link address corresponding to the audio and video conference;

the first joining unit is used for joining the sender to the audio and video conference within the conference time;

the second joining unit is used for receiving a joining request sent by a receiver through a third party platform based on the link address in the conference time, and joining the receiver into the audio and video conference according to the joining request;

and the sending unit is used for sending the audio and video data generated in the audio and video conference to the sender and sending the audio and video data to the receiver through the third party platform.

A storage medium, wherein the storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor to execute the steps in any audio/video processing method provided by the embodiment of the present invention.

The embodiment of the invention can determine the meeting time selected by the sender according to the received reservation request sent by the sender and generate the link address corresponding to the audio and video meeting; then, the sender is added into the audio and video conference within the conference time, and the receiver is added into the audio and video conference according to the received addition request sent by the receiver through the third-party platform based on the link address; at the moment, the audio and video data generated in the audio and video conference can be sent to the sender and sent to the receiver through the third-party platform. According to the scheme, the sender and the receiver can join the same audio and video conference through different platforms, and audio and video data generated in the audio and video conference are respectively sent to the sender and the receiver through different platforms, so that audio and video data interaction of the sender and the receiver in the same audio and video conference through different platforms is realized, the limitation that the same audio and video conference can be joined to carry out audio and video data transmission only by registering a unified account number on a unified platform (namely, application) in the prior art is overcome, and the flexibility and the convenience of audio and video data transmission are greatly improved.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

Fig. 1 is a scene schematic diagram of an audio/video processing system provided by an embodiment of the present invention;

fig. 2 is a schematic flow chart of an audio/video processing method provided by an embodiment of the present invention;

fig. 3 is a schematic display diagram of a reservation interface for reserving a conference according to an embodiment of the present invention;

FIG. 4 is a diagram illustrating an embodiment of the present invention for selecting a recipient for identity verification;

fig. 5 is a schematic diagram of a participant joining an audio-video conference according to an embodiment of the present invention;

fig. 6 is a schematic diagram of a display interface for a sender to perform an audio/video call according to an embodiment of the present invention;

fig. 7 is a schematic diagram of a display interface for a receiving party to perform an audio/video call according to an embodiment of the present invention;

fig. 8 is an architecture diagram of a conference system communicating with other platforms according to an embodiment of the present invention;

fig. 9 is another schematic flow chart of an audio/video processing method provided by an embodiment of the present invention;

fig. 10 is a schematic structural diagram of a conference system processing audio and video data according to an embodiment of the present invention;

fig. 11 is a schematic structural diagram of an audio/video processing apparatus according to an embodiment of the present invention;

fig. 12 is another schematic structural diagram of an audio-video processing device according to an embodiment of the present invention;

fig. 13 is another schematic structural diagram of an audio/video processing apparatus according to an embodiment of the present invention;

fig. 14 is another schematic structural diagram of an audio-video processing device according to an embodiment of the present invention;

fig. 15 is another schematic structural diagram of an audio/video processing apparatus according to an embodiment of the present invention;

fig. 16 is a schematic structural diagram of a server according to an embodiment of the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

The embodiment of the invention provides an audio and video processing method, an audio and video processing device and a storage medium.

Referring to fig. 1, fig. 1 is a scene schematic diagram of an audio and video processing system according to an embodiment of the present invention, where the audio and video processing system may include an audio and video processing device, and the audio and video processing device may be specifically integrated in a server, and the server may be configured to receive a reservation request sent by a sender, determine meeting time selected by the sender according to the reservation request, and generate a link address corresponding to an audio and video meeting according to the reservation request, for example, an audio and video meeting may be established according to the reservation request, a meeting identifier is allocated to the audio and video meeting, and a link address corresponding to the audio and video meeting is generated according to the meeting identifier. Then, the sender is added into the audio and video conference within the conference time, for example, a login request sent by the sender within the conference time is received, the sender is added into the audio and video conference according to the login request, a join request sent by the receiver within the conference time through a third-party platform based on the link address is received, the receiver is added into the audio and video conference according to the join request, the identity of the receiver can be verified according to the join request, and the receiver is added into the audio and video conference when the identity is verified, wherein the third-party platform can comprise WeChat, QQ, skype, WhatsApp, Facebook, Google Account, Twitter and the like. At this time, an audio-video call can be performed between the sender and the receiver, for example, audio-video data generated in the audio-video conference is sent to the sender and sent to the receiver through a third party platform, that is, the audio-video data sent by the sender through the audio-video conference is received and sent to the receiver through the third party platform, or the audio-video data sent by the receiver through the audio-video conference is received and sent to the sender; and so on.

It should be noted that the scene schematic diagram of the audio/video processing system shown in fig. 1 is only an example, and the audio/video processing system and the scene described in the embodiment of the present invention are for more clearly illustrating the technical solution of the embodiment of the present invention, and do not form a limitation on the technical solution provided in the embodiment of the present invention.

The following are detailed below.

In this embodiment, the description will be made from the perspective of an audio/video processing apparatus, which may be specifically integrated in a network device such as a server or a gateway.

An audio and video processing method, comprising: receiving a reservation request sent by a sender, determining meeting time selected by the sender according to the reservation request, and generating a link address corresponding to the audio and video meeting; adding a sender into the audio and video conference within the conference time; receiving a joining request sent by a receiver through a third-party platform based on a link address in the meeting time, and joining the receiver into the audio and video meeting according to the joining request; and sending the audio and video data generated in the audio and video conference to a sender and sending the audio and video data to a receiver through a third-party platform.

Referring to fig. 2, fig. 2 is a schematic flow chart of an audio/video processing method according to an embodiment of the present invention. The audio and video processing method can comprise the following steps:

in step S101, a reservation request sent by a sender is received, a meeting time selected by the sender is determined according to the reservation request, and a link address corresponding to the audio and video meeting is generated.

The sender may be a client used by the user to reserve the audio/video conference, and the client may include a wechat client, a QQ client, a browser client, and the like, and may also include other types of clients, and specific content is not limited herein. The link address may be a URL corresponding to the audio-video conference, for example, the link address may be http:// www.meeting.com/123456 or the like. The audio and video conference can be a network video conference, and audio and video calls, character communication and the like can be carried out between a sender and a receiver in the network video conference.

The sender and the receiver can appoint the meeting time in various ways, for example, the sender and the receiver can appoint the time of audio-video conversation (namely meeting time) in a way of leaving a message on a renting platform, or the sender and the receiver can appoint the time of audio-video conversation in a way of mail, and the like, and then the sender can appoint an audio-video meeting on an application or a webpage provided by the audio-video processing device. When the audio and video conference needs to be reserved, the sender may send a reservation request to the audio and video processing device, and the audio and video processing device may receive the reservation request sent by the sender, where the reservation request may carry related information selected by the sender, for example, conference time selected by the sender, or identity verification selected by the sender, and the like. At this time, the audio/video processing device may determine the meeting time selected by the sender according to the reservation request, and generate a link address corresponding to the audio/video meeting according to the reservation request.

Specifically, the sender can apply for an account in an audio and video conference system provided by the audio and video processing device, and log in the account to enter the audio and video conference system, and after entering the audio and video conference system, the sender can send a reservation request to the audio and video processing device. The audio and video processing device can receive a reservation request sent by a sender, display a reservation interface according to the reservation request, receive meeting time selected by the sender in the reservation interface, and enable the sender and a receiver to join an audio and video meeting within a meeting time range. For example, as shown in fig. 3, the av processing device may receive, in the reservation interface, a conference subject, a start time of the conference, an end time of the conference, participants, and the like, which are input by the sender, and may set the start time of the conference to 2018, 3, month, 20, day, 18:00, the end time of the conference to 2018, 3, month, 20, day, 20:00, and the participants to 3, and the like.

In some embodiments, the step of generating a link address corresponding to the audio/video conference according to the reservation request may include: establishing an audio and video conference according to the reservation request, and distributing a conference identifier for the audio and video conference; and generating a link address corresponding to the audio and video conference according to the conference identifier.

After the reservation is successful, the audio/video processing device can establish an audio/video conference, and allocate a conference identifier to the audio/video conference, where the conference identifier may be a name or a number of the audio/video conference, and the conference identifier may be composed of numbers and/or letters, and at this time, a link address URL corresponding to the audio/video conference may be generated according to the conference identifier and information carried in the reservation request, and for example, when the link address is http:// www.meeting.com/123456, where 123456 is a conference identifier, the link address may be valid within a preset time range before and after the conference time, or may receive a setting instruction, and set the valid number of times of the link address in the setting interface according to the setting instruction, and the like. The sender can send the link address to the receiver, for example, the sender can send a private message to the receiver on a renting platform to inform the receiver of the link address corresponding to the audio and video conference; or the sender can tell the corresponding link address of the audio and video conference and the like in the forms of telephone, mail, WeChat and the like, so that the receiver can join the audio and video conference through the link address after learning the link address.

In step S102, the sender is added to the audio video conference within the conference time.

The conference time can include a range from a start time to a stop time of the audio and video conference, and can also include a first preset time range before the start time and a second preset time range after the stop time, and the like, and the conference time, the first preset time range, the second preset time range and the like can be flexibly set according to time requirements, for example, when the reserved conference time is 3 and 21 days in 2018 from 9:00 to 11:00, a sender and a receiver can join the audio and video conference within 3 and 21 months in 2018 from 7:00 to 18:00, or quit the audio and video conference within the conference time in advance, and the like.

In some embodiments, the step of joining the sender to the audiovisual conference during the conference time may include: receiving a login request sent by a sender in a meeting time; and adding the sender into the audio and video conference according to the login request.

The audio and video processing device can add the sender to the audio and video conference within the conference time, for example, when the conference start time is reached, the sender can send a login request to the audio and video processing device to log in an audio and video conference system provided by the audio and video processing device, and at the moment, the audio and video processing device can add the sender to the audio and video conference to enter the audio and video conference according to the login request.

In step S103, a join request sent by the receiver through the third party platform based on the link address in the conference time is received, and the receiver is joined to the audio and video conference according to the join request.

The receiving party may include one or more than one receiving party, the receiving party may invite the sending party to join a client used by the audio and video conference, the client may include a WeChat client, a QQ client, a browser client, and the like, and may also include other types of clients, and specific content is not limited herein. Third party platforms may include WeChat, QQ, skype, WhatsApp, Facebook, Google Account, Microsoft ID, Twitter, and the like.

It should be noted that, in a certain scenario, the receiver may serve as the sender, and the sender may also serve as the receiver, and after the receiver and the sender exchange roles, the audio/video data processing flow is similar to that in the embodiment of the present invention, and is not described herein again.

After the link address corresponding to the audio and video conference provided by the sender is acquired, the receiver can join the audio and video conference based on the link address within the conference time, for example, after the conference time is reached, the receiver can click or activate the link address through a third-party platform to generate a joining request, and send the joining request to an audio and video processing device. The audio and video processing device can receive a joining request sent by a receiver through a third-party platform based on the link address, the joining request can carry a conference identifier of the audio and video conference, an identity identifier of the receiver and the like, and the receiver can be joined into the audio and video conference according to the joining request.

In some embodiments, receiving a join request sent by the receiver through the third party platform based on the link address during the conference time, and joining the receiver to the audio/video conference according to the join request may include:

receiving a joining request sent by a receiver through a third-party platform based on a link address in the meeting time, and carrying out identity verification on the receiver according to the joining request; and when the identity authentication is passed, adding the receiver into the audio and video conference.

Because the audio and video processing device can allow the user to carry out a conference anonymously, in order to improve the communication effect in a stranger scene, the identity of the other party can be verified by an effective means so as to establish a trust relationship. Specifically, in the meeting time, all receivers clicking the URL can join the audio and video conference, in order to improve the security, the receivers joining the audio and video conference can be required to be authenticated when the conference is reserved, at this time, the audio and video processing device can authenticate the receivers according to the joining request after receiving the joining request sent by the receivers through the third-party platform based on the link address, when the identity authentication of the receivers is passed, the receivers can be joined to the audio and video conference, and when the identity authentication of the receivers is not passed, the receivers can be refused to join the audio and video conference.

In some embodiments, before the step of generating the link address corresponding to the audio-video conference, the audio-video processing method may further include: receiving an identity authentication request sent by a sender, and displaying an authentication reservation interface according to the identity authentication request; receiving a third party account type and an account identification which are sent by a sender and correspond to a third party platform by the receiver in a verification reservation interface; and generating meeting information according to the type and the identification of the third party account.

In the process of establishing the audio and video conference, the sender can set requirements for identity verification of a participant (namely, a receiver) according to actual needs, for example, the sender can specify a third party account type corresponding to the receiver on a third party platform on a reservation page, and input an account identification of the third party account, for example, a nickname of the receiver on the third party account, and the like.

For example, as shown in fig. 4, the sender may select a third party account type in a drop-down list of an identity verification platform, for example, select that the third party account type is Facebook, and input a nickname (i.e., an account identifier) in a nickname input text box of the platform by an input participant (i.e., a receiver), for example, fill in the nickname of the Facebook account corresponding to the receiver as Mike Green, and click a determination button, at this time, the audio/video processing apparatus may receive an identity verification request sent by the sender, display a verification reservation interface according to the identity verification request, then receive, in the verification reservation interface, the receiver sent by the sender at the third party account type and the account identifier corresponding to the third party platform, and generate conference information according to the third party account type and the account identifier.

In some embodiments, the step of receiving a join request sent by the receiving party through the third party platform based on the link address during the meeting time, and authenticating the receiving party according to the join request may include:

receiving a join request sent by a receiver through a third-party platform based on a link address in the meeting time, and acquiring meeting information according to the join request; when the identity authentication is determined to be needed according to the conference information, receiving account information sent by a receiver through a third-party platform; and when the account information is matched with the third party account type and the account identification, determining that the identity authentication is passed.

In order to avoid disturbance, before accessing an audio and video conference, the identity of a receiver can be verified, after the receiver opens a link address to generate a join request, the audio and video processing device can receive the join request sent by the receiver through a third-party platform based on the link address, and obtain conference information according to the join request, for example, corresponding conference information can be searched according to a conference identifier carried in the link address, the conference information can comprise a conference subject, whether the conference requires identity verification, a third-party account type, an account identifier and the like, then, whether identity verification is needed or not is judged according to the conference information, and when the identity verification is not needed according to the conference information, the receiver can be directly added into the audio and video conference; when the fact that identity authentication is needed is determined according to the conference information, the audio and video processing device can prompt the receiver to log in a third party account for identity authentication, for example, prompt the receiver to log in a Facebook account for identity authentication.

And the receiver clicks a login button, a login page of the third party account is opened, a user name and a password corresponding to the third party account are input in the login page to log in the third party account, and after the login is successful, the audio and video processing device can be authorized to acquire relevant information of the receiver on the third party platform, such as a nickname, a head portrait and the like. At this time, the audio/video processing apparatus may receive account information sent by the receiver through the third party platform, where the account information may include a third party account type, an account identifier, authorization information, a nickname, a head portrait, and the like, for example, a Facebook account, a nickname of the Facebook account, and the like. At this time, the acquired account information such as the third party account type, the account identification and the like can be verified with the third party account type and the account identification provided by the sender, if the acquired account information such as the third party account type, the account identification and the like is consistent with the third party account type and the account identification provided by the sender, the acquired account information such as the third party account type, the account identification and the like is matched with the third party account type and the account identification provided by the sender, and at this time, the identity authentication can be determined to be passed, and; if the obtained account information such as the third party account type, the account identification and the like is not matched with the third party account type and the account identification provided by the sender, the identity authentication can be determined not to pass at the moment, and the receiver is refused to join the audio and video conference. For example, the obtained nickname of the Facebook account corresponding to the receiver is Mike Green, and the nickname of the Facebook account corresponding to the receiver provided by the sender is also Mike Green. And the two are consistent, and the receiver can be added into the audio and video conference at the moment. For example, as shown in fig. 5, both the sender and the receiver may have access to an audiovisual conference.

In step S104, the audio and video data generated in the audio and video conference is sent to the sender and sent to the receiver through the third party platform.

After the sender and the receiver join the audio and video conference, the sender and the receiver can perform audio and video data interaction in the audio and video conference, for example, the sender can send the audio and video data to an audio and video processing device in the audio and video conference, and the audio and video processing device can send the audio and video data to the receiver through a third party platform; the receiver can send the audio and video data to the audio and video processing device through the third-party platform in the audio and video conference, and the audio and video processing device can send the audio and video data to the sender.

In some embodiments, the step of sending the audio and video data generated in the audio and video conference to the sender and sending the audio and video data to the receiver through the third party platform may include:

acquiring language types required by a sender and a receiver, and processing audio data in the audio and video data according to the language types to obtain processed audio data; and sending the video data in the audio and video data and the processed audio and video data to a sender and a receiver through a third-party platform.

Before sending the audio and video data to the sender and the receiver, the audio and video processing device can process the audio and video data in advance, for example, the sender and the receiver can turn on the translation mode by themselves, set the type of language required by themselves, wherein the language types may include chinese, english, german, french, thai, russian, etc., and when audio/video data is received in the audio/video conference, the audio and video processing device can identify the audio data in the audio and video data, identify the semantics of the audio data, and acquiring the language types required by the sender and the receiver, and according to the language type required by the sender, the audio data in the audio and video data are converted and/or translated to obtain processed audio data, and the video data in the audio and video data and the processed audio and video data are sent to a sender; and according to the language type required by the receiver, the audio data in the audio and video data is converted and/or translated to obtain processed audio data, and the video data in the audio and video data and the processed audio and video data are sent to the receiver through a third party platform. At the moment, the sender can display the video picture and the subtitle in the display screen and output the voice through a loudspeaker and the like, and the receiver can display the video picture and the subtitle in the display screen through a third-party platform and output the voice through the loudspeaker and the like, so that the simultaneous interpretation function is realized, and the sender can conveniently carry out video call with a strange receiver of foreign countries based on an identity verification mode.

In some embodiments, the step of obtaining the processed audio data by processing the audio data in the audio and video data according to the language type includes:

acquiring a first language type required by a sender and a second language type required by a receiver; identifying audio data in the audio and video data to obtain identified audio data; and converting the recognized audio data according to the first language type and the second language type to obtain converted audio data, and/or translating the recognized audio data to obtain translated characters.

For example, as shown in fig. 6 and 7, if the user corresponding to the sender is ABC, the first language type required by ABC is chinese, the user corresponding to the receiver is XYZ, and the second language type required by XYZ is english, in the process that ABC establishes an audio-video conference with XYZ through a QQ platform and performs audio-video conversation with XYZ through a Facebook platform, when ABC says "do you love me" in the audio-video conference in chinese, the audio-video data needs to be processed, first, the audio-video processing device may identify the audio data in the audio-video data, and identify that the audio data is chinese spoken by ABC: "do you love me". Then, according to the language of the user ABC as Chinese, translating the recognized audio data to obtain translated Chinese characters: "do you love me" and according to the language of user XYZ for English, translate the audio data after discerning, obtain the English characters after translating: "Do you love me". At the moment, the audio and video processing device can also convert Chinese speech into English speech according to actual needs. Finally, the audio/video processing device can send the obtained Chinese characters, Chinese voice, video pictures and the like to the sender, and the sender can display the Chinese characters, English characters and video pictures in the display screen and output the Chinese voice or English voice in the figure 6. And the audio and video processing device can send the obtained English characters, English voice, video pictures and the like to a receiver through a third party platform, and the receiver can display the English characters, Chinese characters and video pictures in a display screen and output Chinese voice or English voice.

When user XYZ speaks "Yes, I love you" in the audio/video conference using english, the audio/video processing apparatus may identify audio data in the audio/video data, and identify that the audio data is english spoken by user XYZ: "Yes, I love you". Then, according to the language of the user ABC as Chinese, translating the recognized audio data to obtain translated Chinese characters: "yes, i love you", and according to the language of user XYZ being english, translating the recognized audio data to obtain translated english characters: "Yes, I love you". At the moment, the audio and video processing device can also convert English speech into Chinese speech according to actual needs. Finally, the audio and video processing device can send the obtained Chinese characters, Chinese voice, video pictures and the like to the sender, and the sender can display the Chinese characters and the video pictures in the display screen and output the Chinese voice or the English voice. And the audio and video processing device can send the obtained English characters, English voice, video pictures and the like to the receiving party through the third party platform, and the receiving party can display the English characters and the video pictures in the display screen and output Chinese voice or English voice in the picture 7.

The invention provides convenience for communication among different native language users, does not need to install uniform application, does not need to register uniform account numbers, can carry out video call, and provides simultaneous interpretation function for different native language users, so that internet users in different countries and regions can communicate more conveniently, the problem that the communication is difficult for users in different regions and different languages on the internet due to different application software, different account number systems, different languages and the like is solved, and the communication is more convenient for each user.

As will be described in detail in the following examples, as the internet is more widely used, communication between users in different native languages is more and more required, for example, a chinese user a issues a house information on a house renting platform, a foreign user B wishes to rent a house in china during a chinese trip, and then the user B sees the house source of the user a on the house renting platform, wishes to communicate with the user a, and views the house internal facilities. Because the user A only masters the read-write capacity of simple English and the user B only masters the read-write capacity of simple Chinese, the two parties can not communicate even if the two parties make calls, the user A can not use Facebook, and the user B uses Facebook, under the condition, the audio and video processing device (such as an audio and video conference system) provided by the invention can provide a convenient communication mode for the user A and the user B. Specifically, taking an audio/video processing device as an audio/video conference system as an example, a user a may subscribe a conference time with a user B in advance, then register an account in the audio/video conference system, log in the audio/video conference system using the registered account, enter a conference subscription interface, select a conference time, and select to authenticate the identity of the user B, where the authentication platform of the user a needs to designate the user B as a Facebook platform and a nickname on the Facebook platform. After the reservation is successful, the audio and video conference system can establish the audio and video conference and generate a link address URL of the audio and video conference, and the user A can send the URL to the user B in a mode outside the system, for example, a private letter carrying the URL is sent to the user B on a renting platform. After the meeting time is up, the user A can log in the audio and video meeting system to enter the pre-established audio and video meeting. After the meeting time is up, when the user B opens the URL, the audio and video meeting system prompts the user B to log in by using a Facebook account for identity verification, and the user B logs in the Facebook account and authorizes the audio and video meeting system to acquire information such as a nickname, a head portrait and the like. And the audio and video conference system verifies the obtained nickname with the nickname filled by the user A, and if the nickname is consistent with the nickname filled by the user A, the user B can join the audio and video conference. After the user A and the user B both join the audio and video conference, the user A can carry out audio and video conversation with the user B in a Facebook in the audio and video conference. In the process of carrying out audio and video conversation, the audio and video conference system can carry out conversion and/or translation and other processing on audio and video data according to the translation mode opened by the user A and the user B and the set language type to obtain the processed audio and video data, the processed audio and video data are sent to the user A and sent to the user B through a Facebook platform, and at the moment, the user A and the user B can see the subtitles of the opposite party on the screen, so that the simultaneous interpretation function is realized, the language pace is broken, and more opportunities are provided for wider cooperation.

As can be seen from the above, the embodiment of the present invention can determine the conference time selected by the sender according to the received reservation request sent by the sender, and generate the link address corresponding to the audio and video conference; then, the sender is added into the audio and video conference within the conference time, and the receiver is added into the audio and video conference according to the received addition request sent by the receiver through the third-party platform based on the link address; at the moment, the audio and video data generated in the audio and video conference can be sent to the sender and sent to the receiver through the third-party platform. According to the scheme, the sender and the receiver can join the same audio and video conference through different platforms, and audio and video data generated in the audio and video conference are respectively sent to the sender and the receiver through different platforms, so that audio and video data interaction of the sender and the receiver in the same audio and video conference through different platforms is realized, the limitation that the same audio and video conference can be joined to carry out audio and video data transmission only by registering a unified account number on a unified platform (namely, application) in the prior art is overcome, and the flexibility and the convenience of audio and video data transmission are greatly improved.

The method described in the above embodiments is further illustrated in detail by way of example.

In this embodiment, an audio and video processing device is taken as an example, the server may be integrated in an audio and video conference system (which will be referred to as a conference system hereinafter), and a sender is taken as a client a and a receiver is taken as a client B, where both the client a and the client B may use a browser to communicate with the conference system. For example, as shown in fig. 8, the conference system, the browser used by the client a, and the browser used by the client B may be connected in pairs, and the browser used by the client a and the browser used by the client B may communicate with each other through a Web Real-Time Communication (WebRTC) protocol, that is, the WebRTC protocol is used to transfer audio and video data between the browsers.

It should be noted that the sender, the receiver, the third party platform, and the like may be flexibly configured according to actual needs, and this embodiment is only an example given for convenience of description, and should not be construed as a limitation on the sender, the receiver, the third party platform, and the like, but the process of audio/video processing is similar regardless of the specific reason platform, and the like, and can be understood according to this example.

Referring to fig. 9, fig. 9 is a schematic flowchart of an audio/video processing method according to an embodiment of the present invention. The method flow can comprise the following steps:

and S201, meeting time is agreed among the client A and the client B in other modes.

The client a and the client B may agree on the conference time in various ways, for example, the client a and the client B may agree on the conference time in a message-leaving manner on a rental platform, or the client a and the client B may agree on the time of an audio/video call in a mail manner, and then the client a may subscribe to an audio/video conference on an application or a web page provided by the conference system.

The conference time can include a range from a start time to a stop time of the audio and video conference, and can also include a first preset time range before the start time, a second preset time range after the stop time, and the like, and the conference time, the first preset time range, the second preset time range, and the like can be flexibly set according to time requirements, for example, when the reserved conference time is 8:00 to 16:00 in 2018, 1, 23, 7:00 to 21:00 in 2018, the client a and the client B can join the audio and video conference in 23, 7:00 to 21:00 in 2018, or quit the audio and video conference in advance in the conference time, and the like.

S202, the client A logs in an account of the conference system, enters an audio and video conference interface to reserve a conference and inputs conference time.

The client A can register an account in the conference system and log in the registered account to enter the conference system, and after entering the conference system, the client A can enter an audio and video conference interface (namely a reservation interface) to select conference time or execute other operations in the reservation interface so as to reserve a conference. For example, as shown in fig. 3, the conference system may receive a conference subject, a start time of the conference, an end time of the conference, and participants and the like input by the client a in a reservation interface, and may set the start time of the conference to 2018, 3, 20, 00, the end time of the conference to 2018, 3, 20, 00, the participants to 3, and the like.

S203, when the client B joins the audio and video conference, the client A requires identity verification on the client B: and (3) specifying the type of the third party account corresponding to the client B, and inputting the nickname of the account of the client B (namely the participant).

The client a may request the conference system to authenticate the identity of a required participant (e.g., the client B) according to actual needs, for example, the client a may specify, on a reservation page, a third party account type corresponding to the client B on a third party platform, and input a nickname of the third party account. For example, as shown in fig. 4, the client a may select a third party account type of the client B in a drop-down list of the identity authentication platform, for example, select that the third party account type of the client B is a Facebook account, and input a nickname in an input text box of the nickname of the platform by an input participant (i.e., the client B), for example, input the nickname of the Facebook account corresponding to the client B as Mike Green, and click a determination button.

And S204, the conference system generates a URL.

S205, the conference system sends the URL to the client A.

S206, the client A sends the URL to the client B.

After the reservation is successful, the conference system can establish the audio and video conference and allocate a conference identifier for the audio and video conference, the conference identifier can be the name or number of the audio and video conference, and a link address URL corresponding to the audio and video conference can be generated according to the conference identifier and other information, for example, when the link address is http:// www.meeting.com/123456, 123456 is a conference identifier. The conference system may send the generated URL to the client a, and the client a may send the link address to the client B, for example, the client a may send the URL to the client B on a rental house platform, or the client a may send the URL to the client B in the form of a phone call, a mail, a WeChat, or the like, so that the client B may join the audio and video conference through the URL after knowing the URL.

And S207, the client A logs in an account of the conference system in the conference time and joins in the audio and video conference, and waits for the client B to join in.

The conference system can add the client a into the audio and video conference within the conference time, for example, when the conference start time is reached, the client a can log in an account of the conference system to enter the conference system and send an addition request to the conference system, and at this time, the conference system can add the sender into the audio and video conference according to the addition request to enter the audio and video conference.

S208, the client B clicks the URL and opens a conference system interface.

And S209, prompting the client B to log in by using a third party account through the conference system.

S210, the client B logs in a third party account and authorizes the conference system to obtain related information.

And S211, the third-party platform sends the nickname, the head portrait, the ID of the client B and the authorization information to the conference system.

After the client B opens the URL through the browser to generate the join request, the conference system may receive the join request sent by the client B through the third-party platform based on the URL, and at this time, the client B may enter the conference system interface, and when it is determined that authentication is required, the conference system may prompt the client B to log in a third-party account for authentication, for example, prompt the client B to log in a Facebook account for authentication. The client B clicks a login button, a login page of the third party account is opened, a user name and a password corresponding to the third party account are input in the login page to log in the third party account, and after the login is successful, the conference system can be authorized to acquire relevant information, such as a nickname, a head portrait and the like, of the client B on the third party platform. At this time, the conference system may receive account information sent by the client B through the third party platform, where the account information may include a third party account type, an account identifier, authorization information, a nickname, a head portrait, and the like, for example, a Facebook account, a nickname of the Facebook account, and the like.

And S212, the conference system verifies the account type and the account nickname.

And S213, when the verification is passed, the conference system adds the client B into the audio and video conference.

The conference system can check the third party account type and the account identification and the like sent by the third party platform with the third party account type and the account identification provided by the client A, if the third party account type and the account identification are consistent with the third party account type and the account identification provided by the client A, the identity verification can be determined to be passed, and the client B can be added into the audio and video conference; if the identity authentication is inconsistent with the authentication verification result, the client B is refused to join the audio-video conference. For example, the nickname of the Facebook account corresponding to the client B sent by the third party platform is Mike Green, and the nickname of the Facebook account corresponding to the client B provided by the client a is also Mike Green. The two are consistent, at this time, the client B may be added to the audio and video conference, and after the client B is added to the audio and video conference, it is necessary to wait for the client a to join the audio and video conference (conference for short), for example, as shown in fig. 5, both the sender and the receiver may join the audio and video conference.

And S214, the client A and the client B carry out video call in the audio and video conference through a third-party platform.

After the client a and the client B both access the audio and video conference, the client a and the client B can perform audio and video data interaction in the audio and video conference, that is, perform audio and video call, for example, the client a can log in an account of a conference system in the audio and video conference, send the audio and video data to the conference system through a conference system platform, the conference system can forward the received audio and video data to a third party platform, and the third party platform sends the audio and video data to the client B; the client B can send the audio and video data to the conference system through the third-party platform in the audio and video conference, and the conference system sends the audio and video data to the client A.

It should be noted that, before sending the audio and video data to the sender and the receiver, the conference system may process the audio and video data in advance, for example, as shown in fig. 10, the conference system may include an audio and video communication server, a voice recognition server, a translation server, and the like, and when the client a generates an audio stream and a video stream (that is, the audio and video data includes an audio stream and a video stream), the audio stream and the video stream may be sent to the audio and video communication server in the conference system, the audio and video communication server stores the audio stream and the video stream, and sends the audio stream and the video stream to the client B through a third-party platform, and transmits the audio stream to the voice recognition server. When transmitting the audio stream to the voice recognition server, the conference identifier RoomID and the client identifier UserID (i.e., speaker UserID) may be set for the audio stream, so that the voice recognition server and the translation server can distinguish the received audio stream. The voice recognition server performs voice recognition on the audio stream to obtain recognized characters, then the voice recognition server sends the recognized characters to the translation server according to the conference identification RoomID and the client identification UserID, the translation server can translate the characters according to the language types required by the client A and the client B to obtain a translated text (for example, translating Chinese into English), and at the moment, the translation server can send the translated text and the original text to the client A according to the client identification UserID and send the translated text and the original text to the client B through a third-party platform. The audio streams of the client A and the client B can be respectively and independently sent to the voice recognition server, so that the situation that the recognition rate is reduced when the two parties speak simultaneously is avoided, and in addition, when the voice recognition server recognizes the voice, the noise reduction processing can be carried out on the audio, so that the recognition rate is improved.

For example, as shown in fig. 6, if the nickname of the QQ account corresponding to the client a is ABC, and the Facebook account corresponding to the client B is XYZ, in the process that the client a establishes an audio-video conference and performs an audio-video call with the client B through the Facebook platform by the QQ platform, when the client a says "do you like me" in the audio-video conference by using the chinese language, the conference system may identify the audio data in the audio-video data, and translate the identified audio data to obtain the translated chinese language: "do you love me" and english: the Do you love me conference system can convert Chinese speech into English speech according to actual needs. Then, the conference system can send the processed audio and video data to the client A, and the client A can display characters and video pictures in a display screen and output Chinese voice or English voice; and the conference system can send the processed audio and video data to the client B through a third-party platform, and the client B can display characters and video pictures in a display screen and output Chinese voice or English voice and the like. Therefore, convenience is provided for communication among different native language users, and internet users in different countries and regions can communicate more conveniently.

The embodiment of the invention adds each client into the same audio and video conference through different platforms, and respectively sends the audio and video data generated in the audio and video conference to each client through different platforms, thereby realizing that each client can carry out audio and video data interaction in the same audio and video conference through different platforms, providing convenience for communication among different native language users, having no need of installing uniform application and registering uniform account numbers, the video call can be carried out, so that internet users in different countries and regions can communicate more conveniently, the problem that the communication is difficult due to different platforms, different languages and the like is solved, the flexibility and the convenience of audio and video data transmission are improved, and the client can be anonymously added into the audio and video conference, and can also carry out identity verification through an account system owned by the client, thereby providing convenience and improving safety.

In order to better implement the audio and video processing method provided by the embodiment of the invention, the embodiment of the invention also provides a device based on the audio and video processing method. The meanings of the nouns are the same as those in the audio and video processing method, and specific implementation details can refer to the description in the method embodiment.

Referring to fig. 11, fig. 11 is a schematic structural diagram of an audio/video processing device according to an embodiment of the present invention, where the audio/video processing device may include a processing unit 301, a first adding unit 302, a second adding unit 303, a sending unit 304, and the like.

The processing unit 301 is configured to receive a reservation request sent by a sender, determine a meeting time selected by the sender according to the reservation request, and generate a link address corresponding to the audio/video meeting.

The sender may be a client used by the user to reserve the audio/video conference, and the client may include a wechat client, a QQ client, a browser client, and the like, and may also include other types of clients, and specific content is not limited herein. The link address may be a URL corresponding to the audio-video conference. The audio and video conference can be a network video conference, and audio and video calls, character communication and the like can be carried out between a sender and a receiver in the network video conference.

The sender and the receiver can appoint the meeting time in various ways, for example, the sender and the receiver can appoint the meeting time in a room renting platform in a message mode, or the sender and the receiver can appoint the audio and video conversation time in a mail mode, and the like, and then the sender can appoint an audio and video meeting on an application or a webpage provided by the audio and video processing device. When the audio and video conference needs to be reserved, the sender may send a reservation request to the processing unit 301, and the processing unit 301 may receive the reservation request sent by the sender, where the reservation request may carry related information selected by the sender, for example, conference time selected by the sender, or identity verification selected by the sender. At this time, the processing unit 301 may determine a conference time selected by the sender according to the reservation request, and generate a link address corresponding to the audio/video conference according to the reservation request.

In some embodiments, as shown in fig. 14, the processing unit 301 may include:

the establishing subunit 3011 is configured to establish an audio/video conference according to the reservation request, and allocate a conference identifier to the audio/video conference;

and the generating subunit 3012 is configured to generate a link address corresponding to the audio/video conference according to the conference identifier.

After the reservation is successful, the establishing subunit 3011 may establish an audio/video conference, and assign a conference identifier to the audio/video conference, where the conference identifier may be a name or a number of the audio/video conference, and the conference identifier may be composed of numbers and/or letters, and the like, and at this time, the generating subunit 3012 may generate a link address URL corresponding to the audio/video conference according to the conference identifier and information and the like carried in the reservation request, for example, when the link address is http:// www.meeting.com/123456, where 123456 is a conference identifier, the link address may be valid within a preset time range before and after the conference time, or may receive a setting instruction, set the valid number of times of the link address in the setting interface according to the setting instruction, and the like. The sender can send the link address to the receiver, for example, the sender can send a private message to the receiver on a renting platform to inform the receiver of the link address corresponding to the audio and video conference; or the sender can tell the corresponding link address of the audio and video conference and the like in the forms of telephone, mail, WeChat and the like, so that the receiver can join the audio and video conference through the link address after learning the link address.

And the first joining unit 302 is configured to join the sender in the audio/video conference within the conference time.

In some embodiments, the first adding unit 302 is specifically configured to: receiving a login request sent by a sender in a meeting time; and adding the sender into the audio and video conference according to the login request.

And the second joining unit 303 is configured to receive a joining request sent by the receiver through the third party platform based on the link address within the meeting time, and join the receiver into the audio/video meeting according to the joining request.

After acquiring the link address corresponding to the audio and video conference provided by the sender, the receiver may join the audio and video conference based on the link address within the conference time, for example, after the conference time is reached, the receiver may click or activate the link address through a third party platform to generate a join request, and send the join request to the second joining unit 303. The second joining unit 303 may receive a joining request sent by the receiver through the third party platform based on the link address, where the joining request may carry a conference identifier of the audio/video conference, an identity identifier of the receiver, and the like, and at this time, the receiver may join the audio/video conference according to the joining request.

In some embodiments, as shown in fig. 12, the second adding unit 303 may include:

the verification subunit 3031 is configured to receive an join request sent by the receiver through the third-party platform based on the link address within the meeting time, and perform identity verification on the receiver according to the join request;

and the adding subunit 3032 is configured to add the receiving party to the audio/video conference when the identity authentication is passed.

Because the user can be allowed to carry out a conference anonymously, in order to improve the communication effect in a stranger scene, the identity of the other party can be verified by an effective means so as to establish a trust relationship. Specifically, in the meeting time, all recipients who click the URL may join the audio/video conference, in order to improve security, when the conference is reserved, the recipients who join the audio/video conference may be required to perform authentication, at this time, the authentication subunit 3031 may perform authentication on the recipients according to the joining request after receiving the joining request sent by the recipients through the third party platform based on the link address, when the authentication on the recipients is passed, the joining subunit 3032 may join the recipients to the audio/video conference, and when the authentication on the recipients is not passed, the recipients may be denied to join the audio/video conference.

In some embodiments, as shown in fig. 13, the audio/video processing apparatus may further include:

a display unit 305, configured to receive an authentication request sent by a sender, and display a verification reservation interface according to the authentication request;

a receiving unit 306, configured to receive, in the verification reservation interface, a third party account type and an account identifier, which are sent by the sending party and correspond to the third party platform by the receiving party;

and a generating unit 307, configured to generate meeting information according to the third party account type and the account identifier.

For example, as shown in fig. 4, the sender may select a third party account type in a drop-down list of a selected authentication platform, for example, select that the third party account type is Facebook, and input a nickname (i.e., an account identifier) in a nickname input text box of the platform by an input participant (i.e., a receiver), for example, fill in the nickname of the Facebook account corresponding to the receiver as Mike Green, and click a determination button, at this time, the display unit 305 may receive an authentication request sent by the sender, display an authentication reservation interface according to the authentication request, then the receiving unit 306 receives the third party account type and the account identifier, which are sent by the sender, corresponding to the third party platform within the authentication reservation interface, and the generating unit 307 generates meeting information according to the third party account type and the account identifier.

In some embodiments, the verification subunit 3031 may be specifically configured to: receiving a join request sent by a receiver through a third-party platform based on a link address in the meeting time, and acquiring meeting information according to the join request; when the identity authentication is determined to be needed according to the conference information, receiving account information sent by a receiver through a third-party platform; and when the account information is matched with the third party account type and the account identification, determining that the identity authentication is passed.

In order to avoid disturbance, before accessing the audio and video conference, the identity of the receiver may be verified, after the receiver opens the link address to generate a join request, the verification subunit 3031 may receive the join request sent by the receiver through the third party platform based on the link address, and obtain conference information according to the join request, for example, may search for corresponding conference information according to a conference identifier carried in the link address, where the conference information may include a conference subject, whether the conference requires identity verification, a third party account type, an account identifier, and the like, and then determine whether identity verification is required according to the conference information, and when it is determined that identity verification is not required according to the conference information, the receiver may be directly added to the audio and video conference; when the fact that identity authentication is needed is determined according to the conference information, the audio and video processing device can prompt the receiver to log in a third party account for identity authentication, for example, prompt the receiver to log in a Facebook account for identity authentication.

And the receiver clicks a login button, a login page of the third party account is opened, a user name and a password corresponding to the third party account are input in the login page to log in the third party account, and after the login is successful, the audio and video processing device can be authorized to acquire relevant information of the receiver on the third party platform, such as a nickname, a head portrait and the like. At this time, the verification sub-unit 3031 may receive account information sent by the receiver through the third party platform, where the account information may include a third party account type, an account identifier, authorization information, a nickname, a head portrait, and the like, for example, a Facebook account, a nickname of the Facebook account, and the like. At this time, the verification subunit 3031 may verify the acquired account information such as the third party account type and the account identifier with the third party account type and the account identifier provided by the sender, and if the acquired account information such as the third party account type and the account identifier is consistent with the third party account type and the account identifier provided by the sender, it is determined that the identity verification is passed, and the receiver may join the audio and video conference; if the obtained account information such as the third party account type, the account identification and the like is not matched with the third party account type and the account identification provided by the sender, the identity authentication can be determined not to pass at the moment, and the receiver is refused to join the audio and video conference.

And the sending unit 304 is configured to send the audio and video data generated in the audio and video conference to a sender, and send the audio and video data to a receiver through a third-party platform.

After the sender and the receiver join the audio and video conference, the sender and the receiver can perform audio and video data interaction in the audio and video conference, for example, the sender can send the audio and video data to the sending unit 304 in the audio and video conference, and the sending unit 304 can send the audio and video data to the receiver through a third-party platform; the receiving side may send the audio and video data to the sending unit 304 through the third party platform in the audio and video conference, and the sending unit 304 may send the audio and video data to the sending side.

In some embodiments, as shown in fig. 15, the sending unit 304 may include:

a processing subunit 3041, configured to obtain language types required by the sender and the receiver, and process the audio data in the audio and video data according to the language types to obtain processed audio data;

the sending subunit 3042 is configured to send the video data in the audio and video data and the processed audio and video data to a sender, and send the processed audio and video data to a receiver through a third-party platform.

Before transmitting the audio and video data to the sender and the receiver, the processing subunit 3041 may process the audio and video data in advance, for example, the sender and the receiver can turn on the translation mode by themselves, set the type of language required by themselves, wherein the language types may include chinese, english, german, french, thai, russian, etc., and when audio/video data is received in the audio/video conference, the processing subunit 3041 may identify the audio data in the audio/video data, identify the semantics of the audio data, and acquiring the language types required by the sender and the receiver, and according to the language type required by the sender, the transmitting subunit 3042 transmits the video data in the audio and video data and the processed audio and video data to the transmitting party; and, according to the language type required by the receiving party, the audio data in the audio and video data is converted and/or translated to obtain processed audio data, and the sending subunit 3042 sends the video data in the audio and video data and the processed audio and video data to the receiving party through the third party platform. At the moment, the sender can display the video picture and the subtitle in the display screen and output the voice through a loudspeaker and the like, and the receiver can display the video picture and the subtitle in the display screen through a third-party platform and output the voice through the loudspeaker and the like, so that the simultaneous interpretation function is realized, and the video call between the sender and a strange receiver in foreign countries can be facilitated based on an identity verification mode.

In some embodiments, the processed audio data includes converted audio data and/or translated words, and the processing subunit 3041 may be specifically configured to: acquiring a first language type required by a sender and a second language type required by a receiver; identifying audio data in the audio and video data to obtain identified audio data; and converting the recognized audio data according to the first language type and the second language type to obtain converted audio data, and/or translating the recognized audio data to obtain translated characters.

For example, as shown in fig. 6, if the user corresponding to the sender is ABC, the first language type required by ABC is chinese, the user corresponding to the receiver is XYZ, and the second language type required by XYZ is english, in the process that ABC establishes an audio-video conference with XYZ through a QQ platform and performs an audio-video call with XYZ through a Facebook platform, when ABC says "do me you in the audio-video conference" in chinese, the audio-video data needs to be processed, first, the processing subunit 3041 may identify the audio data in the audio-video data, and identify that the audio data is chinese spoken by ABC: "do you love me". Then, according to the language of the user ABC as Chinese, translating the recognized audio data to obtain translated Chinese characters: "do you love me" and according to the language of user XYZ for English, translate the audio data after discerning, obtain the English characters after translating: "Do you love me". At the moment, the audio and video processing device can also convert Chinese speech into English speech according to actual needs. Finally, the sending subunit 3042 can send the obtained chinese characters, chinese speech, and video pictures to the sender, and the sender can display the chinese characters, english characters, and video pictures on the display screen and output the chinese speech or english speech. The transmitting subunit 3042 transmits the obtained english text, english speech, and video picture to the receiving party through the third party platform, and the receiving party can display the english text, chinese text, and video picture on the display screen and output the chinese speech or english speech.

As can be seen from the above, in the embodiment of the present invention, the processing unit 301 may determine, according to the received reservation request sent by the sender, the meeting time selected by the sender, and generate a link address corresponding to the audio/video meeting; then the first joining unit 302 joins the sender to the audio and video conference within the conference time, and the second joining unit 303 joins the receiver to the audio and video conference according to the received joining request sent by the receiver through the third party platform based on the link address; at this time, the sending unit 304 may send the audio and video data generated in the audio and video conference to the sender, and send the audio and video data to the receiver through the third party platform. According to the scheme, the sender and the receiver can join the same audio and video conference through different platforms, and audio and video data generated in the audio and video conference are respectively sent to the sender and the receiver through different platforms, so that audio and video data interaction of the sender and the receiver in the same audio and video conference through different platforms is realized, the limitation that the same audio and video conference can be joined to carry out audio and video data transmission only by registering a unified account number on a unified platform (namely, application) in the prior art is overcome, and the flexibility and the convenience of audio and video data transmission are greatly improved.

An embodiment of the present invention further provides a server, as shown in fig. 16, which shows a schematic structural diagram of the server according to the embodiment of the present invention, specifically:

the server may include components such as a processor 401 of one or more processing cores, memory 402 of one or more computer-readable storage media, a power supply 403, and an input unit 404. Those skilled in the art will appreciate that the server architecture shown in FIG. 16 is not meant to be limiting, and may include more or fewer components than those shown, or some components may be combined, or a different arrangement of components. Wherein:

the processor 401 is a control center of the server, connects various parts of the entire server using various interfaces and lines, and performs various functions of the server and processes data by running or executing software programs and/or modules stored in the memory 402 and calling data stored in the memory 402, thereby performing overall monitoring of the server. Optionally, processor 401 may include one or more processing cores; preferably, the processor 401 may integrate an application processor, which mainly handles operating systems, user interfaces, application programs, etc., and a modem processor, which mainly handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into the processor 401.

The memory 402 may be used to store software programs and modules, and the processor 401 executes various functional applications and data processing by operating the software programs and modules stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data created according to the use of the server, and the like. Further, the memory 402 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 access to the memory 402.

The server further includes a power supply 403 for supplying power to each component, and preferably, the power supply 403 may be logically connected to the processor 401 through a power management system, so as to implement functions of managing charging, discharging, and power consumption through the power management system. The power supply 403 may also include any component of one or more dc or ac power sources, recharging systems, power failure detection circuitry, power converters or inverters, power status indicators, and the like.

The server may also include an input unit 404, the input unit 404 being operable to receive input numeric or character information and to generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

Although not shown, the server may further include a display unit and the like, which will not be described in detail herein. Specifically, in this embodiment, the processor 401 in the server loads the executable file corresponding to the process of one or more application programs into the memory 402 according to the following instructions, and the processor 401 runs the application program stored in the memory 402, thereby implementing various functions as follows:

receiving a reservation request sent by a sender, determining meeting time selected by the sender according to the reservation request, and generating a link address corresponding to the audio and video meeting; adding a sender into the audio and video conference within the conference time; receiving a joining request sent by a receiver through a third-party platform based on a link address in the meeting time, and joining the receiver into the audio and video meeting according to the joining request; and sending the audio and video data generated in the audio and video conference to a sender and sending the audio and video data to a receiver through a third-party platform.

Optionally, the step of receiving a join request sent by the receiver through the third party platform based on the link address within the meeting time, and joining the receiver into the audio/video meeting according to the join request may include: receiving a joining request sent by a receiver through a third-party platform based on a link address in the meeting time, and carrying out identity verification on the receiver according to the joining request; and when the identity authentication is passed, adding the receiver into the audio and video conference.

In the foregoing embodiments, the descriptions of the respective embodiments have respective emphasis, and parts that are not described in detail in a certain embodiment may refer to the above detailed description of the audio and video processing method, and are not described herein again.

As can be seen from the above, the embodiment of the present invention can determine the conference time selected by the sender according to the received reservation request sent by the sender, and generate the link address corresponding to the audio and video conference; then, the sender is added into the audio and video conference within the conference time, and the receiver is added into the audio and video conference according to the received addition request sent by the receiver through the third-party platform based on the link address; at the moment, the audio and video data generated in the audio and video conference can be sent to the sender and sent to the receiver through the third-party platform. According to the scheme, the sender and the receiver can join the same audio and video conference through different platforms, and the audio and video data generated in the audio and video conference are respectively sent to the sender and the receiver through different platforms, so that the audio and video data interaction of the sender and the receiver in the same audio and video conference through different platforms is realized, and the flexibility and the convenience of audio and video data transmission are greatly improved.

It will be understood by those skilled in the art that all or part of the steps of the methods of the above embodiments may be performed by instructions or by associated hardware controlled by the instructions, which may be stored in a computer readable storage medium and loaded and executed by a processor.

To this end, the embodiment of the present invention provides a storage medium, in which a plurality of instructions are stored, where the instructions can be loaded by a processor to execute the steps in any one of the audio/video processing methods provided by the embodiments of the present invention. For example, the instructions may perform the steps of:

The above operations can be implemented in the foregoing embodiments, and are not described in detail herein.

Wherein the storage medium may include: read Only Memory (ROM), Random Access Memory (RAM), magnetic or optical disks, and the like.

Since the instructions stored in the storage medium can execute the steps in any audio/video processing method provided in the embodiment of the present invention, the beneficial effects that can be achieved by any audio/video processing method provided in the embodiment of the present invention can be achieved, which are detailed in the foregoing embodiments and will not be described herein again.

The audio and video processing method, device and storage medium provided by the embodiment of the present invention are described in detail above, and a specific example is applied in the text to explain the principle and the implementation of the present invention, and the description of the above embodiment is only used to help understanding the method and the core idea of the present invention; meanwhile, for those skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.

Claims

1. An audio/video processing method, comprising:

adding the sender to the audio and video conference within the conference time;

receiving a joining request sent by a receiver through a third-party platform based on the link address in the conference time, joining the receiver into the audio and video conference according to the joining request, wherein the third-party platform comprises an instant communication platform, and the joining request is generated by clicking or activating the link address by the receiver through the third-party platform;

audio and video data generated in the audio and video conference are sent to the sender and sent to the receiver through the third-party platform;

wherein roles may be interchanged between the receiver and the sender.

2. The audio and video processing method according to claim 1, wherein the step of receiving a join request sent by a receiver through a third party platform based on the link address within the conference time, and joining the receiver to the audio and video conference according to the join request comprises:

receiving a joining request sent by a receiver through the third-party platform based on the link address in the meeting time, and performing identity authentication on the receiver according to the joining request;

and when the identity authentication is passed, adding the receiver into the audio and video conference.

3. The audio-video processing method according to claim 2, wherein before the step of generating the link address corresponding to the audio-video conference, the method further comprises:

receiving an identity authentication request sent by the sender, and displaying an authentication reservation interface according to the identity authentication request;

receiving a third party account type and an account identification, which are sent by the sender and correspond to a third party platform, of the receiver in the verification reservation interface;

and generating conference information according to the third party account type and the account identification.

4. The audio and video processing method according to claim 3, wherein the step of receiving the join request sent by the receiver through the third party platform based on the link address within the meeting time and authenticating the identity of the receiver according to the join request comprises:

receiving a join request sent by a receiver through the third-party platform based on the link address in the meeting time, and acquiring meeting information according to the join request;

when the identity authentication is determined to be needed according to the conference information, receiving account information sent by the receiver through the third-party platform;

and when the account information is matched with the third party account type and the account identification, determining that the identity authentication is passed.

5. The audio-video processing method according to claim 1, wherein the step of joining the sender to the audio-video conference within the conference time comprises:

receiving a login request sent by the sender within the meeting time;

and adding the sender into the audio and video conference according to the login request.

6. The audio/video processing method according to claim 1, wherein the step of generating a link address corresponding to the audio/video conference according to the reservation request comprises:

establishing an audio and video conference according to the reservation request, and distributing a conference identifier for the audio and video conference;

and generating a link address corresponding to the audio and video conference according to the conference identifier.

7. The audio-video processing method according to any one of claims 1 to 6, wherein the step of sending audio-video data generated in the audio-video conference to the sender and to the receiver through the third party platform comprises:

acquiring language types required by the sender and the receiver, and processing the audio data in the audio and video data according to the language types to obtain processed audio data;

and sending the video data in the audio and video data and the processed audio and video data to the sender and the receiver through the third-party platform.

8. The audio/video processing method according to claim 7, wherein the processed audio data includes converted audio data and/or translated text, the step of obtaining the language type required by the sender and the receiver, and processing the audio data in the audio/video data according to the language type to obtain the processed audio data includes:

acquiring a first language type required by the sender and a second language type required by the receiver;

identifying audio data in the audio and video data to obtain identified audio data;

and converting the recognized audio data according to the first language type and the second language type to obtain converted audio data, and/or translating the recognized audio data to obtain translated characters.

9. An audio-video processing apparatus, characterized by comprising:

the second joining unit is used for receiving a joining request sent by a receiver through a third-party platform based on the link address in the conference time, and joining the receiver into the audio and video conference according to the joining request, wherein the third-party platform comprises an instant communication platform, and the joining request is generated by clicking or activating the link address by the receiver through the third-party platform;

and the sending unit is used for sending the audio and video data generated in the audio and video conference to the sender and sending the audio and video data to the receiver through the third party platform, wherein roles can be exchanged between the receiver and the sender.

10. The audio-video processing device according to claim 9, wherein the second adding unit includes:

the verification subunit is used for receiving a join request sent by a receiver through the third-party platform based on the link address in the meeting time, and performing identity verification on the receiver according to the join request;

and the adding subunit is used for adding the receiving party into the audio and video conference when the identity verification is passed.

11. The audio-video processing device according to claim 10, further comprising:

the display unit is used for receiving the identity authentication request sent by the sender and displaying an authentication reservation interface according to the identity authentication request;

the receiving unit is used for receiving a third party account type and an account identification, corresponding to a third party platform, of the receiver sent by the sender in the verification reservation interface;

and the generating unit is used for generating conference information according to the third party account type and the account identification.

12. The audio-video processing device according to claim 9, wherein the processing unit comprises:

the establishing subunit is used for establishing an audio and video conference according to the reservation request and distributing a conference identifier for the audio and video conference;

and the generating subunit is used for generating a link address corresponding to the audio and video conference according to the conference identifier.

13. The audio-video processing device according to any one of claims 9 to 12, wherein the transmission unit includes:

the processing subunit is used for acquiring the language types required by the sender and the receiver, and processing the audio data in the audio and video data according to the language types to obtain processed audio data;

and the sending subunit is used for sending the video data in the audio and video data and the processed audio and video data to the sender and sending the processed audio and video data to the receiver through the third-party platform.

14. The audio/video processing device according to claim 13, wherein the processed audio data includes converted audio data and/or translated words, and the processing subunit is specifically configured to:

15. A storage medium storing a plurality of instructions adapted to be loaded by a processor to perform the steps of the audio/video processing method according to any one of claims 1 to 8.