CN111901552B - Multimedia data transmission method and device and electronic equipment - Google Patents

Abstract

The present application discloses a multimedia data transmission method, device and electronic device, which belong to the technical field of communication. The method includes: during the real-time transmission of multimedia data, determining whether target multimedia data exists in the multimedia data to be transmitted; if it is determined that the target multimedia data exists, canceling the transmission of the target multimedia data; wherein, the target multimedia data includes at least one of the following: The audio data within the non-target volume range, the audio data of the non-target timbre, the video image data of the first area, and the image data of the non-target object in the second area; the first area is the area where the transmission of video image data is cancelled in the video picture, The second area is the area of the transmission target object in the video picture.

Description

Multimedia data transmission method, device and electronic device

technical field

The present application belongs to the field of communication technologies, and in particular relates to a multimedia data transmission method, device and electronic device.

Background technique

With the popularization of Internet technology, there are more and more ways of acquiring and disseminating information. For example, users can conduct distance learning, remote meetings, and remote office at home.

Usually, each remote participant is in different environments. During the process of remote information transmission, there may be unexpected situations in their respective environments. For example, during the process of remote video teaching between teachers and students at home, pets or family members suddenly enter the teaching video screen. , or sudden noise in the surrounding environment.

However, in the current process of remote information transmission, electronic devices transmit all multimedia data collected by cameras and microphones in real time, and images and audios of emergencies are also transmitted to other participants, resulting in poor remote experience.

SUMMARY OF THE INVENTION

The purpose of the embodiments of the present application is to provide a multimedia data transmission method, device, and electronic device, which can solve the problem of poor remote experience in the process of remote information transmission.

In order to solve the above technical problems, this application is implemented as follows:

In a first aspect, an embodiment of the present application provides a method for transmitting multimedia data. The method includes: during the real-time transmission of multimedia data, determining whether target multimedia data exists in the multimedia data to be transmitted; if it is determined that the target multimedia data exists, then Cancel the transmission of target multimedia data; wherein, the target multimedia data includes at least one of the following: audio data within a non-target volume range, audio data of non-target timbre, video image data in the first area, and images of non-target objects in the second area data; the first area is the area in the video picture where the transmission of video image data is canceled, and the second area is the area of the video picture where the target object is to be transmitted.

In a second aspect, an embodiment of the present application provides a device for transmitting multimedia data, wherein the device for transmitting multimedia data includes: a determination module and a transmission module; Whether there is target multimedia data in the multimedia data of , the audio data of the non-target timbre, the video image data of the first area, and the image data of the non-target object in the second area; The area of the target object.

In a third aspect, an embodiment of the present application provides an electronic device, the electronic device includes a processor, a memory, and a program or instruction stored in the memory and executable on the processor, the program or instruction being executed by the processor When executed, the steps of the method as described in the first aspect are implemented.

In a fourth aspect, an embodiment of the present application provides a readable storage medium, where a program or an instruction is stored on the readable storage medium, and when the program or instruction is executed by a processor, the steps of the method according to the first aspect are implemented .

In a fifth aspect, an embodiment of the present application provides a chip, the chip includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is configured to run a program or an instruction, and implement the first aspect the method described.

In the embodiment of the present application, during the real-time transmission of multimedia data, the electronic device can determine whether target multimedia data exists in the multimedia data to be transmitted, and if there is target multimedia data, the electronic device can cancel the transmission of the target multimedia data. Because the target multimedia data includes at least one of the following: audio data within a non-target volume range, audio data of a non-target timbre, video image data of the first area, and portrait data of a non-target object in the second area. Therefore, during the transmission of remote information, before transmitting multimedia data, the electronic device can first determine whether there is target multimedia data that affects video transmission or audio transmission, and then cancel the transmission of the target multimedia data. In the process of user voice, It can cancel the sudden loud noise. In the process of user video, it can cancel the transmission of image data of other objects that interfere with the video screen, which not only avoids the transmission of images or audios of sudden situations to other people participating in the remote event, but also reduces the participation in the event. The probability of damage to the person's image in the remote event, which can make the remote experience better.

Description of drawings

1 is a schematic flowchart of a method for transmitting multimedia data according to an embodiment of the present application;

FIG. 2 is one of the interface schematic diagrams provided by the embodiment of the present application;

FIG. 3 is the second interface schematic diagram provided by the embodiment of the present application;

FIG. 4 is the third interface schematic diagram provided by the embodiment of the present application;

FIG. 5 is the fourth interface schematic diagram provided by the embodiment of the present application;

FIG. 6 is a schematic structural diagram of a possible structure of a multimedia data transmission apparatus provided by an embodiment of the present application;

7 is a schematic structural diagram of a possible electronic device provided by an embodiment of the present application;

FIG. 8 is a schematic hardware diagram of an electronic device according to an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

The terms "first", "second" and the like in the description and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It is to be understood that the data so used are interchangeable under appropriate circumstances so that the embodiments of the present application can be practiced in sequences other than those illustrated or described herein, and distinguish between "first", "second", etc. The objects are usually of one type, and the number of objects is not limited. For example, the first object may be one or more than one. In addition, "and/or" in the description and claims indicates at least one of the connected objects, and the character "/" generally indicates that the associated objects are in an "or" relationship.

It should be noted that, the multimedia data transmission method provided by the embodiments of the present application can be applied to scenarios where a user performs remote information transmission through an electronic device, for example, a user uses an electronic device to conduct a remote conference (such as a voice conference, , distance learning (such as online classes) and other scenarios.

The multimedia data transmission method provided by the embodiments of the present application will be described in detail below with reference to the accompanying drawings through specific embodiments and application scenarios thereof.

Fig. 1 is a kind of multimedia data transmission method that the embodiment of this application provides, this method comprises following S101 and S102:

S101. During the real-time transmission of multimedia data, the electronic device determines whether target multimedia data exists in the multimedia data to be transmitted.

Optionally, the multimedia data to be transmitted may be multimedia data collected in real time, or may be pre-recorded multimedia data transmitted in real time, which is not specifically limited in this embodiment of the present application.

It should be noted that, in this embodiment of the present application, the multimedia data to be transmitted may include at least one of audio data and video data.

The target multimedia data includes at least one of the following: audio data within a non-target volume range, audio data of non-target timbres, video image data of the first area, and image data of non-target objects in the second area.

(a) Audio data in a non-target volume range

It can be understood that, in this embodiment of the present application, the audio in the non-target volume range is the interfering audio that affects the transmission of multimedia data.

Optionally, the target volume range may be any of the following: less than or equal to the first volume threshold, greater than or equal to the second volume threshold, and between the first volume threshold and the second volume threshold.

Wherein, the first volume threshold is smaller than the second volume threshold.

Optionally, the target volume range may be manually set by the user, or configured by default in the electronic device, or automatically matched by the electronic device according to the audio data transmission scenario, which is not specifically limited in this embodiment of the present application.

For example, during the video conference, most of the voices of user A are within [20, 40]. When a car horn sound with a volume greater than 40 suddenly appears in the environment of user A during the video conference, the electronic device can determine the audio data of the car horn sound with a volume greater than 40 as the target multimedia data; If the voice of user A speaks to his family, the electronic device may determine the audio data of user A speaking at a volume of less than 20 as the target multimedia data.

(b) Audio data of non-target timbres

It can be understood that, in this embodiment of the present application, the audio data of the target timbre may be the audio data that the user expects to transmit normally, and the audio data of the non-target timbre may be the audio data that the user does not expect to transmit.

Optionally, the target timbres may include timbres of one or more different users. For example, the user enters his own voice, or the user enters the voices of multiple different users that are allowed to be transmitted.

Exemplarily, the target timbre may be the timbre determined by the user through the electronic device that can be transmitted in this remote event before the multimedia data starts to be transmitted. For example, the user can set the default to only transmit the audio of their own timbre this time.

It can be understood that the user can set it once before each transmission, or set it once by the system, and all subsequent settings will take effect.

For example, during the video conference, user A wishes to transmit only his own voice and not output the voice other than his own voice in the environment. When a non-user A suddenly exists in the environment of user A during the video conference the voice of user B, such as the voice of user B, the voice of user B is the target multimedia data.

(c) Video image data of the first area

Wherein, the first area is an area in the video picture where the transmission of video image data is cancelled.

It can be understood that, in this embodiment of the present application, the first area may be an area set by the user that is not allowed to transmit video images, and the first area is a part of the video image.

Combined with (b) in Figure 2, it is assumed that the first area is the area where the butterfly is located. In the process of video transmission, the electronic device may not transmit the video images corresponding to the area where the butterfly is located. For example, in the process of real-time transmission of video images, the video image data collected in real time by the camera corresponding to the area where the butterfly is located may not be transmitted.

(d) Image data of non-target objects in the second area

Wherein, the second area is the area of the transmission target object in the video picture.

It can be understood that the second area is an area set by the user that allows transmission of video images, and the second area is used to transmit the target object.

Exemplarily, the target object may be a user's biometric image (eg, a face image), a courseware image, or the like.

With reference to (b) in Figure 2, the area where the person is located is the second area, and the second area can be used to transmit the facial image of user A. If the facial images of other users are detected, the transmission of the facial images of other users can be cancelled. .

S102. If the target multimedia data exists, the electronic device cancels the transmission of the target multimedia data.

Specifically, in the case where the electronic device transmits multimedia data collected in real time, that is, the electronic device collects and transmits the multimedia data in real time, the electronic device can determine whether the collected multimedia data has target multimedia data before transmission after collecting the multimedia data. If it exists, cancel the transmission of the target multimedia data.

For example, cancel the transmission of the voice of other users except the voice of the local user in the environment where the local user is located during the video conference, and cancel the transmission of the image of the pet that suddenly breaks into the screen during the video teaching process.

In the multimedia data transmission method provided by the embodiment of the present application, during the real-time transmission of multimedia data, the electronic device can determine whether there is target multimedia data in the multimedia data to be transmitted, and if there is target multimedia data, the electronic device can cancel the transmission of the target multimedia data. . Because the target multimedia data includes at least one of the following: audio data within a non-target volume range, audio data of a non-target timbre, video image data of the first area, and portrait data of a non-target object in the second area. Therefore, during the transmission of remote information, before transmitting multimedia data, the electronic device can first determine whether there is target multimedia data that affects video transmission or audio transmission, and then cancel the transmission of the target multimedia data. In the process of user voice, It can cancel the sudden loud noise. In the process of user video, it can cancel the transmission of image data of other objects that interfere with the video screen, which not only avoids the transmission of images or audios of sudden situations to other people participating in the remote event, but also reduces the participation in the event. The probability of damage to the person's image in the remote event, which can make the remote experience better.

Optionally, in the multimedia data transmission method provided by the embodiment of the present application, the above S102 may be specifically performed by the following S102a:

S102a: If it is determined that target multimedia data exists in the collected multimedia data, the electronic device cancels transmission of the target multimedia data, and transmits other multimedia data in the collected multimedia data.

Specifically, if it is determined that the target multimedia data is collected before the multimedia data is transmitted, the electronic device cancels the transmission of the target multimedia data, and transmits other multimedia data in the collected multimedia data.

Exemplarily, in the process of online teaching by the teacher, if the voices of other people appear in the environment where the teacher is located, the voices of other people can be filtered out, and only the voices of the teacher are transmitted. If a cat appears in the image area where the teacher's face is located, the transmission of the cat's image can be canceled, so that the teacher's image before the cat appears is repeatedly transmitted.

It can be understood that, in the process of real-time transmission of remote information, the electronic device collects data while performing the multimedia data transmission method provided by the embodiment of the present application, which can eliminate the target multimedia data in the collected multimedia data, and the transmission excludes the target multimedia data. other multimedia data, which improves the quality of multimedia data transmission.

Based on this solution, in the process of real-time transmission, the electronic device can collect real-time multimedia data and process it according to the transmission method provided in this application, that is, determine whether there is target multimedia data, and then do not affect the transmission of other real-time multimedia data. Under the circumstance, the transmission of the target multimedia data can be cancelled, so that the transmission of other real-time multimedia data can be ensured, and the transmission effect is better.

Optionally, in the multimedia data transmission method provided in this embodiment of the present application, before the above-mentioned S102, the following S103 and S104 may be further included:

S103, the electronic device receives the setting input of the user.

Optionally, the setting input may be the setting input in the system setting interface or the setting input in the application program.

Optionally, the setting input may be the user's input before transmitting the multimedia data, or may be the user's input during the process of transmitting the multimedia data, which is not specifically limited in this embodiment of the present application.

S104. In response to the setting input, the first device determines a multimedia data transmission rule.

Wherein, determining the multimedia data transmission rule includes at least one of the following: determining that the transmission volume is within a preset range, determining the transmission tone, determining the first area in the video image, and determining the object of the first area to replace the video image. The picture corresponding to the first area is the picture whose transmission is canceled during the video transmission process.

The user can set the multimedia data transmission rules as needed, and the following (1) to (7) are setting examples provided by the embodiment of the present application:

(1) Set the area where video images are allowed to be transmitted.

The user can select the video transmission management in the electronic device setting interface, the electronic device turns on the camera and displays the video preview interface, as shown in (a) in Figure 2, if the user can click the "Intercept Area" button in the video preview interface, then The electronic device can display the two closed-loop areas shown in (b) in FIG. 2 , one area is the area where the butterfly is located, and the other area is the area where the user image is located, and the user can select one of the closed-loop areas in the current video display result. , and click the "Confirm" button to confirm that the closed loop area is an area that allows the transmission of video images.

(2) Set the replacement area and replacement object

After setting the region where video transmission is allowed, the user can also set a replacement region in the region where video transmission is not allowed, and a replacement object in the replacement region to replace the video picture in the region.

It should be noted that the number of replacement regions may be one or more.

Wherein, the replacement object may be a picture, a video, an office document (such as ppt/word/pdf), and the like.

Exemplarily, in conjunction with (b) in FIG. 2 , if the user intercepts the area where his own image is located and the area where the butterfly is located, as shown in (a) in FIG. 3 , the electronic device can divide the video screen into Three sub-areas: the area where the user's own image is located, the area where the butterfly is located, and the area other than the user's own image and the butterfly. The electronic device can identify the current sub-areas, name each area as area A, area B, and area C, and then display them on the interface. Each area can be selected to allow recording by default. When the user selects it, the electronic device does not allow the recording of video images in this area.

After the user checks the area B and the area C, as shown in (a) in FIG. 3 , the electronic device can display the “Replace” control at the corresponding position, and the electronic device can display the “Replace” control in FIG. 3 after the user clicks the “Replace” control. In the interface shown in (b), the user can select "Smart Replacement", "Replace Image" or "Replace Video". The replacement picture and the replacement video may be provided by the electronic device by default for the user, or may be customized by the user.

It should be noted that, if the user does not select a replacement picture after checking the box, the electronic device can display a still picture in the current state, for example, the butterfly area keeps displaying the butterfly image during the subsequent video process.

It can be understood that if the user chooses to display a static picture in the video screen, the electronic device can transmit only the video image data of the designated area during video data transmission, which can reduce the load of data transmission, reduce traffic consumption, and make the video process smoother.

In this embodiment of the present application, after the user selects "smart replacement", the electronic device can perform intelligent replacement based on the current usage scenario.

Exemplarily, during remote teaching, the electronic device can locate the number of pages and specific knowledge points of the textbook corresponding to the current class by intelligently recognizing the lecturer's voice, and the replacement area can prompt the number of pages and specific knowledge points. Even if you are interrupted by the outside world during the online class, you can know the current teaching progress as soon as possible.

Exemplarily, during a remote conference, the electronic device can identify the current usage scenario as a remote conference, and can display the conference theme in the replacement area. When a participant of the electronic device expresses an opinion, the electronic device can automatically extract the The opinions expressed by the participants (for example, the user's audio is detected by the microphone), displayed in the replacement area and transmitted to other participants.

(3) Set the timbres that are allowed to be transferred

Exemplarily, the user can select audio transmission management by timbre in the settings, and the electronic device can activate the microphone and display a piece of text on the interface, as shown in (a) in Figure 4, prompting the user "Please read the text: XXXXXX". After the user reads the text according to the prompt and clicks the "I have read it" button, the electronic device stops the voice data collection, and starts to analyze and record the timbre of the input voice. After the user clicks the "Complete setting" button, the electronic device can Make sure the user setup is complete.

It should be noted that the user may choose to record the voices of multiple users, so that the voices of the multiple users are not filtered in the scenario of subsequent multimedia data transmission.

(4) Set the volume to allow transmission

Exemplarily, the user can select audio volume transmission management in the settings, and the electronic device can activate the microphone and display a piece of text in the interface, as shown in (b) in Figure 4, prompting the user "Please read the text: XXXXXX". After the user reads the text according to the prompt and clicks the "I have read" button, the electronic device stops the voice data collection and starts to determine the volume intensity of the entered voice. After the determined value, as shown in Figure 4 (b) As shown, the current volume level is 34, the user can refer to the volume value and input the transmittable volume intensity range as [20, 40]. After inputting, the electronic device can play the voice entered by the user with the volume intensity of 20 and 40 respectively, so that User adjusts the volume range allowed for recording.

It should be noted that, the above (4) and (5) can be set respectively after performing an audio collection process.

(5) Set the voice to end data transmission

Exemplarily, the user can select audio end transmission management in the settings, the electronic device can turn on the microphone to prompt the user to enter a voice, the user can trigger the electronic device to start collecting the data input by the user's voice, and after the user clicks confirm after reading, the electronic device can play After the user's recorded voice is confirmed, the setting is completed.

Of course, the user can also set which applications to use the ending voice.

(6) Set up abnormal data response mechanism

Exemplarily, the user can trigger the electronic device to display the abnormal audio and video data management interface in the setting interface, and then select the abnormal data response mechanism of "automatic rejection" or "barrage confirmation" in the interface. If the user selects "Auto Reject", it will be intercepted by default when the electronic device recognizes abnormal data input; if the user selects "Barrage Confirmation", it will prompt to confirm whether to transmit when the electronic device recognizes abnormal data input.

(7) Set up applications that use multimedia data transmission rules

Exemplarily, if the user clicks the “application” control in the interface shown in FIG. 3(a), FIG. 4(a) or FIG. 4(b), the electronic device can display the control shown in FIG. The interface shown, in which the user can select the applications that can be used using the multimedia data transmission rules.

It should be noted that the above-mentioned interfaces in FIGS. 3 to 5 may be triggered and displayed by the user in the system setting interface, or may be triggered and displayed by the user in an application program of multimedia data transmission, which is not specifically limited in this embodiment of the present application.

It can be understood that, if the user performs settings in the first application program, the settings take effect in the first application program by default, and the first application program may be any application program with a remote information transmission function in the electronic device.

Based on this solution, users can set multimedia data transmission rules from multiple angles, the setting method is simple and flexible, and can provide users with settable options from multiple angles, so that users can be more flexible in the process of participating in remote events according to their needs. Setting how to transmit multimedia data can improve the user experience.

Optionally, in the multimedia data transmission method provided in the embodiment of the present application, after determining that the target multimedia data exists, the following S105 and S106 may also be included:

S105. If it is determined that the target multimedia data exists, the electronic device displays prompt information.

The prompt information is used to prompt whether to transmit the target multimedia data.

S106, the electronic device receives the first input of the user.

It can be understood that the first input is the input of the user determining to cancel the transmission target multimedia data according to the prompt information.

Further, the above-mentioned S102 can be specifically executed by the following S102b:

S102b, the electronic device cancels the transmission of the target multimedia data in response to the first input.

Exemplarily, if in the process of video teaching, the cat has entered the capture area of the camera, the electronic device can prompt the user whether to transmit the image data of the cat. transmission.

Based on this solution, when the electronic device determines that there is target multimedia data in the multimedia data to be transmitted, the electronic device can first prompt the user whether to cancel the transmission of the target multimedia data, so that the user can determine whether the target multimedia data can be transmitted according to his needs. In the case of transmission, the transmission of the target multimedia data may not be cancelled, and if the user determines that the transmission is not required, the transmission of the target multimedia data may be cancelled, and the transmission mode is more flexible.

Optionally, in the case where the first file is displayed in the first area, the multimedia data transmission method provided by the embodiment of the present application may further include the following S107:

S107. The electronic device updates the content of the first file displayed in the first area in real time according to the target keyword in the audio data collected in real time.

Specifically, the electronic device may locate the position of the target keyword in the first file according to the target keyword.

Exemplarily, during the course of the teacher's online class, the courseware PPT is displayed synchronously in the first area, and the electronic device can determine whether to turn the PPT page according to the keywords in the teacher's speech, for example, detecting that the teacher "Let's look at the first page". Eight pages of content", the electronic device can display the eighth page of the PPT in the first area.

Based on this solution, in the process of multimedia data transmission, the electronic device can locate the content in the first file according to the target keyword in the audio data collected in real time, and display the located position in the first file. On the one hand, it is convenient for local users to view the content and speak according to the content;

Optionally, the multimedia data transmission method provided in this embodiment of the present application may further include the following S108:

S108. In the case of multi-user remote, if the first audio data entered in real time by the local user is collected, the electronic device displays text information corresponding to the first audio data in the video screen transmitted in real time according to the first audio data.

Specifically, the electronic device may perform the foregoing S108 when the microphone captures the speech of the local user.

For example, if the first audio data entered in real time by the local user is collected, text information corresponding to the first audio data is displayed in the first area according to the first audio data.

Exemplarily, during the video conference between user 1 and other users, the electronic device 1 of user 1 can recognize the sentence spoken by user 1, and display the words spoken by user 1 in the form of text in the replacement area selected by user 1, The electronic device 1 will include electronic devices transmitted to other users, and other electronic devices can display the text of the user 1 in the corresponding replacement position on the screen corresponding to the user 1, and can prompt other users of the views expressed by the user 1.

Based on this solution, the electronic device can perform text conversion according to the collected first audio in the case of multi-user remote, when the local user's speech is collected, and display the same as the first audio in the real-time transmission video screen. The text information corresponding to the audio data can facilitate other users to view the opinions of the user who is speaking, and the remote process experience is better.

It should be noted that, in the multimedia data transmission method provided by the embodiments of the present application, the execution body may be a multimedia data transmission device, or a control module in the multimedia data transmission device for executing the method for multimedia data transmission. In the embodiments of the present application, a method for performing multimedia data transmission by a multimedia data transmission apparatus is used as an example to describe the apparatus for transmitting multimedia data provided by the embodiments of the present application.

As shown in FIG. 6, an embodiment of the present application provides a multimedia data transmission apparatus. As shown in FIG. 6, the multimedia data transmission apparatus 600 includes: a determination module 601 and a transmission module 602; In the transmission process, determine whether there is target multimedia data in the multimedia data to be transmitted; the transmission module 602 is used to cancel the transmission of the target multimedia data if the determination module 601 determines that the target multimedia data exists; wherein, the target multimedia data includes at least one of the following : audio data within the non-target volume range, audio data of non-target timbre, video image data in the first area, and image data of non-target objects in the second area; the first area is the area in the video screen where the transmission of video image data is canceled , and the second area is the area of the transmission target object in the video picture.

Optionally, the transmission module is specifically configured to: if the determination module determines that the target multimedia data exists, cancel the transmission of the target multimedia data, and transmit other real-time multimedia data.

Optionally, the multimedia data transmission device further includes: a receiving module; a receiving module, used for receiving a user's setting input before the determining module determines whether there is target multimedia data in the multimedia data to be transmitted; a determining module, also used for responding to The setting input received by the receiving module determines the multimedia data transmission rule; wherein, determining the multimedia data transmission rule includes at least one of the following: determining that the transmission volume is within a preset range, determining the transmission tone, determining the first area in the video screen, determining the first The region replaces the object of the video picture; the picture corresponding to the first region is the picture whose transmission is canceled in the process of video transmission.

Optionally, the multimedia data transmission device further includes: a display module and a receiving module; the display module is used to display prompt information after the determination module determines that the target multimedia data exists, and the prompt information is used to prompt whether to transmit the target multimedia data; the receiving module, is used for receiving the first input of the user; the transmission module is specifically configured to cancel the transmission of the target multimedia data in response to the first input.

Optionally, the first area displays the first file; the multimedia data transmission device further includes: a display module; the display module is configured to update the first file displayed in the first area in real time according to the target keyword in the audio data collected in real time Content.

Optionally, the multimedia data transmission device further includes: a display module; a display module is used to collect the first audio data entered by the local user in real time in the case of a multi-user remote, and transmit the data in real time according to the first audio data. Text information corresponding to the first audio data is displayed in the video picture of the .

An embodiment of the present application provides a multimedia data transmission device. During the real-time transmission of multimedia data, the multimedia data transmission device can determine whether target multimedia data exists in the multimedia data to be transmitted. If there is target multimedia data, the multimedia data transmission device can Cancel the transfer of target multimedia data. Because the target multimedia data includes at least one of the following: audio data within a non-target volume range, audio data of a non-target timbre, video image data of the first area, and portrait data of a non-target object in the second area. Therefore, the multimedia data transmission device may, in the process of transmitting the remote information, before transmitting the multimedia data, first determine whether there is target multimedia data that affects the video transmission or audio transmission, and then cancel the transmission of the target multimedia data. It can cancel the sudden loud noise, and in the process of user video, can cancel the transmission of image data of other objects that interfere with the video screen, which not only avoids the transmission of sudden images or audio to other people participating in the remote event, but also Reduce the probability of damage to the image of people involved in remote events, which can lead to a better remote experience.

The multimedia data transmission device in this embodiment of the present application may be a device, or may be a component, an integrated circuit, or a chip in a terminal. The apparatus may be a mobile electronic device or a non-mobile electronic device. Exemplarily, the mobile electronic device may be a mobile phone, a tablet computer, a notebook computer, a palmtop computer, an in-vehicle electronic device, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (personal digital assistant). assistant, PDA), etc., the non-mobile electronic device can be a server, a network attached storage (NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., this application Examples are not specifically limited.

The multimedia data transmission device in the embodiment of the present application may be a device with an operating system. The operating system may be an Android (Android) operating system, an ios operating system, or other possible operating systems, which are not specifically limited in the embodiments of the present application.

The multimedia data transmission apparatus provided in the embodiment of the present application can implement the various processes implemented by the method embodiments in FIG. 1 to FIG. 5 , and to avoid repetition, details are not described here.

Optionally, as shown in FIG. 7 , an embodiment of the present application further provides an electronic device 700, including a processor 701, a memory 702, a program or instruction stored in the memory 702 and executable on the processor 701, the program Or, when the instruction is executed by the processor 701, each process of the foregoing multimedia data transmission method embodiments can be implemented, and the same technical effect can be achieved. To avoid repetition, details are not described here.

It should be noted that the electronic devices in the embodiments of the present application include the above-mentioned mobile electronic devices and non-mobile electronic devices.

FIG. 8 is a schematic diagram of a hardware structure of an electronic device implementing an embodiment of the present application.

The electronic device 1000 includes but is not limited to: a radio frequency unit 1001, a network module 1002, an audio output unit 1003, an input unit 1004, a sensor 1005, a display unit 1006, a user input unit 1007, an interface unit 1008, a memory 1009, and a processor 1010, etc. part.

Those skilled in the art can understand that the electronic device 1000 may also include a power source (such as a battery) for supplying power to various components, and the power source may be logically connected to the processor 1010 through a power management system, so that the power management system can manage charging, discharging, and power functions. consumption management and other functions. The structure of the electronic device shown in FIG. 8 does not constitute a limitation on the electronic device. The electronic device may include more or less components than the one shown, or combine some components, or arrange different components, which will not be repeated here. .

Wherein, the processor 1010 is used to, during the real-time transmission of multimedia data, determine whether there is target multimedia data in the multimedia data to be transmitted; the network module 1002 is used to cancel the transmission of the target multimedia data if it is determined that the target multimedia data exists; Wherein, the target multimedia data includes at least one of the following: audio data within the non-target volume range, audio data of non-target timbre, video image data in the first area, and image data of non-target objects in the second area; the first area is The area in the video picture where the transmission of video image data is canceled, and the second area is the area in the video picture where the target object is to be transmitted.

The embodiment of the present application provides an electronic device. During the real-time transmission of multimedia data, the electronic device can determine whether target multimedia data exists in the multimedia data to be transmitted. If the target multimedia data exists, the electronic device can cancel the transmission of the target multimedia data. Because the target multimedia data includes at least one of the following: audio data within a non-target volume range, audio data of non-target timbre, video image data of the first area, and portrait data of a non-target object in the second area. Therefore, during the transmission of remote information, before transmitting multimedia data, the electronic device can first determine whether there is target multimedia data that affects video transmission or audio transmission, and then cancel the transmission of the target multimedia data. In the process of user voice, Can cancel the sudden loud noise, in the process of user video, can cancel the transmission of image data of other objects that interfere with the video screen, which not only avoids the transmission of sudden images or audio to other people participating in the remote event, but also reduces the participation in the event. The probability of damage to the person's image in the remote event, which can make the remote experience better.

Optionally, the network module 1002 is further configured to cancel the transmission of the target multimedia data and transmit other real-time multimedia data if it is determined that the target multimedia data exists.

Based on this solution, in the process of real-time transmission, the electronic device can collect real-time multimedia data and process it according to the transmission method provided in this application, that is, determine whether there is target multimedia data, and then do not affect the transmission of other real-time multimedia data. Under the circumstance, the transmission of the target multimedia data can be cancelled, so that the transmission of other real-time multimedia data can be ensured, and the transmission effect is better.

Optionally, the user input unit 1007 is configured to receive the setting input of the user; the processor 1010 is further configured to determine the multimedia data transmission rule in response to the setting input; wherein, determining the multimedia data transmission rule includes at least one of the following: determining the transmission The volume is in the preset range, the transmission tone is determined, the first area in the video image is determined, and the object of the first area to replace the video image is determined; the image corresponding to the first area is the image that is canceled during the video transmission process.

Based on this solution, users can set multimedia data transmission rules from multiple angles, the setting method is simple and flexible, and can provide users with settable options from multiple angles, so that users can be more flexible in the process of participating in remote events according to their needs. Setting how to transmit multimedia data can improve the user experience.

Optionally, the display unit 1006 is used to display prompt information, and the prompt information is used to prompt whether to transmit the target multimedia data; the user input unit 1007 is used to receive the first input of the user; the network module 1002 is also used to respond to the first input of the user. Enter to cancel the transfer of target multimedia data.

Based on this solution, when the electronic device determines that there is target multimedia data in the multimedia data to be transmitted, the electronic device can first prompt the user whether to cancel the transmission of the target multimedia data, so that the user can determine whether the target multimedia data can be transmitted according to his needs. In the case of transmission, the transmission of the target multimedia data may not be cancelled, and if the user determines that the transmission is not required, the transmission of the target multimedia data may be cancelled, and the transmission mode is more flexible.

Optionally, the display unit 1006 is further configured to update the content of the first file displayed in the first area in real time according to the target keyword in the audio data collected in real time.

Based on this solution, in the process of multimedia data transmission, the electronic device can locate the content in the first file according to the target keyword in the audio data collected in real time, and display the located position in the first file. On the one hand, it is convenient for local users to view the content and speak according to the content;

Optionally, the display unit 1006 is also used to display the target audio data corresponding to the target audio data in the video screen transmitted in real time according to the target audio data if the target audio data entered by the local user in real time is collected in the case of a multi-user remote. text information.

In the case of multi-user remote, the electronic device can perform text conversion according to the collected first audio when the speech of the local user is collected, and display the corresponding first audio data in the video screen transmitted in real time. Text information can facilitate other users to view the point of view of the user who is speaking, and the remote process experience is better.

It should be understood that, in this embodiment of the present application, the input unit 1004 may include a graphics processor (Graphics Processing Unit, GPU) 1041 and a microphone 1042. camera) to process the image data of still pictures or videos. The display unit 1006 may include a display panel 1061, which may be configured in the form of a liquid crystal display, an organic light emitting diode, or the like. The user input unit 1007 includes a touch panel 1071 and other input devices 1072 . The touch panel 1071 is also called a touch screen. The touch panel 1071 may include two parts, a touch detection device and a touch controller. Other input devices 1072 may include, but are not limited to, physical keyboards, function keys (such as volume control keys, switch keys, etc.), trackballs, mice, and joysticks, which will not be repeated here. Memory 1009 may be used to store software programs as well as various data, including but not limited to application programs and operating systems. The processor 1010 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and application programs, and the like, and the modem processor mainly handles wireless communication. It can be understood that, the above-mentioned modulation and demodulation processor may not be integrated into the processor 1010.

Embodiments of the present application further provide a readable storage medium, where a program or an instruction is stored on the readable storage medium, and when the program or instruction is executed by a processor, each process of the foregoing multimedia data transmission method embodiment can be achieved, and the same can be achieved. In order to avoid repetition, the technical effect will not be repeated here.

Wherein, the processor is the processor in the electronic device described in the foregoing embodiments. The readable storage medium includes a computer-readable storage medium, such as a computer read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a magnetic disk or an optical disk, and the like.

An embodiment of the present application further provides a chip, where the chip includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is configured to run a program or an instruction to implement the above multimedia data transmission method embodiments and can achieve the same technical effect, in order to avoid repetition, it will not be repeated here.

It should be understood that the chip mentioned in the embodiments of the present application may also be referred to as a system-on-chip, a system-on-chip, a system-on-a-chip, or a system-on-a-chip, or the like.

It should be noted that, herein, the terms "comprising", "comprising" or any other variation thereof are intended to encompass non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements, It also includes other elements not expressly listed or inherent to such a process, method, article or apparatus. Without further limitation, an element qualified by the phrase "comprising a..." does not preclude the presence of additional identical elements in a process, method, article or apparatus that includes the element. Furthermore, it should be noted that the scope of the methods and apparatus in the embodiments of the present application is not limited to performing the functions in the order shown or discussed, but may also include performing the functions in a substantially simultaneous manner or in the reverse order depending on the functions involved. To perform functions, for example, the described methods may be performed in an order different from that described, and various steps may also be added, omitted, or combined. Additionally, features described with reference to some examples may be combined in other examples.

From the description of the above embodiments, those skilled in the art can clearly understand that the method of the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is better implementation. Based on this understanding, the technical solution of the present application can be embodied in the form of a software product in essence or in a part that contributes to the prior art, and the computer software product is stored in a storage medium (such as ROM/RAM, magnetic disk, CD-ROM), including several instructions to make a terminal (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) execute the methods described in the various embodiments of this application.

The embodiments of the present application have been described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific embodiments, which are merely illustrative rather than restrictive. Under the inspiration of this application, without departing from the scope of protection of the purpose of this application and the claims, many forms can be made, which all fall within the protection of this application.

Claims (10)

1. A multimedia data transmission method, characterized in that,

in the real-time transmission process of the multimedia data, determining whether the target multimedia data exists in the multimedia data to be transmitted;

if the target multimedia data exists, canceling transmission of the target multimedia data;

wherein the target multimedia data comprises at least one of: video image data of a first region, image data of a non-target object in a second region; the first area is an area for canceling transmission of video image data in a video picture, and the second area is an area for transmitting the target object in the video picture;

if the target multimedia data exists, canceling transmission of the target multimedia data, including:

if the target multimedia data exists in the collected multimedia data, canceling transmission of the target multimedia data, and transmitting other multimedia data in the collected multimedia data, wherein the other multimedia data are used for replacing the target multimedia data;

the method further comprises the following steps:

under the condition that a first file is displayed in the first area, updating the content of the first file displayed in the first area in real time according to a target keyword in audio data acquired in real time, wherein the audio data is the audio data aiming at the first file.

2. The method of claim 1, wherein the target multimedia data further comprises at least one of:

audio data in a non-target volume range, audio data of a non-target timbre.

3. The method according to claim 1 or 2, wherein before determining whether the target multimedia data exists in the multimedia data to be transmitted, the method further comprises:

receiving a setting input of a user;

determining a multimedia data transmission rule in response to the setting input;

wherein the determining the multimedia data transmission rule comprises at least one of: determining that the transmission volume is in a preset range, determining the transmission tone, determining a first area in a video picture, and determining that the first area replaces an object of the video picture; and the picture corresponding to the first area is a picture which is cancelled in the video transmission process.

4. The method of claim 1 or 2, wherein after determining that the target multimedia data is present, the method further comprises:

displaying prompt information, wherein the prompt information is used for prompting whether the target multimedia data is transmitted or not;

receiving a first input of a user;

the canceling of the transmission of the target multimedia data comprises:

canceling transmission of the target multimedia data in response to the first input.

5. The method according to claim 1 or 2, characterized in that the method further comprises:

under the condition of multi-user remote, if first audio data recorded by a local user in real time are collected, text information corresponding to the first audio data is displayed in a video picture transmitted in real time according to the first audio data.

6. A multimedia data transmission apparatus, characterized in that the multimedia data transmission apparatus comprises: the device comprises a determining module, a transmission module and a display module;

the determining module is used for determining whether target multimedia data exists in the multimedia data to be transmitted in the real-time transmission process of the multimedia data;

the transmission module is used for canceling transmission of the target multimedia data if the determination module determines that the target multimedia data exists;

wherein the target multimedia data comprises at least one of: video image data of a first region, image data of a non-target object in a second region; the first area is an area for canceling transmission of video image data in a video picture, and the second area is an area for transmitting the target object in the video picture;

the transmission module is specifically configured to cancel transmission of the target multimedia data and transmit other multimedia data in the acquired multimedia data if it is determined that the target multimedia data exists in the acquired multimedia data, where the other multimedia data is used to replace the target multimedia data;

the display module is configured to update the content of the first file displayed in the first area in real time according to a target keyword in audio data acquired in real time under the condition that the first file is displayed in the first area, where the audio data is audio data for the first file.

7. The multimedia data transmission apparatus of claim 6, wherein the target multimedia data further comprises at least one of:

audio data in a non-target volume range, audio data of a non-target timbre.

8. The multimedia data transmission apparatus according to claim 6 or 7, further comprising: a receiving module;

the receiving module is used for receiving the setting input of a user before the determining module determines whether the target multimedia data exists in the multimedia data to be transmitted;

the determining module is further configured to determine a multimedia data transmission rule in response to the setting input received by the receiving module;

wherein the determining the multimedia data transmission rule comprises at least one of: determining that the transmission volume is in a preset range, determining the transmission tone, determining a first area in a video picture, and determining that the first area replaces an object of the video picture; and the picture corresponding to the first area is a picture which is cancelled in the video transmission process.

9. The multimedia data transmission apparatus according to claim 6 or 7, further comprising: the device comprises a display module and a receiving module;

the display module is used for displaying prompt information after the determining module determines that the target multimedia data exists, wherein the prompt information is used for prompting whether the target multimedia data is transmitted or not;

the receiving module is used for receiving a first input of a user;

the transmission module is specifically configured to cancel transmission of the target multimedia data in response to the first input.

10. The multimedia data transmission apparatus according to claim 6 or 7, further comprising: a display module;

the display module is used for displaying text information corresponding to first audio data in a video picture transmitted in real time according to the first audio data if the first audio data input by a local user in real time is collected under the condition of multi-user remote.

Images