CN115705180A

CN115705180A - Audio processing method and device, electronic equipment and storage medium

Info

Publication number: CN115705180A
Application number: CN202110912778.XA
Authority: CN
Inventors: 张宇; 牛坤; 陈雍; 张磊; 颜耀; 周千郁; 孙长长; 许沛龙; 陈龙
Original assignee: Beijing Xiaomi Mobile Software Co Ltd
Current assignee: Beijing Xiaomi Mobile Software Co Ltd
Priority date: 2021-08-10
Filing date: 2021-08-10
Publication date: 2023-02-17

Abstract

The disclosure relates to an audio processing method and apparatus, an electronic device and a storage medium. The audio processing method applied to the first device comprises the following steps: determining one or more target applications; determining an audio stream generated by the target application program from the audio stream generated by the first device; the audio stream generated by the target application program is a second type of audio stream, and the audio streams generated by the rest application programs are first type of audio streams; one of said audio streams being generated by an application; and sending the second type of audio stream to a second device, wherein the second type of audio stream is used for the output of the second device.

Description

Audio processing method and device, electronic equipment and storage medium

Technical Field

The present disclosure relates to the field of audio technologies, and in particular, to an audio processing method and apparatus, an electronic device, and a storage medium.

Background

The audio projection is: techniques to migrate an audio stream of one electronic device to output on another electronic device. For example, the audio stream of a mobile phone is sent to a sound box for output, or the audio stream of a television is sent to the sound box for output.

Illustratively, the smart speaker is an audio device having a network connection function and a relatively strong processing function, and may receive audio projection of a device such as a mobile phone, a tablet computer, a wearable device, or a notebook computer, and output a corresponding audio stream with a relatively large volume and/or a relatively good quality sound quality.

In the related art, when audio projection is performed, audio streams generated by the device are all transferred to the projected device for output, but in some cases, a user does not want all audio streams to be output by the projected device, which results in poor playing effect of the audio projection or low user satisfaction.

Disclosure of Invention

The disclosure provides an audio processing method and apparatus, an electronic device and a storage medium.

According to a first aspect of the embodiments of the present disclosure, there is provided an audio processing method applied in a first device, the method including:

determining one or more target applications;

determining an audio stream generated by the target application program from the audio stream generated by the first device; the audio stream generated by the target application program is a second type of audio stream, and the audio streams generated by the rest application programs are first type of audio streams; one of said audio streams being generated by an application;

and sending the second type of audio stream to a second device, wherein the second type of audio stream is used for the output of the second device.

Based on the above scheme, the determining the audio stream generated by the target application from the audio streams generated by the first device includes:

a media framework layer of the first device receives the audio streams of the first type generated by an application layer;

and determining the second type of audio stream according to the application identification ID of the target application program and the application ID carried by the first type of audio stream.

Based on the above scheme, the method further comprises:

when the first type of audio stream comprises a plurality of audio streams, mixing the plurality of audio streams contained in the first type of audio stream to obtain a third type of audio stream;

and outputting the third type audio stream.

Based on the above scheme, the determining one or more target applications includes:

displaying an access control of an audio projection page of a preset APP;

when detecting the operation of acting on entering the control, show the audio frequency of presetting APP throws the page, wherein, contain in the audio frequency throws the page: device information of one or more alternative devices to the second device;

when the operation of selecting at least one piece of equipment information is detected, determining the preset APP as the target application program; wherein the selected device identifier corresponds to the alternative device being the second device.

Based on the above scheme, the entry control for displaying the audio projection page of the preset APP comprises one of the following components:

displaying a button of the preset APP, wherein the button at least comprises the access control;

displaying an application interface of the preset APP, wherein the application interface at least comprises the access control.

Based on the above scheme, the audio projection page comprises:

the first volume adjusting bar is used for uniformly adjusting the volume of each second device or adjusting the volume of the first device;

and/or the presence of a gas in the gas,

and one of the second volume adjustment bars corresponds to one of the second devices, wherein the second volume adjustment bar is used for adjusting the volume of the second device.

Based on the scheme, the preset APP is an audio playing APP; the button and/or the audio projection page further comprises: a control for audio playing; the control of the audio playing is used for detecting and controlling the operation of the audio playing of the preset APP.

Based on the scheme, the first device has a first mode and a second mode; in the first mode, the number of the target application programs is one, and in the second mode, the number of the target application programs is one or more;

the determining one or more target applications includes:

determining one of said target applications when said first device is in said first mode;

determining one or more of the target applications while the first device is in the second mode.

According to a second aspect of the embodiments of the present disclosure, there is provided an audio processing apparatus, for use in a first device, the apparatus including:

a first determination module for determining one or more target applications;

the second determining module is used for determining the audio stream generated by the target application program from the audio stream generated by the first equipment; the audio stream generated by the target application program is the second type of audio stream, and the audio streams generated by the rest application programs are the first type of audio streams; one of said audio streams being generated by an application;

and the sending module is used for sending the second type of audio stream to second equipment, wherein the second type of audio stream is used for the output of the second equipment.

Based on the above scheme, the second determining module is configured to receive an audio stream generated by an application layer by a media frame layer of the first device; and determining the second type of audio stream according to the application identification ID of the target application program and the application ID carried by the audio stream generated by the application layer.

Based on the above scheme, the apparatus further comprises:

the mixing module is used for mixing a plurality of audio streams contained in the first type of audio stream to obtain a third type of audio stream when the first type of audio stream contains a plurality of audio streams;

and the output module is used for outputting the third type of audio stream.

Based on the above scheme, the first determining module includes:

the display sub-module is used for displaying an access control of an audio projection page of a preset APP; when detecting the operation of acting on entering the control, show the audio frequency of presetting APP throws the page, wherein, contain in the audio frequency throws the page: device information of one or more alternative devices to the second device;

a determining submodule, configured to determine, when an operation of selecting at least one piece of device information is detected, that the preset APP is the target application program; wherein the selected device identifier corresponds to the alternative device being the second device.

Based on the above scheme, the display sub-module is specifically configured to execute one of the following:

Based on the above scheme, the audio projection page comprises:

and/or the presence of a gas in the gas,

Based on the scheme, the preset APP is an audio playing APP; the button and/or the audio projection page further comprises: a control for audio playing; and the control of the audio playing is used for detecting and controlling the operation of the audio playing of the preset APP.

Based on the scheme, the first device has a first mode and a second mode; in the first mode, the number of the target application programs is one, and the number of the second mode can be multiple;

the first determining module is configured to determine one of the target applications when the first device is in the first mode; determining one or more of the target applications while the first device is in the second mode.

According to a third aspect of embodiments of the present disclosure, there is provided an electronic apparatus including:

a memory for storing processor-executable instructions;

a processor coupled to the memory;

wherein the processor is configured to perform the audio processing method as described above.

According to a fourth aspect of embodiments of the present disclosure, there is provided a non-transitory computer-readable storage medium having instructions therein, which when executed by a processor of a computer, enable the computer to perform the audio processing method as described above.

The technical scheme provided by the embodiment of the disclosure can have the following beneficial effects:

in the embodiment of the disclosure, when the second device replaces the first device to play audio, only the second type of audio stream of the target Application program is transmitted to the second device, instead of transmitting all audio streams generated by all Application programs of the first device to the second device, so that audio projection with Application (APP) dimension is realized, audio streams that the second device is not desired to play are also transmitted to the second device to play, finer dimension and better audio playing effect are realized, and user experience is improved.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and together with the description, serve to explain the principles of the disclosure.

FIG. 1 is a flow diagram illustrating an audio processing method according to an exemplary embodiment;

FIG. 2 is a schematic flow diagram illustrating an audio processing method according to an exemplary embodiment;

FIG. 3 is a software hierarchy diagram of a desking system shown in accordance with an exemplary embodiment;

FIG. 4 is a diagram illustrating a determination flow of a target application, according to an exemplary embodiment;

FIG. 5 is a schematic illustration of the display effect of an audio projection page shown in accordance with an exemplary embodiment;

FIG. 6 is a diagram illustrating display effects of buttons and access controls in accordance with an exemplary embodiment; (ii) a

FIG. 7A is a page-switching diagram illustrating an audio processing method according to an exemplary embodiment;

FIG. 7B is a page-switching diagram illustrating an audio processing method according to an exemplary embodiment;

FIG. 7C is a page-switching diagram illustrating an audio processing method according to an exemplary embodiment;

FIG. 7D is a page-switching diagram illustrating an audio processing method according to an exemplary embodiment;

FIG. 8 is a schematic diagram illustrating the structure of an audio processing device according to an exemplary embodiment;

fig. 9 is a schematic structural diagram of an electronic device shown in accordance with an example embodiment.

Detailed Description

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. The following description refers to the accompanying drawings in which the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the exemplary embodiments below are not intended to represent all implementations consistent with the present disclosure. Rather, they are merely examples of devices consistent with certain aspects of the present disclosure, as detailed in the appended claims.

As shown in fig. 1, an embodiment of the present disclosure provides an audio processing method, applied in a first device, the method including:

s110: determining one or more target applications;

s120: determining an audio stream generated by the target application program from the audio stream generated by the first device; the audio stream generated by the target application program is the second type of audio stream, and the audio streams generated by the rest application programs are the first type of audio streams; one of said audio streams being generated by an application;

s130: and sending the second type of audio stream to a second device, wherein the second type of audio stream is used for the output of the second device.

The first device may be various mobile terminals and/or fixed terminals. The mobile terminal includes at least but is not limited to: the mobile phone, the tablet computer, the wearable device, the smart home device, the smart office or the notebook. The fixed terminal may include at least: smart televisions or projectors.

The wearable devices include, but are not limited to: smart band, smart watch, smart foot ring and/or head-mounted device. The head-mounted device includes but is not limited to: smart glasses and/or various head-worn Virtual Reality (VR) devices and/or Augmented Reality (AR) devices, and the like.

The intelligent household equipment comprises: the intelligent cleaning device and/or the intelligent kitchen ware device and/or the intelligent doorbell. The intelligent office equipment includes but is not limited to: intelligent printing equipment and/or network telephones, etc.

The intelligent cleaning device includes but is not limited to: intelligent washing machine, robot of sweeping floor. The intelligent kitchen appliance includes but is not limited to: intelligent refrigerator, intelligent cooking equipment. The intelligent cooking device comprises: intelligent oven, intelligent electric rice cooker etc..

These first devices may connect to a network, download information from the network, and may establish a connection with a second device, thereby transferring the second type of audio stream to the second device over the connection established with the second device.

The second type of audio stream here may comprise one or more audio streams. The number of audio streams contained in the second type of audio stream is equal to the number of the target application programs. For example, when the target application is multiple, the second type of audio stream includes multiple audio streams, and the number of audio streams included in the second type of audio stream is equal to the number of the target application. For another example, when the number of the target applications is 1, the second type of audio stream includes only one audio stream.

The second device may be a device comprising: any device of an audio output component. Illustratively, the audio output component may include: a loudspeaker. Illustratively, the second device may be various speakers, such as a smart speaker, a wireless headset, or a smart television.

Typically, the first device has a greater maximum output volume of the audio output component of the second device than the second device, and/or the second device has a higher audio output quality than the first device.

For example, a mobile phone as the first device may play music, but the maximum volume and sound quality of music played by the mobile phone may be limited by the size and/or hardware limitations of the mobile phone itself, and may not be as effective as a smart sound box. If the audio processing method of the embodiment of the present disclosure is used, only the second type audio stream of the music application program of the mobile phone may be transmitted to the smart sound box. Therefore, the situation that the called ring tone is suddenly inserted when the sound box plays music when the mobile phone is called currently is reduced. If the volume of the called ring tone is large and is usually a sound other than the sound expected by the user, the user may be frightened, and the sudden insertion of the called ring tone may interfere with the original music playing.

Meanwhile, in some cases, the ring of the person close to the user may be special, so privacy may also be involved, if the ring is directly played by the second device, the user may generate embarrassed emotion under the condition that other people are present, so that the privacy of the user is better protected, and the use satisfaction of the user is improved again.

For another example, the user is cooking with the intelligent cooking device, but the user does not want to stay in the kitchen and to stay in the bedroom, and then the second device may be an audio device located in the bedroom. The intelligent cooking equipment may be provided with: cooking App, recipe App, etc. If the smart device downloads a new recipe, an alert tone is generated, which is not the alert tone the user wants to hear, but the user may only want to hear the alert tone of the cooking completion level. By using the audio processing method provided by the embodiment of the disclosure, only the cooking App can be regarded as a target application program, and the audio stream corresponding to the prompt tone generated by the cooking App according to the cooking completion degree can be transmitted to the intelligent sound box in a bedroom for output by the intelligent sound box, so that the interference to a user is reduced, and the user experience is improved.

The target application may be a part of the applications installed in the first device, and the remaining applications may be any applications other than the target application.

In the disclosed embodiment, a connection is established between the first device and the second device, and the connection may be a wireless connection. The wireless connection includes, but is not limited to:

connecting by Bluetooth;

and connecting WiFi.

In some embodiments, the first device and the second device are a direct point-to-point connection, e.g., wiFi direct. In other embodiments, the first device and the second device may be connected via a gateway device. For example, the first device and the second device may be located within a local area network formed by the same WiFi router or wireless modem.

In some embodiments, as shown in fig. 2, the S120 may include:

s121: a media framework layer of the first device receives an audio stream generated by an application layer;

s122: and determining the second type of audio stream according to the application identification ID of the target application program and the application ID carried by the audio stream generated by the application layer.

The operating system of the first device may be various operating systems, such as an IOS system, an android system, or a saiban system, or a Windows system provided by microsoft.

And if the operating system of the first device is an android system, the application program of the installation system is located in an application layer. The android system further includes: media Framework (Framework). Fig. 3 is a schematic diagram illustrating a software hierarchy of a first device of an Android system. Located at the top most layer is an application layer, which may include: one or more applications, such as a social application, an instant messaging application, a music playing application, and the like. The application programs of each application program layer may generate audio streams, the audio streams generated by the remaining application programs except the target application program in the application layers are classified into a first type of audio stream which is positioned at the second highest layer and is a media Framework layer (Framework), the media Framework layer is the next layer of the application layers, and the audio streams generated by the application programs of the application layers are transmitted to the media Framework layer for processing. In the media framework layer, various components are laid out, which are shared by the applications of the application layer. As shown in fig. 3, there are also a library, a desk runtime library layer and a kernel layer under the media framework layer.

Typically, the audio output hardware (e.g., speakers) of the first device is common to all applications within the first device. If there are multiple applications generating audio streams at a time, in order to avoid omission, a mixing component is disposed in the media frame, and the mixing component mixes the audio streams generated by the multiple applications, and the audio stream output by the first device is a mixed audio stream, so that a user can hear two or more sounds in the audio stream output by the first device.

After an application is installed in the first device, the operating system assigns an application ID to the application to identify data interactions between the operating system and the application. Thus, different applications within the first device have different application IDs. The application ID is assigned by the operating system of the first device and may be different from an ID provided by the application store or developer after the application program is developed.

Of course, in another embodiment, the original ID of a different application (e.g., application store or developer) may also be the ID carried in the audio stream.

Generally, the ID reassigned by the first device is smaller relative to the original ID of the application, and thus has the characteristic of small bit overhead.

In the embodiment of the present disclosure, in order to implement audio projection of APP dimensions, before performing an original audio stream of each application program in a first device, an application ID carried by each audio stream is detected, and it is determined which audio streams belong to an audio stream generated by a target application program, and if the audio streams belong to the audio stream generated by the target application program, the audio streams do not participate in mixing of multiple audio streams generated by remaining application programs. In this manner, the second type audio stream of the target application is separated from the first type audio stream generated by the non-target application before entering the mixing component and is transmitted to the second device.

If the second type of audio stream has only one audio stream, the audio stream can bypass the mixing component (i.e. not enter the mixing component) and be directly submitted to the communication component, and be sent to the second device for playing by the second device. The communication components may include, but are not limited to: various wireless communication components. The communication component may be a wireless communication chip of a hardware layer of the first device, or the like.

If the second type of audio stream comprises multiple audio streams,

in the embodiment of the disclosure, according to the application ID of the target application program, simple and convenient separation of the first type of audio stream and the second type of audio stream is realized in the media framework layer, so that audio projection of App dimension can be realized.

The method further comprises the following steps:

and outputting the third type audio stream.

If the first type of audio stream comprises a plurality of audio streams, the plurality of audio streams included in the first type of audio stream need to be mixed into one audio stream before the first device outputs the audio streams, and the mixed audio stream is referred to as a third type of audio stream.

In this way, the audio stream generated by the first device can be synchronously output with other devices.

In an embodiment, if there are a plurality of target applications, the mixing parameters may be used to mix audio streams included in a plurality of audio streams of the second type according to the persistence of the audio streams provided by each target application.

For example, only one target application of the plurality of target applications may provide an audio stream stably and continuously, and other target applications may provide the audio stream suddenly.

Assume that the plurality of target applications include: when the application program A continuously provides the audio stream and at least one application program B which can provide the audio stream in a burst mode is further included, when the second type audio streams of a plurality of target application programs are mixed at a time point, the following processing can be carried out:

according to the mixing parameters, the audio stream provided by the application program B is shaped, the volume of the shaped audio stream is gradually increased to a preset value from the burst insertion moment, and the audio stream provided by the application program A is not shaped;

mixing the audio stream provided by the unshaped application A and the audio stream of the application B after shaping;

and sending one audio stream obtained by mixing a plurality of audio streams contained in the second type of audio stream to the second device. The audio stream of the second type, which is a mixture of multiple audio streams, may be referred to as a fourth audio stream.

By shaping before mixing, the playing effect of the audio stream provided by the application program B which is inserted into the audio stream in a burst way can be gradually achieved, so that the situation that the user is threatened by the playing of the audio stream which is inserted into the application program B in a burst way is reduced.

Typically, the application a may be a music playing application, a webcast playing application, or a language learning playing application.

And application B may be a lifestyle service type application. For example, a user orders takeout through a life service application, and when takeout is performed to a certain extent, a corresponding alert tone may be configured, but the alert tone is not continuous on the one hand, and on the other hand, the insertion time may be determined by the takeout completion degree, so that the takeout is bursty, and if the takeout is directly inserted, a bad playing effect of the user may be startled by an excessively large volume of the bursty insertion or an excessively large difference from a playing effect of an audio stream provided by the application a, and therefore, a gradual effect of audio data playing of the bursty application B may be achieved by shaping the audio stream before mixing.

For another example, the application B may further include: the timing application program is provided with a timing prompt on the first device, if the timing prompt needs to be played by the second device, the timing prompt is also sudden obviously, and the prompt can be gradually input by shaping before the prompt is mixed with audio streams of other target application programs.

The shaping of the audio stream having the bursty characteristic in the second type of audio stream may be to perform a reduction process on the amplitude of the audio data included in the audio stream according to the shaping parameter. The shaping parameter here may indicate a volume profile of the shaped audio stream.

In some embodiments, the S110 may include at least one of:

determining the application program corresponding to the application interface with the detected projection operation as the target application program;

determining the target application program according to the selection operation of the application program icon in the configuration interface;

determining a first type of application program in the first device as the target application program according to a first configuration;

and according to the second configuration, determining the application programs except the second type of application programs in the first equipment as the target application programs.

The projection operation may be a user operation detected by the first device at the human-computer interaction interface. For example, a casting operation is detected on an application interface of an application playing class application program, and the application program corresponding to the application interface where the casting operation is detected is the target application program.

In some embodiments, after the user may arrive at home, the user may configure the audio playback of the mobile phone in the setting page of the first device if the user wants to transfer the audio playback of the mobile phone to a device such as a sound box, and does not want to manually trigger each time. In this scenario, if the distance between the first device and the second device is close enough, or the first device detects that the connection with the second device is successfully established, the projection can be automatically performed, and the configuration information is obtained according to the selection operation of the configuration page, so as to determine the target application program.

In some embodiments, the first device may be automatically configured when shipped from the factory or when the operating system in the device is updated, and the configuration may be generated without manual operation by the user or without execution by the user on a configuration page.

The first configuration may include IDs of one or more applications, and the IDs may be presented in a list, and the application corresponding to the ID in the list is the first type of application.

Illustratively, the first type of application includes at least one of:

an audio playback-type application;

network broadcast type application programs;

language learning class APP.

The language learning type APP can be used for language learning of users, such as an English learning type APP, a French learning type APP and the like.

The second configuration is equivalent to defining which applications cannot be projected directly to the second device without additional manipulation by the user. However, if the first device initiates the automatic audio stream projection mode, applications other than the second application may be determined to be the target application according to the second configuration.

Illustratively, the second type of application includes at least one of:

a telephone APP;

instant messaging APP.

For example, the telephone APP here may include: a circuit switched domain based application.

Typical instant messaging APPs include, but are not limited to: wechat, QQ, microblog, etc.

In some embodiments, as shown in fig. 4, the S110 may include:

s111: displaying an access control of an audio projection page of a preset APP;

s112: when detecting the operation of acting on entering the control, show the audio frequency of presetting APP throws the page, wherein, contain in the audio frequency throws the page: device information of one or more alternative devices to the second device;

s113: when the operation of selecting at least one piece of equipment information is detected, determining the preset APP as the target application program; wherein the selected device identifier corresponds to the alternative device being the second device.

The audio projection page may be a control page that sends an audio stream of a preset APP to other devices for playing.

The preset APP may be an APP capable of providing an audio stream, and specifically may be the APP mentioned in any of the foregoing embodiments, for example, the preset APP may be an audio playing APP or a network broadcasting APP.

In one embodiment, the audio projection page may be different from an application interface of the preset APP.

FIG. 5 is a schematic diagram of an audio projection page.

For example, the display area of the audio projection page may be smaller than the area of the display screen of the first device.

Through the display of the access control, a user can conveniently trigger the first device to quickly access the audio projection page through the operation of acting on the access control. FIG. 6 illustrates a button containing an access control.

Device information of the alternative devices of the plurality of second devices is displayed on the audio projection page. The device information may be information that may identify or distinguish the alternative device, such as a device icon and/or a device name and/or description information of the device. As shown in fig. 5 and fig. 7A to 7D, the description information of the device may include: the favorite sound box of the living room and the like. Illustratively, as shown in fig. 5 and fig. 7A to 7D, the lower half of the audio projection page displays device information of the device alternative as the second device.

In some embodiments, the alternative device may be an audio device that has established a connection with the first device, illustratively an audio device that has established a wireless connection with the first device. The audio device may be any device capable of audio playback. The wireless connections include, but are not limited to: and (4) WiFi connection.

The device information of the alternative devices is displayed in the audio projection page, and if the selection operation of the device information acting on at least one alternative device is detected, the current preset APP is determined to be the target application program.

There may be a plurality of display positions of the entry control, and for example, the S111 may include:

or alternatively

In some embodiments, if the current preset APP is not started, the preset APP may not be started, and the touch operation based on a desktop or a negative screen or other pages may also be performed to trigger the first device to display the button of the preset APP, so that the access control can be quickly found.

If the current preset APP is started, the first device displays the application interface of the preset APP, and the entering control is displayed in the application interface of the preset APP, so that the first device can be triggered to be switched to display the audio projection page conveniently and rapidly under the condition that the application interface of the preset APP does not need to be exited. The 1 st page in fig. 7D is an application interface (play interface) of the music APP, and the entry control is displayed in the play interface.

In the embodiment of the disclosure, the display screen of the first device may display a button of the preset APP. The button may be a button having an area smaller than a display area of a display screen of the first device. In general, the preset APP button may be a strip-shaped chamfered rectangular button. One or more controls are included within the button.

In some embodiments, the area of the audio projection page may be larger than the display area of the button.

In some embodiments, the button displaying the preset APP comprises at least one of:

displaying the button of the preset APP in the control center;

displaying a button of the preset APP on a notification bar;

and displaying the preset APP button on a screen locking interface.

The display positions of the buttons are multiple, and the specific implementation is not limited to the above examples.

Because the button of presetting the APP can show everywhere such as control center, lock screen interface and notice fence, consequently can make things convenient for the user to call out conveniently under the current state of first equipment the button to the audio frequency of control first equipment is thrown.

For example, if the preset APP is playing audio through the first device, when a screen locking event is detected, the screen locking state is entered, the first device displays a screen locking interface, the button of the preset APP is displayed on the screen locking interface, a control in the button is still in an activated state when the first device is in the screen locking state, namely, the user operation can still be detected, and the first device is triggered to respond to the user operation.

Namely, when the first device is in a screen locking state corresponding to the screen locking interface, the audio playing of the preset APP can still be controlled through the button, and the audio projection page is called through the access control.

Similarly, when the first device is in the screen locking state, the audio projection page is displayed on the screen locking interface in a suspended manner, the control controls included in the screen locking interface are all in the activated state, the control controls in the activated state can detect various user operations, and the first device is triggered to respond to the user operations.

In some embodiments, as shown in FIG. 5, the audio projection page includes:

a first volume adjusting bar 11, configured to adjust the volume of each second device or adjust the volume of the first device in a unified manner;

and/or the presence of a gas in the atmosphere,

and a second volume bar 12, one of which corresponds to one of the second devices, wherein the second volume bar is used for adjusting the volume of the second device.

For example, when a first device is connected to a second device, the first volume adjustment bar 11 is used to uniformly adjust the volume of all the second devices. The first volume adjustment bar 11 is used to adjust the volume of the first device when the first device is not connected to the second device.

That is, the first volume adjustment bar is equivalent to a second volume adjustment bar which totally controls all the second devices, when no second device exists, the volume of the first device is controlled, and for the second volume adjustment bar of each second device, a control which adjusts the output volume of any one corresponding second device can be detected, so that a user can select the second device which plays the audio stream provided by the first device, and can also control the playing volume of the audio stream generated by the target application program from multiple dimensions and the like through the audio projection page.

And after detecting the operation acting on the first volume adjusting bar, the first device generates a first volume adjusting instruction, the first volume adjusting instruction is used for controlling all second volume adjusting bars of the second device in a total manner, and the volume of the first device is controlled when no second device exists.

And after the first equipment detects the operation acting on the second volume adjusting bar, generating a second volume adjusting instruction, transmitting the second volume adjusting instruction to the second equipment, and adjusting the equipment volume of the second equipment after the second equipment receives the second volume adjusting instruction.

In some embodiments, the preset APP is an audio playing APP; the button and/or the audio projection page further comprises: a control for audio playing; and the control of the audio playing is used for detecting and controlling the operation of the audio playing of the preset APP.

The control controls include, but are not limited to, at least one of:

the playing control is used for controlling the starting or stopping of the audio playing of the preset APP;

the next control is used for controlling the audio playing of the preset APP to be switched to the next audio stream;

and the last control is used for controlling the audio playing of the preset APP to be switched to the last audio stream.

In some embodiments, the first device has a first mode and a second mode; in the first mode, the number of the target application programs is one, and in the second mode, the number of the target application programs is one or more.

The first mode may be referred to as a single APP audio stream projection mode, and the second mode may be a multiple APP audio projection mode.

In the first mode, only 1 target application program is provided, so that the first device can only transmit the audio stream generated by one application program to the second device.

In the second mode, one or more target applications may be provided. At this time, the second mode may be a multiple APP audio projection mode.

Whether the first device is operating in the first mode or the second mode may be determined based on detecting a user operation. For example, on a setup page of the first device, an operating mode currently used by the first device may be configured, e.g., the first mode is selected, the first device will be in the first mode, otherwise the first device may operate in the second mode.

Correspondingly, the S110 may include: determining one of said target applications when said first device is in said first mode; determining one or more of the target applications while the first device is in the second mode.

Since the first device is in the first mode, the first device will only have one target application program, and if the first device is in the second mode, the target application programs are allowed to be one or more.

Correspondingly, the determining one of the target applications when the first device is in the first mode includes: detecting a selection operation acting on the application identifier;

and when the first electronic equipment is in the first mode, determining the application program corresponding to the last application identifier acted by the selection operation as the target application program.

Exemplarily, when the first electronic device is in the first mode, determining an application program corresponding to a last application identifier acted by the selecting operation as the target application program, includes:

when the first electronic equipment is in the first mode and no target application program exists currently, the selection operation for determining a second application program as a target application program is detected, and the target application program currently of the first electronic equipment is switched to the second application program

When the first electronic device is in the first mode and the target application is the first application, the selection operation for determining the second application as the target application is detected, and the target application of the first electronic device is switched to the second application.

The application identifier here includes, but is not limited to, a text identifier such as an application icon and/or an application name displayed by the first device.

The selection operation includes, but is not limited to, any operation acting on a User Interface (UI) of the first device, for example, a touch operation acting on a touch screen of the first device.

In the embodiment of the present disclosure, since the first device is in the first mode, if there is no target application currently, and multiple applications are selected based on the discovery in one selection process, the last selected application is taken as the target application.

If the target application program exists at present, and the selection operation for selecting the target application program is detected again, the switching operation of the target application program is determined, and the switching of the target application program is realized.

In some embodiments, said determining one or more of said target applications while said first device is in said second mode comprises:

detecting a selection operation acting on the application identifier;

when the first electronic device is in the second mode and the target application program does not exist currently, determining the application program corresponding to the one or more application identifications acted by the selection operation as the target application program;

when the first electronic equipment is in the second mode and at least one target application program is determined, the selection operation is detected, and a third application program corresponding to the application identification acted by the selection operation is added as the target application program.

If the first device is operating in the second mode, the second device supports a plurality of target applications. If a plurality of application programs are determined to be selected in the process of selecting the target application program once, the application programs are all determined to be the target application program, and if the target application program exists, the selection operation of the target application program is detected again, the addition operation of the target application program can be determined, and the application program acting on the selection operation is added to be the target application program.

In some embodiments, regardless of whether the first device is in the first mode or the second mode, the method further comprises:

detecting a deletion operation of a target application program;

and deleting the application program subjected to the deletion operation from the target application program.

In some embodiments, the method further comprises:

receiving a control instruction returned by the second equipment;

transmitting the control instruction to the target application program, wherein the control instruction is used for the target application program to execute at least one of the following:

stopping providing the audio stream;

switching the audio stream;

and adjusting the playing volume.

For example, the second device may receive a user action that is returned by the second device to the first device via a connection with the first device. After receiving the information, the wireless communication module of the first device transmits the information to the corresponding target application program, and the target application program controls the wireless communication module.

For example, the second device is a smart speaker, and if the smart speaker receives a voice command instructing to stop playing, the smart speaker returns command information of the voice command to the mobile phone serving as the first device. The handset, upon receipt, transmits the audio stream to the music playback application, which stops the audio stream from being generated. If the music playing application program is an audio stream downloaded from the network, the audio stream downloading from the network is stopped after the stop instruction is received, and if the audio stream is read from the local, the audio stream reading is stopped, and the audio stream sending to the media frame of the first device is also stopped, so that unnecessary operations of the first device are reduced.

For another example, the smart speaker may also receive: similarly, the voice instruction for indicating audio stream switching, such as 'change one song', and the like, is returned to the mobile phone after being received by the smart sound box, and is returned to the target application program through the media framework and the like after being received by the wireless communication module of the mobile phone, so that the target application program can execute.

The above is only illustrated by a voice instruction, and in a specific implementation process, the control instruction returned by the second device may be returned based on any user operation. The user operation may include: manual operation and/or voice operation, etc.

In some embodiments, if there are a plurality of second devices, one device detects that a user operation performs a response without returning to the first device, and possibly other second devices may still perform the response, thereby generating a poor user control effect.

As shown in fig. 8, an embodiment of the present disclosure provides an audio processing apparatus, applied in a first device, the apparatus including:

a first determining module 110 for determining one or more target applications;

a second determining module 120, configured to determine, from the audio stream generated by the first device, an audio stream generated by the target application; the audio stream generated by the target application program is the second type of audio stream, and the audio streams generated by the rest application programs are the first type of audio streams; one of the audio streams is generated by an application;

a sending module 130, configured to send the second type of audio stream to a second device, where the second type of audio stream is used for the second device to output.

In some embodiments, the first determining module 110, the second determining module 120, and the sending module 130 may each include a program module; the program module, when executed by the processor, enables audio transfer in the APP dimension to the second device for playback.

In other embodiments, the first determining module 110, the second determining module 120, and the sending module 130 may be a combination of hardware and software modules; the soft and hard combining module includes but is not limited to: various programmable arrays; the programmable array includes, but is not limited to: field programmable arrays and/or complex programmable arrays.

In still other embodiments, the first determining module 110, the second determining module 120, and the sending module 130 may further include, but are not limited to: a pure hardware module; the pure hardware module includes, but is not limited to, an application specific integrated circuit.

It is to be understood that the determining module is configured to receive an audio stream generated by an application layer by a media framework layer of the first device; and determining the second type of audio stream according to the application identification ID of the target application program and the application ID carried by the audio stream generated by the application layer.

It is to be understood that the apparatus further comprises:

and outputting the third type audio stream.

As can be appreciated, the first determining module includes:

the display sub-module is used for displaying a button of a preset APP, wherein the button at least comprises a first control; when the operation acting on the first control is detected, displaying an audio projection page of the preset APP, wherein the audio projection page contains: device information of one or more alternative devices to the second device;

In some embodiments, the display sub-module is specifically configured to perform one of:

displaying the button of the preset APP in the control center;

displaying a button of the preset APP on a notification bar;

displaying the button of the preset APP on a screen locking interface;

and displaying the button of the preset APP on the application page of the preset APP.

In some embodiments, the audio projection page includes:

and/or the presence of a gas in the atmosphere,

It is to be understood that the apparatus further comprises:

a first mixing module, configured to mix a plurality of second-type audio streams to obtain an audio stream when the second-type audio streams include a plurality of audio streams; the audio stream may be referred to as a fourth type of audio stream;

the sending module 130 is configured to send the mixed audio stream to the second device.

As can be appreciated, the first determining module 110 is configured to perform at least one of:

determining the application program corresponding to the application interface of which the projection operation is detected as the target application program;

It is to be understood that the first type of application includes at least one of:

an audio playback-type application;

network broadcast type application programs;

a language learning class APP;

and/or the presence of a gas in the gas,

the second type of application program comprises:

a telephone APP;

instant messaging APP.

In some embodiments, the audio projection page includes:

and/or the presence of a gas in the gas,

In some embodiments, the first device has a first mode and a second mode; in the first mode, the number of the target application programs is one; the number of the target application programs in the second mode is one or more

The first determining module 110 is configured to determine one of the target applications when the first device is in the first mode; determining one or more of the target applications while the first device is in the second mode.

In some embodiments, the first determining module 110 is specifically configured to detect a selection operation acting on an application identifier;

when the first electronic equipment is in the first mode and the target application program does not exist currently, determining the application program corresponding to the last application identifier of the selection operation function as the target application program;

In some embodiments, the first determining module 110 is configured to detect a selection operation acting on an application identifier; when the first electronic device is in the second mode and the target application program does not exist currently, determining the application program corresponding to the one or more application identifications acted by the selection operation as the target application program; when the first electronic equipment is in the second mode and at least one target application program is determined, the selection operation is detected, and a third application program corresponding to the application identifier acted by the selection operation is added as the target application program.

An embodiment of the present disclosure provides an electronic device, including:

a memory for storing processor-executable instructions;

a processor connected with the memory;

wherein the processor is configured to execute the audio processing method provided by any of the preceding claims.

The processor may include various types of storage media, non-transitory computer storage media capable of continuing to remember to store the information thereon after a power loss to the communication device.

The processor may be connected to the memory via a bus or the like for reading the executable program stored on the memory, for example, capable of performing at least one of the methods as shown in any of fig. 1, 2-4.

The electronic device may comprise the first device and/or the second device.

Fig. 9 is a block diagram illustrating an electronic device 800 in accordance with an example embodiment. For example, the electronic device 800 may be included in a terminal device such as a mobile phone or a mobile computer, or a device such as a server.

Referring to fig. 9, electronic device 800 may include one or more of the following components: processing component 802, memory 804, power component 806, multimedia component 808, audio component 810, input/output (I/O) interface 812, sensor component 814, and communication component 816.

The processing component 802 generally controls overall operation of the electronic device 800, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. The processing components 802 may include one or more processors 820 to execute instructions to perform all or a portion of the steps of the methods described above. Further, the processing component 802 can include one or more modules that facilitate interaction between the processing component 802 and other components. For example, the processing component 802 can include a multimedia module to facilitate interaction between the multimedia component 808 and the processing component 802.

The memory 804 is configured to store various types of data to support operation at the device 800. Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phonebook data, messages, pictures, videos, and so forth. The memory 804 may be implemented by any type or combination of volatile or non-volatile memory devices such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks.

The power supply component 806 provides power to the various components of the electronic device 800. The power components 806 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power for the electronic device 800.

The multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and a user. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front facing camera and/or a rear facing camera. The front camera and/or the rear camera may receive external multimedia data when the device 800 is in an operating state, such as a shooting state or a video state. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

The audio component 810 is configured to output and/or input audio signals. For example, the audio component 810 includes a Microphone (MIC) configured to receive external audio signals when the electronic device 800 is in an operational state, such as a call state, a recording state, and a voice recognition state. The received audio signals may further be stored in the memory 804 or transmitted via the communication component 816. In some embodiments, audio component 810 also includes a speaker for outputting audio signals.

The I/O interface 812 provides an interface between the processing component 802 and peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to: a home button, a volume button, a start button, and a lock button.

The sensor assembly 814 includes one or more sensors for providing various aspects of state assessment for the electronic device 800. For example, sensor assembly 814 may detect the open/closed status of device 800, the relative positioning of components, such as a display and keypad of device 800, sensor assembly 814 may also detect a change in position of device 800 or a component of device 800, the presence or absence of user contact with device 800, orientation or acceleration/deceleration of device 800, and a change in temperature of device 800. Sensor assembly 814 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

The communication component 816 is configured to facilitate wired or wireless communication between the electronic device 800 and other devices. The electronic device 800 may access a wireless network based on a communication standard, such as Wi-Fi,2G, or 3G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, communications component 816 further includes a Near Field Communications (NFC) module to facilitate short-range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, ultra Wideband (UWB) technology, bluetooth (BT) technology, and other technologies.

In an exemplary embodiment, the apparatus 800 may be implemented by one or more Application Specific Integrated Circuits (ASICs), digital Signal Processors (DSPs), digital Signal Processing Devices (DSPDs), programmable Logic Devices (PLDs), field Programmable Gate Arrays (FPGAs), controllers, micro-controllers, microprocessors, or other electronic components for performing the above-described methods.

In an exemplary embodiment, a non-transitory computer-readable storage medium comprising instructions, such as the memory 804 comprising instructions, executable by the processor 820 of the device 800 to perform the above-described method is also provided. For example, the non-transitory computer readable storage medium may be a ROM, a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

The disclosed embodiments provide a non-transitory computer-readable storage medium, wherein instructions in the storage medium, when executed by a processor of a UE, enable the UE or a base station to perform the audio processing method provided in any of the foregoing embodiments, and to perform at least one of the methods illustrated in any of fig. 1, 2 to 4.

The audio processing method comprises the following steps: determining one or more target applications; determining an audio stream generated by the target application program from the audio stream generated by the first device; the audio stream generated by the target application program is a second type of audio stream, and the audio streams generated by the rest application programs are first type of audio streams; one of said audio streams being generated by an application; and sending the second type of audio stream to a second device, wherein the second type of audio stream is used for the output of the second device.

It is to be understood that, the determining the audio stream generated by the target application from the audio streams generated by the first device includes:

a media framework layer of the first device receives an audio stream generated by an application layer;

and determining the second type of audio stream according to the application identification ID of the target application program and the application ID carried by the audio stream generated by the application layer.

It is to be understood that the method further comprises:

and outputting the third type audio stream.

As will be appreciated, the determining one or more target applications includes:

displaying a button of a preset APP, wherein the button at least comprises an access control;

It is to be understood that the button displaying the preset APP includes at least one of:

displaying the button of the preset APP in the control center;

displaying a button of the preset APP on a notification bar;

displaying the button of the preset APP on a screen locking interface;

As can be appreciated, the audio projection page includes:

and/or the presence of a gas in the gas,

As can be understood, the preset APP is an audio playing APP; the button and/or the audio projection page further comprises: a control for audio playing; and the control of the audio playing is used for detecting and controlling the operation of the audio playing of the preset APP.

As can be appreciated, the method further comprises:

when the second type audio stream comprises a plurality of audio streams, mixing the plurality of audio streams contained in the second type audio stream;

the sending the second type of audio stream to a second device includes:

and mixing a plurality of audio streams contained in the second type of audio stream and then sending the mixed audio stream to the second equipment.

It is to be appreciated that the determining one or more of the target applications includes at least one of:

an audio playback-type application;

network broadcast type application programs;

a language learning class APP;

and/or the presence of a gas in the gas,

the second type of application includes at least one of:

a telephone APP;

instant messaging APP.

Understandably, the first device has a first mode and a second mode; in the first mode, the number of the target application programs is one; the number of target applications in the second mode is one or more of the determining one or more target applications, including:

It is to be understood that said determining one of said target applications when said first device is in said first mode comprises:

detecting a selection operation acting on the application identifier;

when the first electronic equipment is in the first mode, determining an application program corresponding to the last application identifier of the selection operation function as the target application program;

when the first electronic device is in the first mode and the target application is the first application, the selection operation for determining the second application as the target application is detected, and the target application currently on the first electronic device is switched to the second application.

It is to be understood that said determining one or more of said target applications when said first device is in said second mode comprises:

detecting a selection operation acting on the application identifier;

when the first electronic equipment is in the first mode and the target application program does not exist currently, determining the application program corresponding to the one or more application identifications acted by the selection operation as the target application program;

when the first electronic equipment is in the second mode and at least one target application program is determined, the selection operation is detected, and a third application program corresponding to the application identifier acted by the selection operation is added as the target application program.

Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

It will be understood that the present disclosure is not limited to the precise arrangements that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. An audio processing method applied to a first device, the method comprising:

determining one or more target applications;

determining an audio stream generated by the target application program from the audio stream generated by the first device; the audio stream generated by the target application program is a second type of audio stream, and the audio streams generated by the rest application programs are first type of audio streams; one of the audio streams is generated by an application;

2. The method of claim 1, wherein determining the audio stream generated by the target application from the audio streams generated by the first device comprises:

3. The method of claim 2, further comprising:

and outputting the third type audio stream.

4. The method of any of claims 1 to 3, wherein the determining one or more target applications comprises:

displaying an access control of an audio projection page of a preset APP;

when the operation of selecting at least one piece of equipment information is detected, determining the preset APP as the target application program; wherein the candidate device corresponding to the selected device identifier is the second device.

5. The method of claim 4, wherein the entry control for displaying the audio projection page of the preset APP comprises one of:

6. The method of claim 4, wherein the audio projection page comprises:

and/or the presence of a gas in the gas,

and one of the second volume adjusting bars corresponds to one of the second devices, wherein the second volume adjusting bar is used for adjusting the volume of the second device.

7. The method of claim 4, wherein the predetermined APP is an audio playback type APP; the button and/or the audio projection page further comprises: a control for audio playing; and the control of the audio playing is used for detecting and controlling the operation of the audio playing of the preset APP.

8. The method of any of claims 1 to 3, wherein the first device has a first mode and a second mode; in the first mode, the number of the target application programs is one; the number of the target application programs in the second mode is one or more;

the determining one or more target applications includes:

9. An audio processing apparatus, applied in a first device, the apparatus comprising:

a first determination module for determining one or more target applications;

the second determining module is used for determining the audio stream generated by the target application program from the audio stream generated by the first equipment; the audio stream generated by the target application program is the second type of audio stream, and the audio streams generated by the rest application programs are the first type of audio streams; one of the audio streams is generated by an application;

10. The apparatus of claim 9, wherein the second determining module is configured to receive the audio streams of the first type generated by an application layer at a media framework layer of the first device; and determining the second type of audio stream according to the application identification ID of the target application program and the application ID carried by the first type of audio stream.

11. The apparatus of claim 11, further comprising:

and the output module is used for outputting the third type of audio stream.

12. The apparatus of any of claims 9 to 11, wherein the first determining module comprises:

the display sub-module is used for displaying an access control of an audio projection page of a preset APP; when detecting the operation of acting on entering the control, show the audio frequency of presetting APP throws the page, wherein, contain in the audio frequency throws the page: device information of one or more devices alternative to the second device;

the determining submodule is used for determining the preset APP as the target application program when the operation of selecting at least one piece of equipment information is detected; wherein the selected device identifier corresponds to the alternative device being the second device.

13. The apparatus of claim 12, wherein the display sub-module is configured to perform one of:

14. The apparatus of claim 12, wherein the audio projection page comprises:

and/or the presence of a gas in the atmosphere,

15. The apparatus of claim 12, wherein the predetermined APP is an audio playback-like APP; the button and/or the audio projection page further comprises: a control for audio playing; and the control of the audio playing is used for detecting and controlling the operation of the audio playing of the preset APP.

16. The apparatus of any of claims 9 to 11, wherein the first device has a first mode and a second mode; in the first mode, the number of the target application programs is one; the number of the target application programs in the second mode is one or more

17. An electronic device, comprising:

a memory for storing processor-executable instructions;

a processor coupled to the memory;

wherein the processor is configured to perform the audio processing method as provided in any one of claims 1 to 8.

18. A non-transitory computer-readable storage medium, instructions in which, when executed by a processor of a computer, enable the computer to perform the audio processing method of any of claims 1 to 8.