WO2023130991A1

WO2023130991A1 - Collaborative calling method and apparatus, device, storage medium, and program product

Info

Publication number: WO2023130991A1
Application number: PCT/CN2022/141752
Authority: WO
Inventors: 刘能宾
Original assignee: 荣耀终端有限公司
Priority date: 2022-01-10
Filing date: 2022-12-26
Publication date: 2023-07-13
Also published as: CN114500716A

Abstract

The present application relates to the technical field of terminals, and discloses a collaborative calling method and apparatus, a device, a storage medium, and a program product. The method comprises: if a first device receives a calling operation when the first device and a second device are in a multi-screen collaborative state, establishing a calling audio link, setting an input device of the calling audio link as an audio acquisition hardware device of the first device, and setting an output device of the calling audio link as an audio playback hardware device of the first device. When receiving the calling operation, the first device also detects the state of a collaborative calling function, and if the collaborative calling function is in an enabled state, the input device of the calling audio link is switched from the audio acquisition hardware device to a virtual input module, and the output device of the calling audio link is switched from the audio playback hardware device to a virtual output module, so that the audio acquisition hardware device and the audio playback hardware device of the first device are turned off during the collaborative calling, thereby reducing hardware power consumption while meeting the collaborative calling requirement.

Description

Cooperative call method, device, equipment, storage medium and program product

This application claims the priority of the Chinese patent application submitted to the State Intellectual Property Office on January 10, 2022, with the application number 202210023658.9, and the application name "Collaborative call method, device, equipment, storage medium and program product", all of which The contents are incorporated by reference in this application.

technical field

The present application relates to the technical field of terminals, and in particular to a method, device, device, storage medium and program product for a collaborative call.

Background technique

With the rapid development of terminal technology, multi-screen collaboration technology has been widely used. Multi-screen collaboration refers to mirroring and displaying the screen image of the first device on the interface of the second device after the first device (such as a mobile phone) establishes a connection with the second device (such as a tablet computer). In this case, the user can make the first device execute a corresponding function by manipulating the screen image of the first device displayed on the interface of the second device.

In the scenario of multi-screen collaboration, if the first device is in a call, it can be switched to the second device to collect and play the voice of the call, that is, a collaborative call can be made. Specifically, when the first device and the second device conduct a coordinated call, the microphone of the second device collects the call voice of the local user and sends it to the first device, and the first device sends the call voice to the remote call device; The end call device sends the call voice of the remote user to the first device, and the first device sends the call voice to the second device, which is played by the speaker of the second device.

However, in the related art, during the coordinated call between the first device and the second device, the first device still turns on its own hardware such as a microphone, a speaker, and an earpiece, thereby consuming power consumption.

Contents of the invention

The present application provides a coordinated call method, device, device, storage medium and program product, which can turn off hardware such as a microphone, a loudspeaker, and an earpiece during a coordinated call to reduce hardware power consumption. Described technical scheme is as follows:

In the first aspect, a coordinated call method is provided. In this method, the first device receives a call operation, wherein the first device and the second device are in a multi-screen coordinated state. Afterwards, the first device establishes a call audio link in response to the call operation, and sets the input device of the call audio link as the audio collection hardware device of the first device, and sets the output device of the call audio link as the first device's audio collection device. Audio playback hardware device, and the first device detects the state of the coordinated call function, if it detects that the coordinated call function is in an open state, then the input device of the call audio link is switched from the audio collection hardware device to a virtual input module, and the call audio The output device of the link is switched from an audio playback hardware device to a virtual output module.

When the first device and the second device are in a multi-screen coordinated state, the screen image of the first device is displayed on the interface of the second device. In this case, the user can make the first device execute a corresponding function by manipulating the screen image of the first device displayed on the interface of the second device.

The first device can start a coordinated call, so as to collect and play the call voice through the second device. For example, in the case that the call operation is triggered by the user through the screen image of the first device displayed by the second device, the first device starts the coordinated call function when receiving the call operation. Or, before receiving the call operation, the first device may automatically enable the coordinated call function when it detects that some specific modes are activated during the multi-screen coordination process with the second device.

The audio collection hardware device of the first device is hardware for collecting audio, for example, the audio collection hardware device may be a microphone or the like. The audio playing hardware device of the first device is hardware for playing audio, for example, the audio playing hardware device may be a loudspeaker or an earpiece.

After the first device enables the coordinated call function, since the collection and playback of call voices are performed on the second device, the first device no longer needs to collect and play call voices. Therefore, in this application, the first device detects When the collaborative call function is turned on, the input device of the call voice can be switched from the audio collection hardware device to the virtual input module, and the output device of the call voice can be switched from the audio playback hardware device to the virtual output module, so that during the collaborative call process The audio collection hardware device and the audio playback hardware device are turned off in the middle, so as to reduce power consumption.

Both the virtual input module and the virtual output module are virtual software modules, not hardware devices. The virtual input module does not perform audio collection, for example, the virtual input module can be an IN_PROXY module, and the virtual output module does not perform audio playback, for example, the virtual output module can be an OUT_PROXY module. In this way, after the first device switches the input device of the call voice to the virtual input module, the first device no longer collects audio, and after the first device switches the output device of the call voice to the virtual output module, the first device no longer plays audio, In this way, while meeting the requirements of cooperative calls, the power consumption of hardware is also reduced.

It is worth noting that when the first device receives a call operation, it detects the status of the coordinated call function, and if it detects that the coordinated call function is off, it does not switch between the input device and the output device of the call audio link, and still maintains the call The input device of the audio link is the audio collection hardware device of the first device, and the output device of the call-hold audio link is the audio playback hardware device of the first device. If the cooperative call function is turned off, it means that the first device needs to collect and play the call voice, so the input device of the call audio link needs to be the audio collection hardware device of the first device, and the output of the call audio link should be kept The device is an audio playback hardware device of the first device, so as to ensure the normal progress of the call.

For example, in the case that the call operation is triggered by the user on the first device, the first device closes the coordinated call function when receiving the call operation. Or, before receiving the call operation, the first device may automatically turn off the coordinated call function when it detects that some specific modes are closed during the multi-screen coordination process with the second device.

In a possible situation, after the first device establishes the call audio link, if it detects that the cooperative call function is switched from the on state to the off state, then the input device of the call audio link is switched from the virtual input module to the audio collection Hardware device, switch the output device of the call audio link from a virtual output module to an audio playback hardware device.

After the coordinated call between the first device and the second device, the coordinated call can also be turned off, so that the first device can continue to collect and play the voice of the call. For example, after the first device establishes a call audio link, if it detects that the user has turned off the coordinated call switch displayed by the first device or the second device, it switches the coordinated call function from the on state to the off state.

After the first device closes the coordinated call, it switches back the collection and playback of the call voice to the first device for execution. In this case, the first device needs to collect and play the voice of the call. Therefore, in this application, when the first device detects that the coordinated call function is switched from the on state to the off state, the input device of the call audio link can be Switch from the virtual input module to the audio collection hardware device, switch the output device of the call audio link from the virtual output module to the audio playback hardware device, to open the audio collection hardware device for audio collection, and turn on the audio playback hardware The device is used for audio playback, so as to ensure the normal progress of the call.

In a possible situation, after the first device establishes the call audio link, if it detects that the coordinated call function is switched from the off state to the on state, it will switch the input device of the call audio link from the audio collection hardware device to the virtual The input module switches the output device of the call audio link from an audio playback hardware device to a virtual output module.

After the first device closes the coordinated call, it can also turn on the coordinated call to switch to the second device to collect and play the call voice. For example, after the first device establishes a call audio link, if it detects that the user turns on the coordinated call switch displayed by the first device or the second device, it switches the coordinated call function from the off state to the on state.

After the first device and the second device have a coordinated call, since the collection and playback of the call voice are performed on the second device, the first device no longer needs to collect and play the call voice. Therefore, in this application, the first device When it is detected that the cooperative call function is switched from the closed state to the open state, the input device of the call audio link can be switched from the audio collection hardware device to a virtual input module, and the output device of the call audio link can be replaced by the audio playback hardware device Switch to a virtual output module to turn off the audio collection hardware device and the audio playback hardware device during a cooperative call to reduce power consumption.

It should be noted that in this application, the first device may also turn off the collaborative call function when the call ends. For example, the first device receives the call end operation, and in response to the call end operation, the first device closes the coordinated call function, and sets the input device of the call audio link as the audio collection hardware device, and sets the call audio link output device Set the hardware device for this audio playback.

The call end operation is an operation for instructing to end a call, for example, the call end operation may be an operation of hanging up the phone. The call ending operation may be directly triggered by the user in the first device, for example, it may be triggered after the user clicks an end button in the call interface on the first device. Alternatively, the call ending operation may be triggered by the user through the screen image of the first device displayed on the second device, for example, the user may click on the second device in the call interface of the first device displayed on the second device. Fired after the hangup button.

It can be seen from the above description that if the first device conducts a coordinated call during the call, the first device will shut down the audio playback hardware device and the audio playback hardware device. In this case, after the call audio link is established, the first device receives the call uplink voice data collected by the second device sent by the second device when the coordinated call function is turned on, and sends the call uplink voice data to to the remote communication device, and receive the communication downlink voice data sent by the remote communication device, and send the communication downlink voice data to the second device for playback, so as to realize the cooperative communication with the second device.

If the first device does not make a coordinated call during the call, for example, the first device has not started the coordinated call after starting the call, or the first device turns off the coordinated call after starting the coordinated call function during the call. call, the first device will turn on the audio playback hardware device and the audio playback hardware device. In this case, after the call audio link is established, the first device collects the uplink voice data of the call through the audio collection hardware device, and sends the uplink voice data of the call to the remote call device when the cooperative call function is turned off. And receive the call downlink voice data sent by the remote call device, and play the call downlink voice data through the audio playback hardware device, so as to realize the normal progress of the call.

In the second aspect, a coordinated calling device is provided, and the coordinated calling device has the function of realizing the behavior of the coordinated calling method in the above first aspect. The coordinated calling device includes at least one module, and the at least one module is used to implement the coordinated calling method provided in the first aspect above.

In the third aspect, a cooperative communication device is provided, the structure of the coordinated communication device includes a processor and a memory, and the memory is used to store a program that supports the coordinated communication device to execute the coordinated communication method provided in the first aspect above, And storing the data involved in realizing the cooperative calling method described in the first aspect above. The processor is configured to execute programs stored in the memory. The cooperative communication device may further include a communication bus for establishing a connection between the processor and the memory.

In a fourth aspect, a computer-readable storage medium is provided, wherein instructions are stored in the computer-readable storage medium, and when the computer-readable storage medium is run on a computer, the computer is made to execute the collaborative calling method described in the above-mentioned first aspect.

A fifth aspect provides a computer program product containing instructions, which when run on a computer, causes the computer to execute the coordinated calling method described in the first aspect above.

The technical effects obtained by the above-mentioned second aspect, third aspect, fourth aspect and fifth aspect are similar to those obtained by the corresponding technical means in the above-mentioned first aspect, and will not be repeated here.

Description of drawings

FIG. 1 is a schematic structural diagram of a terminal provided in an embodiment of the present application;

FIG. 2 is a block diagram of a software system of a terminal provided in an embodiment of the present application;

FIG. 3 is a schematic diagram of a call data flow in a non-multi-screen collaborative scenario provided by an embodiment of the present application;

FIG. 4 is a schematic diagram of a call data flow in a multi-screen collaborative scenario provided by an embodiment of the present application;

Fig. 5 is a schematic diagram of an interface of a tablet computer provided by an embodiment of the present application;

Fig. 6 is a schematic diagram of the interface of a mobile phone provided by the embodiment of the present application;

Fig. 7 is a schematic interface diagram of another tablet computer provided by the embodiment of the present application;

Fig. 8 is a schematic diagram of the interface of another mobile phone provided by the embodiment of the present application;

FIG. 9 is a schematic diagram of an interface in a multi-screen collaboration scenario provided by an embodiment of the present application;

Fig. 10 is a schematic diagram of an interface in another multi-screen collaboration scenario provided by the embodiment of the present application;

Fig. 11 is a schematic diagram of a multi-screen collaboration system provided by an embodiment of the present application;

Fig. 12 is a flow chart of a coordinated call method provided by an embodiment of the present application;

FIG. 13 is a flow chart of a coordinated call process provided by an embodiment of the present application;

Fig. 14 is a schematic structural diagram of a collaborative communication device provided by an embodiment of the present application.

Detailed ways

In order to make the purpose, technical solution and advantages of the present application clearer, the implementation manner of the present application will be further described in detail below in conjunction with the accompanying drawings.

It should be understood that the "plurality" mentioned in this application means two or more. In the description of this application, unless otherwise specified, "/" means or means, for example, A/B can mean A or B; "and/or" in this article is just a description of the relationship between associated objects, It means that there can be three kinds of relationships, for example, A and/or B, which can mean: A exists alone, A and B exist simultaneously, and B exists alone. In addition, in order to clearly describe the technical solution of the present application, words such as "first" and "second" are used to distinguish the same or similar items with basically the same function and effect. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit the difference.

Phrases such as "one embodiment" or "some embodiments" described in this application mean that a particular feature, structure, or characteristic described by the embodiment is included in one or more embodiments of the present application. Thus, appearances of "in one embodiment," "in some embodiments," "in other embodiments," "in other embodiments," etc. in various places in this application are not necessarily All refer to the same embodiment, but mean "one or more but not all embodiments" unless specifically stated otherwise. In addition, the terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless specifically stated otherwise.

The terminal involved in the embodiment of the present application will be described below.

FIG. 1 is a schematic structural diagram of a terminal provided by an embodiment of the present application. 1, the terminal 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (universal serial bus, USB) interface 130, a charging management module 140, a power management module 141, a battery 142, and an antenna 1 , antenna 2, mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, earphone jack 170D, sensor module 180, button 190, motor 191, indicator 192, camera 193, display screen 194, and a subscriber identity module (subscriber identity module, SIM) card interface 195, etc. Among them, the sensor module 180 may include a pressure sensor 180A, a gyro sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an environmental Light sensor 180L, bone conduction sensor 180M, etc.

It can be understood that, the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the terminal 100 . In other embodiments of the present application, the terminal 100 may include more or fewer components than shown in the figure, or combine some components, or separate some components, or arrange different components. The illustrated components can be realized in hardware, software or a combination of software and hardware.

The processor 110 may include one or more processing units, for example: the processor 110 may include an application processor (application processor, AP), a modem processor, a graphics processing unit (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), controller, memory, video codec, digital signal processor (digital signal processor, DSP), baseband processor, and/or neural network processor (neural-network processing unit, NPU) wait. Wherein, different processing units may be independent devices, or may be integrated in one or more processors.

Wherein, the controller may be the nerve center and command center of the terminal 100 . The controller can generate an operation control signal according to the instruction opcode and timing signal, and complete the control of fetching and executing the instruction.

A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. The memory may hold instructions or data that the processor 110 has just used or recycled. If the processor 110 needs to use the instruction or data again, it can be directly recalled from the memory. Repeated access is avoided, and the waiting time of the processor 110 is reduced, thereby improving the efficiency of the system.

The charging management module 140 is configured to receive a charging input from a charger. Wherein, the charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 can receive charging input from the wired charger through the USB interface 130 . In some wireless charging embodiments, the charging management module 140 may receive wireless charging input through the wireless charging coil of the terminal 100 . While the charging management module 140 is charging the battery 142 , it can also supply power to the terminal 100 through the power management module 141 .

The power management module 141 is used for connecting the battery 142 , the charging management module 140 and the processor 110 . The power management module 141 receives the input from the battery 142 and/or the charging management module 140 to provide power for the processor 110 , the internal memory 121 , the external memory, the display screen 194 , the camera 193 and the wireless communication module 160 . The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle times, and battery health status (leakage, impedance). In some other embodiments, the power management module 141 may also be disposed in the processor 110 . In some other embodiments, the power management module 141 and the charging management module 140 may also be set in the same device.

The wireless communication function of the terminal 100 can be realized by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

The mobile communication module 150 can provide wireless communication solutions including 2G/3G/4G/5G applied on the terminal 100 . The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (low noise amplifier, LNA) and the like. The mobile communication module 150 can receive electromagnetic waves through the antenna 1, filter and amplify the received electromagnetic waves, and send them to the modem processor for demodulation. The mobile communication module 150 can also amplify the signals modulated by the modem processor, and convert them into electromagnetic waves and radiate them through the antenna 1 . In some embodiments, at least part of the functional modules of the mobile communication module 150 may be set in the processor 110 . In some embodiments, at least part of the functional modules of the mobile communication module 150 and at least part of the modules of the processor 110 may be set in the same device.

The wireless communication module 160 can provide wireless local area networks (wireless local area networks, WLAN) (such as wireless fidelity (Wi-Fi) network), bluetooth (bluetooth, BT), global navigation satellite system, etc. (global navigation satellite system, GNSS), frequency modulation (frequency modulation, FM), near field communication technology (near field communication, NFC), infrared technology (infrared, IR) and other wireless communication solutions. The wireless communication module 160 may be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2 , frequency-modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110 . The wireless communication module 160 can also receive the signal to be sent from the processor 110 , frequency-modulate it, amplify it, and convert it into electromagnetic waves through the antenna 2 for radiation.

The terminal 100 realizes the display function through the GPU, the display screen 194, and the application processor. The GPU is a microprocessor for image processing, and is connected to the display screen 194 and the application processor. GPUs are used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

The terminal 100 can realize the shooting function through the ISP, the camera 193 , the video codec, the GPU, the display screen 194 and the application processor.

The external memory interface 120 may be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the terminal 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function. Such as saving music, video and other files in the external memory card.

The internal memory 121 may be used to store computer-executable program codes including instructions. The processor 110 executes various functional applications and data processing of the terminal 100 by executing instructions stored in the internal memory 121 . The internal memory 121 may include an area for storing programs and an area for storing data. Wherein, the stored program area can store an operating system, at least one application program required by a function (such as a sound playing function, an image playing function, etc.) and the like. The storage data area can store data (such as audio data, phone book, etc.) created by the terminal 100 during use. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, flash memory device, universal flash storage (universal flash storage, UFS) and the like.

The terminal 100 can implement audio functions, such as music playing and recording, through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the earphone interface 170D, and the application processor.

The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signal. The audio module 170 may also be used to encode and decode audio signals. In some embodiments, the audio module 170 may be set in the processor 110 , or some functional modules of the audio module 170 may be set in the processor 110 .

The SIM card interface 195 is used for connecting a SIM card. The SIM card can be connected and separated from the terminal 100 by inserting it into the SIM card interface 195 or pulling it out from the SIM card interface 195 . The terminal 100 may support 1 or N SIM card interfaces, where N is an integer greater than 1. SIM card interface 195 can support Nano SIM card, Micro SIM card, SIM card etc. Multiple cards can be inserted into the same SIM card interface 195 at the same time. The types of multiple cards may be the same or different. The SIM card interface 195 is also compatible with different types of SIM cards. The SIM card interface 195 is also compatible with external memory cards. The terminal 100 interacts with the network through the SIM card to implement functions such as calling and data communication. In some embodiments, the terminal 100 adopts an eSIM, that is, an embedded SIM card. The eSIM card can be embedded in the terminal 100 and cannot be separated from the terminal 100 .

Next, the software system of the terminal 100 will be described.

The software system of the terminal 100 may adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture. In this embodiment of the present application, the software system of the terminal 100 is exemplarily described by taking an Android system with a layered architecture as an example.

FIG. 2 is a block diagram of a software system of a terminal 100 provided by an embodiment of the present application. Referring to Figure 2, the layered architecture divides the software into several layers, and each layer has a clear role and division of labor. Layers communicate through software interfaces. In some embodiments, the Android system is divided into application program layer, application program framework layer, Android runtime (Android runtime) and system layer, extension layer, and kernel layer from top to bottom.

The application layer can consist of a series of application packages. As shown in Figure 2, the application package may include applications such as multi-screen collaboration, camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, and short message. The multi-screen collaboration application is used to enable the multi-screen collaboration function.

The application framework layer provides an application programming interface (application programming interface, API) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions. As shown in Figure 2, the application framework layer can include audio framework (Audio Framework), distributed mobile sensing development platform (distributed mobile sensing development platform, DMSDP), window manager, content provider, view system, phone manager, resource manager, notification manager, etc.

The audio framework is responsible for the output of playback data, the collection of recording data, and the comprehensive management of audio affairs. DMSDP is used to provide functional support for the multi-screen collaborative process, for example, it can realize the collaborative call in the multi-screen collaborative process. A window manager is used to manage window programs. The window manager can get the size of the display screen, determine whether there is a status bar, lock the screen, capture the screen, etc. Content providers are used to store and retrieve data, and make these data accessible to applications. These data can include video, images, audio, calls made and received, browsing history and bookmarks, phonebook, etc. The view system includes visual controls, such as controls for displaying text, controls for displaying pictures, and so on. The view system can be used to build the display interface of the application, and the display interface can be composed of one or more views, for example, including a view for displaying SMS notification icons, a view for displaying text, and a view for displaying pictures. The phone manager is used to provide communication functions of the terminal 100, such as management of call status (including connecting, hanging up, etc.). The resource manager provides various resources for the application, such as localized strings, icons, pictures, layout files, video files, etc. The notification manager enables the application to display notification information in the status bar, which can be used to convey notification-type messages, and can automatically disappear after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be notifications that appear in the status bar at the top of the system in the form of charts or scrolling text, such as notifications for applications running in the background. The notification manager can also be a notification that appears on the screen in the form of a dialog window, such as prompting text information in the status bar, making a prompt sound, vibrating the electronic device, and flashing an indicator light.

Android Runtime includes core library and virtual machine. The Android runtime is responsible for the scheduling and management of the Android system. The core library consists of two parts: one part is the function function that the java language needs to call, and the other part is the core library of Android. The application layer and the application framework layer run in virtual machines. The virtual machine executes the java files of the application program layer and the application program framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

The system layer can include multiple functional modules, such as: audio services (Audio Services), surface manager (surface manager), media library (Media Libraries), 3D graphics processing library (such as: OpenGL ES), 2D graphics engine (such as : SGL) and so on. The audio service is the maker of the audio strategy, responsible for the strategy selection of audio device switching, volume adjustment strategy, etc., and the executor of the audio strategy, responsible for the management of input and output stream devices and the processing and transmission of audio stream data. The surface manager is used to manage the display subsystem and provides the fusion of 2D and 3D layers for multiple applications. The media library supports playback and recording of various commonly used audio and video formats, as well as still image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc. The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing, etc. 2D graphics engine is a drawing engine for 2D drawing.

The extension layer can also be called the hardware abstraction layer (hardware abstracting layer, HAL), which can realize the encapsulation of the kernel driver, provide an interface upward, and shield the implementation details of the lower layer. The extension layer is connected upwards to Android Runtime and Framework, and downwards to the driver. The extension layer can include the Audio Hardware Abstraction Layer (Audio HAL), which is responsible for interacting with audio hardware devices.

The kernel layer is the layer between hardware and software. The kernel layer includes at least a display driver, a camera driver, an audio driver, a sensor driver, and a pulse code modulation (PCM) device.

The application scenario involved in the embodiment of the present application will be described below by taking multi-screen collaboration between a mobile phone and a tablet computer as an example.

When the mobile phone is making a call or receiving an incoming call, it can communicate with the remote call device. The call may include a voice call, a video call, etc., which is not limited in this embodiment of the present application. For example, the call may be a call established through an operator service, such as a 2G call, a 3G call, a 4G call, or a 5G call.

In the case of a phone call, it involves the collection of uplink (uplink) voice data and the playback of downlink (downlink) voice data.

The uplink voice data of the call refers to the call voice data of the local user collected by the mobile phone or the tablet computer that conducts a cooperative call with the mobile phone during the call on the mobile phone. The uplink voice data of the call needs to be sent to the remote call device. The downlink voice data of the call refers to the call voice data of the remote user sent by the remote call device received by the mobile phone during the call on the mobile phone. The downlink voice data of the call needs to be played on the mobile phone or the tablet computer that conducts a coordinated call with the mobile phone .

In the case that the mobile phone does not perform multi-screen collaboration with the tablet computer, if the mobile phone makes a call, the microphone (Mic) of the mobile phone collects the call voice of the local user, and sends the call voice to the remote call device; the remote call device Send the call voice of the remote user to the mobile phone, and the mobile phone plays the call voice through its own speaker (Speaker) or earpiece.

The call data flow in the non-multi-screen collaborative scenario is shown in Figure 3. The uplink and downlink voice data of the call is transferred between the audio chip and the device (microphone, speaker, receiver) of the mobile phone. The audio chip can be ADSP (Audio DSP), etc. . Specifically, the microphone of the mobile phone collects the call voice of the local user, and sends the call voice to the audio chip of the mobile phone, and the audio chip of the mobile phone sends the call voice to the remote call via the modem processor of the mobile phone. equipment. At the same time, the audio chip of the mobile phone receives the voice of the remote user sent by the remote communication device through the modem processor of the mobile phone, and sends the voice of the voice to the speaker or earpiece of the mobile phone for playback.

In the case of multi-screen collaboration between the mobile phone and the tablet computer, the mobile phone can be switched to the tablet computer to collect and play the voice of the call, that is, the collaborative call can be performed. Specifically, when a mobile phone and a tablet computer make a cooperative call, the microphone of the tablet computer collects the call voice of the local user and sends it to the mobile phone, and the mobile phone sends the call voice to the remote call device; the remote call device transmits the call voice of the remote user The voice is sent to the mobile phone, and the mobile phone sends the call voice to the tablet computer, which is played by the speaker of the tablet computer.

The call data flow in the multi-screen collaboration scenario is shown in Figure 4. The uplink and downlink voice data of the call is transferred between the audio chip of the mobile phone and the Audio HAL. Specifically, the microphone of the tablet computer collects the call voice of the local user, and sends the call voice to the mobile phone. After the Audio HAL of the mobile phone receives the call voice, it sends the call voice to the audio chip of the mobile phone, and the audio chip of the mobile phone sends the call voice to the remote call device through the modem processor of the mobile phone. At the same time, the audio chip of the mobile phone receives the call voice of the remote user sent by the remote call device through the modem processor of the mobile phone, and sends the call voice to the Audio HAL of the mobile phone, and the Audio HAL of the mobile phone sends the call voice to the tablet computer, played through the tablet's speakers.

In this way, when the mobile phone and the tablet computer are in a coordinated call, the microphone of the mobile phone no longer picks up sound, and the speaker and earpiece of the mobile phone no longer produce sound. In related technologies, the mobile phone directly mutes the microphone, speaker and earpiece by calling the device mute interface, so that the sound collected by the microphone of the mobile phone will not be transmitted, and the speaker and earpiece of the mobile phone will not play sound. However, in this case, the mobile phone will still turn on hardware such as its own microphone, speaker, and earpiece, thereby consuming power consumption.

To this end, the embodiment of the present application provides a coordinated call method, which can directly turn off hardware such as a microphone, a speaker, and an earpiece of the mobile phone when the mobile phone and the tablet computer are in a coordinated call, thereby reducing power consumption.

Several possible connection modes of multi-screen collaboration are described below.

1. Establish a connection via Bluetooth.

For example, if the user wants to cooperate with the mobile phone and the tablet computer, the Bluetooth in both the mobile phone and the tablet computer can be turned on first. Then, the user manually turns on the multi-screen collaboration function in the mobile phone. For example, the user can find the switch of "multi-screen collaboration" through the path of "Settings"-"More Connections"-"Multi-screen Collaboration" in the interface of the mobile phone. screen coordination function.

Referring to the schematic diagram of the interface of the tablet computer shown in FIG. 5 , as shown in (a) of FIG. 5 , the user slides out of the notification panel from the status bar of the tablet computer, and the notification panel includes the option 51 of "multi-screen collaboration". The user clicks on the "multi-screen collaboration" option 51, and the tablet computer displays a first prompt window in response to the user's trigger operation on the "multi-screen collaboration" option 51. The first prompt window includes instructions for instructing the user how to operate to achieve multi-screen collaboration The prompt message for the first operation. For example, as shown in (b) in Figure 5, the first operation prompt information includes "1. Turn on the Bluetooth of your mobile phone and approach this machine, and click "Connect" after finding this machine. 2. After connecting, you can Operate the mobile phone on the tablet computer to realize data sharing between devices." prompt content. In this way, the user can perform corresponding operations according to the first operation prompt information, such as bringing the mobile phone close to the tablet computer.

In one example, referring to the schematic diagram of the interface of the mobile phone shown in Figure 6, when the mobile phone finds the tablet when the mobile phone is close to the tablet, the mobile phone displays a second prompt window, as shown in (a) in Figure 6 , the second prompt window includes the prompt content of “whether to establish a cooperative connection with the discovered device”, and a “connect” option 61 and a “cancel” option 62 . When the user clicks on the "Connect" option 61, it means that the user confirms to establish a collaborative connection, and the mobile phone responds to the user's trigger operation on the "Connect" option 61, and establishes a collaborative connection with the tablet computer via Bluetooth. When the user clicks the "cancel" option 62, it means that the user does not want to establish a coordinated connection, and the mobile phone does not perform the operation of establishing a coordinated connection in response to the user's trigger operation on the "cancel" option 62. In another example, when the mobile phone is close to the tablet computer, when the mobile phone finds the tablet computer, it may not display the second prompt window, but automatically establishes a cooperative connection with the tablet computer via Bluetooth.

As an example and not limitation, in the process of establishing a cooperative connection between the mobile phone and the tablet computer through Bluetooth, in order to display the progress of establishing the cooperative connection, the mobile phone can also display a third prompt window for indicating that it is connecting, for example, it can be displayed as shown in Figure 6 The third prompt window shown in (b) figure. Optionally, the third prompt window includes a "cancel" option, so that the user can cancel the connection at any time if necessary.

2. Establish a connection by scanning the code.

For example, the user can find the button of "scan code connection" through the path of "my mobile phone"-"connect now"-"scan code connection" in the interface of the tablet computer. The trigger operation of the button displays a two-dimensional code for establishing a cooperative connection, for example, the two-dimensional code shown in FIG. 7 may be displayed. Optionally, the tablet computer may also display second operation prompt information for prompting the user how to operate to realize multi-screen collaboration. For example, as shown in FIG. ".

In one example, referring to the schematic diagram of the interface of the mobile phone shown in Figure 8, the user can enter the interface displaying the "scan" option in the browser (or smart vision) of the mobile phone, for example, can enter the interface shown in Figure 8 (a) The interface of the browser shown in the figure, in which the option 81 of "scan" is displayed. The user can click on the "scan" option 81, and the mobile phone starts the camera in response to the user's trigger operation on the "scan" option 81, and displays the code scanning interface shown in (b) in Figure 8, so that the user can Point the camera at the QR code displayed on the tablet to scan the code.

In one example, after the mobile phone scans the code successfully, it sends a request for establishing a cooperative connection to the tablet computer. After the tablet computer receives the request sent by the mobile phone, it can display a fourth prompt window, which includes prompt information for prompting the user whether to agree to establish a collaborative connection. For example, the prompt information can include "xx device request and this Do you agree to establish a collaborative connection?”, as well as the options of “Agree” and “Deny”. When the user clicks the "Agree" option, it means that the user allows the mobile phone to establish a collaborative connection with the tablet computer, and the tablet computer establishes a collaborative connection with the mobile phone in response to the user's trigger operation on the "Agree" option. When the user clicks the "deny" option, it means that the user does not allow the mobile phone to establish a cooperative connection with the tablet computer, and the tablet computer notifies the mobile phone that the cooperative connection establishment fails in response to the user's trigger operation on the "deny" option.

It should be noted that the above is only an example of the user opening the QR code on the tablet computer through the path of "My Phone"-"Connect Now"-"Scan Code Connection". Optionally, the QR code can also be opened through other paths. For example, as shown in (b) in Figure 5, in addition to the first operation prompt information, the first prompt window also includes the prompt content "Can't find this machine? You can also scan the code to connect", in which " The four words "scan code connection" can be triggered. The user can click on the "scan code to connect" content in the first prompt window, and the tablet computer displays the QR code shown in Figure 7 in response to the user's trigger operation on the "scan code to connect" content. In this way, the user can scan the QR code displayed on the tablet computer through the mobile phone, so as to establish a collaborative connection by scanning the code.

3. Establish a connection by touching.

Users can enable NFC and multi-screen collaboration functions in both mobile phones and tablets. After that, the user touches the NFC area on the back of the mobile phone (usually located around the camera on the back of the mobile phone) to the NFC area of the tablet (usually located in the lower right corner of the tablet), and the mobile phone and the tablet respond to the user's touch operation. collaborative connection. Optionally, before establishing a coordinated connection through NFC, the tablet computer and the mobile phone may also prompt the user whether to agree to establish the coordinated connection, and after the user agrees to establish the coordinated connection, the mobile phone and the tablet computer execute the operation of establishing the coordinated connection. In an example, when the cooperative connection between the mobile phone and the tablet computer is successfully established, the mobile phone can also remind the user by vibrating or ringing.

It should be noted that, the foregoing several possible connection modes are described by taking wireless connection as an example. In another embodiment, it can also be implemented through a wired connection, for example, it can be implemented through a Type-C to high definition multimedia interface (high definition multimedia interface, HDMI) connection line, which is not limited in this embodiment of the present application.

After the mobile phone and the tablet computer successfully establish a collaborative connection, as shown in Figure 9, the tablet computer will mirror the screen image of the mobile phone. In this way, the user can make the mobile phone perform corresponding functions by operating the screen image of the mobile phone displayed on the tablet computer in the tablet computer as required. In one example, the mobile phone and the tablet computer synchronously display the main interface of the mobile phone as shown in Figure 9. If the user wants to make a call, he can click the icon of making a call in the main interface of the mobile phone displayed on the tablet computer to open The dialing interface of the mobile phone. At this time, the mobile phone and the tablet computer will display the dialing interface of the mobile phone synchronously. Then the user can perform a dialing operation in the dialing interface of the mobile phone displayed on the tablet computer, so as to realize calling in the mobile phone.

After making a call in the mobile phone, if the mobile phone starts to make a call, as shown in FIG. 10 , the mobile phone and the tablet computer will synchronously display the call interface of the mobile phone. When the mobile phone is in a call, you can choose to switch to the tablet computer to collect and play the call voice, that is, you can make a collaborative call. Specifically, when a mobile phone and a tablet computer make a cooperative call, the microphone of the tablet computer collects the call voice of the local user and sends it to the mobile phone, and the mobile phone sends the call voice to the remote call device; the remote call device transmits the call voice of the remote user The call voice is sent to the mobile phone, and the phone sends the call voice to the tablet computer, which is played by the speaker of the tablet computer.

For example, after the multi-screen coordination between the mobile phone and the tablet computer, if the mobile phone starts to make a call, as shown in Figure 10, after the user pulls down the notification bar of the tablet computer on the tablet computer, the notification bar of the tablet computer may display "Collaborated to mobile phone", the notification bar may also include a switch for switching the call voice to the tablet computer, and the user can operate the switch as required to indicate whether to switch the call voice to the tablet computer, that is, whether to perform a collaborative call. Alternatively, the user can also pull down the notification bar of the mobile phone, and the notification bar of the mobile phone can display the prompt content of "already coordinated to the tablet computer", and the notification bar can also include a switch for switching the call voice to the tablet computer, The user can operate the switch as required to indicate whether to make a collaborative call. If during the call on the mobile phone, the user instructs to switch the call voice to the tablet computer by operating the switch for switching the call voice to the tablet computer on the tablet computer or the mobile phone, that is, to instruct a coordinated call, then the ongoing call of the mobile phone The collection and playback of the call voice of the call is performed on the tablet computer side. If during the call on the mobile phone, the user instructs not to switch the call voice to the tablet computer by operating the switch for switching the call voice to the tablet computer on the tablet computer or mobile phone, that is, to indicate not to make a coordinated call, the mobile phone is The collection and playback of the call voice of the ongoing call is still performed on the mobile phone side.

It is worth noting that, in some embodiments, in a scenario of multi-screen collaboration, the mobile phone may automatically perform a collaborative call by default. That is to say, after the multi-screen collaboration between the mobile phone and the tablet computer, if the mobile phone starts to make a call, no user operation is required, and the coordinated call can be carried out automatically, that is, the collection and playback of the call voice is automatically switched to the tablet side for execution. In this case, if the user does not want to make a coordinated call, he can operate the switch for switching the call voice to the tablet computer on the tablet computer or mobile phone to instruct not to switch the call voice to the tablet computer, that is, to instruct to turn off the coordinated call, Switch the call voice back to the mobile phone, at this time the collection and playback of the call voice of the ongoing call on the mobile phone will be switched back to the mobile phone side to continue execution.

When the mobile phone and the tablet computer are in a coordinated call, the voice of the call is switched to the tablet computer side, and it is expected that the phone side will no longer make a sound. To this end, the embodiment of the present application provides a coordinated call method, which can turn off hardware such as a microphone, a loudspeaker, and an earpiece of a mobile phone during a coordinated call, thereby reducing power consumption.

The coordinated call method provided in the embodiment of the present application can be applied to a multi-screen coordinated system, and the multi-screen coordinated system will be described below.

Fig. 11 is a schematic diagram of a multi-screen collaboration system provided by an embodiment of the present application. Referring to FIG. 11 , the multi-screen collaboration system may include a first device 1101 and a second device 1102 .

The first device 1101 and the second device 1102 may communicate through a wired connection or a wireless connection.

Both the first device 1101 and the second device 1102 may be terminals, and the terminals may be the terminals described in the embodiments of FIGS. 1-2 above. For example, the terminal can be a mobile phone, a tablet computer, a wearable device, a vehicle-mounted device, an augmented reality (augmented reality, AR)/virtual reality (virtual reality, VR) device, a notebook computer, a personal computer (personal computer, PC), a super Mobile personal computer (ultra-mobile personal computer, UMPC), netbook, personal digital assistant (personal digital assistant, PDA), television, etc., the embodiments of the present application are not limited to this.

The first device 1101 and the second device 1102 can perform multi-screen collaboration. After the first device 1101 and the second device 1102 perform multi-screen collaboration, the screen image of the first device 1101 may be displayed on the interface of the second device 1102 .

The first device 1101 and the second device 1102 may be terminals of different types, or terminals of the same type, which is not limited in this embodiment of the present application. For example, both may be terminals such as a mobile phone or a tablet computer.

In a possible implementation, the screen size of the first device 1101 is smaller than the screen size of the second device 1102, so that when the small screen and the large screen perform multi-screen coordination, the screen image of the small screen is displayed on the large screen as a window. The interface of the large screen allows the user to operate the screen images of the small screen in the interface of the large screen, thereby improving the user's operating experience. For example, the first device 1101 is a mobile phone, and the second device 1102 is a tablet computer or a TV. Alternatively, the first device 1101 is a tablet computer, and the second device 1102 is a TV. Certainly, the screen size of the first device 1101 may also be larger than the screen size of the second device 1102 . For example, the first device 1101 is a tablet computer, and the second device 1102 is a mobile phone.

In the case of multi-screen collaboration between the first device 1101 and the second device 1102, if the first device 1101 starts a call, the coordinated call can be started to switch to the second device 1102 to collect and play the call voice.

The coordinated call method provided in the embodiment of the present application is applied in a scenario where the first device 1101 and the second device 1102 perform multi-screen coordination. In this case, the first device 1101 can turn off the microphone and speaker of the first device 1101 when the coordinated call is started in the case of multi-screen collaboration with the second device 1102 by executing the coordinated call method provided in the embodiment of the present application. and handset and other hardware to reduce power consumption.

The coordinated call method provided by the embodiment of the present application will be described below.

The coordinated call method provided by the embodiment of the present application is applied to the first device, and the first device and the second device can perform multi-screen coordination, and the multi-screen cooperation refers to displaying the screen of the first device on the interface of the second device.

After the first device and the second device perform multi-screen collaboration, the screen image of the first device is displayed on the interface of the second device. In this case, the user can make the first device execute a corresponding function by manipulating the screen image of the first device displayed on the interface of the second device. The first device and the second device can realize multi-screen collaboration in many possible ways. For example, multi-screen collaboration can be realized through Bluetooth, code scanning, touch, etc. These methods have been described in detail above. This embodiment of the present application will not describe it in detail.

The first device is a device capable of talking. One or more SIM cards may be installed in the first device, and the first device may use any one of the one or more SIM cards to make a call. For example, if only one SIM card is installed in the first device, the first device can directly use this SIM card to make a call, such as using this SIM card to make a call, or use this SIM card to receive an incoming call. In the case that the first device is equipped with multiple SIM cards, the first device can use one of the multiple SIM cards to make calls, such as using this SIM card to make calls, or using this SIM card to answer calls call.

Next, when the first device and the second device perform multi-screen collaboration, the involved call scene and call voice switching scene will be described.

Call scene:

In the case of multi-screen collaboration between the first device and the second device, the user can directly operate on the first device to make the first device start a call, or the user can operate on the second device to make the first The device starts talking.

As an example, several possible operation modes are described by taking the first device as an example to start a call by making a call.

In the first possible operation mode, if the user wants to make a call on the first device, the user may directly perform an operation on the first device.

For example, the user can click the icon of making a call in the main interface of the first device to open the dialing interface of the first device, and then the user can perform a dialing operation in the dialing interface of the first device to make a call in the first device to start a call. Wherein, in the case that the first device is only equipped with a SIM card, the user can directly perform a dialing operation in the dialing interface of the first device, so that the SIM card is used to make a call in the first device; When the device is equipped with multiple SIM cards, the user can first select a SIM card in the dialing interface of the first device, and then perform a dialing operation on the dialing interface of the first device. In this way, the selected SIM card is used in the first device. This SIM card to make calls.

In the second possible operation mode, if the user wants to make a call on the first device, the operation can be implemented on the second device that performs multi-screen coordination with the first device.

For example, after the multi-screen collaboration between the first device and the second device, the first device and the second device display the main interface of the first device synchronously, and the user can click to make a call in the main interface of the first device displayed on the second device icon to open the dialing interface of the first device, then the first device and the second device will display the dialing interface of the first device synchronously, and then the user can dial in the dialing interface of the first device displayed on the second device operation, to implement making a call in the first device to start a call. Wherein, in the case that only one SIM card is installed in the first device, the user can directly perform the dialing operation in the dialing interface of the first device displayed on the second device, so that the SIM card can be used in the first device To make a call; in the case of multiple SIM cards installed in the first device, the user can first select a SIM card in the dialing interface of the first device displayed on the second device, and then select a SIM card on the second device displayed on the second device. The dialing interface of a device performs a dialing operation, so that the selected SIM card can be used in the first device to make a call.

In the third possible operation mode, if the user wants to make a call on the first device, it can be realized by performing operations on both the first device and the second device that performs multi-screen collaboration with the first device.

For example, after the multi-screen collaboration between the first device and the second device, the first device and the second device display the main interface of the first device synchronously, and the user can click to make a call in the main interface of the first device displayed on the second device icon to open the dialing interface of the first device. At this time, the first device and the second device will display the dialing interface of the first device synchronously, and then the user can perform dialing operations in the dialing interface of the first device to Make a call from your device to start a call. Wherein, in the case that the first device is only equipped with a SIM card, the user can directly perform a dialing operation in the dialing interface of the first device, so that the SIM card is used to make a call in the first device; When the device is equipped with multiple SIM cards, the user can first select a SIM card in the dialing interface of the first device, and then perform a dialing operation on the dialing interface of the first device. In this way, the selected SIM card is used in the first device. use this SIM card to make calls.

For another example, after the first device and the second device perform multi-screen collaboration, the first device and the second device display the main interface of the first device synchronously, and the user can click the icon of making a call in the main interface of the first device to open the The dialing interface of the first device, at this time, the first device and the second device will display the dialing interface of the first device synchronously, and then the user can perform dialing operations in the dialing interface of the first device displayed on the second device, so as to realize A phone call is made in the first device to start the conversation. Wherein, in the case that only one SIM card is installed in the first device, the user can directly perform the dialing operation in the dialing interface of the first device displayed on the second device, so that the SIM card can be used in the first device To make a call; in the case of multiple SIM cards installed in the first device, the user can first select a SIM card in the dialing interface of the first device displayed on the second device, and then select a SIM card on the second device displayed on the second device. The dialing interface of a device performs a dialing operation, so that the selected SIM card can be used in the first device to make a call.

As an example, several possible operation modes are described by taking the first device as an example of answering an incoming call and starting a call.

In the first possible operation mode, when there is an incoming call on the first device, if the user wants to answer the incoming call on the first device, he can directly perform an operation on the first device.

For example, if there is an incoming call in the first device, the first device will display an incoming call interface, and the user can click an answer button on the incoming call interface of the first device to answer the incoming call in the first device to start a call. Wherein, under the situation that only a SIM card is installed in the first device, the incoming call in the first device is the incoming call for this SIM card, so is to use this SIM card to answer the incoming call in the first device; When the device is equipped with multiple SIM cards, the incoming call in the first device is for one of the multiple SIM cards, so that the first device uses this SIM card to answer the incoming call.

In the second possible operation mode, when there is an incoming call on the first device, if the user wants to answer the call on the first device, the operation can be implemented on the second device that performs multi-screen coordination with the first device.

For example, after the first device and the second device perform multi-screen collaboration, if there is an incoming call on the first device, the first device and the second device will simultaneously display the incoming call interface of the first device, and the user can display the first Click the answer button in the incoming call interface of the device to realize answering the incoming call in the first device to start the call. Wherein, under the situation that only a SIM card is installed in the first device, the incoming call in the first device is the incoming call for this SIM card, so is to use this SIM card to answer the incoming call in the first device; When the device is equipped with multiple SIM cards, the incoming call in the first device is for one of the multiple SIM cards, so that the first device uses this SIM card to answer the incoming call.

Call voice switching scene:

In the case of multi-screen collaboration between the first device and the second device, both the first device and the second device include a coordinated call switch, and the coordinated call switch is used to indicate whether the second device will perform the call when the first device is in a call. The collection and playback of call voice indicates whether to enable the collaborative call function. For example, both the drop-down notification bar of the first device and the drop-down notification bar of the second device may include a coordinated call switch, so that the user can turn on or off the coordinated call switch in the first device or the second device. Of course, the cooperative call switch can also be set in other interfaces of the first device and the second device, which is not limited in this embodiment of the present application.

When the coordinated call switch is turned on, the coordinated call function is turned on. In this case, if the first device is in a call with the second device during the multi-screen collaborative call, the coordinated call will be made during the call with the first device, that is, the The call voice is collected and played through the second device. Specifically, when the first device and the second device conduct a coordinated call, the microphone of the second device collects the call voice of the local user and sends it to the first device, and the first device sends the call voice to the remote call device; The remote call device sends the call voice of the remote user to the first device, and the first device sends the call voice to the second device, which is played by a loudspeaker of the second device.

When the coordinated call switch is turned off, the coordinated call function is turned off. In this case, if the first device is in a call with the second device during the multi-screen collaborative call, the coordinated call will not be made during the call with the first device, that is, during the call with the first device The collection and playback of the call voice is still performed by the first device. Specifically, the microphone of the first device collects the call voice of the local user and sends it to the remote call device; the remote call device sends the call voice of the remote user to the first device, which is played by the speaker of the first device.

In some embodiments, when the first device and the second device start multi-screen collaboration, the coordinated call function is enabled by default, that is, when the first device and the second device establish a multi-screen coordinated connection, the coordinated call function is enabled. call function. In some other embodiments, during the multi-screen collaboration between the first device and the second device, the activation of the coordinated call function may also be triggered by a specific mode association, for example, if the user plays audio or video on the second device, Then the first device can enable the collaborative call function. In this case, when the coordinated call function is turned on, the first device and the second device can also turn on the coordinated call switch synchronously, and when the coordinated call is turned off, the first device and the second device can also synchronously turn off the coordinated call switch.

As an example, if the user wants to turn off the coordinated call function, he can directly turn off the coordinated call switch on the first device during the multi-screen coordinated process between the first device and the second device, or directly turn off the coordinated call switch on the second device. Turn off the coordinated call switch to disable the coordinated call function. If the user wants to turn on the coordinated call function after turning off the coordinated call function, he can directly turn on the coordinated call switch on the first device during the multi-screen coordinated process between the first device and the second device, or directly on the second device. Turn on the coordinated call switch on the device to enable the coordinated call function.

For example, the collaborative call function has been enabled during the multi-screen collaboration between the first device and the second device. If the first device has not started a call and the user wants to turn off the coordinated call function, the coordinated call switch can be directly turned off in the first device or the second device. In this way, if the first device subsequently starts a call, the first device still collects and plays the call voice.

Or, the collaborative call function has been turned off during the multi-screen coordination between the first device and the second device. If the first device has not started a call and the user wants to enable the coordinated call function, the user can directly turn on the coordinated call switch in the first device or the second device. In this way, if the first device subsequently starts a call, the second device will collect and play the call voice.

For another example, the collaborative call function has been enabled during the multi-screen collaboration between the first device and the second device. If the first device has started a call, the second device collects and plays the call voice at this time. If the user wants to turn off the coordinated call function, he can directly turn off the coordinated call switch in the first device or the second device. In this way, the call voice of the ongoing call of the first device will be switched to the first device for collection and playback.

Or, the collaborative call function has been turned off during the multi-screen coordination between the first device and the second device. If the first device has started talking, it is now the first device that collects and plays the voice of the call. If the user wants to enable the coordinated call function, he can directly turn on the coordinated call switch in the first device or the second device. In this way, the call voice of the ongoing call of the first device will be switched to the second device for collection and playback.

As another example, the enabling and disabling of the collaborative call function may also be determined by the user's behavior of making a call or answering an incoming call. Specifically, if the user directly makes a call or answers an incoming call on the first device during the multi-screen collaboration between the first device and the second device, the coordinated call function will be turned off, that is, during the call on the first device The call voice is collected and played by the first device, and at this time, the cooperative call switch in the first device and the second device is also in an off state. Or, if during the multi-screen collaboration between the first device and the second device, the user operates the screen of the first device displayed on the second device to make a call or answer an incoming call on the first device, the Coordinated call function, that is, the second device collects and plays the call voice during the call of the first device, and at this time, the coordinated call switch in the first device and the second device will also be turned on.

It can be seen from the above description that the user can enable and disable the collaborative call function in two ways.

Specifically, if the first device receives the first instruction triggered by the user during the multi-screen coordination with the second device, it may turn off the coordinated calling function. The first instruction is used to instruct the first device to collect and play the call voice when the first device is in a call. Optionally, the first instruction is triggered when the user turns off the coordinated call switch in the first device or the second device; or, the first instruction is triggered when the user makes a call or receives an incoming call on the first device.

If the first device receives the second instruction triggered by the user during the multi-screen coordination with the second device, it may enable the coordinated calling function. The second instruction is used to instruct the second device to collect and play the call voice when the first device is in an operation call. Optionally, the second instruction is triggered when the user turns on the coordinated call switch in the first device or the second device; or, the second instruction is the screen image of the first device displayed by the user through the operation of the second device, so that Fired when the first device makes a call or receives an incoming call.

In addition, the first device may also automatically enable the collaborative calling function. For example, when the first device establishes a multi-screen collaborative connection with the second device, it can automatically enable the coordinated call function; or, during the multi-screen collaborative process with the second device, the first device detects that some specific modes , the collaborative call function can be automatically enabled.

The first device may also automatically disable the collaborative call function. For example, in the process of multi-screen collaboration between the first device and the second device, when it detects that some specific modes are closed, it can automatically turn off the coordinated call function; or, when the first device ends the call, that is, when it hangs up the phone , you can automatically turn off the collaborative call function.

Next, the overall flow of the coordinated call method provided by the embodiment of the present application will be described.

Fig. 12 is a flow chart of a coordinated call method provided by an embodiment of the present application. Referring to Figure 12, the method includes:

Step 1201: The first device performs multi-screen collaboration with the second device.

After the first device and the second device perform multi-screen coordination, the first device and the second device are in a multi-screen coordination state, and the screen image of the first device is displayed on the interface of the second device. In this case, the user can make the first device execute a corresponding function by manipulating the screen image of the first device displayed on the interface of the second device. The first device and the second device can realize multi-screen collaboration in many possible ways. For example, multi-screen collaboration can be realized through Bluetooth, code scanning, touch, etc. These methods have been described in detail above. This embodiment of the present application will not describe it in detail.

Step 1202: the first device receives a call operation.

The call operation is an operation for instructing the first device to make a call, for example, the call operation may be an operation of making a call or answering an incoming call. The call operation may be directly triggered by the user in the first device, or the call operation may be triggered by the user through a screen image of the first device displayed on the second device. Various possible triggering modes of the call operation have been described in detail above, and will not be repeated in this embodiment of the application.

Step 1203: The first device establishes a call audio link in response to the call operation, and sets the input device of the call audio link as the audio collection hardware device of the first device, and sets the output device of the call audio link as the first The device's audio playback hardware.

After receiving the call operation, the first device determines that it needs to make a call, and thus needs to establish a call audio link, and set an input device and an output device of the call audio link. The input device of the call audio link is a device for collecting the uplink voice data of the call, and the output device of the call audio link is a device for playing the downlink voice data of the call. When the first device receives the call operation, it will set the input device of the call audio link as its own audio collection hardware device, so as to collect audio through the audio collection hardware device, and will set the output device of the call audio link to An audio playback hardware device for itself to play audio through the audio playback hardware device.

Step 1204: The first device detects the status of the coordinated call function. If it detects that the coordinated call function is on, the input device of the call audio link is switched from the audio collection hardware device to a virtual input module, and the call audio link is switched to a virtual input module. The output device is switched from the audio playback hardware device to a virtual output module.

When the first device receives a call operation, it may also detect the state of the coordinated call function, so as to determine whether to switch the input device and the output device of the call audio link according to the state of the coordinated call function.

The first device can enable the coordinated call function, so as to collect and play the call voice through the second device. The first device may enable the collaborative call function in various possible ways, which have been described in detail above, and will not be repeated in this embodiment of the present application.

For example, in the case that the call operation is triggered by the user through the screen image of the first device displayed by the second device, the first device starts the coordinated call function when receiving the call operation. Or, before receiving the call operation, the first device may automatically enable the coordinated call function when it detects that some specific modes are activated during the multi-screen coordination process with the second device.

After the first device activates the coordinated call function, it is about to switch the collection and playback of the call voice to the second device for execution. Specifically, during the coordinated call between the first device and the second device, the second device collects the call uplink voice data of the local user (for example, the second device can collect call uplink voice data through its own microphone), and the call The uplink voice data is sent to the first device, and the first device sends the call uplink voice data to the remote call device. At the same time, the remote call device collects the call downlink voice data of the remote user, sends the call downlink voice data to the first device, and the first device sends the call downlink voice data to the second device, and the second device plays it ( For example, the second device can play the call downlink voice data through its own speaker).

After the first device and the second device have a cooperative call, since the collection and playback of the call voice are performed on the second device, the first device no longer needs to collect and play the call voice. Therefore, in the embodiment of this application, the second When a device detects that the coordinated call function is on, it can switch the input device of the call audio link from the audio collection hardware device to a virtual input module, and switch the output device of the call audio link from the audio playback hardware device to a virtual input module. The virtual output module is used to turn off the audio collection hardware device and the audio playback hardware device during the coordinated call to reduce power consumption.

Both the virtual input module and the virtual output module are virtual software modules, not hardware devices. The virtual input module does not perform audio collection, such as the virtual input module can be an IN_PROXY module, and the virtual output module does not perform audio playback, such as the virtual output module can be an OUT_PROXY module. In this way, after the first device switches the input device of the call audio link to a virtual input module, the first device no longer collects audio, and after the first device switches the output device of the call audio link to a virtual output module, the first device does not Play the audio again, so as to meet the needs of collaborative calls and reduce hardware power consumption.

In some embodiments, if the first device disables the coordinated call function during the call, the following step 1205 may be performed.

Step 1205: After the first device establishes the call audio link, if it detects that the cooperative call function is switched from the on state to the off state, then the input device of the call audio link is switched from the virtual input module to the audio collection hardware device, and the The output device of the call audio link is switched from the virtual output module to the audio playback hardware device.

After the coordinated call between the first device and the second device, the coordinated call can also be turned off, so that the first device can continue to collect and play the voice of the call. The first device may disable the coordinated call in various possible ways, which have been described in detail above, and will not be repeated in this embodiment of the present application. For example, after the first device establishes a call audio link, if it detects that the user has turned off the coordinated call switch displayed by the first device or the second device, it switches the coordinated call function from the on state to the off state.

After the first device closes the coordinated call, it switches back the collection and playback of the call voice to the first device for execution. In this case, the first device needs to collect and play the voice of the call. Therefore, in this embodiment of the application, when the first device detects that the coordinated call function is switched from the on state to the off state, it can switch the audio link of the call to The input device is switched from the virtual input module to the audio collection hardware device, and the output device of the call audio link is switched from the virtual output module to the audio playback hardware device to open the audio collection hardware device for audio collection, and open the audio Play hardware devices for audio playback, so as to ensure the normal progress of the call.

In some embodiments, if the first device enables the coordinated call function during the call, the following step 1206 may be performed.

Step 1206: After the first device establishes the call audio link, if it detects that the cooperative call function is switched from the off state to the on state, then the input device of the call audio link is switched from the audio collection hardware device to a virtual input module, and the The output device of the call audio link is switched from the audio playback hardware device to a virtual output module.

After the first device closes the coordinated call, it can also turn on the coordinated call to switch to the second device to collect and play the call voice. The first device may enable the collaborative call in multiple possible ways, which have been described in detail above, and will not be repeated in this embodiment of the present application. For example, after the first device establishes a call audio link, if it detects that the user turns on the coordinated call switch displayed by the first device or the second device, it switches the coordinated call function from the off state to the on state.

After the first device and the second device have a cooperative call, since the collection and playback of the call voice are performed on the second device, the first device no longer needs to collect and play the call voice. Therefore, in the embodiment of this application, the second When a device detects that the coordinated call function is switched from off to on, it can switch the input device of the call audio link from the audio collection hardware device to a virtual input module, and use the audio to play the output device of the call audio link The hardware device is switched to a virtual output module, so that the audio collection hardware device and the audio playback hardware device are turned off during the collaborative call process to reduce power consumption.

It should be noted that in this embodiment of the present application, the first device may also turn off the collaborative call function when the call ends. For example, the first device receives the call end operation, and in response to the call end operation, the first device closes the coordinated call function, and sets the input device of the call audio link as the audio collection hardware device, and sets the call audio link output device Set the hardware device for this audio playback.

If the first device does not make a coordinated call before receiving the call end operation, that is, the coordinated call function is turned off, then the first device will keep the coordinated call function turned off when receiving the call end operation, and keep the call audio link The input device of the channel is the audio collection hardware device, and the output device of the call audio link is the audio playback hardware device. If the first device performs a coordinated call before receiving the call end operation, that is, the coordinated call function is in the on state, then the first device will switch the coordinated call function from the on state to the off state when receiving the call end operation, and will The input device of the call audio link is switched from the virtual input device to the audio collection hardware device, and the output device of the call audio link is switched from the virtual output device to the audio playback hardware device.

If the first device does not make a coordinated call during the call, for example, the first device has not started the coordinated call after starting the call, or the first device turns off the coordinated call after starting the coordinated call function during the call. call, the first device will turn on the audio playback hardware device and the audio playback hardware device. In this case, after the first device establishes the call audio link, when the cooperative call function is turned off, the first device collects the call uplink voice data of the local user through the audio collection hardware device, and sends the call uplink The voice data is sent to the remote communication device. At the same time, the remote call device collects the call downlink voice data of the remote user, and sends the call downlink voice data to the first device. After receiving the call downlink voice data sent by the remote call device, the first device sends the call downlink voice data. The data is played through the audio playing hardware device, so as to realize the normal progress of the call.

For ease of understanding, the above-mentioned cooperative calling method will be illustrated below in combination with the software system shown in FIG. 2 and the flow chart of the coordinated calling process shown in FIG. 13 .

Referring to Figure 2, the software system of the first device may include a call application and a multi-screen collaborative application in the application layer, an audio framework and DMSDP in the application framework layer, an audio service in the system layer, and an extension layer The Audio HAL in, and the PCM device in the kernel layer, for example, the Audio HAL can be the Primary Audio HAL. In addition, the first device also includes an audio chip, which can communicate with the remote communication device, for example, the audio chip can communicate with the remote communication device through a modem processor, that is, through the modem processor Send the call uplink voice data to the remote call device, and receive the call downlink voice data sent by the remote call device through the modem processor.

Referring to FIG. 13 , the coordinated call process may include the following steps 1301 - 1311 .

Step 1301: The call application program detects the operation of making a call or answering an incoming call on the call interface, and then instructs the audio service to call the Audio HAL to establish a call audio link through the audio framework.

The call application program detects the operation of making a call or receiving an incoming call on the call interface (ie, the incall UI interface), and then determines that the first device needs to make a call, so that the call establishment process can be started. At this time, the calling application program can instruct the audio service through the audio framework to call the Audio HAL, so that the Audio HAL starts a voice call (voice call) process to realize call establishment, specifically as described in step 1302.

Step 1302: Audio HAL instructs the audio chip to establish a call audio link, and sets the input device of the call audio link as the microphone of the first device, and sets the output device of the call audio link as the earpiece of the first device.

After the Audio HAL starts the voice call process, it selects the sound emitting device and performs routing switching to realize the establishment of the call audio link. Wherein, selecting the sound emitting device is to instruct the audio chip to set the input device of the call audio link as the microphone of the first device, and set the output device of the call audio link as the earpiece of the first device. Routing switching is to instruct the audio chip to send the call uplink voice data collected by the microphone of the first device to the remote call device, and send the received call downlink voice data sent by the remote call device to the earpiece of the first device for playback .

In step 1302, the input device of the call audio link is set to the microphone of the first device, and the output device of the call audio link is set to the earpiece of the first device. The audio collection hardware device of the first device, the operation of setting the output device of the call audio link as the audio playback hardware device of the first device is the same, which will not be repeated in this embodiment of the present application.

Step 1303: When the first device and the second device are in the multi-screen collaborative state, the multi-screen collaborative application program monitors the call status, and if it detects that the call state changes from idle to active, then judges whether the collaborative call function is enabled.

The multi-screen collaborative application program will continue to monitor the call status of the first device during the multi-screen collaboration process. If it detects that the call status has changed from idle (idle) to call (offhook), it will determine that the first device has started a call. At this time The multi-screen collaborative application program can determine whether the collaborative call function is enabled. If the coordinated call function is enabled, you need to make a coordinated call during this call; if the coordinated call switch function is turned off, you do not need to perform a coordinated call during this call.

When the coordinated call function is turned on, it means that the user is allowed to establish a coordinated call, and when the coordinated call function is turned off, it means that the user is not allowed to establish a coordinated call.

In one possible manner, the user can manually enable or disable the collaborative call function. For example, the user may click on the coordinated call switch to turn on and off the coordinated call switch, so as to enable and disable the coordinated call function.

In another possible manner, the enabling and disabling of the coordinated call function may also be determined by the user's behavior of making a call or answering an incoming call. Specifically, if the user directly makes a call or answers an incoming call on the first device during the multi-screen collaboration between the first device and the second device, the first device will automatically turn off the coordinated call function. Or, if during the multi-screen collaboration between the first device and the second device, the user makes a call or answers an incoming call on the first device by manipulating the screen of the first device displayed on the second device, the first The device will automatically enable the collaborative calling function.

Step 1304: When the coordinated call function is enabled, the multi-screen coordinated application program sends a first switching instruction to the DMSDP to instruct the DMSDP to establish the coordinated call.

The first switching instruction is used to instruct the DMSDP to establish a coordinated call, that is, to instruct to switch the voice of the call to the second device, so that the second device can collect and play the voice of the call.

It is worth noting that the above step 1303 and step 1304 are only described by taking the multi-screen cooperative application program as an example to send the first switching instruction to the DMSDP when it detects that the call state has changed from idle to in-call, and the coordinated call function is enabled. Optionally, the multi-screen coordinated application program may also send the first switching instruction to the DMSDP in other cases. For example, the multi-screen coordinated application may also send a first switching instruction to the DMSDP when it detects that the coordinated call function is switched from the off state to the on state, and the monitored call state of the first device is in call.

Step 1305: After DMSDP receives the first switching instruction sent by the multi-screen collaborative application program, if it determines that the call status it monitors is in-call, it calls the Audio HAL to establish a collaborative call.

The DMSDP will also continuously monitor the call state of the first device during the coordinated call. After receiving the first switching instruction, the DMSDP determines that the multi-screen coordinated application indicates that a coordinated call needs to be established. At this time, the DMSDP will determine whether to establish a cooperative call according to the call status monitored by itself.

Specifically, when the DMSDP determines that the call state monitored by itself is idle, and determines that the received first switching instruction does not match the call state it has monitored, it does not establish a coordinated call. When the DMSDP determines that the conversation status monitored by itself is in the conversation, it determines that the first switching instruction received matches the conversation status monitored by itself, and then calls the Audio HAL to set up a coordinated conversation, specifically as described in step 1306.

Step 1306: Audio HAL creates the Stream stream corresponding to the uplink voice data of the call, to open the PCM device for obtaining the uplink voice data of the call and the corresponding channel in the audio chip, and creates the Stream stream corresponding to the downlink voice data of the call, to open the channel for the uplink voice data of the call Obtain the corresponding path in the PCM device and the audio chip of the downlink voice data of the call, and realize the establishment of the cooperative call.

In this case, during the coordinated call between the first device and the second device, DMSDP receives the call uplink voice data collected by the second device sent by the second device, and DMSDP sends the call uplink voice data to the Audio HAL, Audio HAL writes the uplink voice data of the call into the PCM device, and the PCM device sends the uplink voice data of the call to the audio chip, and the audio chip sends the uplink voice data of the call to the remote call device through the corresponding channel.

At the same time, the audio chip receives the call downlink voice data sent by the remote call device through the corresponding channel, and sends the call downlink voice data to the PCM device, and Audio HAL reads the call downlink voice data from the PCM device, and sends the call downlink voice data to the PCM device. The voice data is sent to the DMSDP, and the DMSDP sends the call downlink voice data to the second device for playback by the second device.

Step 1307: After the cooperative call is established, the Audio HAL instructs the audio chip to switch the input device of the call audio link from the microphone to the IN_PROXY module, and switch the output device of the call audio link from the earpiece to the OUT_PROXY module.

After the first device and the second device have a cooperative call, since the collection and playback of the call voice are performed on the second device, the first device no longer needs to collect and play the call voice. Therefore, in the embodiment of this application, the second After a device starts a cooperative call, it can switch the input device of the call audio link from the microphone of the first device to the IN_PROXY module, and switch the output device of the call audio link from the earpiece of the first device to the OUT_PROXY module, so as to Turn off the microphone and earpiece of the first device during a call. In this way, the first device no longer collects audio and plays audio, thereby reducing hardware power consumption while meeting the requirements of collaborative calls.

In step 1307, the input device of the call audio link is switched from the microphone to the IN_PROXY module, and the output device of the call audio link is switched from the earpiece to the OUT_PROXY module. The operations of switching the collection hardware device to the virtual input module and switching the output device of the call audio link from the audio playback hardware device to the virtual output module are the same, and will not be repeated in this embodiment of the present application.

Step 1308: When the first device and the second device are in the multi-screen collaborative state, if the multi-screen collaborative application program detects that the call state has changed from in-call to idle, or detects that the collaborative call function has changed from the on state to the off state, Then send a second switching instruction to the DMSDP to instruct to close the coordinated call.

The second switching instruction is used to instruct the DMSDP to close the coordinated call, that is, to instruct to switch the call voice back to the first device, so that the first device can collect and play the call voice.

If the multi-screen collaborative application detects that the call status of the first device has changed from in-call to idle, it means that the first device has ended the call, and at this time may instruct to close the coordinated call. For example, after the user hangs up the phone, the call state of the first device changes from in-call to idle. At this time, the multi-screen collaborative application monitors the change of the call state and can instruct to close the collaborative call.

If the multi-screen collaborative application program detects that the collaborative call function has been turned from the on state to the off state, it means that the user wants to collect and play the call voice on the first device, and at this time can instruct to turn off the collaborative call. For example, the user can manually click the coordinated call switch to turn off the coordinated call function. At this time, after the multi-screen collaborative application detects that the coordinated call switch is switched from on to off, it can instruct to turn off the coordinated call.

Step 1309: After receiving the second switching instruction sent by the multi-screen cooperative application program, the DMSDP invokes the Audio HAL to close the cooperative call.

After receiving the second switching instruction, the DMSDP determines that the multi-screen coordinated application indicates that the coordinated call needs to be terminated. Therefore, at this time, the DMSDP can call the Audio HAL to close the coordinated call, as described in step 1310.

Step 1310: Audio HAL closes the Stream stream corresponding to the uplink voice data of the call, so as to close the PCM device and the corresponding path in the audio chip used to obtain the uplink voice data of the call, and close the Stream stream corresponding to the downlink voice data of the call, so as to close the channel used for Obtain the corresponding path in the PCM device and the audio chip of the downlink voice data of the call to realize the closing of the cooperative call.

After the collaborative call is closed, the collection and playback of the call voice returns to the original state, that is, it is switched back to the first device for execution. In this case, if the first device is still talking after closing the coordinated call, the first device needs to collect and play the call voice by itself.

Step 1311: While closing the coordinated call, the Audio HAL instructs the audio chip to switch the input device of the call audio link from the IN_PROXY module to its own microphone, and switch the output device of the call audio link from the OUT_PROXY module to its own earpiece.

In this case, if the first device is still talking, the microphone of the first device collects the uplink voice data of the call and sends it to the audio chip, and the audio chip sends the uplink voice data of the call collected by the microphone of the first device to the remote call equipment. The remote call device sends the call downlink voice data to the audio chip, and the audio chip sends the received call downlink voice data sent by the remote call device to the earpiece of the first device for playback.

In step 1311, the input device of the call audio link is switched from the IN_PROXY module to its own microphone, and the output device of the call audio link is switched from the OUT_PROXY module to its own listening operation. The operation of switching the device from the virtual input module to the audio collection hardware device and switching the output device of the call audio link from the virtual output module to the audio playback hardware device is the same, and will not be repeated in this embodiment of the application.

In this embodiment of the application, when the first device and the second device are in a multi-screen coordinated state, if the first device receives a call operation, it will establish a call audio link, and set the input device of the call audio link to As the audio collection hardware device of the first device, the output device of the call audio link is set as the audio playback hardware device of the first device. At the same time, when the first device receives the call operation, it can also detect the state of the coordinated call function, and if the coordinated call function is turned on, the input device of the call audio link is switched from the audio collection hardware device to a virtual input module , switching the output device of the call audio link from the audio playback hardware device to a virtual output module, so as to close the audio collection hardware device and the audio playback hardware device during the coordinated call, so that the first device no longer collects audio, Audio is no longer played, thereby reducing hardware power consumption while meeting the needs of collaborative calls.

Fig. 14 is a schematic structural diagram of a collaborative communication device provided by an embodiment of the present application. The device can be realized by software, hardware or a combination of the two as part or all of a computer device. The computer device can be implemented as shown in Fig. 1-Fig. 2 terminal as shown in the example. Referring to FIG. 14 , the device includes: a receiving module 1401 , an establishing module 1402 , and a switching module 1403 .

The receiving module 1401 is configured to receive a call operation, wherein the device and the second device are in a multi-screen coordinated state;

The establishment module 1402 is used to establish a call audio link in response to a call operation, and set the input device of the call audio link as the audio collection hardware device of the device, and set the output device of the call audio link as the audio of the device Play hardware equipment;

The switching module 1403 is used to switch the input device of the call audio link from an audio collection hardware device to a virtual input module, and switch the output device of the call audio link from an audio playback hardware device if it is detected that the coordinated call function is in an open state It is a virtual output module, and the coordinated call function is used to instruct the second device to collect and play the call voice. The virtual input module does not collect audio, and the virtual output module does not play audio.

Optionally, the switching module 1403 is also used for:

After the call audio link is established, if it is detected that the cooperative call function is switched from the on state to the off state, the input device of the call audio link is switched from a virtual input module to an audio collection hardware device, and the output device of the call audio link Switch from a virtual output module to an audio playback hardware device.

Optionally, the switching module 1403 is also used for:

After the call audio link is established, if it is detected that the coordinated call function is switched from off to on, the input device of the call audio link is switched from an audio collection hardware device to a virtual input module, and the output device of the call audio link is switched to a virtual input module. Switch from an audio playback hardware device to a virtual output module.

Optionally, the device also includes:

The first call module is configured to receive the call uplink voice data collected by the second device sent by the second device and send the call uplink voice data to The remote communication device receives the communication downlink voice data sent by the remote communication device, and sends the communication downlink voice data to the second device for playing.

Optionally, the device also includes:

The second call module is used to collect the uplink voice data of the call through the audio collection hardware device, and send the uplink voice data of the call to the remote call device after the call audio link is established, and when the coordinated call function is in a closed state, and Receive the call downlink voice data sent by the remote call device, and play the call downlink voice data through the audio playback hardware device.

Optionally, the device also includes:

The opening module is used to enable the cooperative call function in response to the call operation if the call operation is triggered by the user through the screen image of the device displayed by the second device;

The closing module is used to close the coordinated calling function in response to the calling operation if the calling operation is triggered by the user on the device.

Optionally, the device also includes:

The closing module is used to switch the coordinated calling function from the on state to the off state if it is detected that the user turns off the cooperative calling switch displayed by the device or the second device after the call audio link is established;

The opening module is used to switch the cooperative calling function from the off state to the on state if it is detected that the user turns on the cooperative calling switch displayed by the device or the second device after the call audio link is established.

Optionally, the device also includes:

The receiving module 1401 is also used to receive a call end operation;

The setting module is used to close the coordinated call function in response to the call end operation, and set the input device of the call audio link as an audio collection hardware device, and set the output device of the call audio link as an audio playback hardware device.

Optionally, the audio collection hardware device is a microphone, and the audio playback hardware device is a speaker or an earpiece.

In this embodiment of the application, when the second device is in a multi-screen cooperative state, if a call operation is received, a call audio link is established, and the input device of the call audio link is set as its own audio collection hardware Device, set the output device of the call audio link as its own audio playback hardware device. At the same time, when the call operation is received, the state of the coordinated call function can also be detected. If the coordinated call function is turned on, the input device of the call audio link is switched from the audio collection hardware device to a virtual input module, and the call The output device of the audio link is switched from the audio playback hardware device to a virtual output module, so as to close the audio collection hardware device and the audio playback hardware device during the cooperative call, so that it no longer collects audio itself, nor plays audio , thereby reducing hardware power consumption while meeting the cooperative call requirements.

It should be noted that: the coordinated call device provided by the above-mentioned embodiments only uses the division of the above-mentioned functional modules as an example for illustration during the coordinated call. In practical applications, the above-mentioned function allocation can be completed by different functional modules according to needs. The internal structure of the device is divided into different functional modules to complete all or part of the functions described above.

The functional units and modules in the above-mentioned embodiments can be integrated into one processing unit, or each unit can exist separately physically, or two or more units can be integrated into one unit, and the above-mentioned integrated units can use hardware It can also be implemented in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the embodiments of the present application.

The coordinated calling device provided by the above embodiments and the coordinated calling method embodiments belong to the same concept. The specific working process and technical effects of the units and modules in the above embodiments can be found in the method embodiments, and will not be repeated here.

In the above embodiments, all or part may be implemented by software, hardware, firmware or any combination thereof. When implemented using software, it may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on the computer, the processes or functions according to the embodiments of the present application will be generated in whole or in part. The computer can be a general purpose computer, a special purpose computer, a computer network or other programmable devices. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be accessed from a website, computer, server, or data center Transmission to another website site, computer, server or data center by wired (such as: coaxial cable, optical fiber, Digital Subscriber Line (DSL)) or wireless (such as: infrared, wireless, microwave, etc.). The computer-readable storage medium may be any available medium that can be accessed by a computer, or may be a data storage device such as a server or a data center integrated with one or more available media. The available medium may be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a digital versatile disc (Digital Versatile Disc, DVD)) or a semiconductor medium (such as a solid state disk (Solid State Disk, SSD)) wait.

The above-mentioned optional embodiments provided by the application are not intended to limit the application. Any modifications, equivalent replacements, improvements, etc. made within the technical scope of the disclosure of the application shall be included in the scope of the application. within the scope of protection.

Claims

A method for coordinated conversation, characterized in that the method comprises:

The first device receives a call operation, wherein the first device and the second device are in a multi-screen coordinated state;

In response to the call operation, the first device establishes a call audio link, and sets the input device of the call audio link as an audio collection hardware device of the first device, and sets the call audio link The output device is set as the audio playback hardware device of the first device;

If the first device detects that the coordinated call function is in an open state, the input device of the call audio link is switched from the audio collection hardware device to a virtual input module, and the output device of the call audio link is switched from The audio playback hardware device is switched to a virtual output module, and the coordinated call function is used to instruct the second device to collect and play call voice, the virtual input module does not perform audio collection, and the virtual output module does not Play audio.
The method of claim 1, further comprising:

After the first device establishes the call audio link, if it detects that the coordinated call function is switched from the on state to the off state, then the input device of the call audio link is switched from the virtual input module to The audio collection hardware device switches the output device of the call audio link from the virtual output module to the audio playback hardware device.
The method according to claim 1 or 2, further comprising:

After the first device establishes the call audio link, if it detects that the coordinated call function is switched from the off state to the on state, then the input device of the call audio link is switched by the audio collection hardware device For the virtual input module, switch the output device of the call audio link from the audio playback hardware device to the virtual output module.
The method according to any one of claims 1 to 3, wherein the method further comprises:

After the call audio link is established, the first device receives the call uplink voice data sent by the second device and collected by the second device when the cooperative call function is turned on, and sends The call uplink voice data is sent to the remote call device, and the call downlink voice data sent by the remote call device is received, and the call downlink voice data is sent to the second device for playing.
The method according to any one of claims 1 to 4, wherein the method further comprises:

After the call audio link is established, the first device collects call uplink voice data through the audio collection hardware device and sends the call uplink voice data to The remote communication device receives the communication downlink voice data sent by the remote communication device, and plays the communication downlink voice data through the audio playback hardware device.
The method according to any one of claims 1 to 5, wherein the method further comprises:

If the call operation is triggered by the user through the screen of the first device displayed by the second device, in response to the call operation, the first device enables the coordinated call function;

If the call operation is triggered by the user on the first device, in response to the call operation, the first device disables the coordinated call function.
The method according to any one of claims 1 to 6, further comprising:

After the first device establishes the call audio link, if it detects that the user has turned off the coordinated call switch displayed by the first device or the second device, the coordinated call function is switched from on to off state;

After the first device establishes the call audio link, if it detects that the user turns on the coordinated call switch displayed by the first device or the second device, the coordinated call function is switched from an off state is on.
The method according to any one of claims 1 to 7, wherein the method further comprises:

The first device receives a call end operation;

In response to the call end operation, the first device closes the coordinated call function, and sets the input device of the call audio link as the audio collection hardware device, and sets the output device of the call audio link to Set as the audio playback hardware device.
The method according to any one of claims 1 to 8, wherein the audio collection hardware device is a microphone, and the audio playback hardware device is a speaker or an earpiece.
A cooperative communication device, characterized in that the device comprises:

a receiving module, configured to receive a call operation, wherein the device and the second device are in a multi-screen coordinated state;

The establishment module is used to establish a call audio link in response to the call operation, and set the input device of the call audio link as the audio collection hardware device of the device, and set the output device of the call audio link set as the audio playback hardware device of the device;

The switching module is used to switch the input device of the call audio link from the audio collection hardware device to a virtual input module if it is detected that the coordinated call function is on, and switch the output device of the call audio link from The audio playback hardware device is switched to a virtual output module, and the coordinated call function is used to instruct the second device to collect and play call voice, the virtual input module does not perform audio collection, and the virtual output module does not Play audio.
A computer device, characterized in that the computer device includes a memory, a processor, and a computer program stored in the memory and operable on the processor, and the computer program is implemented when executed by the processor. The method according to any one of claims 1 to 9.
A computer-readable storage medium, characterized in that instructions are stored in the computer-readable storage medium, and when the computer-readable storage medium is run on a computer, the computer is made to execute the method according to any one of claims 1 to 9.
A computer program product comprising instructions, characterized in that, when it is run on a computer, it causes the computer to execute the method according to any one of claims 1 to 9.
A method for coordinated conversation, characterized in that the method comprises:

The first device receives a call operation, wherein the first device and the second device are in a multi-screen coordinated state;

In response to the call operation, the first device establishes a call audio link, and sets the input device of the call audio link as an audio collection hardware device of the first device, and sets the call audio link The output device is set as the audio playback hardware device of the first device;

If the first device detects that the coordinated call function is in an open state, the input device of the call audio link is switched from the audio collection hardware device to a virtual input module, and the output device of the call audio link is switched from The audio playback hardware device is switched to a virtual output module to close the audio collection hardware device and the audio playback hardware device during the coordinated call, and the coordinated call function is used to instruct the second device to perform a call voice collection and playback, the virtual input module and the virtual output module are virtual software modules, the virtual input module does not perform audio collection, and the virtual output module does not perform audio playback;

After the first device establishes the call audio link, if it detects that the coordinated call function is switched from the on state to the off state, then the input device of the call audio link is switched from the virtual input module to The audio collection hardware device switches the output device of the call audio link from the virtual output module to the audio playback hardware device, so as to open the audio collection hardware device for audio collection and open the audio playback hardware The device performs audio playback;

After the first device establishes the call audio link, if it detects that the coordinated call function is switched from the off state to the on state, then the input device of the call audio link is switched by the audio collection hardware device For the virtual input module, switch the output device of the call audio link from the audio playback hardware device to the virtual output module, so as to close the audio collection hardware device and the audio playback during the coordinated call hardware equipment.
The method of claim 14, further comprising:

After the call audio link is established, the first device receives the call uplink voice data sent by the second device and collected by the second device when the cooperative call function is turned on, and sends The call uplink voice data is sent to the remote call device, and the call downlink voice data sent by the remote call device is received, and the call downlink voice data is sent to the second device for playing.
The method according to claim 14 or 15, further comprising:

After the call audio link is established, the first device collects call uplink voice data through the audio collection hardware device and sends the call uplink voice data to The remote communication device receives the communication downlink voice data sent by the remote communication device, and plays the communication downlink voice data through the audio playback hardware device.
The method according to any one of claims 14 to 16, further comprising:

If the call operation is triggered by the user through the screen of the first device displayed by the second device, in response to the call operation, the first device enables the coordinated call function;

If the call operation is triggered by the user on the first device, in response to the call operation, the first device disables the coordinated call function.
The method according to any one of claims 14 to 17, further comprising:

After the first device establishes the call audio link, if it detects that the user has turned off the coordinated call switch displayed by the first device or the second device, the coordinated call function is switched from on to off state;

After the first device establishes the call audio link, if it detects that the user turns on the coordinated call switch displayed by the first device or the second device, the coordinated call function is switched from an off state is on.
The method according to any one of claims 14 to 18, further comprising:

The first device receives a call end operation;

In response to the call end operation, the first device closes the coordinated call function, and sets the input device of the call audio link as the audio collection hardware device, and sets the output device of the call audio link to Set as the audio playback hardware device.
The method according to any one of claims 14 to 19, wherein the audio collection hardware device is a microphone, and the audio playback hardware device is a loudspeaker or an earpiece.