WO2022228573A1

WO2022228573A1 - Display device and method for outputting audio

Info

Publication number: WO2022228573A1
Application number: PCT/CN2022/090563
Authority: WO
Inventors: 卢平光; 刘儒茜; 王建亮; 齐消消; 贾亚洲; 张永芳; 陈俊宁; 崔文华; 李斌
Original assignee: 海信视像科技股份有限公司
Priority date: 2021-04-30
Filing date: 2022-04-29
Publication date: 2022-11-03

Abstract

Disclosed in some embodiments of the present application are a display device and a method for outputting audio. The display device comprises a display and an audio output interface controller; the display is used for displaying a user interface, and the user interface comprises a plurality of windows; there are a plurality of audio output interfaces, which are used for connecting multiple types of audio devices; and the controller is configured as: in response to a user-input display instruction indicating to display a plurality of windows, controlling the display to display the plurality of windows; detecting a focused window on which a user-controlled focus is located; and in response to a user-input switch instruction indicating to switch an audio device corresponding to the focused window, switching an audio device corresponding to the focused window, and adjusting audio devices corresponding to remaining windows.

Description

Display device and audio output method

CROSS-REFERENCE TO RELATED APPLICATIONS

This application requires filing on April 30, 2021 with application number 202110481435.2; filed on April 30, 2021 with application number 202110480084.3; filed on April 30, 2021 with application number 202110481408.5; in 2021 The priority of the Chinese patent application with application number 202110481495.4 filed on April 30, 2019, the entire contents of which are incorporated herein by reference.

technical field

The present application relates to display device technology, and in particular, to a display device and an audio output method.

Background technique

For some display devices, the screen displayed on the display can be split-screened to form multiple windows, and each window displays different screen contents, so that the user can watch different program screens at the same time. Although the display device can display different pictures through multiple windows, it will only select one of the windows to play the sound, that is, the user is allowed to watch multiple program pictures, but the sound of one of the program pictures will be played, and the rest of the program pictures cannot be played. Play sound, which seriously affects the user experience.

SUMMARY OF THE INVENTION

Some embodiments of the present application provide a display device, including: a display, an audio output interface, and a controller. Wherein, the display is used to display a user interface, and the user interface includes multiple windows; there are multiple audio output interfaces, which are used to connect multiple types of audio devices; the controller is configured to: display the display of multiple windows in response to an instruction input by the user instruction, control the display to display multiple windows; detect the focus window on which the focus controlled by the user is located; and switch the audio device corresponding to the focus window in response to the switching instruction of switching the audio device corresponding to the focus window in response to the instruction input by the user, And adjust the audio devices corresponding to the remaining windows so that each window uses a different audio device.

Some embodiments of the present application further provide an audio output method, applied to a display device, the method includes: in response to a display instruction of displaying a plurality of windows input by a user, controlling the display to display a plurality of windows; detecting that the user-controlled focus is on the focus window; in response to the switching instruction of switching the audio device corresponding to the focus window in response to the instruction input by the user, the audio device corresponding to the focus window is switched, and the audio devices corresponding to the remaining windows are adjusted so that each window uses different audio equipment.

Some embodiments of the present application provide a display device, including: a display, an audio output interface, and a controller. Wherein, the display is used to display a user interface, and the user interface includes a plurality of windows; the audio output interface can be connected to a local speaker and/or an external audio device for outputting audio to form an audio output path. The controller is configured to: obtain a control instruction input by a user for adjusting audio parameters; and in response to the control instruction, detect a current focus window, where the focus window is a plurality of windows in the user interface with focus The marked window; obtain the current playback audio parameters of the focus window, and obtain the audio parameter adjustment amount input by the user; calculate the adjusted audio parameters according to the current playback audio parameters and the audio parameter adjustment amount, so as to adjust the audio parameters according to the adjustment The post audio parameter configures the audio output path of the focus window.

Some embodiments of the present application provide a display device including a display, an audio processor, an audio output interface, and a controller. Wherein, the display is used to display a user interface, and the user interface includes multiple windows; the audio processor is used to generate audio signals; there are multiple audio output interfaces, used to connect multiple types of audio devices; the controller is configured to: respond to The display instruction input by the user indicating the display of multiple windows controls the display to display multiple windows; the focus window where the focus controlled by the user is located is detected; in response to the audio and video synchronization instruction input by the user, the audio device corresponding to the focus window is detected; When the audio device corresponding to the focus window is the first type of audio device, acquire the first audio and video synchronization value; when the audio device corresponding to the focus window is the second type of audio device, acquire the second audio and video synchronization value; according to The audio-visual synchronization value sets the audio playback parameter corresponding to the audio signal, so that the picture information presented by the focus window is synchronized with the audio information played by the audio device corresponding to the focus window.

Some embodiments of the present application provide a display device, including: a display, an audio output interface, and a controller. Wherein, the display is used to display a user interface, and the user interface includes multiple windows; the audio output interface can be connected to audio devices such as a local speaker and/or an external audio device for outputting audio to form volume output path. Wherein, the audio output interface connected to the local speaker is a local interface, and the audio output interface connected to an external audio device is an external interface. The controller is configured to: obtain a control instruction input by the user for waking up the voice broadcast; in response to the control instruction, detect a current response window, where the response window is a local interface in multiple windows of the user interface. A window for outputting audio; performing mute processing on the signal source of the response window; and broadcasting feedback voice through the audio output path of the response window, where the feedback voice is the voice information fed back according to the voice command input by the user during the mute period.

Description of drawings

1 is a schematic diagram of an operation scenario between a display device and a control device according to some embodiments;

FIG. 2 is a block diagram of a hardware configuration of the control apparatus 100 according to some embodiments;

3 is a block diagram of a hardware configuration of a display device 200 according to some embodiments;

FIG. 4 is a schematic diagram of software configuration in the display device 200 according to some embodiments;

5 is a schematic diagram of a screen in a display in a multi-window mode according to some embodiments of the present application;

6 is a schematic diagram of an audio output mode in a multi-window mode according to some embodiments of the present application;

7 is a schematic diagram of an audio device corresponding to a focus window displayed on a display according to some embodiments of the present application;

8 is a schematic diagram of an audio device corresponding to a focus window displayed on a display according to some embodiments of the present application;

9 is a schematic flowchart of a method for performing audio parameter adjustment according to a focus window according to some embodiments of the present application;

10 is a schematic flowchart of querying currently playing audio parameters according to some embodiments of the present application;

11 is a schematic diagram of an interface effect of performing audio parameter adjustment according to a focus window according to some embodiments of the present application;

FIG. 12 is a schematic flowchart of an embodiment of the present application according to a method for adjusting audio parameters synchronously in multiple windows according to some embodiments of the present application;

13 is a schematic diagram of an interface effect of performing audio parameter adjustment synchronously with multiple windows according to some embodiments of the present application according to an embodiment of the present application;

14 is a schematic diagram of the effect of displaying a volume adjustment control on a global interface according to some embodiments of the present application according to an embodiment of the present application;

15 is a schematic flowchart of a method for performing audio parameter adjustment according to a target window according to some embodiments of the present application;

16 is a schematic diagram of an interface effect of performing audio parameter adjustment according to a target window according to some embodiments of the present application;

17 is a schematic diagram of a sound effect adjustment window according to some embodiments of the present application;

FIG. 18 is a schematic diagram showing the synchronization adjustment of audio and video in a focus window according to some embodiments of the present application;

19 is a schematic flowchart of a multi-window voice broadcast method according to some embodiments of the present application;

20 is a schematic flowchart of a detection response window according to some embodiments of the present application;

21 is a schematic flowchart of obtaining feedback speech according to some embodiments of the present application;

22 is a schematic flowchart of detecting a response window according to a database according to some embodiments of the present application;

23 is a schematic flowchart of displaying a voice interaction control in a focus window according to some embodiments of the present application;

FIG. 24 is a schematic flowchart of synchronously broadcasting feedback speech according to some embodiments of the present application.

Detailed ways

In order to make the objectives, implementations and advantages of the present application clearer, the exemplary embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings in the exemplary embodiments of the present application. Obviously, the exemplary embodiments described It is only a part of the embodiments of the present application, but not all of the embodiments.

FIG. 1 is a schematic diagram of an operation scenario between a display device and a control apparatus according to an embodiment. As shown in FIG. 1 , a user can operate the display device 200 through the smart device 300 or the control device 100 . In some embodiments, the control apparatus 100 may be a remote controller, and the communication between the remote controller and the display device includes infrared protocol communication or Bluetooth protocol communication, and other short-range communication methods, and the display device 200 is controlled wirelessly or wiredly. The user can control the display device 200 by inputting user instructions through keys on the remote control, voice input, control panel input, and the like. In some embodiments, a smart device 300 (eg, a mobile terminal, a tablet computer, a computer, a notebook computer, etc.) can also be used to control the display device 200 . For example, the display device 200 is controlled using an application running on the smart device. In some embodiments, the display device 200 can also be controlled in a manner other than the control apparatus 100 and the smart device 300. For example, the module for acquiring voice commands configured inside the display device 200 can directly receive the user's voice command for control. , the user's voice command control can also be received through a voice control device provided outside the display device 200 device. In some embodiments, the display device 200 is also in data communication with the server 400 . The display device 200 may be allowed to communicate via local area network (LAN), wireless local area network (WLAN), and other networks. The server 400 may provide various contents and interactions to the display device 200 .

FIG. 2 exemplarily shows a configuration block diagram of the control apparatus 100 according to an exemplary embodiment. As shown in FIG. 2 , the control device 100 includes a controller 110 , a communication interface 130 , a user input/output interface 140 , a memory, and a power supply. The control apparatus 100 can receive the user's input operation instruction, and convert the operation instruction into an instruction that the display device 200 can recognize and respond to, so as to play a role of intermediary between the user and the display device 200 .

FIG. 3 is a block diagram showing a hardware configuration of the display apparatus 200 according to an exemplary embodiment. As shown in FIG. 3, in some embodiments, the display device 200 includes a tuner demodulator 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output interface 270, a memory, a power supply At least one of a power supply and a user interface. In some embodiments, the display 260 includes a display screen component for presenting a picture, and a driving component for driving the image display, for receiving the image signal output from the controller, for displaying the video content, the image content and the menu manipulation interface Components and user manipulation UI interface. The display 260 may be a liquid crystal display, an OLED display, and a projection display, and may also be a projection device and a projection screen. In some embodiments, the controller 250, through various software control programs stored on the memory, controls the operation of the display device and responds to user operations. The controller 250 controls the overall operation of the display apparatus 200 . For example, in response to receiving a user command for selecting a UI object to be displayed on the display 260, the controller 250 may perform an operation related to the object selected by the user command. In some embodiments, the object may be any of the selectable objects, such as hyperlinks, icons, or other operable controls.

FIG. 4 is a schematic diagram of software configuration in the display device 200 according to some embodiments. Referring to FIG. 4 , in some embodiments, the system is divided into four layers, from top to bottom, the applications layer (referred to as “application layer”) ”), the Application Framework layer (referred to as the “framework layer”), the Android runtime (Android runtime) and the system library layer (referred to as the “system runtime layer”), and the kernel layer. In some embodiments, in the application layer

There is at least one application running, which can be a window program, a system setting program or a clock program that comes with the operating system; the framework layer provides an application programming interface (API) and a programming framework for the application . The application framework layer includes some predefined functions. The kernel layer is the layer between hardware and software. The kernel layer at least includes at least one of the following drivers: audio driver, display driver, Bluetooth driver, etc.

Some embodiments of the present application provide a display device including a display, an audio output interface, and a controller. The display is used to display the user interface; the audio output interface includes multiple types of audio interfaces, which are respectively used to connect multiple types of audio devices. The audio output interface in the display device may include a first-type audio interface and a second-type audio interface, the first-type audio interface is used for connecting the first-type audio device, and the second-type audio interface is used for connecting the second-type audio device. The first type of audio devices may be audio devices such as speakers or sound boxes built into the display device. The second type of audio device may be a Bluetooth speaker, a USB speaker, or a Soundbar speaker connected to the display device.

In some embodiments, the display device may be provided with one audio interface of the first type and a plurality of audio interfaces of the second type. That is, the display device can be connected with a first-type audio device and a plurality of second-type audio devices, and play audio content. For example, a display device can have a built-in speaker and two external Bluetooth speakers.

In some embodiments, the display device has a split-screen function, which can divide the user interface displayed on the display into multiple windows, and each window can display different screen contents, that is, the display device has a multi-window mode. In the multi-window mode, the display device can display multiple windows at the same time, so that the user can view different screen contents in one display device.

In some embodiments, when the display device displays a user screen and there is only one window in the user screen, the user may input a display instruction to display multiple windows, that is, a multi-window mode instruction, to the display device. When the controller receives the multi-window mode instruction, it can control the display from displaying one window to displaying multiple windows. For example, the correspondence between the multi-window mode instruction and the remote control buttons is pre-bound, and the user can send the multi-window mode instruction to the display device by operating the designated button of the remote control. For example, for example, a multi-window mode button is set on the remote control. When the user touches the button, the remote control sends a multi-window mode command to the controller. At this time, the controller controls the display device to enter the multi-window mode and controls the display to display multiple windows. When the user touches the button again, the controller can control the display device to exit the multi-window mode. In some embodiments, the correspondence between the multi-window mode command and multiple remote control buttons can also be pre-bound. When the user touches multiple buttons bound to the multi-window mode command, the remote control sends out the multi-window mode command. instruction. E.g,

In some embodiments, when the user uses the smart device to control the display device, such as using a mobile phone, a multi-window mode instruction can also be sent to the display device. For example, a split-screen control can be set in the mobile phone. You can choose whether to enter the multi-window mode and set the number of windows through the split-screen control, so as to send the multi-window mode command to the controller. At this time, the controller controls the display device to enter the multi-window mode. mode, and control the display to display multiple windows.

In some embodiments, at least two windows may be displayed in a multi-window mode. 5 is a schematic diagram of a screen in a display in a multi-window mode according to some embodiments of the present application. After the display device enters the multi-window mode, a first window and a second window are displayed in the display at the same time, wherein the first window and the second window are The windows can display different screen contents.

In some embodiments, when the screen content of the first media asset is displayed in the first window, the screen content of the second media asset may be displayed in the second window. At this time, the screen types displayed in the two windows are the same. for media resources. It is also possible to display a UI interface or a game interface in the second window when the screen content of the media resource is displayed in the first window, and at this time, the screen types displayed by the two windows are different. Thus, different needs of users can be met, and user experience can be enhanced.

In some embodiments, the multi-window mode can be set according to the usage status of the user. When it is detected that the user has turned on the display device, the display device defaults to single-window mode, that is, only the first window is displayed on the display. The first window can be displayed in full screen or windowed, and the display device can play in the first window. First media resource. When the first media resource is played in the first window, if the user sends a multi-window mode instruction to the display device, the user may also enter the program recommendation interface first.

In some embodiments, when the display device is in the multi-window mode, while the multi-windows simultaneously display different picture contents, the audio contents corresponding to each picture can also be separately output. The display device can use a first-class audio device and multiple second-class audio devices to play audio at the same time. For example, the display device is connected to a built-in speaker and two external Bluetooth speakers at the same time. At this time, the display device can use these three audio devices respectively. Output the audio content of the three windows. At the same time, each window can output audio to the audio device corresponding to each window through its own audio channel, and play audio content through the audio device.

In some embodiments, the display device can set two audio device modes. One is the built-in playback mode, which can use the first type of audio equipment built in the display device to play the audio content corresponding to the window. One is the external playback mode, by using the second type of audio device, such as a Bluetooth speaker, etc., to play the audio content corresponding to the window.

FIG. 6 is a schematic diagram of an audio output manner in a multi-window mode according to some embodiments of the present application. Wherein, the display device is in a multi-window mode, wherein the first media resource is played in the first window, and the first type of audio device is used to output the audio; the second media resource is played in the second window, and the second type of audio device is used to output the audio . It is also possible that both the first window and the second window use the second type of audio device to output audio, that is, the first window uses the first device to output audio, and the second window uses the second device to output audio. The first device and the second device in the embodiments of the present application are both audio devices of the second type. It should be noted that since each window plays audio through different audio devices, the audio of each screen in the multi-window mode can be played independently, so as to achieve the purpose of not interfering with each other.

In some embodiments, the controller can receive various control instructions input by the user during the use of the user, and respectively perform corresponding interactive actions according to these control instructions, so as to control each window, such as image quality control and sound control Wait. The display device can also detect the focused window in real time. The focus window refers to the window where the user-controlled focus is located.

In some embodiments, while the display device detects the focused window in real time, the controller may also acquire the audio device currently corresponding to the focused window. For example, it can be obtained through a database in the display device for recording the audio device currently used by each window. The controller can determine the focus window according to the focus position, and if the focus window is the first window, it can obtain the audio output device currently used by the first window from the database.

In some embodiments, the display device may display the audio device mode status to the user, where the audio device mode status refers to: the audio device currently corresponding to the focus window and the switchable audio device mode, so that the user can focus on the focus window. Switch the audio device mode. 7 is a schematic diagram of an audio device corresponding to a focus window displayed on a display according to some embodiments of the present application, wherein the user interface includes a first window and a second window, and the first window is the focus window. When the user's display instruction is detected, the audio device currently corresponding to the first window, that is, the first type of audio device, and the switchable audio device mode of the first window, that is, the second type of audio device, are displayed in the first window. When detecting a switch instruction instructing to switch the audio device corresponding to the focus window, the display device may switch the audio device corresponding to the focus window in response to the switch instruction input by the user. At the same time, in order to ensure that each window can still play audio independently, you can also adjust the audio devices corresponding to the other windows, so as to ensure that each window uses a different audio device.

In some embodiments, if the audio device currently corresponding to the focus window is the second type of audio device, and the user instructs to switch the audio device corresponding to the focus window from the second type of audio device to the first type of audio device, the display device responds to this The switch command can switch the audio device corresponding to the focus window.

Before switching the audio device, the controller may first set the first audio channel of the focus window to a mute state, so as to prevent the popping phenomenon from occurring when the audio device is switched. After the first audio channel is set to the mute state, the current usage state of the first type of audio device can be determined. If the first type of audio device is not in use, it means that there is currently no window using the first type of audio device, you can directly connect the first audio channel and the first type of audio device, and unmute the first audio channel. So that the focus window can output audio to the first type of audio device through the first audio channel, and use the first type of audio device to play the audio.

If the first type of audio device is in use, it means that other windows are using the first type of audio device. At this time, the window and the audio device of the focus window can be exchanged, so that the focus window can use the first type of audio device. Before switching audio devices, the controller can detect the window currently corresponding to the first type of audio device and set it as the first switching window, and at the same time set the second audio channel of the first switching window to a mute state to prevent popping sound. The controller then connects the second audio channel with the second type of audio device corresponding to the focus window, and connects the first audio channel with the first type of audio device. At this point, the audio devices are exchanged between the focus window and the first switching window. The controller then unmutes the first audio channel and the second audio channel, so that the focus window can use the first type of audio device to play audio, and at the same time, each window can still play audio independently without being affected.

In some embodiments, if the audio device currently corresponding to the focus window is the first type of audio device, and the user instructs to switch the audio device corresponding to the focus window from the first type of audio device to the second type of audio device, the display device responds to this The switch command can switch the audio device corresponding to the focus window.

Before switching the audio device, the controller may first set the first audio channel of the focus window to a mute state, so as to prevent the popping phenomenon from occurring when the audio device is switched.

After the first audio channel is set to a mute state, considering that the display device can be connected to multiple second-type audio devices, it can be determined whether the display device is currently connected to an unused second-type audio device. If there is currently an unused second-class audio device, you can directly connect the first audio channel to an unused second-class audio device, so that the focus window can output audio to the second audio channel through the first audio channel Class 2 audio device and play audio using the second class audio device.

If the display device is not connected to an unused second-class audio device, it means that all second-class audio devices are used by other windows. At this time, a second-class audio device can be randomly selected for use, so that the focus window can be used. Class II audio equipment. Before switching the audio device, the controller can randomly select a second-type audio device, that is, the first device, and determine the second switching window currently corresponding to the second-type audio device, and at the same time select the third audio channel of the second switching window Set to mute to prevent popping.

The controller then connects the third audio channel with the first type of audio device, and connects the first audio channel with the first device. At this point, the audio devices are exchanged between the focus window and the second switching window. The controller then unmutes the first audio channel and the third audio channel, so that the focus window can use the first device to play audio, and at the same time, each window can still play audio independently without being affected.

In some embodiments, the display device may display the audio device currently corresponding to the focus window and the usage of the displayed audio device to the user. The usage of the audio device may include all audio devices that can be used by the display device and a window currently corresponding to each audio device. The user selects the audio device of the focus window according to the usage of the audio device, and switches to the selected specific audio device.

8 is a schematic diagram of an audio device corresponding to a focus window displayed on a display according to some embodiments of the present application, wherein the user interface includes a first window, a second window and a third window, and the first window is the focus window. When the user's query instruction is detected, the audio device currently corresponding to the first window is displayed in the first window, which is the first type of audio device. And the usage of the audio device, that is: the first type of audio device currently corresponds to the first window, the first device currently corresponds to the second window, and the second device currently corresponds to the third window.

In some embodiments, if the audio device currently corresponding to the focus window is the first type of audio device, and the user instructs to switch the audio device corresponding to the focus window from the first type of audio device to a specific second type of audio device, the display device responds to The switching instruction can switch the audio device corresponding to the focus window.

After setting the first audio channel to the mute state, the controller may determine the usage state of the specific second-type audio device. If the specific type 2 audio device is not in use, it means that there is currently no window using the type 1 audio device, you can directly connect the first audio channel with the specific type 2 audio device, and unmute the first audio channel state. So that the focus window can output audio to a specific second-type audio device through the first audio channel, and use the specific second-type audio device to play the audio.

If a specific second-class audio device is in use, it means that other windows are using the specific second-class audio device. At this time, the window and the audio device of the focus window can be exchanged, so that the focused window can use the specific second-class audio device. Audio equipment. Before exchanging the audio devices, the controller can detect the window currently corresponding to the specific second-type audio device, and set it as a specific switching window, and at the same time set the specific audio channel of the specific switching window to a mute state to prevent popping sound. The controller then connects the specific audio channel with the first type of audio equipment, and connects the first audio channel with the specific second type of audio equipment. At this point, the audio devices are swapped between the focused window and the specific toggle window. The controller then unmutes the first audio channel and the second audio channel, so that the focused window can use a specific second type of audio device to play audio, and at the same time, each window can still play audio independently without being affected.

In some embodiments, when it is detected that a specific second-type audio device currently corresponds to a specific switching window, it may also be detected first whether an unused second-type audio device is currently connected to the display device. If so, connect the specific switch window with the unused second-class audio device, and connect the focused window with the specific second-class audio device. If there are no unused second-class audio devices, swap the audio devices between the focused window and the specific toggle window.

In some embodiments, if the audio device currently corresponding to the focus window is the second type of audio device, and the user instructs to switch the audio device corresponding to the focus window from the second type of audio device to the first type of audio device, the display device responds to this The switch command can switch the audio device corresponding to the focus window. The controller can exchange the audio device between the focus window and the window currently corresponding to the first type of audio device. For the specific exchange method, refer to the above-mentioned embodiment to switch the audio device corresponding to the focus window from the first type of audio device to a specific second type of audio device. Audio equipment.

In some embodiments, if the audio device currently corresponding to the focus window is a second type audio device, and the user instructs to switch the audio device corresponding to the focus window from the second type audio device to a specific second type audio device, the display device responds to The switching instruction can switch the audio device corresponding to the focus window.

The controller may first determine the usage of the specific second type of audio equipment. If the specific second type audio device is not in use, the first audio channel and the specific second type audio device are directly connected. If the specific second-type audio device is in use, the audio device can be directly exchanged between the focus window and the specific switching window currently corresponding to the specific second-type audio device. Repeat.

In some embodiments, the display device can package the audio data of the window and transmit it to the second type of audio device. When the display device is connected to a second type of audio device and is connected to a plurality of second type of audio devices, the packaging and processing methods for audio data are also different.

When the display device is connected to a second-class audio device, the audio data of the window can be packaged separately for the left and right channels, and the left-channel packaged data and the right-channel packaged data are respectively transmitted to the second-class audio device. When two second-class audio devices are connected to the display device, the audio data of the same window is packaged together, that is, each window has only one packaged data, and then the packaged data is transmitted to the corresponding second-class audio device.

In some embodiments, when the display device is in the multi-window mode, the user can also control the display device to enter the single-window mode. The user selects the focus window, and can control the display device to enter the single-window mode by touching the single-window mode button twice. At this time, the controller can control the display to display only the focus window, or display the focus window in full screen.

In some embodiments, when the display device is switched from the multi-window mode to the single-window mode, the currently used audio device can be controlled to be the audio device corresponding to the focus window, and the audio channels of all other windows can be set to the mute state at the same time, that is, the display The device only uses the audio device corresponding to the focused window to play audio.

In some embodiments, if the focus window uses the second-type audio device to play audio, and other windows also use the second-type audio device to play audio, when switching the multi-window mode to the single-window mode, it is also necessary to change the audio The data packaging method is changed to separate packaging for the left and right channels of the audio data of the focus window, and send them to the second type of audio equipment respectively.

In some embodiments, when the display device is switched from the multi-window mode to the single-window mode, the first type of audio device may also be set as the default audio device, and the currently used audio device is controlled to be the first type of audio device.

Some embodiments of the present application also provide an audio output method, which is applied to a display device, and the method includes:

In response to a user-input instruction to display a display instruction of multiple windows, the display is controlled to display multiple windows; the focus window on which the user-controlled focus is located is detected; the switching instruction for switching the audio device corresponding to the focus window is switched in response to the user-input instruction, The audio device corresponding to the focus window is switched, and the audio devices corresponding to the remaining windows are adjusted, so that each window uses a different audio device.

In some embodiments, when the switching instruction is: switch the audio device corresponding to the focus window from the second type of audio device to the first type of audio device; switch the audio device corresponding to the focus window and adjust The steps of the audio equipment corresponding to the remaining windows include: setting the first audio channel of the focus window to a mute state, and judging the use state of the first type of audio equipment; the first audio channel is used to set the focus The audio of the window is output to the audio device corresponding to the focus window; when the first type of audio device is not in use, the first audio channel is connected to the first type of audio device; when the first type of audio device is When the audio device is in use, determine the first switching window currently corresponding to the first type of audio device; set the second audio channel of the first switching window to a mute state, and the second audio channel is used to The audio of the first switching window is output to the audio device corresponding to the first switching window; the second audio channel is connected to the second type of audio device corresponding to the focus window, and the first audio The channel is connected to the first type of audio equipment; the mute state of the second audio channel is released; the mute state of the first audio channel is released.

In some embodiments, when the switching instruction is: switch the audio device corresponding to the focus window from the first type of audio device to the second type of audio device; switch the audio device corresponding to the focus window and adjust The step of the audio equipment corresponding to the remaining windows includes: setting the first audio channel of the focus window to a mute state, and judging whether the audio output interface is connected to an unused second type of audio equipment; the first audio channel The audio channel is used to output the audio of the focus window to the audio device corresponding to the focus window; when the audio output interface is connected with an unused second type of audio device, the first audio channel and the unused audio device are connected. When the audio output interface is not connected to the unused second-type audio equipment, randomly select the first device in the second-type audio equipment, and determine the current corresponding to the first device. the second switching window; set the third audio channel of the second switching window to a mute state, and the third audio channel is used to output the audio of the second switching window to the audio corresponding to the second switching window device; connect the third audio channel to the first type of audio device, and connect the first audio channel to the first device; unmute the third audio channel; The mute state of an audio channel.

In some embodiments, before performing the step of switching the audio device corresponding to the focus window and adjusting the audio devices corresponding to the remaining windows, the method includes: querying the audio device corresponding to the focus window in response to an instruction input by the user. query instruction to control the display to display the usage of the audio device, so that the user can switch the audio device corresponding to the focus window to a specific audio device according to the usage of the audio device; the usage of the audio device includes the audio device that can be used by the display device All audio devices and the current window for each audio device.

In some embodiments, when the switching instruction is: switch the audio device corresponding to the focus window from a first type of audio device to a specific second type of audio device; switch the audio device corresponding to the focus window, and The step of adjusting the audio devices corresponding to the remaining windows includes: setting the first audio channel of the focus window to a mute state, and judging the use state of the specific second type of audio device; the first audio channel is used for Output the audio of the focus window to the audio device corresponding to the focus window; when the specific second-type audio device is not in use, perform the first audio channel and the specific second-type audio device. connect; when the specific second-type audio device is in use, determine a specific switching window currently corresponding to the specific second-type audio device; set a specific audio channel of the specific switching window to a mute state, the specific switching window The audio channel is used to output the audio of the specific switching window to the audio device corresponding to the specific switching window; connect the specific audio channel to the first type of audio device, and connect the first audio channel to all audio devices. The specific second type of audio equipment is connected; the mute state of the specific audio channel is released; the mute state of the first audio channel is released.

In some embodiments, the user interface includes a first window and a second window; when the audio device corresponding to the first window is a first type of audio device, the audio device corresponding to the second window is a second audio device Class audio device; when the audio device corresponding to the first window is the second class audio device, the audio device corresponding to the second window is the first class audio device or the remaining second class audio devices.

In some embodiments, before executing the step of controlling the display to display a plurality of windows, the step includes: in response to an instruction input by the user to turn on a power-on instruction of the display device, controlling the display to display a first window, where the first window is used to play the first window. Media resources; in the step of controlling the display to display multiple windows, the step includes: in response to the display instruction, controlling the display to simultaneously display a first window and a second window, where the second window is used to display a recommended list of media resources , so that the user selects the second media asset according to the media asset recommendation list; the second media asset is played in the second window.

In some embodiments, when the switching instruction is: switching the audio device corresponding to the focus window from the first device in the second type of audio device to the second device in the second type of audio device; wherein, the The focus window is the first window, and the second device is the audio device currently corresponding to the second window; the steps of switching the audio device corresponding to the focus window and adjusting the audio devices corresponding to the remaining windows include: Connect the audio channel of the first window to the second device, and connect the audio channel of the second window to the first device.

In some embodiments, the method further includes: in response to a control instruction of switching the user interface from multiple windows to a single window in response to an instruction input by the user, controlling the display to display only the focused window; and setting the audio channels of the remaining windows to a mute state.

When the display device 200 is playing media asset data or other data with audio and video content, it can also perform interactive actions to control the audio and video data output process, including image quality control and sound control. For example, the display device 200 can receive the user's control instruction, call the setting key UI menu according to the control instruction, display the volume adjustment control of the current window in the OSD layer menu, and receive the user's instruction to select volume adjustment in this option again, thereby completing the display Device 200 volume adjustment. The user can set the volume to any specific value in the range of 0-100 according to the UI, and the display device 200 adjusts the gain of the audio output channel according to the set volume value, so that the local speaker or the external audio output device outputs sound according to the volume value. .

In the multi-window mode, the display device 200 will simultaneously output content corresponding to multiple windows. Therefore, when adjusting the volume of multiple windows according to the traditional audio parameter adjustment method, the adjustment operation cannot be performed accurately for each window. To this end, a display device 200 is provided in some embodiments of the present application. The display device 200 includes a display 260 , an audio output interface 270 and a controller 250 . The display 260 is used for displaying a user interface, and the displayed user interface may include multiple windows when the display device 200 is in a multi-window mode. The audio output interface 270 can be connected to a local speaker and/or an external audio device for outputting audio to form an audio output path. The controller 250 is configured to execute a multi-window audio parameter adjustment method for performing accurate audio parameter adjustment operations on the display device 200 in the multi-window mode.

FIG. 9 is a schematic flowchart of a method for performing audio parameter adjustment according to a focus window according to some embodiments of the present application. As shown in FIG. 9 , the specific implementation of the multi-window audio parameter adjustment method will be described below by taking volume adjustment as an example. It should be understood that, the audio parameter adjustment method provided in the embodiment of the present application also includes other audio parameter adjustment besides volume adjustment. For example, surround parameters, bass parameters, mid-range parameters, treble parameters, timbre parameters, pitch parameters, etc. Among them, the method for adjusting the volume includes the following:

Get the control command entered by the user to adjust the volume. The controller 250 of the display device 200 can receive various control instructions input by the user during the use of the user, and different control instructions have different functions correspondingly. Among them, some control commands can be used to adjust the volume. For example, the user can press the volume "+" or volume "-" key on the remote control to input a control command to adjust the volume; and can also control the display device 200 to display the volume setting by calling up the menu option and selecting "sound-volume" menu to enter control commands for adjusting the volume.

In addition, when the display device 200 has a built-in intelligent voice system, the user can also control the display device 200 to adjust the volume by inputting a voice command. For example, the user can input voice content such as "adjust the volume to 75", "increase the volume", "decrease the volume" through a far-field microphone or an external sound collection device, so that the intelligent voice system responds to the corresponding voice content to form a The control instruction for adjusting the volume controls the display device 200 to perform volume adjustment settings.

It should be noted that the above-mentioned control instructions for adjusting the volume are only used as several interaction examples, and do not limit the interaction modes of the solution of the present application. Enter a control command for adjusting the volume. For example, the control instruction can also be input through a somatosensory operation, a touch operation, and a physical key operation on the display device 200.

After receiving the control instruction for adjusting the volume, the display device 200 may detect the current focus window in response to the control instruction input by the user.

According to the detection result of the focused window, the controller 250 may acquire the current playback volume for the focused window. The current playback volume can be acquired through a database in the display device 200 for recording setting parameters or playback parameters. For example, the controller 250 can obtain whether the first window or the second window needs to be adjusted according to the focus position of the window. If the focus is marked on the first window, the system can obtain the current audio output device from the database corresponding to the first window. , and get the current volume value according to the audio output device.

The current playback volume can also be obtained in other ways. For example, for an external audio output device that does not support the database of the display device 200, the output volume of the audio output device can also be collected through the built-in microphone of the display device 200 or an external audio input device to obtain Current volume value.

While acquiring the current volume value, the controller 250 may also acquire the volume adjustment amount input by the user. The volume adjustment amount can be obtained by parsing the control instruction input by the user, or can be obtained by parsing the interactive instruction input by the user after the focus window is detected. Obviously, for different volume adjustment situations, the volume adjustment amount can be presented as different values. That is, when the user controls to increase the output volume, the volume adjustment amount is a positive value; when the user controls to decrease the output volume, the volume adjustment amount is a negative value. For example, while obtaining the current volume value of 30, it can be obtained through the analysis of the control command input by the user that the user wants to increase the output volume by 20, then the volume adjustment amount is +20; similarly, it is obtained through the analysis of the control command input by the user that the user wants to lower the output volume. If the output volume is 20, the volume adjustment amount is -20.

The adjusted volume is then calculated according to the current playback volume and the volume adjustment amount, where the adjusted volume is equal to the sum of the current volume and the volume adjustment amount. For example, the adjusted volume=30+20=50, that is, the controller can set the audio output channel gain of the focus window to 50% according to the adjusted volume.

It can be seen that the display device 200 provided by the above embodiment can first detect the window where the current focus mark is located after acquiring the control instruction for adjusting the volume, acquire the current playback volume of the focus window, and then calculate the adjusted volume in combination with the volume adjustment amount input by the user , so as to set the gain of the audio output channel of the focus window according to the adjusted volume to complete the volume adjustment. When the display device 200 is in the multi-window mode, the playback volume in the window is individually adjusted according to the position of the focus mark, so as to accurately complete the volume adjustment.

In the above-mentioned embodiment, the controller 250 can adjust the volume individually for the focus window, and the determination of the focus window will directly affect the target of volume adjustment. Therefore, in order to determine the focus window and detect the current volume of the focus window, in some embodiments, the controller 250 may detect the position of the focus mark after acquiring the control instruction input by the user, and determine that the window where the focus mark is located is The current focus window. FIG. 10 is a schematic flowchart of querying currently playing audio parameters according to some embodiments of the present application. As shown in FIG. 10 , after the focus window is determined, the identification ID of the focus window can be obtained. The identification ID refers to a character string set according to the number of windows in the current user interface and used to distinguish each window.

The controller 250 then queries the current playing volume in the database according to the identification ID. Wherein, the database may record the current setting parameters of the display device 200 , including setting parameters such as playback volume, playback brightness, contrast, color mode, and image quality processing parameters. In the multi-window mode, each window may correspond to a set of setting parameters, and each set of setting parameters and the identification ID of the window may form a table entry and be stored in the database. To this end, after the window identification ID is obtained, matching can be performed in the database according to the identification ID as an index value to obtain an entry corresponding to the identification ID, and then the current playback volume can be queried.

For the displayed volume adjustment control, in some embodiments, after the current playback volume is queried, the volume adjustment control pattern may also be set according to the current playback volume. The volume adjustment control may be set to a specific shape according to the UI layout characteristics of the display device 200 . For example, the volume control can be a scroll bar or a slide ring with a float that indicates the current volume. Therefore, after obtaining the current volume, the position of the float pattern can be set according to the current volume, so as to obtain a volume adjustment control pattern that conforms to the specific value of the current volume.

After the volume adjustment control pattern is set, the controller 250 may further display the volume adjustment control after the adjustment pattern in the focus window, so as to indicate the window in which the user is currently performing volume adjustment through the displayed adjustment pattern. It can be seen that in this embodiment, different volume adjustment control patterns can be set according to different current volumes, and the volume adjustment control can be displayed in the focus window. While realizing the individual adjustment for the window, it is convenient for the user to distinguish the current volume adjustment window. Reduce the possibility of user misoperation.

In order to enable the controller 250 to calculate the adjusted volume according to the current volume and the volume adjustment amount input by the user, after the control displays the volume adjustment control in the focus window, the controller 250 can further obtain the volume adjustment amount input by the user. Wherein, when the control command input by the user includes the volume adjustment value, the controller 250 can directly obtain the volume adjustment value from the control command; when the control command input by the user does not include the volume adjustment value, the controller 250 needs to further receive The interactive action entered by the user, and the volume adjustment amount is determined according to the interactive action.

In some embodiments, the controller 250 may also first acquire the interactive action instruction input by the user on the volume adjustment control, and parse the interactive action instruction to obtain the action amplitude in the interactive action instruction, and then calculate the volume adjustment amount according to the action amplitude.

Obviously, according to different specific interaction modes, the interactive action instructions that the user can input are also different. Moreover, according to the specific shape of the volume adjustment control, the user can use different interactive actions to complete the input of the volume adjustment amount. For example, after the volume adjustment control is displayed in the first window, the float position in the volume adjustment control can be adjusted by a remote control or a touch operation, and the controller 250 calculates the volume adjustment amount according to the moving distance of the float position.

When the control instruction for adjusting the volume is input by the user pressing the volume key of the control device, the controller 250 can also obtain the volume adjustment amount by analyzing the pressing action. The pressing action may include a clicking action and/or a long pressing action. For example, when the control command for adjusting the volume is input by the user by pressing the volume "+" key on the remote control, the specific adjustment amount of the current volume adjustment can be parsed through the control command, that is, the volume can be detected by detecting the volume The number of button clicks or the long-press time to obtain the volume adjustment amount.

Therefore, in order to obtain the volume adjustment amount, in some embodiments, the controller 250 may parse the control instruction first, and calculate the volume adjustment amount according to the analysis result. If the pressing action is a click action, obtain the volume adjustment step corresponding to a single click action to obtain the volume adjustment amount. For example, the control system of the display device 200 sets the volume adjustment step of a single click to 5, and the user clicks the volume "+" key once, and the corresponding volume adjustment is +5%. When the user wants to increase the volume by 20%, the input of the volume adjustment amount can be completed by clicking the volume "+" key four times.

If the pressing action is a long pressing action, record the pressing duration of the long pressing action, and calculate the volume adjustment amount according to the pressing duration. For example, it is set in the control system of the display device 200 that every 0.1s increase in the pressing time corresponds to an increase or decrease of the volume by 1%. After the user long presses the volume "+" key for 1s, the volume adjustment amount can be increased by 10%. %.

In some embodiments, after the control instruction is input, the display device 200 may further set a parameter adjustment control pattern according to the audio parameter adjustment amount, wherein the parameter adjustment control includes a floating logo pattern and a stroke pattern. For example, when performing volume adjustment, the controller 250 may set the volume adjustment control pattern according to the control instruction after the user inputs the control instruction. If the pressing action corresponding to the control command is a click action, after each click action is input, the position of the floating sign pattern on the stroke pattern is modified according to the adjustment step; if the pressing action is a long pressing action, during the duration of the long pressing action, the The preset speed moves the position of the floating logo pattern on the stroke pattern.

The above-mentioned embodiment can adjust the volume of each window independently of each other for the display device 200 in the multi-window mode. Obviously, in order to reduce the amount of data processing and the invalid judgment, it is also possible to detect whether the display device 200 is in the multi-window mode. In some embodiments, the controller 250 can traverse the current number of playing processes in response to the control instruction. Usually, a window can correspond to one playback process, and a window without a playback process usually does not output an audio signal. Therefore, by traversing the current number of playback processes, it can be determined whether the current display device 200 needs to individually adjust the volume for each window.

According to the traversal result of the playback process, if the number of playback processes is greater than or equal to 2, it is determined that the display device 200 is currently in the multi-window mode, and the multi-windows can output audio signals correspondingly. The controller 250 may perform the step of detecting the current focus window when it is detected that the number of current playing processes is greater than or equal to 2.

If the number of playback processes is less than or equal to 1, it is determined that the display device 200 is not in the multi-window mode or that only one of the multiple windows corresponds to outputting an audio signal. At this time, it is not necessary to perform volume adjustment separately for each window, therefore, the current overall volume of the display device 200 may be acquired to calculate the adjusted volume according to the current overall volume and the volume adjustment amount. Wherein, to obtain the current overall volume of the display device 200, the current overall volume may also be queried in the database according to the method in the above-mentioned embodiment. After obtaining the current overall volume, the adjusted volume is also calculated according to the volume adjustment input by the user, and the gain of the audio output channel of the speaker or the external audio output device is modified according to the adjusted volume to realize the adjustment of the overall volume.

In addition, regardless of whether the overall volume adjustment is performed or the volume adjustment is performed separately for a single window, after setting the overall audio output path gain of the display device 200 or adjusting the audio output path gain of the focus window according to the adjusted volume, the controller 250 can also adjust the adjusted audio output path gain. The volume is recorded, that is, the adjusted volume is recorded to update the database according to the adjusted volume, so that the adjusted volume can be used as the current volume of the next volume adjustment process.

Based on the above-mentioned display device 200, some embodiments of the present application further provide a multi-window audio parameter adjustment method, which is applied to the above-mentioned display device 200. Then, the current focus window can be detected in response to the control instruction. And get the current playback volume of the focused window, and get the volume adjustment amount input by the user. Then calculate the adjusted volume according to the current playback volume and the volume adjustment amount, so as to set the audio output channel gain of the focus window according to the adjusted volume.

11 is a schematic diagram of an interface effect of performing audio parameter adjustment according to a focus window according to some embodiments of the present application. As shown in FIG. 11 , it can be known from the above technical solutions that the display device 200 and the multi-window audio parameter adjustment method provided by the above embodiments can After obtaining the control command to adjust the volume, first detect the window where the current focus mark is located, and obtain the current playback volume of the focus window, and then calculate the adjusted volume in combination with the volume adjustment amount input by the user, so as to set the audio of the focus window according to the adjusted volume. Output channel gain, complete volume adjustment. In the method, when the display device 200 is in the multi-window mode, the playback volume in the window can be adjusted independently according to the position of the focus mark, so as to accurately complete the volume adjustment.

The above-mentioned embodiment can perform separate volume adjustment for the focus window. In some embodiments of the present application, unified volume adjustment is also supported, that is, after the user inputs a control command to adjust the volume, the volume adjustment is performed for all windows synchronously, as shown in FIG. 12 . The embodiment of the present application is a schematic flowchart of a method for adjusting audio parameters synchronously in multiple windows according to some embodiments of the present application. As shown in FIG. 12 , the method for adjusting the volume synchronously specifically includes: obtaining a control command input by a user for adjusting the volume; responding to Based on the control instruction, obtain the current playback volume of each window, and obtain the volume adjustment amount input by the user; calculate the adjusted volume according to the current playback volume and the volume adjustment amount, so as to set the audio output channel gain of each window according to the adjusted volume .

The difference between this embodiment and the above-mentioned embodiments is that the controller 250 of the display device 200 can obtain the current playback volume of each window at the same time after obtaining the control instruction input by the user, and calculate the volume adjustment volume input by the user in combination with the volume adjustment volume input by the user. The adjusted volume of each window, so as to set the volume output channel gain of multiple windows at the same time according to the calculated adjusted volume, so that in one input process, the corresponding output volume of multiple windows can be adjusted at the same time, reducing the number of repeated operations by the user.

For example, after the user inputs a control command by clicking the volume "-" key on the control device 100, the controller 250 of the display device 200 may, in response to the control command, query the database for the current volume corresponding to the first window and the second window, 25 and 35 respectively. At the same time, the controller 250 can also analyze the volume adjustment amount from the control instruction, that is, the volume adjustment amount corresponding to clicking the volume "-" key is reduced by 10, so it can be calculated that the adjusted volumes of the first window and the second window are respectively 25 -10=15; 35-10=25. Finally, the corresponding volume output channel gains of the first window and the second window are adjusted to 15% and 25% respectively, so that the volume of the first window and the second window can be adjusted at the same time during one operation, so as to avoid the user's second operation.

Based on the above-mentioned display device 200, in some embodiments, a multi-window audio parameter adjustment method is further provided, which is applied to the above-mentioned display device 200. Then, in response to the control instruction, obtain the current playback volume of each window, and obtain the volume adjustment amount input by the user; then calculate the adjusted volume according to the current playback volume and the volume adjustment amount, so as to set each window according to the adjusted volume. gain of the audio output path.

FIG. 13 is a schematic diagram of the interface effect of the multi-window synchronously performing audio parameter adjustment according to some embodiments of the present application according to an embodiment of the present application. As shown in FIG. 13 , the above-mentioned multi-window audio parameter adjustment method may, after obtaining a control command for adjusting the volume, At the same time, the current playback volume of multiple windows is obtained, and the adjusted volume of each window is calculated in combination with the volume adjustment amount input by the user, so as to set the audio output channel gain of each window according to the adjusted volume to complete the volume adjustment. The method can adjust the playback volume of multiple windows at the same time through one volume adjustment interaction when the display device 200 is in the multi-window mode, so as to accurately complete the volume adjustment, reduce the number of user interactions, and improve user experience.

Similarly, the display device 200 may indicate the volume adjustment process through the volume adjustment control, and change the specific pattern shape along with the volume adjustment process, so as to obtain a better interactive experience. That is, in some embodiments, after acquiring the currently playing audio parameters of each window, the display device 200 can set a parameter adjustment control pattern according to the currently playing audio parameters; and display each parameter adjustment control in the global interface. For example, FIG. 14 is a schematic diagram showing the effect of displaying volume adjustment controls on a global interface according to some embodiments of the present application. As shown in FIG. 14 , after inputting a control command for adjusting audio parameters, the display device 200 may Obtain the current playback volume of the two windows at the same time, and set the pattern of the two volume adjustment controls according to the current playback volume. And after the user inputs the volume to be adjusted, the patterns of the two volume adjustment controls are modified simultaneously according to the input volume.

In addition to the foregoing volume adjustment methods, the display device 200 may also allow the user to select a window for which volume adjustment is performed through a window after the user inputs a control instruction. FIG. 15 is a schematic flowchart of a method for adjusting audio parameters according to a target window according to some embodiments of the present application. As shown in FIG. 15 , in some embodiments, the controller 250 of the display device 200 may obtain a user input for adjusting After the volume control command, in response to the control command, the display is controlled to display a window selection interface. After the user selects a window on the window selection interface, the current playback volume of the target window and the volume adjustment amount input by the user are obtained. Finally, the adjusted volume is calculated according to the current playback volume and the volume adjustment amount, so as to set the audio output channel gain of the target window according to the adjusted volume.

The difference between this embodiment and the foregoing embodiments is that after the user inputs a control instruction, the display device 200 can jump to a window selection interface. The window selection interface may include multiple window options. After the user selects any option, the target window can be determined, and the volume adjustment operation can be performed for the target window.

For example, after entering the dual-window mode, the user inputs a control instruction for adjusting the volume by calling the volume setting menu, and the display device 200 may display the window selection interface before displaying the volume adjustment control. Two window options, the first window and the second window, may be included in the window selection interface. The user can control the movement of the focus mark through the direction keys on the control device 100, select the first window and press the "OK" key, at this time it is determined that the first window is the window whose volume needs to be adjusted, that is, the target window. Then, according to the method in the above-mentioned embodiment, the current volume 25 of the first window and the volume adjustment amount input by the user+10 are obtained, and the adjusted volume is calculated as 25+10=35. Therefore, the gain of the audio output path of the first window can be set as: 35%.

Based on the above-mentioned display device 200, some embodiments of the present application further provide a multi-window audio parameter adjustment method. The multi-window audio parameter adjustment method package may first obtain a control instruction for adjusting the volume input by the user, and then respond to the The control instruction controls the display to display a window selection interface for the user to select a target window. Then obtain the current playback volume of the target window, and obtain the volume adjustment amount input by the user, so as to calculate the adjusted volume according to the current playback volume and the volume adjustment amount, so as to set the audio output channel gain of the target window according to the adjusted volume.

16 is a schematic diagram of an interface effect of performing audio parameter adjustment according to a target window according to some embodiments of the present application. As shown in FIG. 16 , it can be known from the above technical solutions that the display device 200 and the multi-window audio parameter adjustment method provided by the above embodiments can After acquiring the control instruction for adjusting the volume, a window selection interface is displayed in the user interface for the user to select a window for volume adjustment. Then, the adjusted volume is calculated according to the current playback volume of the selected window and the input volume adjustment amount, so as to set the audio output channel gain of each window according to the adjusted volume to complete the volume adjustment. The method can enable the user to select the window for adjusting the volume by himself through the window selection interface, so that the user can accurately adjust the volume for each window.

It should be noted that, for the multiple window audio parameter adjustment methods provided in the above embodiments, suitable volume adjustment methods can be selected according to different usage environments in practical applications. For example, when the multiple windows are all windows for playing media asset images, the method of performing volume adjustment is adopted for the focus window; when the multiple windows are all UI interface windows, the method of adjusting the overall volume of the display device 200 is adopted; When the multiple windows include both the playback window and the UI interface window, a volume adjustment method of selecting according to the window selection interface to adjust the target window is adopted.

In addition to the above-mentioned audio parameter adjustment methods, a specific volume adjustment method can also be determined for the audio signal output mode corresponding to each window in the multi-window mode. For example, when the audio signal of the first window is output through the speaker in the display device 200; and the second window is output through the external audio output device, after obtaining the volume adjustment control instruction input by the user, it can be determined that the audio signal is performed through the local speaker. The outputted first window is the adjusted window, and the current volume of the first window is obtained, and the adjusted volume is calculated in combination with the volume adjustment amount input by the user, so that the volume output channel gain corresponding to the first window is set according to the adjusted volume, and the adjustment is completed. The volume adjustment of the display device 200 is displayed.

With the use process of the user, the display device 200 can also be controlled to exit the multi-window mode. For example, after the user selects a window and presses the OK key, the display device 200 switches from the multi-window mode to the normal mode for display, that is, the displayed content becomes a single-screen display. In the process of exiting the multi-window mode, the display device 200 may also modify the output volume of the display device 200 to the output volume of the corresponding window according to the switched display form.

That is, the display device 200 can obtain the output device and the corresponding volume of the corresponding window from the multi-window mode database according to the user's selection, and refresh the output device and volume of the current window mode when switching to the normal mode. For example, if the user selects the first window with the focus, and presses the "OK" key, the display device 200 is controlled to switch to the normal mode, and the content in the first window is displayed in full screen. Then, obtain the output device Out1 of the first window and the corresponding volume Vol1 from the dual-window mode database, so as to set the output device Out1 and the volume to Vol1 in the single-window mode.

It should be noted that, the multiple windows in the multi-window mode described in the embodiments of the present application may be displayed on the same display 260 of the display device 200 , or may be displayed on different multiple displays 260 . For example, the display device 200 may be a dual-screen smart spot TV, that is, may include a first display and a second display. Wherein, the first display can display a specific user interface as a main screen, and the second display can display a specific interface as a secondary screen. After entering the multi-window mode, the display device 200 may control to display different windows on the first display and the second display respectively, so as to output specific display screen content at the same time. In the case where multiple windows are displayed on different displays, the display device 200 can also adjust the volume of the display device 200 by running the above-mentioned multi-window audio parameter adjustment method to improve user experience.

In addition, the above-mentioned embodiment uses volume adjustment as an example to describe the multi-window audio parameter adjustment process. It should be understood that the adjusted audio parameters are not limited to the above-mentioned volume, but may also be other audio parameters. When the display device 200 is performing the adjustment process of other related audio parameters, the general process of the parameter adjustment method is the same, and the difference is only that for different audio parameters, the parameters executed by the user may be different, or the specific parameter setting methods may be different.

FIG. 17 is a schematic diagram of a sound effect adjustment window according to some embodiments of the present application. As shown in FIG. 17 , taking the adjustment method in which the audio parameter is the sound effect mode as an example, after the user inputs a control command for adjusting the sound effect mode, the display device 200 also In response to the control instruction, the current focus window can be detected, and then the current sound effect mode corresponding to the focus window can be obtained, for example, the current sound effect mode is the "high-resolution human voice" mode. Then obtain the sound effect mode input by the user, such as the "3D surround" mode. Since the current sound effect mode is different from the sound effect mode set by the user, the adjusted sound effect mode, that is, the "3D surround" mode, can be calculated according to the current sound effect mode and the sound effect mode set by the user, and then according to the corresponding audio analysis of the "3D surround" mode, Processing method, set the audio output channel to obtain the audio playback effect of "3D Surround" mode.

For the same reason, you can also refer to the corresponding embodiment of the above-mentioned unified volume adjustment method, by obtaining the current sound effect mode of each window and the sound effect mode input by the user, and calculating the adjusted sound effect mode, so as to set each sound effect mode according to the adjusted sound effect mode. sound mode for each window. And, referring to the above-mentioned embodiment of adjusting the volume according to the target window, after inputting the control instruction for adjusting the sound effect mode, a window selection interface is displayed to determine the target window. Then, the adjusted sound effect mode is calculated by acquiring the current sound effect mode of the target window and the new sound effect mode input by the user, thereby adjusting the sound effect mode of the target window to the specified sound effect mode.

In some embodiments, the user may send an audio and video synchronization instruction to the display device, where the audio and video synchronization instruction is a control instruction for enabling audio and video synchronization. When detecting the audio and video synchronization instruction input by the user, the display device may detect the audio device currently corresponding to the focus window in response to the instruction. When the focus window uses different types of audio devices, that is, the display device is in different audio device modes, the controller can set different audio and video synchronization values. When the audio device corresponding to the focus window is the first type of audio device, the controller can obtain the first audio and video synchronization value; when the audio device corresponding to the focus window is the second type of audio device, the controller can obtain the second audio and video synchronization value value. The audio playback parameters corresponding to the audio signal are set according to the current audio-visual synchronization value of the focus window, so that the picture information presented by the focus window is synchronized with the audio information played by the audio device corresponding to the focus window. Considering that the sound effect processing flow of the display device for the first type of audio device and the second type of audio device may be different, the delay parameters of the sound effect processing in the two audio device modes may also be different. Therefore, in some embodiments of the present application, the Different audio and video synchronization values are set for the two audio device modes, so that the audio and video synchronization effect of the display device is better, so as to enhance the user experience.

In some embodiments, when it is detected that the audio device corresponding to the focus window is the first type of audio device, the controller may acquire the first audio-visual synchronization value. At this time, the controller first obtains the image quality processing delay parameters and the sound effect processing delay parameters of the focus window. Among them, the picture quality processing delay parameter refers to the time consumed by the display device to perform picture quality processing (pq) on the picture content to be displayed in the window, and the sound effect processing delay parameter refers to the display device transmitting the audio content to be played in the window to the first In the process of a class of audio equipment, the time spent in sound effect processing (aq). It should be noted that when the display device uses the first type of audio device to play audio, the display device needs to perform sound effect processing on the audio signal and send it to the first type of audio device. Generally speaking, the process of image quality processing by the display device is more complicated than that of sound effect processing, so the delay parameter of image quality processing will be greater than the delay parameter of sound effect processing.

When the controller obtains the image quality processing delay parameter of the focus window, it can query the resolution information and frame rate information corresponding to the focus window in the database. Wherein, the database may record the current setting parameters of the display device, including playback volume, playback brightness, contrast, color mode, image quality processing parameters, resolution, frame rate and other setting parameters. According to the resolution information, the controller can query the database for all processes of image quality processing under the file resolution and the number of frames to be processed in each process. The controller can calculate the time T required to process one frame of data according to the frame rate information. For example, when the frame rate is 60HZ, the processing time of one frame of data is 1/60s. The processing time of each processing process can be calculated according to the time T required to process one frame of data and the number of frames to be processed by each processing process, so as to obtain the total image quality processing delay time of all processes, that is, the image quality processing delay parameter.

When the controller obtains the sound effect processing delay parameter of the focus window, it can query the database for the sound effect processing type corresponding to the focus window, such as Dolby sound effect or DTS sound effect. It should be noted that the sound effect processing delay parameters corresponding to each sound effect processing type are fixed. Therefore, the sound effect processing delay parameter of the focus window can be directly obtained according to the current sound effect processing type.

The image quality processing delay parameter is greater than the sound effect processing delay parameter. When calculating the audio and video synchronization value, the audio and video synchronization value can be obtained by subtracting the sound effect processing delay parameter from the image quality processing delay parameter. For example, the image quality processing delay parameter is 300ms, and the sound effect processing delay parameter is 70ms. Then the audio and video synchronization value at this time is 230ms.

In some embodiments, when it is detected that the audio device corresponding to the focus window is the second type of audio device, the controller may acquire the second audio-visual synchronization value. At this time, the controller can set the audio and video synchronization value to the preset value. It should be noted that when the display device uses the second type of audio device, such as a Bluetooth speaker, the second type of audio device can choose to perform sound effect processing by itself. The display device needs to send the audio signal to the second-class audio device, and then the second-class audio device will play the audio. Compared with the first type of audio equipment, the sound delay of these second type of audio equipment will be much larger than the image quality processing delay parameter. Therefore, it is necessary to advance the audio playback time as much as possible, and set the audio and video synchronization value to a small value. .

In some embodiments, when it is detected that the audio device corresponding to the focused window is a second type of audio device, such as a Bluetooth speaker. The controller can close the sound effect processing process corresponding to the focus window, and set the audio and video synchronization value to 0.

In some embodiments, the controller may directly set audio playback parameters corresponding to the audio signal according to the audio-visual synchronization value. For example, if the audio and video synchronization value is Nms, it means that the display device will play the audio faster than the display screen by Nms. At this time, an audio playback parameter can be set so that the audio signal can be played after a delay of Nms.

In some embodiments, considering that users have different perception sensitivities to audio and pictures, the user may perform adjustment of the audio-visual synchronization degree, that is, the user may set the audio-visual synchronization adjustment parameters by himself. The display device can display the current audio and video synchronization adjustment situation of the focus window to the user, wherein the audio and video synchronization adjustment situation includes the current audio and video synchronization adjustment parameters of the focus window and the adjustment range of the audio and video synchronization adjustment parameters. For example, FIG. 18 is a schematic diagram showing the adjustment of audio and video synchronization in the focus window of some embodiments of the present application. As shown in FIG. 18 , the adjustment range can be -10-10, and the default audio and video synchronization adjustment parameter of the display device is 0.

After acquiring the audio and video synchronization adjustment parameters, the controller can calculate the audio and video synchronization adjustment time. A step value may be preset, and the step value represents: the audio and video synchronization adjustment time when the audio and video synchronization adjustment parameter is 1. Multiply the currently set audio and video synchronization adjustment parameters and the step value to obtain the audio and video synchronization adjustment time.

In some embodiments, after acquiring the audio-visual synchronization value of the focus window and the current audio-visual synchronization adjustment time, the controller may further set audio playback parameters corresponding to the audio signal. For example, the audio and video synchronization value is Nms, and the audio and video synchronization adjustment time is Mms. It means that the audio played by the display device is Nms earlier than the display screen, and the user expects the audio to be Mms earlier than the display screen. At this time, an audio playback parameter can be set so that the audio signal can be played after a delay of (N+M)ms.

In some embodiments, the display device may display the audio device currently corresponding to the focus window and the switchable audio device mode to the user, so that the user can switch the audio device mode of the focus window. When detecting a switch instruction instructing to switch the audio device corresponding to the focus window, in response to the switch instruction input by the user, the controller switches the audio device corresponding to the focus window. At the same time, the controller can adjust the audio and video synchronization value, and reset the audio playback parameters according to the adjusted audio and video synchronization value.

In some embodiments, when the audio device corresponding to the focus window is switched from the first type of audio device to the second type of audio device, the controller can close the sound effect processing flow corresponding to the focus window, and set the audio and video synchronization value to 0. Since the second type of audio device is used in the focus window, the sound delay will be large, so you need to set the audio and video synchronization value to 0 to ensure the audio and video synchronization.

In some embodiments, when the audio device corresponding to the focus window is switched from the second type of audio device to the first type of audio device, the controller may first determine the usage state of the first type of audio device.

When the first type of audio device is in an unused state, the controller switches the audio device corresponding to the focus window to the first type of audio device, and can reset the audio and video synchronization value. The controller obtains the resolution information, frame rate information and sound effect processing type corresponding to the focus window, obtains image quality processing delay parameters according to the resolution information and frame rate information, and obtains sound effect processing delay parameters according to the sound effect processing type. By subtracting the sound effect processing delay parameter from the image quality processing delay parameter, the current audio and video synchronization value of the focus window can be obtained.

When the first type of audio device is in use, the controller may first determine the switching window currently corresponding to the first type of audio device, and at this time, the switching window and the audio device of the focus window need to be exchanged. For the switch window, since the audio device after the switch window is switched is switched to the second type of audio device, it is necessary to close the sound effect processing flow corresponding to the switch window, and set the audio and video synchronization value of the switch window to 0. For the focus window, the audio device of the focus window is the first type of audio device, so the audio and video synchronization value of the focus window needs to be recalculated. The display device may also allow the user to select a window for performing audio and video synchronization through a window after the user inputs the audio and video synchronization instruction, and perform audio and video synchronization processing on the target window selected by the user.

In some embodiments, the controller controls the display to display a window selection interface in response to the audio-visual synchronization instruction. After the user selects a window on the window selection interface, the audio device corresponding to the target window selected by the user is detected, and the audio and video synchronization value is set. The audio playback parameters corresponding to the audio signal are set according to the audio-visual synchronization value, so that the picture information presented by the target window is synchronized with the audio information played by the audio device corresponding to the target window.

Some embodiments of the present application provide an audio output method, which is applied to a display device, and the method includes:

In response to a user-input instruction to display a display instruction of multiple windows, the display is controlled to display multiple windows; the focus window where the user-controlled focus is located is detected; in response to the audio-visual synchronization instruction input by the user, the audio device corresponding to the focus window is detected When the audio equipment corresponding to the focus window is the first type of audio equipment, obtain the first audio and picture synchronization value; When the audio equipment corresponding to the focus window is the second type of audio equipment, obtain the second audio and picture synchronization value ; Set the audio playback parameters corresponding to the audio signal according to the audio-visual synchronization value, so that the picture information presented by the focus window is synchronized with the audio information played by the audio device corresponding to the focus window.

In some embodiments, performing the step of acquiring the first audio and video synchronization value further includes: acquiring an image quality processing delay parameter and a sound effect processing delay parameter of the focus window, and according to the image quality processing delay parameter and all The first audio-visual synchronization value is calculated using the sound effect processing delay parameter; in the step of acquiring the second audio-visual synchronization value, the controller is configured to: set the second audio-visual synchronization value to a preset value.

In some embodiments, performing the step of setting audio playback parameters corresponding to the audio signal according to the audio and video synchronization value, further includes: acquiring audio and video synchronization adjustment parameters set by a user; adjusting the audio and video synchronization according to the audio and video synchronization value The parameter calculates the audio-visual synchronization adjustment time, so as to calculate the audio playback parameter corresponding to the audio signal according to the audio-visual synchronization adjustment time and the audio-visual synchronization value.

In some embodiments, the method further includes: switching the audio device corresponding to the focus window in response to an instruction to switch the audio device corresponding to the focus window in response to an instruction input by the user, and adjusting the audio and video synchronization value to Reset the audio playback parameters according to the adjusted audio and video synchronization value.

In some embodiments, when it is detected that the audio device corresponding to the focus window is switched from the first type of audio device to the second type of audio device, the step of adjusting the audio and video synchronization value further includes: closing the focus The sound effect processing flow corresponding to the window, and the audio and video synchronization value is set to 0; when it is detected that the audio device corresponding to the focus window is switched from the second type of audio device to the first type of audio device, the adjustment of the audio and video synchronization value is performed. in the step of: obtaining the resolution information, frame rate information and sound effect processing type corresponding to the focus window; obtaining image quality processing delay parameters according to the resolution information and the frame rate information; according to the sound effect processing type Obtaining a sound effect processing delay parameter; subtracting the sound effect processing delay parameter from the image quality processing delay parameter to obtain the current sound and picture synchronization value of the focus window.

In some embodiments, the method further includes: when it is detected that the switching instruction instructs to switch the audio device corresponding to the focus window from the second type of audio device to the first type of audio device, judging the use state; when the first type of audio device is not in use, switch the audio device corresponding to the focus window to the first type of audio device; when the first type of audio device is in use, determine the first type of audio device The switching window currently corresponding to the class audio device; switching the audio device corresponding to the switching window to the second class audio device, and closing the sound effect processing flow corresponding to the switching window, and setting the audio and video synchronization value of the switching window to 0.

In some embodiments, the method includes: in response to a query instruction input by the user to inquire about the audio and video synchronization adjustment parameters corresponding to the focus window, controlling the display to display the audio and video synchronization adjustment status, so that the user can synchronize the audio and video according to the audio and video synchronization The adjustment situation resets the audio and video synchronization adjustment parameters; the audio and video synchronization adjustment situation includes the current audio and video synchronization adjustment parameters of the focus window and the adjustment range of the audio and video synchronization adjustment parameters.

Some embodiments of the present application provide an audio output method, including: in response to a user-input instruction to display a display instruction of multiple windows, controlling a display to display multiple windows; detecting a focus window on which a user-controlled focus is located; The audio and video synchronization query instruction controls the display to display the audio and video synchronization opening state; so that the user can choose to open the audio and video synchronization of the focus window according to the audio and video synchronization opening state, or close the audio and video synchronization of the focus window; in response to The audio and video synchronization instruction input by the user sets the audio playback parameters corresponding to the audio signal, so that the picture information presented by the focus window is synchronized with the audio information played by the audio device corresponding to the focus window.

Some embodiments of the present application provide an audio output method, including: in response to a user-input instruction to display a display instruction of multiple windows, controlling a display to display multiple windows; in response to a user-input audio-visual synchronization instruction, controlling the display to display A window selection interface, so that the user selects a target window in the window selection interface, and detects the audio device corresponding to the target window; when the audio device corresponding to the focus window is the first type of audio device, obtain the first audio device When the audio device corresponding to the focus window is the second type of audio device, obtain the second audio and video synchronization value; set the audio playback parameter corresponding to the audio signal according to the audio and video synchronization value, so that the target The picture information presented by the window is synchronized with the audio information played by the audio device corresponding to the target window.

The above-mentioned audio output method can set the audio playback parameters corresponding to the audio signal according to the audio-visual synchronization value, so that the picture information presented by the focus window is synchronized with the audio information played by the audio device corresponding to the focus window, so that the audio information of the window selected by the user can be accurately performed. The synchronous processing of pictures improves the user experience.

When using the intelligent voice system, the interaction process between the display device 200 and the user also relies on real-time audio interaction, that is, the user inputs control commands by inputting voice commands, and the intelligent voice system reports the interaction results through voice broadcast. In this process, the display device 200 can process two types of voice information, ie, the voice input by the user and the voice fed back to the user. However, the voice input process is easily affected by environmental factors. In the process of broadcasting the feedback voice, the media sound played by the display device 200 will also affect the output effect, so that the user cannot hear the broadcast voice content clearly. In some embodiments, the display device 200 may perform muting processing on the audio played by the display device 200 when the user inputs a voice interaction instruction, so as to reduce the influence of the played audio on the input voice. At the same time, when the feedback voice is broadcast, the sound of the media material during the broadcast period can be muted or reduced, so that the user can hear the broadcast voice clearly.

However, when the display device 200 is in the multi-window mode, the display device 200 can output audio by means of the device, and different output modes will have different effects on the input and broadcast of the voice. For example, when the display device 200 is in a multi-window mode, the first window plays audio through a local speaker; the second window plays audio through an external audio device. The distance between the speaker of the display device itself and the user is relatively short, and the output sound is likely to affect the voice interaction process, and the above-mentioned muting and/or sound reduction processing needs to be performed during the voice interaction process. The audio played by the external audio device usually does not affect the interaction process, so the original audio output mode can be maintained to reduce the impact of the voice interaction process on normal movie viewing.

In order to improve the accuracy of the voice interaction process and reduce the impact on the user's normal viewing process, some embodiments of the present application further provide a display device 200. Including: display 260 , audio output interface 270 and controller 250 . The display 260 is used for displaying a user interface, and the displayed user interface may include multiple windows when the display device 200 is in a multi-window mode. The audio output interface 270 can be connected to a local speaker and/or an external audio device for outputting audio to form a volume output path. The controller 250 is configured to run a multi-window voice announcement method for precise voice control in the multi-window mode. FIG. 19 is a schematic flowchart of a multi-window voice broadcast method according to some embodiments of the present application. As shown in FIG. 19 , the multi-window voice broadcast method specifically includes the following contents:

Get the control command input by the user to wake up the voice broadcast. The controller 250 of the display device 200 can receive various control instructions input by the user during the use of the user, and different control instructions have different functions correspondingly. Among them, some control instructions can be used to wake up the voice broadcast. After receiving the control instruction, the controller 250 of the display device 200 may detect the current response window in response to the control instruction. The response window is a window that outputs audio through the local interface among the multiple windows of the user interface. That is, after receiving the control instruction input by the user, the reality device 200 in the multi-window mode can detect the window that uses the local interface for audio output among the multiple windows.

For example, FIG. 20 is a schematic flowchart of detecting a response window according to some embodiments of the present application. As shown in FIG. 20 , after the user inputs the wake-up word "Hi! xx", the controller 250 may first detect the content contained in the current user interface The windows are the first window and the second window respectively, and then the audio output path of each window is detected. When it is detected that the audio output device of the first window is a local speaker and the audio output device of the second window is an external audio device, it may be determined that the first window is a response window.

After detecting and determining the response window, the controller 250 may perform mute processing on the signal source of the response window, so that the user can input a specific voice command. For example, after determining that the first window is the response window, the display device 200 may mute the sound of the media assets played in the first window, so that the local speaker of the display device 200 no longer outputs sound, and it is convenient for the user to input a specific voice instruction.

Among them, the mute processing performed on the signal source of the response window can be implemented in a number of different ways. For example, after the response window is determined, the local speaker may be temporarily turned off, so that the local speaker does not output any sound during the user inputting a specific voice command. It is also possible to directly control the transmission process of the audio data stream, so that the video part in the media asset data to be played is played normally, and the audio part is not parsed or the parsed audio data stream is not sent to the local speaker. The mute effect can also be achieved by setting the gain of the volume output channel of the response window so that the output gain corresponding to the response window is 0%.

During the mute period, the display device 200 may also receive a voice control instruction input by the user, and generate a feedback voice for the voice control instruction input by the user. For example, if the user inputs the voice "adjust the brightness to 50%", the recognition result output by the voice control system is "control/brightness/50%", so a control command for adjusting the brightness can be generated, and the adjustment target value is 50% .

FIG. 21 is a schematic flowchart of obtaining feedback speech according to some embodiments of the present application. After a response command is generated, as shown in FIG. 21 , the feedback speech can be matched in the corresponding speech broadcast template. The feedback voice content corresponding to different response commands may be stored in the voice broadcast template. Feedback speech can be composed of fixed sentence patterns and parametric words. For example, for the control command to adjust the brightness, the fixed sentence pattern in the feedback voice is "adjusted the brightness to", and the parameter word is the adjustment target value, that is, "50%", so the content of the feedback voice can be matched as "the brightness has been adjusted to" Adjust to 50%". Then, the feedback voice is broadcast through the audio output path of the response window, that is, the feedback voice is broadcast through the local speaker of the display device 200 .

In the above-mentioned embodiment, it is provided that the display device 200 can perform mute processing for the response window that uses the local interface for audio output in the multi-window, and broadcast the feedback voice through the local speaker, so that the input and broadcast of the voice are in the most easily recognizable state, which is convenient for the user. The intelligent voice system realizes accurate control and can guide the user to input the correct voice interaction command.

In some embodiments, FIG. 22 is a schematic flowchart of detecting a response window according to a database according to some embodiments of the present application. As shown in FIG. 22 , in order to detect the current response window, the controller 250 of the display device 200 may receive a user input After the control instruction, call the database. Wherein, the audio output path of each window in the user interface is recorded in the database. After calling the database, the controller 250 may traverse the audio output paths of each window in the database to obtain a response window that outputs audio through the local interface. For example, when it is determined that the corresponding audio output path of the first window is the local interface by traversing the audio data path items of the entries in the database, the first window is determined to be the response window.

In some embodiments, the display device 200 may further guide the user to input and allow the user to verify the input voice information by displaying prompt content during the user's voice interaction. For example, after the user inputs the voice command, the recognition result for the voice command may be displayed through the voice interaction control, that is, the text content corresponding to the voice command is displayed.

In the multi-window mode, the display device 200 may display the voice interaction control in the global interface, or may display the voice interaction control in a specified window. In the case of displaying the voice interaction control in a specified window, the specified window may be a certain response window, that is, during the muting process on the signal source of the response window, the voice interaction control is displayed in the response window. For example, after it is determined that the first window is the response window, the voice interaction control may be displayed in the first window. The specified window can also be another window, such as the focus window.

Obviously, since a UI layout interface can be displayed in each window, FIG. 23 is a schematic flowchart of displaying a voice interaction control in a focus window according to some embodiments of the present application. As shown in FIG. 23 , in some embodiments, the focus window The flag is not on the window but on any option within the window. Therefore, when detecting the focus window, it is also possible to further detect whether there is a focus mark within the scope of the window. When the focus mark is within the range of the first window, the focus window is determined to be the first window; when the focus mark is within the range of the second window, the focus window is determined to be the second window.

After the focused window is detected, the controller 250 may display a voice interaction control in the focused window during the mute processing. For example, if the focus mark is on the first window, even if the window currently using the local interface to output audio is the second window, the voice interaction control is still displayed at the lower left of the first window, that is, the voice assistant prompt information is displayed so that the user can perform interactive actions. For the displayed voice interaction control, the presented specific display content may include text content and graphic shapes. For example, a voice interaction control may include a rounded rectangular outer frame and text within the outer frame. In different interaction stages, the shape and text content of the outer frame are different. For example, the controller 250 may add a first prompt text to the voice interaction control before receiving the voice interaction instruction input by the user, for prompting the user to input a voice.

In order to achieve the above-mentioned display effect, the display device 200 may receive the voice interaction instruction input by the user after displaying the voice interaction control, and convert the voice in the voice interaction instruction into text information. Converting speech into text information can be done through the built-in recognition engine in the intelligent voice assistant, that is, the recognition engine can analyze the speech waveform and match it in the thesaurus to determine the text contained in the speech, and then combine the Chinese syntactic rules to identify the recognized text. information is corrected to obtain textual information that matches the input speech. The recognition engine can use open source engines, self-developed engines, and third-party engines.

After obtaining the converted text information, the display device 200 may replace the text content in the voice interaction control with the text information. When the corresponding sentence of the speech input by the user is long, the recognition engine can also split the speech according to the pause when the user input, and convert the text information of each segment in turn, and display the text segments obtained by the conversion in real time in real time. voice interaction command. When displaying the text content corresponding to the voice in real time, the display device 200 can also adjust the graphic shape of the voice interaction control in real time according to the displayed text content. That is, the controller 250 can detect the number of characters included in the text information, and then modify the graphic shape of the voice interactive control according to the number of characters. For example, when the voice content input by the user is less, it can be displayed by a graphic shape with a shorter length, and when the voice content input by the user is more, it can be displayed by a graphic shape with a longer length. The shape-changeable voice interactive control not only has a certain aesthetics to adapt to different input contents, but also facilitates the user to identify the position of the voice interactive control through the change of the graphic shape of the voice interactive control, and guides the user to complete the verification of the recognition result. .

In some embodiments, in order to prompt the user to complete the voice interaction, the display device 200 may further detect in real time whether a voice interaction instruction input by the user is received during the process of displaying the first prompt text through the voice interaction control. If the voice interaction instruction input by the user is not received within the preset time, the first prompt text in the voice interaction control is replaced with the second prompt text.

The display process of the above-mentioned voice interaction control can also be applied to the process of broadcasting the feedback voice. That is, in some embodiments, after the display device 200 obtains the feedback voice content, it can obtain the text corresponding to the feedback voice, and then display the text corresponding to the feedback voice through the voice interaction control. Similarly, according to the number of characters of the text corresponding to the feedback voice, the shape of the voice interaction control can also be modified in real time, that is, when the feedback voice content is long, the text display is completed through a larger voice interaction control.

During the broadcast of the feedback voice, the display device 200 may always display the voice interaction control, that is, the controller 250 may control the display 260 to display the voice interaction control, until the voice interaction control is no longer displayed after the feedback voice broadcast ends.

During the process of broadcasting the feedback voice, the display device 200 may perform muting or sound reduction processing on the audio output channel of the response window, so that the user can hear the voice broadcasting content clearly. In order to reduce the impact of the voice broadcast on the normal viewing process, in some embodiments, sound reduction processing may be performed on the corresponding audio output channel, that is, when the feedback voice is broadcast, the media playback volume in the corresponding output channel is reduced. In this regard, the controller 250 of the display device 200 may first acquire the first signal source data of the response window in the step of broadcasting the feedback voice through the audio output path of the response window. The sound mixing process is then performed on the first signal source data and the feedback voice data to generate first sound mixing result data. Finally, the first mixing result data is played through the audio output path of the response window.

For example, the controller 250 can obtain the signal source and output device corresponding to the first window from the dual-window mode database. If the output device of the first window is a local speaker, the current signal source is muted. Mute. The audio fed back by the intelligent voice system is then mixed with the current signal source data, and the mixed data is broadcast through the output device corresponding to the first window, that is, through the local speaker of the display device 200 . If the output device of the first window is a Bluetooth speaker, obtain the audio output device of the second window from the second window database; if the audio output device of the second window is a local speaker, obtain the signal source of the second window from the second window database , and mute the second window signal source output. After the radio analysis of the intelligent voice system is finished, unmute. Then, the audio fed back by the intelligent voice system is mixed with the signal source data of the second window, and the mixed data is broadcast through the output device of the second window.

By broadcasting the mixing result of the feedback voice and the media sound through the audio output path of the response window, it is not only possible to keep using the local speaker to broadcast the feedback voice, which is more convenient for users to listen to, but also can obtain a more coordinated broadcast effect and reduce the impact of the voice broadcast process on the original sound. The media sound playback process has an impact and improves the user experience.

In some embodiments, the feedback voice may also be output synchronously through multiple audio devices. 24 is a schematic flowchart of synchronously broadcasting feedback speech according to some embodiments of the present application. As shown in FIG. 24 , in the step of broadcasting feedback speech through the audio output path of the response window, the controller 250 of the display device 200 can detect the synchronization window. The synchronization window is a window that outputs audio through an external audio device among the multiple windows of the user interface. When each window corresponds to a separate audio output device, the synchronization window is another window other than the response window. Then obtain the second signal source data of the synchronization window, and perform sound mixing on the second signal source data and the feedback voice data to generate the second sound mixing result data, and finally play the second sound mixing result data through the audio output path of the response window. .

For example, if the first window output device is a local speaker and the second window output device is an external audio device, it can be determined that the first window is a response window and the second window is a synchronization window. Therefore, after the audio output path corresponding to the first window is muted, radioed, and unmuted, the feedback voice is obtained through the intelligent voice system. The feedback voice is then mixed with the signal source of the first window and/or the second window to obtain a mixed sound and complete the broadcast according to the corresponding audio output path respectively.

By using different audio output paths to complete the broadcast of the feedback voice, users who pay attention to different windows can be connected to the results of voice interaction without affecting the viewing process of different users, and the user experience can be improved.

In some embodiments, the display device 200 may further determine whether it is currently in a multi-window mode before performing the voice interaction. For example, the controller 250 may traverse the current number of playing processes in response to the control instruction. Usually, a window can correspond to one playback process, and a window without a playback process usually does not output an audio signal. Therefore, by traversing the current number of playback processes, it can be determined whether the current display device 200 needs to individually adjust the volume for each window.

According to the traversal result of the playback process, if the number of playback processes is greater than or equal to 2, it is determined that the display device 200 is currently in the multi-window mode, and the multi-windows are capable of correspondingly outputting audio signals, so the step of detecting the current response window can be performed. If the number of playback processes is less than or equal to 1, it is determined that the display device 200 is not in the multi-window mode or that only one of the multiple windows corresponds to outputting an audio signal. At this time, the system sound of the current display device 200 is muted, and the feedback voice is broadcast through the local speaker, that is, the interaction is completed according to the conventional voice interaction mode.

Some embodiments of the present application provide a multi-window voice broadcast method. The method first obtains a control command input by a user to wake up the voice broadcast; and then, in response to the control command, detects the current response window and performs mute on the signal source of the response window. Processing; finally, the feedback voice is broadcast through the audio output path of the response window. The step of detecting the current response window includes: calling a database, where the audio output path of each window in the user interface is recorded; traversing the audio output path of each window in the database to obtain audio output through a local interface response window.

In some embodiments, the step of performing muting processing on the signal source of the response window includes: detecting a position of a focus mark in the user interface, so as to identify a focus window according to the position of the focus mark, the focus window A window with a focus mark among the plurality of windows of the user interface; during the muting process on the signal source of the response window, a voice interaction control is displayed in the focus window.

In some embodiments, the step of performing muting processing on the signal source of the response window includes: displaying a voice interaction control in the response window during the muting processing on the signal source of the response window.

In some embodiments, the displayed content of the voice interaction control includes text content and/or graphic shapes, and after displaying the voice interaction control, it includes: receiving a voice interaction instruction input by a user; converting the voice in the voice interaction instruction into text information; use the text information to replace the text content in the voice interaction control; detect the number of characters contained in the text information; modify the graphic shape of the voice interaction control according to the number of characters.

In some embodiments, the method includes: before receiving the voice interaction command input by the user, adding a first prompt text to the voice interaction control; detecting whether the voice interaction command input by the user is received; if the voice interaction command input by the user is received at a preset time If the voice interaction instruction input by the user is not received, the first prompt text in the voice interaction control is replaced with the second prompt text.

In some embodiments, the step of broadcasting the feedback voice through the audio output path of the response window includes: acquiring the text corresponding to the feedback voice; adding the text corresponding to the feedback voice in the voice interaction control; During the speech, the display is controlled to display the voice interaction control; after the broadcast of the feedback speech is finished, the display is controlled to no longer display the voice interaction control.

In some embodiments, the step of broadcasting the feedback voice through the audio output path of the response window includes: acquiring first signal source data of the response window; performing mixing between the first signal source data and the feedback voice data audio processing to generate first sound mixing result data; and playing the first sound mixing result data through the audio output path of the response window.

In some embodiments, the step of broadcasting the feedback voice through the audio output path of the response window includes: detecting a synchronization window, where the synchronization window is a window for outputting audio through an external audio device among the multiple windows of the user interface; Acquiring the second signal source data of the synchronization window; performing sound mixing on the second signal source data and the feedback voice data to generate second sound mixing result data;

It can be seen that the second audio mixing result data is played through the audio output path of the response window, and the multi-window voice broadcasting method provided by the embodiment can detect the response of outputting audio through the local interface after receiving the control instruction input by the user window and mute its source so that the user can enter voice interaction commands during mute. After the display device forms the feedback voice for the voice interaction instruction, it broadcasts the feedback voice through the audio output path of the response window. The method can detect the response window for outputting audio through the local interface after the display device enters the multi-window mode, thereby muting the local speaker, alleviating the influence of the played sound on the voice control, and improving the accuracy of the intelligent voice control.

For the convenience of explanation, the above description has been made in conjunction with specific embodiments. However, the above discussion in some embodiments is not intended to be exhaustive or to limit implementations to the specific forms disclosed above. Numerous modifications and variations are possible in light of the above teachings. The above embodiments have been chosen and described to better explain the principles and practical applications, so as to enable those skilled in the art to better utilize the embodiments and various modified embodiments suitable for specific use considerations.

Claims

A display device comprising:

a display for displaying a user interface, the user interface including a plurality of windows;

Audio output interface, the audio output interface is multiple, used to connect multiple types of audio equipment;

Controller, configured as:

Controlling the display to display the plurality of windows in response to a display instruction inputted by the user indicating to display the plurality of windows;

Detects the focused window that the user-controlled focus is on;

In response to the switching instruction of switching the audio device corresponding to the focus window in response to the instruction input by the user, the audio device corresponding to the focus window is switched, and the audio devices corresponding to the remaining windows are adjusted, so that each window uses a different audio device .
The display device according to claim 1, wherein the switching instruction is: switching the audio device corresponding to the focus window from the second type of audio device to the first type of audio device;

The controller is configured to:

In the step of switching the audio device corresponding to the focus window and adjusting the audio devices corresponding to other windows, the first audio channel of the focus window is set to a mute state, and the use of the first type of audio device is judged state; the first audio channel is used to output the audio of the focus window to the audio device corresponding to the focus window;

When the first type of audio equipment is not in use, connecting the first audio channel and the first type of audio equipment;

When the first type of audio device is in use, determine the first switching window currently corresponding to the first type of audio device; set the second audio channel of the first switching window to a mute state, and the second The audio channel is used to output the audio of the first switching window to the audio device corresponding to the first switching window; connect the second audio channel and the second type of audio device corresponding to the focus window, and, connecting the first audio channel with the first type of audio equipment; releasing the mute state of the second audio channel;

Unmute the first audio channel.
The display device according to claim 1, wherein the switching instruction is: switching the audio device corresponding to the focus window from the first type of audio device to the second type of audio device;

The controller is configured to:

In the step of switching the audio device corresponding to the focus window and adjusting the audio devices corresponding to other windows, the first audio channel of the focus window is set to a mute state, and it is determined whether the audio output interface is connected There is an unused second-class audio device; the first audio channel is used to output the audio of the focus window to the audio device corresponding to the focus window;

When the audio output interface is connected with an unused second-class audio device, connecting the first audio channel and the unused second-class audio device;

When the audio output interface is not connected to an unused second-type audio device, randomly select a first device in the second-type audio device, and determine the second switching window currently corresponding to the first device; The third audio channel of the second switching window is set to a mute state, and the third audio channel is used to output the audio of the second switching window to the audio device corresponding to the second switching window; Connect with the first type of audio equipment, and connect the first audio channel with the first equipment; remove the mute state of the third audio channel;

Unmute the first audio channel.
The display device of claim 1, the controller further configured to:

Before performing the step of switching the audio device corresponding to the focus window and adjusting the audio devices corresponding to the other windows, in response to the user's input of an instruction to query the audio device corresponding to the focus window, control the display to display the audio device usage, so that the user can switch the audio device corresponding to the focus window to a specific audio device according to the usage of the audio device; the usage of the audio device includes all audio devices that can be used by the display device and each audio device The current corresponding window.
The display device according to claim 4, wherein the switching instruction is: switching the audio device corresponding to the focus window from a first-type audio device to a specific second-type audio device;

The controller is configured to:

In the step of switching the audio device corresponding to the focus window and adjusting the audio devices corresponding to other windows, the first audio channel of the focus window is set to a mute state, and the specific second type of audio is determined The use state of the device; the first audio channel is used to output the audio of the focus window to the audio device corresponding to the focus window;

when the specific second type audio device is not in use, connecting the first audio channel and the specific second type audio device;

When the specific second-type audio device is in use, determine a specific switching window currently corresponding to the specific second-type audio device; set a specific audio channel of the specific switching window to a mute state, and the specific audio channel for outputting the audio of the specific switching window to the audio device corresponding to the specific switching window; connecting the specific audio channel and the first type of audio device, and connecting the first audio channel and the specific audio device The second type of audio device connection; unmute the specific audio channel;

Unmute the first audio channel.
The display device according to claim 4, wherein the user interface comprises a first window and a second window;

When the audio device corresponding to the first window is a first-class audio device, the audio device corresponding to the second window is a second-class audio device;

When the audio device corresponding to the first window is an audio device of the second type, the audio device corresponding to the second window is the audio device of the first type or the remaining audio devices of the second type.
The display device of claim 6, the controller configured to:

Before executing the step of controlling the display to display a plurality of windows, in response to a user input instructing to turn on the power-on instruction of the display device, controlling the display to display a first window, where the first window is used to play the first media resource;

In performing the step of controlling the display to display a plurality of windows, the controller is configured to:

In response to the display instruction, the display is controlled to display a first window and a second window at the same time, and the second window is used to display a media asset recommendation list, so that the user can select a second media asset according to the media asset recommendation list ;

A second media asset is played in the second window.
The display device according to claim 6, wherein the switching instruction is: switching the audio device corresponding to the focus window from the first device in the second type of audio device to the second device in the second type of audio device; wherein , the focus window is the first window, and the second device is the audio device currently corresponding to the second window;

The controller is configured to:

In the step of switching the audio device corresponding to the focus window and adjusting the audio devices corresponding to the remaining windows, the audio channel of the first window is connected to the second device, and the audio channel of the second window is connected to the first device to connect.
The display device of claim 1, the controller configured to:

Controlling the display to display only the focus window in response to a control instruction for switching the user interface from a plurality of windows to a single window in response to an instruction input by the user;

Mute the audio channels of the remaining windows.
An audio output method, applied to a display device, the method comprising:

Controlling the display to display the plurality of windows in response to a display instruction inputted by the user indicating to display the plurality of windows;

Detects the focused window that the user-controlled focus is on;

In response to the switching instruction of switching the audio device corresponding to the focus window in response to the instruction input by the user, the audio device corresponding to the focus window is switched, and the audio devices corresponding to the remaining windows are adjusted, so that each window uses a different audio device .
The method of claim 10, comprising:

The switching instruction is: switching the audio device corresponding to the focus window from the second type of audio device to the first type of audio device;

In the step of switching the audio device corresponding to the focus window and adjusting the audio devices corresponding to other windows, the first audio channel of the focus window is set to a mute state, and the use of the first type of audio device is judged state; the first audio channel is used to output the audio of the focus window to the audio device corresponding to the focus window;

When the first type of audio equipment is not in use, connecting the first audio channel and the first type of audio equipment;

When the first type of audio device is in use, determine the first switching window currently corresponding to the first type of audio device; set the second audio channel of the first switching window to a mute state, and the second The audio channel is used to output the audio of the first switching window to the audio device corresponding to the first switching window; connect the second audio channel and the second type of audio device corresponding to the focus window, and, connecting the first audio channel with the first type of audio equipment; releasing the mute state of the second audio channel;

Unmute the first audio channel.
The method of claim 10, comprising:

The switching instruction is: switching the audio equipment corresponding to the focus window from the first type of audio equipment to the second type of audio equipment;

In the step of switching the audio device corresponding to the focus window and adjusting the audio devices corresponding to other windows, the first audio channel of the focus window is set to a mute state, and it is determined whether the audio output interface is connected There is an unused second-class audio device; the first audio channel is used to output the audio of the focus window to the audio device corresponding to the focus window;

When the audio output interface is connected with an unused second-class audio device, connecting the first audio channel and the unused second-class audio device;

When the audio output interface is not connected to an unused second-type audio device, randomly select a first device in the second-type audio device, and determine the second switching window currently corresponding to the first device; The third audio channel of the second switching window is set to a mute state, and the third audio channel is used to output the audio of the second switching window to the audio device corresponding to the second switching window; Connect with the first type of audio equipment, and connect the first audio channel with the first equipment; remove the mute state of the third audio channel;

Unmute the first audio channel.
The method of claim 10, comprising:

Before performing the step of switching the audio device corresponding to the focus window and adjusting the audio devices corresponding to the other windows, in response to the user's input of an instruction to query the audio device corresponding to the focus window, control the display to display the audio device usage, so that the user can switch the audio device corresponding to the focus window to a specific audio device according to the usage of the audio device; the usage of the audio device includes all audio devices that can be used by the display device and each audio device The current corresponding window.