CN114302070A

CN114302070A - Display device and audio output method

Info

Publication number: CN114302070A
Application number: CN202110481435.2A
Authority: CN
Inventors: 陈俊宁; 卢平光; 张永芳; 崔文华; 齐消消; 贾亚洲
Original assignee: Hisense Visual Technology Co Ltd
Current assignee: Hisense Visual Technology Co Ltd
Priority date: 2021-04-30
Filing date: 2021-04-30
Publication date: 2022-04-08

Abstract

The application provides a display device and an audio output method. The display is used for displaying a user interface, and the user interface comprises a plurality of windows; the audio output interfaces are multiple and are used for connecting multiple types of audio equipment; the controller is configured to: controlling a display to display a plurality of windows in response to a display instruction input by a user to indicate display of the plurality of windows; detecting a focus window in which a user-controlled focus is located; responding to a switching instruction which is input by a user and used for indicating to switch the audio equipment corresponding to the focus window, switching the audio equipment corresponding to the focus window, and adjusting the audio equipment corresponding to the other windows, so that each window uses different audio equipment, each window can independently play audio, and the experience of the user is improved.

Description

Display device and audio output method

Technical Field

The present application relates to the field of display device technologies, and in particular, to a display device and an audio output method.

Background

Along with the rapid development of intelligent display equipment, the functions of the display equipment are more and more abundant, the performance is more and more powerful, the bidirectional human-computer interaction function can be realized, and various functions such as audio and video, entertainment, data and the like are integrated, so that the diversified and personalized requirements of users are met. As the user demands the functions or services provided by the display device to be higher, the screen-division display is also becoming a function frequently used by the user.

For some display devices, the screen division processing can be performed on the pictures displayed by the display to form a plurality of windows, and each window respectively displays different picture contents, so that a user can simultaneously watch different program pictures.

However, the inventor of the present application finds that, although the existing display device can display different pictures through multiple windows, only one of the windows is selected to play sound, that is, the user is allowed to watch multiple program pictures, but the sound of one of the program pictures is played, and the rest of the program pictures cannot play sound, so that the user experience is seriously affected.

Disclosure of Invention

The invention provides a display device and an audio output method. The problem that in an existing display device, only one window in a plurality of windows can play sound, and user experience is poor is solved.

In a first aspect, the present application provides a display device comprising a display, an audio output interface, and a controller. The display is used for displaying a user interface, and the user interface comprises a plurality of windows; the audio output interfaces are multiple and are used for connecting multiple types of audio equipment; the controller is configured to perform the steps of: controlling a display to display a plurality of windows in response to a display instruction input by a user to indicate display of the plurality of windows; detecting a focus window in which a user-controlled focus is located; responding to a switching instruction which is input by a user and used for switching the audio equipment corresponding to the focus window, and adjusting the audio equipment corresponding to the rest windows to enable each window to use different audio equipment.

In some implementations, the switching instruction is: and switching the audio equipment corresponding to the focus window from the second type of audio equipment to the first type of audio equipment.

In the step of switching the audio device corresponding to the focus window and adjusting the audio devices corresponding to the remaining windows, the controller is further configured to: setting a first audio channel of the focus window to be in a mute state, and judging the use state of first-class audio equipment; the first audio channel is used for outputting the audio of the focus window to the audio equipment corresponding to the focus window; when the first type of audio equipment is in an unused state, connecting the first audio channel with the first type of audio equipment; when the first type of audio equipment is in a use state, determining a first switching window corresponding to the first type of audio equipment currently; setting a second audio channel of the first switching window to be in a mute state, wherein the second audio channel is used for outputting the audio of the first switching window to the audio equipment corresponding to the first switching window; connecting the second audio channel with second audio equipment corresponding to the focus window, and connecting the first audio channel with first audio equipment; removing a mute state of the second audio channel; the mute state of the first audio channel is released.

In some implementations, the switching instruction is: and switching the audio equipment corresponding to the focus window from the first type of audio equipment to the second type of audio equipment.

In the step of switching the audio device corresponding to the focus window and adjusting the audio devices corresponding to the remaining windows, the controller is further configured to: setting a first audio channel of the focus window to be in a mute state, and judging whether the audio output interface is connected with unused second-class audio equipment or not; the first audio channel is used for outputting the audio of the focus window to the audio equipment corresponding to the focus window; when the audio output interface is connected with second type of unused audio equipment, connecting the first audio channel with the second type of unused audio equipment; when the audio output interface is not connected with unused second-class audio equipment, randomly selecting first equipment in the second-class audio equipment, and determining a second switching window currently corresponding to the first equipment; setting a third audio channel of the second switching window to be in a mute state, wherein the third audio channel is used for outputting the audio of the second switching window to the audio equipment corresponding to the second switching window; connecting the third audio channel with a first type of audio device, and connecting the first audio channel with the first device; removing a mute state of the third audio channel; the mute state of the first audio channel is released.

In some implementations, before performing the step of switching the audio device corresponding to the focus window and adjusting the audio devices corresponding to the remaining windows, the controller is further configured to: responding to a query instruction which is input by a user and indicates to query the audio equipment corresponding to the focus window, and controlling a display to display the use condition of the audio equipment so that the user can switch the audio equipment corresponding to the focus window into specific audio equipment according to the use condition of the audio equipment; the audio device usage includes all audio devices that can be used by the display device and a window currently corresponding to each audio device.

In some implementations, the switching instruction is: and switching the audio equipment corresponding to the focus window from the first type of audio equipment to the specific second type of audio equipment.

In the step of switching the audio device corresponding to the focus window and adjusting the audio devices corresponding to the remaining windows, the controller is further configured to: setting a first audio channel of the focus window to be in a mute state, and judging the use state of the specific second-class audio equipment; the first audio channel is used for outputting the audio of the focus window to the audio equipment corresponding to the focus window; connecting the first audio channel with the particular second class of audio device when the particular second class of audio device is in an unused state; when the specific second-class audio equipment is in a use state, determining a specific switching window currently corresponding to the specific second-class audio equipment; setting a specific audio channel of the specific switching window to be in a mute state, wherein the specific audio channel is used for outputting the audio of the specific switching window to audio equipment corresponding to the specific switching window; connecting the specific audio channel with a first type of audio device, and connecting the first audio channel with the specific second type of audio device; removing a mute state of the particular audio channel; the mute state of the first audio channel is released.

In some implementations, a first window and a second window are included in the user interface. When the audio equipment corresponding to the first window is first-class audio equipment, the audio equipment corresponding to the second window is second-class audio equipment; and when the audio equipment corresponding to the first window is second-class audio equipment, the audio equipment corresponding to the second window is the first-class audio equipment or the rest second-class audio equipment.

In some implementations, prior to performing the step of controlling the display to display the plurality of windows, the controller is further configured to: and responding to a starting instruction of starting the display equipment by the instruction input by the user, and controlling the display to display a first window, wherein the first window is used for playing the first media resource. In the performing the step of controlling the display to display the plurality of windows, the controller is further configured to: responding to the display instruction, controlling a display to simultaneously display a first window and a second window, wherein the second window is used for displaying a media resource recommendation list so that a user can select a second media resource according to the media resource recommendation list; and playing a second media resource in the second window.

In some implementations, the switching instruction is: switching the audio equipment corresponding to the focus window from first equipment in second class of audio equipment to second equipment in second class of audio equipment; the focus window is a first window, and the second device is an audio device corresponding to the second window. In the step of switching the audio device corresponding to the focus window and adjusting the audio devices corresponding to the remaining windows, the controller is further configured to: and connecting the audio channel of the first window with the second equipment, and connecting the audio channel of the second window with the first equipment.

In some implementations, the controller is further configured to: controlling a display to display only the focus window in response to a control instruction input by a user to switch a user interface from a plurality of windows to a single window; the audio channels of the remaining windows are set to a mute state.

In a second aspect, the present application further provides an audio output method applied to a display device, the method including: controlling a display to display a plurality of windows in response to a display instruction input by a user to indicate display of the plurality of windows; detecting a focus window in which a user-controlled focus is located; responding to a switching instruction which is input by a user and used for switching the audio equipment corresponding to the focus window, and adjusting the audio equipment corresponding to the rest windows to enable each window to use different audio equipment.

According to the technical scheme, the display device and the audio output method can control the display to display a plurality of windows and detect the focus window where the focus controlled by the user is located; the audio equipment corresponding to the focus window is switched, and the audio equipment corresponding to the rest windows is adjusted, so that each window uses different audio equipment, and each window can independently play audio, and the experience of a user is improved.

Drawings

In order to more clearly explain the technical solution of the present application, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious to those skilled in the art that other drawings can be obtained according to the drawings without any creative effort.

FIG. 1 illustrates a usage scenario of a display device according to some embodiments;

fig. 2 illustrates a hardware configuration block diagram of the control apparatus 100 according to some embodiments;

fig. 3 illustrates a hardware configuration block diagram of the display apparatus 200 according to some embodiments;

FIG. 4 illustrates a software configuration diagram in the display device 200 according to some embodiments;

FIG. 5 illustrates an interaction flow diagram for components of a display device in some embodiments;

FIG. 6 is a diagram illustrating a screen of a display in a multi-window mode in one possible embodiment;

FIG. 7 shows a schematic diagram of a multi-window mode in a possible embodiment;

FIG. 8 is a diagram illustrating an audio output manner in a multi-window mode in a possible embodiment;

FIG. 9 shows a schematic view of a focus window in a possible embodiment;

FIG. 10 illustrates an interaction flow diagram for components of a display device in some embodiments;

FIG. 11 is a schematic diagram of an audio device corresponding to a display focus window in a possible embodiment;

FIG. 12 illustrates an interaction flow diagram for components of a display device in some embodiments;

FIG. 13 is a diagram of an audio device corresponding to a display focus window in a possible embodiment;

FIG. 14 is a schematic diagram of the display device switching from a multi-window mode to a single-window mode in one possible embodiment;

FIG. 15 shows a flow diagram of one embodiment of an audio output method.

Detailed Description

To make the purpose and embodiments of the present application clearer, the following will clearly and completely describe the exemplary embodiments of the present application with reference to the attached drawings in the exemplary embodiments of the present application, and it is obvious that the described exemplary embodiments are only a part of the embodiments of the present application, and not all of the embodiments.

It should be noted that the brief descriptions of the terms in the present application are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of the present application. These terms should be understood in their ordinary and customary meaning unless otherwise indicated.

The terms "first," "second," "third," and the like in the description and claims of this application and in the above-described drawings are used for distinguishing between similar or analogous objects or entities and not necessarily for describing a particular sequential or chronological order, unless otherwise indicated. It is to be understood that the terms so used are interchangeable under appropriate circumstances.

The terms "comprises" and "comprising," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a product or apparatus that comprises a list of elements is not necessarily limited to all elements expressly listed, but may include other elements not expressly listed or inherent to such product or apparatus.

The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and/or software code that is capable of performing the functionality associated with that element.

Fig. 1 is a schematic diagram of a usage scenario of a display device according to an embodiment. As shown in fig. 1, the display apparatus 200 is also in data communication with a server 400, and a user can operate the display apparatus 200 through the smart device 300 or the control device 100.

In some embodiments, the control apparatus 100 may be a remote controller, and the communication between the remote controller and the display device includes at least one of an infrared protocol communication or a bluetooth protocol communication, and other short-distance communication methods, and controls the display device 200 in a wireless or wired manner. The user may control the display apparatus 200 by inputting a user instruction through at least one of a key on a remote controller, a voice input, a control panel input, and the like.

In some embodiments, the smart device 300 may include any of a mobile terminal, a tablet, a computer, a laptop, an AR/VR device, and the like.

In some embodiments, the smart device 300 may also be used to control the display device 200. For example, the display device 200 is controlled using an application program running on the smart device.

In some embodiments, the smart device 300 and the display device may also be used for communication of data.

In some embodiments, the display device 200 may also be controlled in a manner other than the control apparatus 100 and the smart device 300, for example, the voice instruction control of the user may be directly received by a module configured inside the display device 200 to obtain a voice instruction, or may be received by a voice control apparatus provided outside the display device 200.

In some embodiments, the display device 200 is also in data communication with a server 400. The display device 200 may be allowed to be communicatively connected through a Local Area Network (LAN), a Wireless Local Area Network (WLAN), and other networks. The server 400 may provide various contents and interactions to the display apparatus 200. The server 400 may be a cluster or a plurality of clusters, and may include one or more types of servers.

In some embodiments, software steps executed by one step execution agent may be migrated on demand to another step execution agent in data communication therewith for execution. Illustratively, software steps performed by the server may be migrated to be performed on a display device in data communication therewith, and vice versa, as desired.

Fig. 2 exemplarily shows a block diagram of a configuration of the control apparatus 100 according to an exemplary embodiment. As shown in fig. 2, the control device 100 includes a controller 110, a communication interface 130, a user input/output interface 140, a memory, and a power supply. The control apparatus 100 may receive an input operation instruction from a user and convert the operation instruction into an instruction recognizable and responsive by the display device 200, serving as an interaction intermediary between the user and the display device 200.

In some embodiments, the communication interface 130 is used for external communication, and includes at least one of a WIFI chip, a bluetooth module, NFC, or an alternative module.

In some embodiments, the user input/output interface 140 includes at least one of a microphone, a touchpad, a sensor, a key, or an alternative module.

Fig. 3 shows a hardware configuration block diagram of the display apparatus 200 according to an exemplary embodiment.

In some embodiments, the display apparatus 200 includes at least one of a tuner demodulator 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output interface 270, a memory, a power supply, a user interface.

In some embodiments the controller comprises a central processor, a video processor, an audio processor, a graphics processor, a RAM, a ROM, a first interface to an nth interface for input/output.

In some embodiments, the display 260 includes a display screen component for displaying pictures, and a driving component for driving image display, a component for receiving image signals from the controller output, displaying video content, image content, and menu manipulation interface, and a user manipulation UI interface, etc.

In some embodiments, the display 260 may be at least one of a liquid crystal display, an OLED display, and a projection display, and may also be a projection device and a projection screen.

In some embodiments, the tuner demodulator 210 receives broadcast television signals via wired or wireless reception, and demodulates audio/video signals, such as EPG data signals, from a plurality of wireless or wired broadcast television signals.

In some embodiments, communicator 220 is a component for communicating with external devices or servers according to various communication protocol types. For example: the communicator may include at least one of a Wifi module, a bluetooth module, a wired ethernet module, and other network communication protocol chips or near field communication protocol chips, and an infrared receiver. The display apparatus 200 may establish transmission and reception of control signals and data signals with the control device 100 or the server 400 through the communicator 220.

In some embodiments, the detector 230 is used to collect signals of the external environment or interaction with the outside. For example, detector 230 includes a light receiver, a sensor for collecting ambient light intensity; alternatively, the detector 230 includes an image collector, such as a camera, which may be used to collect external environment scenes, attributes of the user, or user interaction gestures, or the detector 230 includes a sound collector, such as a microphone, which is used to receive external sounds.

In some embodiments, the external device interface 240 may include, but is not limited to, the following: high Definition Multimedia Interface (HDMI), analog or data high definition component input interface (component), composite video input interface (CVBS), USB input interface (USB), RGB port, and the like. The interface may be a composite input/output interface formed by the plurality of interfaces.

In some embodiments, the controller 250 and the modem 210 may be located in different separate devices, that is, the modem 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

In some embodiments, the controller 250 controls the operation of the display device and responds to user operations through various software control programs stored in memory. The controller 250 controls the overall operation of the display apparatus 200. For example: in response to receiving a user command for selecting a UI object to be displayed on the display 260, the controller 250 may perform an operation related to the object selected by the user command.

In some embodiments, the object may be any one of selectable objects, such as a hyperlink, an icon, or other actionable control. The operations related to the selected object are: displaying an operation connected to a hyperlink page, document, image, or the like, or performing an operation of a program corresponding to the icon.

In some embodiments the controller comprises at least one of a Central Processing Unit (CPU), a video processor, an audio processor, a Graphics Processing Unit (GPU), a RAM Random Access Memory (RAM), a ROM (Read-Only Memory), a first to nth interface for input/output, a communication Bus (Bus), and the like.

A CPU processor. For executing operating system and application program instructions stored in the memory, and executing various application programs, data and contents according to various interactive instructions receiving external input, so as to finally display and play various audio-video contents. The CPU processor may include a plurality of processors. E.g. comprising a main processor and one or more sub-processors.

In some embodiments, a graphics processor for generating various graphics objects, such as: at least one of an icon, an operation menu, and a user input instruction display figure. The graphic processor comprises an arithmetic unit, which performs operation by receiving various interactive instructions input by a user and displays various objects according to display attributes; the system also comprises a renderer for rendering various objects obtained based on the arithmetic unit, wherein the rendered objects are used for being displayed on a display.

In some embodiments, the video processor is configured to receive an external video signal, and perform at least one of video processing such as decompression, decoding, scaling, noise reduction, frame rate conversion, resolution conversion, and image synthesis according to a standard codec protocol of the input signal, so as to obtain a signal displayed or played on the direct display device 200.

In some embodiments, the video processor includes at least one of a demultiplexing module, a video decoding module, an image composition module, a frame rate conversion module, a display formatting module, and the like. The demultiplexing module is used for demultiplexing the input audio and video data stream. And the video decoding module is used for processing the video signal after demultiplexing, including decoding, scaling and the like. And the image synthesis module is used for carrying out superposition mixing processing on the GUI signal input by the user or generated by the user and the video image after the zooming processing by the graphic generator so as to generate an image signal for display. And the frame rate conversion module is used for converting the frame rate of the input video. And the display formatting module is used for converting the received video output signal after the frame rate conversion, and changing the signal to be in accordance with the signal of the display format, such as an output RGB data signal.

In some embodiments, the audio processor is configured to receive an external audio signal, decompress and decode the received audio signal according to a standard codec protocol of the input signal, and perform at least one of noise reduction, digital-to-analog conversion, and amplification processing to obtain a sound signal that can be played in the speaker.

In some embodiments, a user may enter user commands on a Graphical User Interface (GUI) displayed on display 260, and the user input interface receives the user input commands through the Graphical User Interface (GUI). Alternatively, the user may input the user command by inputting a specific sound or gesture, and the user input interface receives the user input command by recognizing the sound or gesture through the sensor.

In some embodiments, a "user interface" is a media interface for interaction and information exchange between an application or operating system and a user that enables conversion between an internal form of information and a form that is acceptable to the user. A commonly used presentation form of the User Interface is a Graphical User Interface (GUI), which refers to a User Interface related to computer operations and displayed in a graphical manner. It may be an interface element such as an icon, a window, a control, etc. displayed in the display screen of the electronic device, where the control may include at least one of an icon, a button, a menu, a tab, a text box, a dialog box, a status bar, a navigation bar, a Widget, etc. visual interface elements.

In some embodiments, user interface 280 is an interface that may be used to receive control inputs (e.g., physical buttons on the body of the display device, or the like).

In some embodiments, a system of a display device may include a Kernel (Kernel), a command parser (shell), a file system, and an application program. The kernel, shell, and file system together make up the basic operating system structure that allows users to manage files, run programs, and use the system. After power-on, the kernel is started, kernel space is activated, hardware is abstracted, hardware parameters are initialized, and virtual memory, a scheduler, signals and interprocess communication (IPC) are operated and maintained. And after the kernel is started, loading the Shell and the user application program. The application program is compiled into machine code after being started, and a process is formed.

Referring to fig. 4, in some embodiments, the system is divided into four layers, which are an Application (Applications) layer (abbreviated as "Application layer"), an Application Framework (Application Framework) layer (abbreviated as "Framework layer"), an Android runtime (Android runtime) and system library layer (abbreviated as "system runtime library layer"), and a kernel layer from top to bottom.

In some embodiments, at least one application program runs in the application program layer, and the application programs may be windows (windows) programs carried by an operating system, system setting programs, clock programs or the like; or an application developed by a third party developer. In particular implementations, the application packages in the application layer are not limited to the above examples.

The framework layer provides an Application Programming Interface (API) and a programming framework for the application program of the application layer. The application framework layer includes a number of predefined functions. The application framework layer acts as a processing center that decides to let the applications in the application layer act. The application program can access the resources in the system and obtain the services of the system in execution through the API interface.

As shown in fig. 4, in the embodiment of the present application, the application framework layer includes a manager (Managers), a Content Provider (Content Provider), and the like, where the manager includes at least one of the following modules: an Activity Manager (Activity Manager) is used for interacting with all activities running in the system; the Location Manager (Location Manager) is used for providing the system service or application with the access of the system Location service; a Package Manager (Package Manager) for retrieving various information related to an application Package currently installed on the device; a Notification Manager (Notification Manager) for controlling display and clearing of Notification messages; a Window Manager (Window Manager) is used to manage the icons, windows, toolbars, wallpapers, and desktop components on a user interface.

In some embodiments, the activity manager is used to manage the lifecycle of the various applications as well as general navigational fallback functions, such as controlling exit, opening, fallback, etc. of the applications. The window manager is used for managing all window programs, such as obtaining the size of a display screen, judging whether a status bar exists, locking the screen, intercepting the screen, controlling the change of the display window (for example, reducing the display window, displaying a shake, displaying a distortion deformation, and the like), and the like.

In some embodiments, the system runtime layer provides support for the upper layer, i.e., the framework layer, and when the framework layer is used, the android operating system runs the C/C + + library included in the system runtime layer to implement the functions to be implemented by the framework layer.

In some embodiments, the kernel layer is a layer between hardware and software. As shown in fig. 4, the core layer includes at least one of the following drivers: audio drive, display driver, bluetooth drive, camera drive, WIFI drive, USB drive, HDMI drive, sensor drive (like fingerprint sensor, temperature sensor, pressure sensor etc.) and power drive etc..

For some display devices, the screen division processing can be performed on the pictures displayed by the display to form a plurality of windows, and each window respectively displays different picture contents, so that a user can simultaneously watch different program pictures. However, although the existing display device can display different pictures through multiple windows, only one of the windows is selected to play sound, that is, the user is allowed to watch multiple program pictures, but the sound of one of the program pictures is played, and the rest of the program pictures cannot play sound, thereby seriously affecting the user experience.

In order to solve the above technical problem, the present application provides a display device including a display, an audio output interface, and a controller. Wherein, the display is used for displaying a user interface; the audio output interface comprises a plurality of types of audio interfaces which are respectively used for connecting a plurality of types of audio equipment.

In some embodiments, the audio output interfaces in the display device may include a first type of audio interface for connecting to a first type of audio device and a second type of audio interface for connecting to a second type of audio device. The first type of audio device may be an audio device such as a speaker or a sound box built in the display device. The second type of audio device may be a bluetooth speaker, a USB speaker, or a Soundbar speaker externally connected to the display device.

In some embodiments, the display device may be provided with one first type of audio interface and a plurality of second type of audio interfaces. That is, the display device may be connected with one first type audio device and a plurality of second type audio devices, and play audio contents. For example, the display device may have a speaker built in the device and two bluetooth speakers connected externally.

In some embodiments, the display device has a split screen function, the user interface displayed in the display may be divided into a plurality of windows, and each window may present different picture content, i.e. the display device has a multi-window mode. In the multi-window mode, the display device may simultaneously display a plurality of windows so that different picture contents can be viewed by a user in one display device.

FIG. 5 is a flow diagram that illustrates the interaction of components of the display device in some embodiments. When a user screen is displayed in the display device and there is only one window in the user screen, the user may input a display instruction to display a plurality of windows, i.e., a multi-window mode instruction to the display device. The controller may control the display to change from displaying one window to displaying multiple windows when the controller receives the multi-window mode command.

In some embodiments, the user may send a multi-window mode command to the display device by manipulating a designated key of the remote controller. And binding the corresponding relation between the multi-window mode command and the remote controller key in advance in the actual application process. For example, a multi-window mode key is set on the remote controller, when a user touches the key, the remote controller sends a multi-window mode command to the controller, and at this time, the controller controls the display device to enter a multi-window mode and controls the display device to display a plurality of windows. When the user touches the key again, the controller may control the display device to exit the multi-window mode.

In some embodiments, the corresponding relationship between the multi-window mode command and the plurality of remote control keys may also be pre-bound, and when the user touches the plurality of keys bound to the multi-window mode command, the remote control sends out the multi-window mode command. In a feasible embodiment, the keys bound by the multi-window mode command are sequentially direction keys (left, down, left, down), that is, the remote controller sends the multi-window mode command to the controller only when the user continuously touches the keys (left, down, left, down) within a preset time. By adopting the binding method, the multi-window mode instruction can be prevented from being sent out due to misoperation of a user.

The embodiment of the application is only exemplary in providing several binding relationships between the multi-window mode command and the key, and the binding relationship between the multi-window mode command and the key can be set according to the habit of a user in the process of practical application, which is not limited herein.

In some embodiments, when the user controls the display device using the smart device, for example using a cell phone, a multi-window mode instruction may also be sent to the display device. In the process of practical application, a split screen control can be set in the mobile phone, whether a multi-window mode is entered or not and the number of windows is set can be selected through the split screen control, so that a multi-window mode instruction is sent to the controller, and at the moment, the controller controls the display device to enter the multi-window mode and controls the display device to display a plurality of windows.

In some embodiments, when the user controls the display device using the cell phone, a continuous click command may be issued to the cell phone. The continuous click command refers to: in a preset period, the number of times that a user clicks the same area of the mobile phone touch screen exceeds a preset threshold value. For example: when the user continuously clicks a certain area of the mobile phone touch screen for 3 times within 1s, the user is regarded as a continuous clicking instruction. After receiving the continuous click command, the mobile phone can send a multi-window mode command to the display device, so that the controller controls the display device to enter a multi-window mode and controls the display device to display a plurality of windows.

In some embodiments, when the user uses the mobile phone to control the display device, the following may also be set: when detecting that a touch pressure value of a certain area of the mobile phone touch screen by a user exceeds a preset pressure threshold, the mobile phone can send a multi-window mode instruction to the display device.

A multi-window mode switching option can be set in a UI interface of the display device, and when a user clicks the switching option, the display device can be controlled to enter or exit the multi-window mode.

In some embodiments, two windows may be displayed in the multi-window mode, or more than two windows may be displayed. Fig. 6 is a schematic diagram of a picture in a display in a multi-window mode in a feasible embodiment, where after a display device enters the multi-window mode, a first window and a second window are simultaneously displayed in the display, where the first window and the second window may respectively display different picture contents, the first window displays picture contents of a first media resource, and the second window displays picture contents of a second media resource.

In some embodiments, when the contents of the pictures displayed in the multiple windows are different, the picture types displayed in the multiple windows can be the same. For example, when the picture content of the first media resource is displayed in the first window, the picture content of the second media resource is displayed in the second window, and at this time, the picture types displayed by the two windows are the same and are both media resources. It can also be set in different use scenes, and the types of the pictures displayed in the multiple windows are different. For example, when the screen content of the media asset is displayed in the first window, a UI interface or a game interface or the like is displayed in the second window, and the types of the screens displayed in the two windows are different. Therefore, different requirements of the user can be met, and the experience of the user is enhanced.

In some embodiments, the multi-window mode may be set according to a user's use state. When it is detected that the user starts the display device, the display device defaults to a single-window mode, that is, only the first window is displayed in the display, the first window may be displayed in a full screen manner or in a windowed manner, and the display device may play the first media resource in the first window.

When the first media resource is played in the first window, if the user sends a multi-window mode instruction to the display equipment, the program recommendation interface can be entered. Fig. 7 is a schematic diagram illustrating a multi-window mode in one possible embodiment, where a media asset recommendation list is displayed on the right side of a first window, where the media asset recommendation list includes multiple recommended program options, and if a user selects and confirms the recommended program 2 option, the display device may be controlled to enter the multi-window mode, where the first media asset is still displayed through the first window, and the recommended program 2 selected by the user is displayed in a second window.

In some embodiments, when the display device is in the multi-window mode, the audio content corresponding to each screen may also be output separately while the different screen contents are simultaneously displayed in multiple windows. The display device may simultaneously use one first type of audio device and a plurality of second type of audio devices to play audio, for example, the display device is simultaneously connected with one built-in speaker and two external bluetooth speakers, and at this time, the display device may respectively output audio contents of three windows by using the three audio devices. Meanwhile, each window can respectively output audio to the audio equipment corresponding to each window through the respective audio channel, and the audio content is played through the audio equipment.

In some embodiments, the display device may set two audio device modes. One is a built-in playing mode, and audio content corresponding to a first type of audio device playing window built in the display device can be used. One is an external playing mode, which plays the audio content corresponding to the window by using a second type of audio device, such as a bluetooth speaker.

Fig. 8 shows a schematic diagram of an audio output mode in a multi-window mode in a possible embodiment. The display equipment is in a multi-window mode, wherein a first media resource is played in a first window, and audio is output by adopting first-class audio equipment; and playing the second media resource in the second window, and outputting audio by adopting second audio equipment.

Or the first window and the second window output audio by using a second type of audio device, that is, the first window outputs audio by using the first device, and the second window outputs audio by using the second device. The first device and the second device in the embodiment of the application are both second-class audio devices. It should be noted that, because each window plays audio through different audio devices, the audio of each picture in the multi-window mode can be played independently, so as to achieve the purpose of mutual noninterference.

Therefore, to ensure that the audio of all windows can be played normally in the multi-window mode, the following settings can be set: and when the audio equipment corresponding to the first window is the first type of audio equipment, the audio equipment corresponding to the second window is the second type of audio equipment. And when the audio equipment corresponding to the first window is second-class audio equipment, the audio equipment corresponding to the second window is the first-class audio equipment or the rest second-class audio equipment. To ensure that the audio equipment used for each window is different.

The controller can receive various control instructions input by a user during the use process of the user and respectively execute corresponding interactive actions according to the control instructions so as to control each window, such as image quality control, voice control and the like.

In some embodiments, the display device may detect the focus window in real-time. Wherein, the focus window refers to a window where the focus controlled by the user is located. The focus is a mark used for assisting a user in man-machine interaction by the display device, and can highlight the currently selected content of the user. For example, the focus may be a selection box, which is displayed by means such as "deepening display" and "highlighting" and can be controlled by the user through "up, down, left and right" direction keys on the remote controller, so as to select different options. The focus can be located on a certain option in the window, or can be located in the whole window, as long as the focus is located in the whole range of a certain window, the window is the focus window. Fig. 9 shows a schematic view of a focus window in a possible embodiment, the user interface comprising a first window and a second window, wherein the focus is in the form of a selection box. At this time, the selection frame selects the option in the first window, so that the first window is the focus window, and the user can also select other options in the first window, thereby adjusting the first window.

The display device can respond to various operations of the user in time by detecting the focus window in real time. FIG. 10 is a flow diagram that illustrates the interaction of components of the display device in some embodiments.

In some embodiments, the controller may also obtain the audio device to which the focus window currently corresponds while the display device detects the focus window in real time. For example, it may be obtained from a database in the display device that records the audio device currently used for each window. The controller may determine the focus window according to the focus position, and if the focus window is the first window, may obtain the audio output device currently used by the first window from the database.

In some embodiments, the display device may present to the user an audio device mode condition that refers to: the audio equipment mode that the focus window corresponds to at present and can switch over, make the user switch over the audio equipment mode of focus window. The user can send a display instruction to the display device by operating the designated key of the remote controller. And binding the corresponding relation between the display instruction and the remote controller key in advance in the actual application process. For example, a display key is set on the remote controller, when a user touches the display key, the remote controller sends a display instruction to the controller, and the controller controls the display to display the audio device currently corresponding to the focus window and the switchable audio device mode.

Or the user calls a setting key UI menu, and the controller controls the OSD layer menu to display the audio equipment corresponding to the current window and the switchable audio equipment mode.

Fig. 11 is a schematic diagram of an audio device corresponding to a display displaying a focus window in a feasible embodiment, wherein a user interface includes a first window and a second window, and the first window is the focus window. When a display instruction of a user is detected, displaying audio equipment currently corresponding to a first window, namely first-class audio equipment, and audio equipment modes which can be switched by the first window, namely second-class audio equipment, in the first window. The user may choose to switch the audio device of the first window from the first type of audio device to the second type of audio device or not.

When a switching instruction indicating to switch the audio device corresponding to the focus window is detected, the display device may switch the audio device corresponding to the focus window in response to the switching instruction input by the user. Meanwhile, in order to ensure that each window can still independently play audio, audio equipment corresponding to other windows can be adjusted, so that each window is ensured to use different audio equipment.

In some embodiments, if the audio device currently corresponding to the focus window is the second type of audio device, and the user instructs to switch the audio device corresponding to the focus window from the second type of audio device to the first type of audio device, the display device may switch the audio device corresponding to the focus window in response to the switching instruction.

Before switching the audio device, the controller may set the first audio channel of the focus window to a mute state to prevent a pop phenomenon from occurring when switching the audio device.

After setting the first audio channel to a mute state, a current usage state of the first type of audio device may be determined. If the first type of audio device is in the unused state, it is indicated that no window uses the first type of audio device currently, the first audio channel and the first type of audio device can be directly connected, and the mute state of the first audio channel is released. So that the focus window can output the audio to the first type of audio device through the first audio channel and play the audio by using the first type of audio device.

If the first type of audio device is in the use state, it indicates that other windows are using the first type of audio device, and at this time, the audio devices of the window and the focus window can be exchanged, so that the focus window can use the first type of audio device. Before the audio devices are exchanged, the controller may detect a current window corresponding to the first type of audio device, and set the current window as a first switching window, and set a second audio channel of the first switching window to a mute state to prevent pop. The controller connects the second audio channel with the second type of audio device corresponding to the focus window, and connects the first audio channel with the first type of audio device. At this time, the audio devices between the focus window and the first switching window are switched. The controller releases the mute state of the first audio channel and the second audio channel, so that the focus window can play audio by using the first type of audio equipment, and simultaneously, each window can still independently play audio without being influenced.

In some embodiments, if the audio device currently corresponding to the focus window is the first type of audio device, and the user instructs to switch the audio device corresponding to the focus window from the first type of audio device to the second type of audio device, the display device may switch the audio device corresponding to the focus window in response to the switching instruction.

After the first audio channel is set to the mute state, it is possible to determine whether or not the display apparatus is currently connected with an unused second type audio apparatus, considering that the display apparatus can be connected with a plurality of second type audio apparatuses. If the second type of unused audio equipment exists currently, the first audio channel and the second type of unused audio equipment can be directly connected, so that the focus window can output audio to the second type of audio equipment through the first audio channel and play audio by using the second type of audio equipment.

If the display device is not connected with the unused second type audio devices, it is indicated that all the second type audio devices are used by other windows, and at this time, one second type audio device can be randomly selected for use, so that the focus window can use the second type audio devices. Before switching the audio devices, the controller may randomly select one second type of audio device, that is, the first device, determine a second switching window currently corresponding to the second type of audio device, and set a third audio channel of the second switching window to a mute state to prevent pop.

The controller then connects the third audio channel to the first type of audio device and connects the first audio channel to the first device. At this time, the audio devices between the focus window and the second switching window are exchanged. The controller releases the mute state of the first audio channel and the third audio channel, so that the focus window can play audio by using the first device, and simultaneously, each window can still independently play audio without being influenced.

FIG. 12 is a flow diagram that illustrates the interaction of components of the display device in some embodiments.

In some embodiments, the display device may present to the user the audio device and display audio device usage to which the focus window currently corresponds. The audio device usage may include all audio devices that can be used by the display device and a window currently corresponding to each audio device. And the user selects the audio equipment in the focus window according to the use condition of the audio equipment and switches to the selected specific audio equipment. The user can send the query instruction to the display device by operating the designated key of the remote controller. And binding the corresponding relation between the query instruction and the remote controller key in advance in the actual application process. For example, an inquiry key is arranged on the remote controller, when a user touches the inquiry key, the remote controller sends an inquiry instruction to the controller, and the controller controls the display to display the audio device corresponding to the current focus window and the use condition of the audio device.

Or the user calls a setting key UI menu, and the controller controls the OSD layer menu to display the audio equipment corresponding to the current window and the use condition of the audio equipment.

Fig. 13 is a schematic diagram of an audio device corresponding to a display displaying a focus window in a possible embodiment, where a user interface includes a first window, a second window, and a third window, and the first window is the focus window. When a query instruction of a user is detected, displaying the audio equipment currently corresponding to the first window in the first window, wherein the audio equipment is the first type of audio equipment. And audio device usage, namely: the first type of audio equipment currently corresponds to the first window, the first equipment currently corresponds to the second window, and the second equipment currently corresponds to the third window. The user may choose to switch the audio device of the first window from the first type of audio device to the first device or the second device, or not.

In some embodiments, if the audio device currently corresponding to the focus window is the first type of audio device, and the user instructs to switch the audio device corresponding to the focus window from the first type of audio device to a specific second type of audio device, the display device may switch the audio device corresponding to the focus window in response to the switching instruction.

After setting the first audio channel to a mute state, the controller may determine a usage state of a particular second type of audio device.

After setting the first audio channel to a mute state, a current usage state of the particular second type audio device may be determined. If the specific second type audio device is in the unused state, it indicates that no window uses the first type audio device currently, the first audio channel and the specific second type audio device can be directly connected, and the mute state of the first audio channel is released. So that the focus window can output the audio to the specific second type audio device through the first audio channel and play the audio by using the specific second type audio device.

If the specific second type audio device is in the use state, it indicates that other windows are using the specific second type audio device, and at this time, the audio devices of the window and the focus window can be exchanged, so that the focus window can use the specific second type audio device. Before the audio devices are exchanged, the controller may detect a window currently corresponding to a specific second type of audio device and set the window as a specific switching window, and set a specific audio channel of the specific switching window to a mute state to prevent pop. The controller then connects the particular audio channel to the first type of audio device and connects the first audio channel to the particular second type of audio device. At this time, the audio devices between the focus window and the specific switching window are exchanged. The controller releases the mute state of the first audio channel and the second audio channel, so that the focus window can play audio by using specific second-class audio equipment, and simultaneously, each window can still independently play audio without being influenced.

In some embodiments, when it is detected that a specific second type of audio device currently corresponds to a specific switching window, it may also be detected whether an unused second type of audio device is currently connected to the display device. If so, connecting the specific switching window with the unused second type audio devices, and connecting the focus window with the specific second type audio devices. If there are no unused audio devices of the second type, the audio devices between the focus window and the particular switching window are swapped.

In some embodiments, if the audio device currently corresponding to the focus window is the second type of audio device, and the user instructs to switch the audio device corresponding to the focus window from the second type of audio device to the first type of audio device, the display device may switch the audio device corresponding to the focus window in response to the switching instruction. The controller may perform audio device switching between the focus window and a window currently corresponding to the first type of audio device, and the specific switching method may refer to switching the audio device corresponding to the focus window from the first type of audio device to a specific second type of audio device in the above embodiments, which is not described herein again.

In some embodiments, if the audio device corresponding to the focus window is currently the second type of audio device, and the user instructs to switch the audio device corresponding to the focus window from the second type of audio device to a specific second type of audio device, the display device may switch the audio device corresponding to the focus window in response to the switching instruction.

The controller may first determine the usage of a particular second type of audio device. The first audio channel is directly connected to the specific second type of audio device if the specific second type of audio device is in an unused state. If the specific second-class audio device is in the use state, the focus window and the specific switching window currently corresponding to the specific second-class audio device may be directly exchanged for the audio device, and the specific exchanging method may refer to the contents in the foregoing embodiments, and will not be described here again.

In some embodiments, the display device may perform the packing process on the audio data of the window and transmit the audio data to the second type of audio device. When the display device is connected with one second-class audio device and a plurality of second-class audio devices, the packing processing methods of the audio data are different.

When the display device is connected with a second type of audio device, the audio data of the window can be separately packed by the left and right channels, and the left channel packed data and the right channel packed data are respectively transmitted to the second type of audio device. When the display device is connected with two second-class audio devices, the audio data of the same window can be packed together, namely, only one packed data exists in each window, and then the packed data is transmitted to the corresponding second-class audio devices.

In some embodiments, the user may also control the display device to enter a single window mode when the display device is in a multi-window mode. And the user selects the focus window, and can control the display equipment to enter a single-window mode through secondary touch control of a single-window mode key. At this time, the controller may control the display to display only the focus window, or may display the focus window in full screen.

In some embodiments, when the display device switches from the multi-window mode to the single-window mode, the currently used audio device may be controlled to be the audio device corresponding to the focus window, and simultaneously, the audio channels of all the remaining windows are set to be in a mute state, that is, the display device only uses the audio device corresponding to the focus window to play audio.

In some embodiments, if the focus window uses the second type of audio device to play audio, and other windows also use the second type of audio device to play audio, when the multi-window mode is switched to the single-window mode, the packing mode of the audio data needs to be changed to separately pack the left and right channels of the audio data of the focus window, and the audio data is transmitted to the second type of audio device.

In some embodiments, when the display device switches from the multi-window mode to the single-window mode, the first type of audio device may also be set as a default audio device and the currently used audio device is controlled to be the first type of audio device.

Fig. 14 is a schematic diagram illustrating a display device switching from a multi-window mode to a single-window mode in a possible embodiment, wherein a user interface includes a first window and a second window, the first window is a focus window, and audio is played through a first type of audio device. When the display device switches from the multi-window mode to the single-window mode, the controller may control the display to display the first window in a full screen and play audio using the first type of audio device.

It should be noted that, in the embodiment of the present application, the multiple windows in the multi-window mode may be displayed on the same display of the display device, or may be displayed on different multiple displays. For example, the display device may be a dual-screen smart-dot television, i.e. may comprise a first display and a second display. The first display is a main screen and can display a specific user interface, and the second display is a secondary screen and can display a specific interface. After entering the multi-window mode, the display device may control to display different windows in the first display and the second display, respectively, so as to output specific display screen contents at the same time. For the case that a plurality of windows are displayed on different displays, the display device may also adjust the volume of the display device 200 by operating the multi-window volume adjustment method, so as to improve user experience.

An embodiment of the present application further provides an audio output method, which is applied to a display device, and as shown in fig. 15, the method includes:

step 1501, controlling a display to display a plurality of windows in response to a display instruction which is input by a user and indicates to display the plurality of windows;

step 1502, detecting a focus window where a focus controlled by a user is located;

and 1503, responding to a switching instruction input by a user and instructing to switch the audio device corresponding to the focus window, switching the audio device corresponding to the focus window, and adjusting the audio devices corresponding to the other windows, so that each window uses different audio devices.

The same and similar parts in the embodiments in this specification may be referred to one another, and are not described herein again.

Those skilled in the art will readily appreciate that the techniques of the embodiments of the present invention may be implemented as software plus a required general purpose hardware platform. Based on such understanding, the technical solutions in the embodiments of the present invention may be essentially or partially implemented in the form of software products, which may be stored in a storage medium, such as ROM/RAM, magnetic disk, optical disk, etc., and include instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method in the embodiments or some parts of the embodiments of the present invention.

Finally, it should be noted that: the above embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present application.

The foregoing description, for purposes of explanation, has been presented in conjunction with specific embodiments. However, the illustrative discussions above are not intended to be exhaustive or to limit the embodiments to the precise forms disclosed above. Many modifications and variations are possible in light of the above teaching. The embodiments were chosen and described in order to best explain the principles and the practical application, to thereby enable others skilled in the art to best utilize the embodiments and various embodiments with various modifications as are suited to the particular use contemplated.

Claims

1. A display device, comprising:

the display is used for displaying a user interface, and the user interface comprises a plurality of windows;

the audio output interfaces are multiple and are used for connecting multiple types of audio equipment;

a controller configured to:

controlling a display to display a plurality of windows in response to a display instruction input by a user to indicate display of the plurality of windows;

detecting a focus window in which a user-controlled focus is located;

responding to a switching instruction which is input by a user and used for switching the audio equipment corresponding to the focus window, and adjusting the audio equipment corresponding to the rest windows to enable each window to use different audio equipment.

2. The display device according to claim 1, wherein the switching instruction is: switching the audio equipment corresponding to the focus window from the second type of audio equipment to the first type of audio equipment;

in the step of switching the audio device corresponding to the focus window and adjusting the audio devices corresponding to the remaining windows, the controller is further configured to:

setting a first audio channel of the focus window to be in a mute state, and judging the use state of first-class audio equipment; the first audio channel is used for outputting the audio of the focus window to the audio equipment corresponding to the focus window;

when the first type of audio equipment is in an unused state, connecting the first audio channel with the first type of audio equipment;

when the first type of audio equipment is in a use state, determining a first switching window corresponding to the first type of audio equipment currently; setting a second audio channel of the first switching window to be in a mute state, wherein the second audio channel is used for outputting the audio of the first switching window to the audio equipment corresponding to the first switching window; connecting the second audio channel with second audio equipment corresponding to the focus window, and connecting the first audio channel with first audio equipment; removing a mute state of the second audio channel;

the mute state of the first audio channel is released.

3. The display device according to claim 1, wherein the switching instruction is: switching the audio equipment corresponding to the focus window from the first type of audio equipment to the second type of audio equipment;

setting a first audio channel of the focus window to be in a mute state, and judging whether the audio output interface is connected with unused second-class audio equipment or not; the first audio channel is used for outputting the audio of the focus window to the audio equipment corresponding to the focus window;

when the audio output interface is connected with second type of unused audio equipment, connecting the first audio channel with the second type of unused audio equipment;

when the audio output interface is not connected with unused second-class audio equipment, randomly selecting first equipment in the second-class audio equipment, and determining a second switching window currently corresponding to the first equipment; setting a third audio channel of the second switching window to be in a mute state, wherein the third audio channel is used for outputting the audio of the second switching window to the audio equipment corresponding to the second switching window; connecting the third audio channel with a first type of audio device, and connecting the first audio channel with the first device; removing a mute state of the third audio channel;

the mute state of the first audio channel is released.

4. The display device of claim 1, wherein before performing the steps of switching the audio device corresponding to the focus window and adjusting the audio devices corresponding to the remaining windows, the controller is further configured to:

responding to a query instruction which is input by a user and indicates to query the audio equipment corresponding to the focus window, and controlling a display to display the use condition of the audio equipment so that the user can switch the audio equipment corresponding to the focus window into specific audio equipment according to the use condition of the audio equipment; the audio device usage includes all audio devices that can be used by the display device and a window currently corresponding to each audio device.

5. The display device according to claim 4, wherein the switching instruction is: switching the audio equipment corresponding to the focus window from the first type of audio equipment to specific second type of audio equipment;

setting a first audio channel of the focus window to be in a mute state, and judging the use state of the specific second-class audio equipment; the first audio channel is used for outputting the audio of the focus window to the audio equipment corresponding to the focus window;

connecting the first audio channel with the particular second class of audio device when the particular second class of audio device is in an unused state;

when the specific second-class audio equipment is in a use state, determining a specific switching window currently corresponding to the specific second-class audio equipment; setting a specific audio channel of the specific switching window to be in a mute state, wherein the specific audio channel is used for outputting the audio of the specific switching window to audio equipment corresponding to the specific switching window; connecting the specific audio channel with a first type of audio device, and connecting the first audio channel with the specific second type of audio device; removing a mute state of the particular audio channel;

the mute state of the first audio channel is released.

6. The display device of claim 4, wherein the user interface includes a first window and a second window;

when the audio equipment corresponding to the first window is first-class audio equipment, the audio equipment corresponding to the second window is second-class audio equipment;

and when the audio equipment corresponding to the first window is second-class audio equipment, the audio equipment corresponding to the second window is the first-class audio equipment or the rest second-class audio equipment.

7. The display device of claim 6, wherein prior to performing the step of controlling the display to display the plurality of windows, the controller is further configured to:

responding to a starting instruction of indicating to start the display equipment input by a user, and controlling a display to display a first window, wherein the first window is used for playing a first media resource;

in the performing the step of controlling the display to display the plurality of windows, the controller is further configured to:

responding to the display instruction, controlling a display to simultaneously display a first window and a second window, wherein the second window is used for displaying a media resource recommendation list so that a user can select a second media resource according to the media resource recommendation list;

and playing a second media resource in the second window.

8. The display device according to claim 6, wherein the switching instruction is: switching the audio equipment corresponding to the focus window from first equipment in second class of audio equipment to second equipment in second class of audio equipment; the focus window is a first window, and the second device is an audio device corresponding to the second window currently;

and connecting the audio channel of the first window with the second equipment, and connecting the audio channel of the second window with the first equipment.

9. The display device of claim 1, wherein the controller is further configured to:

controlling a display to display only the focus window in response to a control instruction input by a user to switch a user interface from a plurality of windows to a single window;

the audio channels of the remaining windows are set to a mute state.

10. An audio output method applied to a display device, the method comprising:

detecting a focus window in which a user-controlled focus is located;