WO2022083357A1

WO2022083357A1 - Display device and camera control method

Info

Publication number: WO2022083357A1
Application number: PCT/CN2021/118076
Authority: WO
Inventors: 孔祥键; 刘晋; 李东航; 祖荫鹏
Original assignee: 海信视像科技股份有限公司
Priority date: 2020-10-22
Filing date: 2021-09-13
Publication date: 2022-04-28
Also published as: CN114390190A; CN114390190B

Abstract

Provided are a display device and a camera control method; in response to receipt of a preset operation, said display device while continuing to display the screen of the playback content, displaying a target option used for adjusting the camera angle; in response to receiving an operation on said target option, while continuing to display the screen of the playback content, displaying an angle adjustment layer; when displaying said angle adjustment layer, receiving an inputted adjustment operation, and adjusting said camera angle according to said adjustment operation. Thus it is possible, without affecting a user viewing playback content, to provide the user with the ability to adjust the angle at which the camera is positioned.

Description

Display device and camera control method

This application claims the priority of the Chinese patent application filed on October 22, 2020, with the application number 202011142108.6 and the invention titled “Method for Activating Camera for Display Device and Monitoring Application”, the entire contents of which are incorporated herein by reference Applying.

technical field

The present application relates to the technical field of display devices, and in particular, to a display device and a method for controlling a camera.

Background technique

The display device can provide the user with playback pictures such as audio, video, and pictures. Today, display devices can not only provide users with the content of live TV programs received through data broadcasting, but also provide users with various application and service content such as online video and online games.

In order to further meet the personalized needs of users, the related technology configures a camera on the display device, obtains the local image data collected by the camera through the controller of the display device, processes and displays it, and realizes functions such as video chatting, photographing, and video recording on the display device. .

SUMMARY OF THE INVENTION

In the first aspect, the display device provided by this application includes:

A display, which is used to display the screen of the playback content;

Camera, used to collect image data;

Controller, configured as:

In response to receiving the preset operation, while continuing to display the screen of the playing content, display a target option for adjusting the angle of the camera;

In response to receiving the operation on the target option, while continuing to display the picture of the playing content, display an angle adjustment layer;

When the angle adjustment layer is displayed, an input adjustment operation is received, and the camera angle is adjusted according to the adjustment operation.

In a second aspect, the present application provides a display device, comprising:

A display, which is used to display the screen of the playback content;

a controller and an external device interface, and the camera is connected to the controller through the external device interface;

Controller, configured as:

In a third aspect, the present application further provides a method for adjusting the angle of a camera, which is applied to a display device, where the display device includes the camera, or the display device includes an external device interface, and the camera is connected to the external device interface through the external device interface. The display device, the method includes:

Display the screen of the playback content;

Description of drawings

In order to illustrate the technical solutions of the present application more clearly, the accompanying drawings that need to be used in the embodiments will be briefly introduced below. Other drawings can also be obtained from these drawings.

FIG. 1 is a schematic diagram of an operation scene between a display device and a control apparatus according to an exemplary embodiment of the present application;

FIG. 2 is a block diagram of a hardware configuration of a display device 200 according to an exemplary embodiment of the present application;

FIG. 3 is a block diagram of the hardware configuration of the control device 100 shown in the present application according to an exemplary embodiment;

FIG. 4 is a schematic diagram of software configuration in a display device 200 according to an exemplary embodiment of the present application;

FIG. 5 is a schematic diagram of displaying an icon control interface of an application in the display device 200 according to an exemplary embodiment of the present application;

6a is a schematic diagram of the appearance of a display device according to an exemplary embodiment of the present application;

FIG. 6b is a schematic diagram of an external camera of a display device according to an exemplary embodiment of the present application;

FIG. 7 is a user interface shown in some embodiments of the present application;

8(a) to 8(c) are user interfaces shown in some embodiments of the present application;

Figures 9(a) to 9(c) are user interfaces shown in some embodiments of the present application;

10 is a flowchart of a method for controlling a camera shown in some embodiments of the present application;

11a to 11f are schematic diagrams showing the angles of the cameras shown in some embodiments of the present application;

Figures 12(a) to 12(i) are user interfaces shown in some embodiments of the present application;

FIG. 13 is a flowchart of a method for controlling a camera according to some embodiments of the present application.

Detailed ways

Embodiments will be described in detail below, examples of which are illustrated in the accompanying drawings. Where the following description refers to the drawings, the same numerals in different drawings refer to the same or similar elements unless otherwise indicated. The implementations described in the following examples are not intended to represent all implementations consistent with this application. are merely exemplary of systems and methods consistent with some aspects of the present application as recited in the claims.

Based on the exemplary embodiments described in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the appended claims of this application. Furthermore, although the disclosures in this application have been presented in terms of illustrative example or instances, it should be understood that various aspects of this disclosure may also constitute a complete embodiment in isolation.

It should be noted that the brief description of the terms in the present application is only for the convenience of understanding the embodiments described below, rather than intended to limit the embodiments of the present application. Unless otherwise specified, these terms are to be understood according to their ordinary and ordinary meanings.

The terms "first", "second", "third", etc. in the description and claims of this application and the above drawings are used to distinguish similar or similar objects or entities, and are not necessarily meant to limit specific Order or precedence unless otherwise indicated. It should be understood that the terms so used are interchangeable under appropriate circumstances, eg, can be implemented in an order other than those given in the illustration or description of the embodiments of the present application.

Furthermore, the terms "comprising" and "having" and any variations thereof, are intended to cover but not exclusively include, for example, a product or device incorporating a series of components is not necessarily limited to those explicitly listed, but may include No other components are expressly listed or inherent to these products or devices.

The term "module" as used in this application refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic or combination of hardware or/and software code capable of performing the function associated with that element.

As used in this application, the term "remote control" refers to a component of an electronic device, such as the display device disclosed in this application, that can wirelessly control the electronic device, usually over a short distance. Generally, infrared and/or radio frequency (RF) signals and/or Bluetooth are used to connect with electronic devices, and functional modules such as WiFi, wireless USB, Bluetooth, and motion sensors may also be included. For example, a hand-held touch remote control replaces most of the physical built-in hard keys in a general remote control device with a user interface in a touch screen.

The term "gesture" used in this application refers to a user's behavior that is used by a user to express an expected thought, action, purpose/or result through an action such as a change of hand shape or hand movement.

FIG. 1 exemplarily shows a schematic diagram of an operation scenario between a display device and a control apparatus according to an embodiment. As shown in FIG. 1 , a user may operate the display apparatus 200 through the mobile terminal 300 and the control apparatus 100 .

In some embodiments, the control device 100 may be a remote controller, and the communication between the remote controller and the display device includes infrared protocol communication or Bluetooth protocol communication, and other short-range communication methods, etc., and controls the display device 200 by wireless or other wired methods. The user can control the display device 200 by inputting user instructions through keys on the remote control, voice input, control panel input, and the like. For example, the user can control the display device 200 by inputting corresponding control commands through the volume up/down key, channel control key, up/down/left/right movement keys, voice input key, menu key, power-on/off key, etc. on the remote control. function.

In some embodiments, mobile terminals, tablet computers, computers, notebook computers, and other smart devices may also be used to control the display device 200 . For example, the display device 200 is controlled using an application running on the smart device. The app can be configured to provide users with various controls in an intuitive user interface (UI) on the screen associated with the smart device.

In some embodiments, the mobile terminal 300 may install a software application with the display device 200 to implement connection communication through a network communication protocol, so as to achieve the purpose of one-to-one control operation and data communication. For example, a control command protocol can be established between the mobile terminal 300 and the display device 200, the remote control keyboard can be synchronized to the mobile terminal 300, and the function of controlling the display device 200 can be realized by controlling the user interface on the mobile terminal 300. The audio and video content displayed on the mobile terminal 300 may also be transmitted to the display device 200 to implement a synchronous display function.

As also shown in FIG. 1 , the display device 200 also performs data communication with the server 400 through various communication methods. The display device 200 may be allowed to communicate via local area network (LAN), wireless local area network (WLAN), and other networks. The server 400 may provide various contents and interactions to the display device 200 . For example, the display device 200 interacts by sending and receiving information, and electronic program guide (EPG), receiving software program updates, or accessing a remotely stored digital media library. The server 400 may be a cluster or multiple clusters, and may include one or more types of servers. Other network service contents such as video-on-demand and advertising services are provided through the server 400 .

The display device 200 may be a liquid crystal display, an OLED display, or a projection display device. The specific display device type, size and resolution are not limited. Those skilled in the art can understand that the display device 200 can make some changes in performance and configuration as required.

The display device 200 may additionally provide a smart IPTV function that provides computer-supported functions, including but not limited to, IPTV, smart TV, Internet Protocol TV (IPTV), and the like, in addition to the broadcast receiving TV function.

FIG. 2 exemplarily shows a block diagram of the hardware configuration of the display device 200 according to the exemplary embodiment.

The display device 200 includes a controller 250, a tuner 210, a communicator 220, a detector 230, an input/output interface 255, a display 275, an audio output interface 285, a memory 260, a power supply 290, a user interface 265, and external devices At least one of the interfaces 240 .

The display 275 includes a display screen component for presenting pictures, and a driving component for driving image display.

The display 275 is used to present a user-manipulated UI interface generated in the display device 200 and used to control the display device 200 .

In some embodiments, the display 275 may be a liquid crystal display, an OLED display, and a projection display, and may also be a projection device and projection screen.

The communicator 220 is a component for communicating with external devices or external servers according to various communication protocol types. For example, the communicator may include at least one of a Wifi chip, a Bluetooth communication protocol chip, a wired Ethernet communication protocol chip and other network communication protocol chips or a near field communication protocol chip, and an infrared receiver.

In some embodiments, the display device 200 may establish control signal and data signal transmission and reception between the communicator 220 and the external control device 100 or the content providing device.

The user interface 265 can be used to receive infrared control signals from the control device 100 (eg, an infrared remote control, etc.).

The detector 230 is a signal used by the display device 200 to collect the external environment or interact with the outside. The detector 230 includes a light receiver, a sensor for collecting ambient light intensity, and can adaptively display parameter changes and the like by collecting ambient light. For example, the detector 230 can also include an image collector, such as a camera, a camera, etc., which can be used to collect external environment scenes, as well as to collect user attributes or interactive gestures with the user, can adaptively change display parameters, and can also identify users. Gestures to enable interaction with users.

In some embodiments, detector 230 may also include a temperature sensor or the like, such as by sensing ambient temperature.

The input/output interface 255 is configured to enable data transfer between the controller 250 and other external devices or other controllers 250 . Such as receiving video signal data and audio signal data of external equipment, or command instruction data, etc.

The external device interface 240 may include, but is not limited to, any one or more interfaces such as a high-definition multimedia interface HDMI interface, an analog or data high-definition component input interface, a composite video input interface, a USB input interface, and an RGB port. It is also possible to form a composite input/output interface by a plurality of the above-mentioned interfaces.

The controller 250 controls the operation of the display device and responds to the user's operation through various software control programs stored in the memory. The controller 250 may control the overall operation of the display apparatus 200 . For example, in response to receiving a user command for selecting a UI object to be displayed on the display 275, the controller 250 may perform an operation related to the object selected by the user command.

In some embodiments, the object may be any of the selectable objects, such as a hyperlink or an icon. Operations related to the selected object, such as displaying operations linked to hyperlinked pages, documents, images, etc., or executing operations corresponding to the icon. The user command for selecting the UI object may be an input command through various input devices (eg, a mouse, a keyboard, a touchpad, etc.) connected to the display device 200 or a voice command corresponding to a voice spoken by the user.

As shown in FIG. 2 , the controller 250 includes a random access memory 251 (Random Access Memory, RAM), a read-only memory 252 (Read-Only Memory, ROM), a video processor 270, an audio processor 280, and other processors 253 (For example: at least one of graphics processing unit (Graphics Processing Unit, GPU), central processing unit 254 (Central Processing Unit, CPU), communication interface (Communication Interface), and communication bus 256 (Bus). Wherein, the communication bus Connect the parts.

In some embodiments, when the power-on signal is received, the power supply of the display device 200 starts to start, and the CPU executes the system start-up instruction in the ROM 252, and copies the temporary data of the operating system stored in the memory to the RAM 251, so as to facilitate startup or operation operating system. After the startup of the operating system is completed, the CPU copies the temporary data of various application programs in the memory to the RAM 251, so as to facilitate starting or running various application programs.

In some embodiments, the user may input user commands on a graphical user interface (GUI) displayed on the display 275, and the user input interface receives the user input commands through the graphical user interface (GUI). Alternatively, the user may input a user command by inputting a specific sound or gesture, and the user input interface recognizes the sound or gesture through a sensor to receive the user input command.

"User interface" is a medium interface for interaction and information exchange between application programs or operating systems and users, which realizes the conversion between the internal form of information and the form acceptable to users. The commonly used form of user interface is Graphical User Interface (GUI), which refers to a user interface related to computer operations displayed in a graphical manner. It can be an icon, window, control and other interface elements displayed on the display screen of the electronic device, wherein the control can include icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, Widgets, etc. visual interface elements.

FIG. 3 exemplarily shows a block diagram of the configuration of the control device 100 according to an exemplary embodiment. As shown in FIG. 3 , the control device 100 includes at least one of a controller 110 , a communication interface 130 , a user input/output interface, a memory, and a power supply.

In some embodiments, a system may include a kernel (Kernel), a command parser (shell), a file system, and applications. Together, the kernel, shell, and file system make up the basic operating system structures that allow users to manage files, run programs, and use the system. After power-on, the kernel starts, activates the kernel space, abstracts hardware, initializes hardware parameters, etc., runs and maintains virtual memory, scheduler, signals and inter-process communication (IPC). After the kernel starts, the shell and user applications are loaded. An application is compiled into machine code after startup, forming a process.

Referring to FIG. 4 , in some embodiments, the system is divided into four layers, from top to bottom, they are an application layer (referred to as “application layer”), an application framework layer (referred to as “framework layer”) ”), the Android runtime and the system library layer (referred to as the “system runtime layer”), and the kernel layer.

In some embodiments, at least one application program runs in the application program layer, and these application programs may be a Window program, a system setting program, a clock program, a camera application, etc. built into the operating system; they may also be developed by a third party The application programs developed by the author, such as the Hijian program, the K song program, the magic mirror program, etc. During specific implementation, the application package in the application layer is not limited to the above examples, and may actually include other application packages, which is not limited in this embodiment of the present application.

The framework layer provides an application programming interface (API) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions. The application framework layer is equivalent to a processing center, which decides to let the applications in the application layer take action. The application program can access the resources in the system and obtain the services of the system during execution through the API interface.

As shown in FIG. 4 , the application framework layer in the embodiment of the present application includes managers (Managers), content providers (Content Provider), etc., wherein the manager includes at least one of the following modules: an activity manager (Activity Manager) uses Interacts with all activities running in the system; Location Manager is used to provide system services or applications with access to system location services; Package Manager is used to retrieve files currently installed on the device Various information related to the application package; Notification Manager (Notification Manager) is used to control the display and clearing of notification messages; Window Manager (Window Manager) is used to manage icons, windows, toolbars, wallpapers on the user interface and desktop widgets.

In some embodiments, the activity manager is used to: manage the life cycle of each application and the usual navigation and fallback functions, such as controlling the exit of the application (including switching the user interface currently displayed in the display window to the system desktop), opening the , back (including switching the currently displayed user interface in the display window to the upper-level user interface of the currently displayed user interface), and the like.

In some embodiments, the window manager is used to manage all window programs, such as obtaining the size of the display screen, judging whether there is a status bar, locking the screen, taking screenshots, and controlling the change of the display window (for example, reducing the display window to display, shaking display, twisting deformation display, etc.), etc.

In some embodiments, the system runtime layer provides support for the upper layer, that is, the framework layer. When the framework layer is used, the Android operating system will run the C/C++ library included in the system runtime layer to implement the functions to be implemented by the framework layer.

In some embodiments, the kernel layer is the layer between hardware and software. As shown in Figure 4, the kernel layer at least includes at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, touch sensors, pressure sensors, etc.), etc.

In some embodiments, the kernel layer further includes a power driver module for power management.

In some embodiments, software programs and/or modules corresponding to the software architecture in FIG. 4 are stored in the first memory or the second memory shown in FIG. 2 or FIG. 3 .

In some embodiments, taking the magic mirror application (photography application) as an example, when the remote control receiving device receives the input operation of the remote control, the corresponding hardware interrupt is sent to the kernel layer. The kernel layer processes the input operation into the original input event (including the value of the input operation, the timestamp of the input operation and other information). Raw input events are stored at the kernel layer. The application framework layer obtains the original input event from the kernel layer, identifies the control corresponding to the input event according to the current position of the focus, and regards the input operation as a confirmation operation, and the control corresponding to the confirmation operation is the control of the magic mirror application icon. The mirror application calls the interface of the application framework layer, starts the mirror application, and then starts the camera driver by calling the kernel layer to capture still images or videos through the camera.

In some embodiments, for a display device with a touch function, taking a split-screen operation as an example, the display device receives an input operation (such as a split-screen operation) performed by the user on the display screen, and the kernel layer can generate corresponding input operations according to the input operation. Enter an event and report the event to the application framework layer. The window mode (such as multi-window mode) and window position and size corresponding to the input operation are set by the activity manager of the application framework layer. The window management of the application framework layer draws the window according to the settings of the activity manager, and then sends the drawn window data to the display driver of the kernel layer, and the display driver displays the corresponding application interface in different display areas of the display screen.

In some embodiments, as shown in FIG. 5, the application layer contains at least one application that can display corresponding icon controls in the display, such as: live TV application icon control, video on demand application icon control, media center application Program icon controls, application center icon controls, game application icon controls, etc.

In some embodiments, the live TV application may provide live TV from different sources. For example, a live TV application may provide a TV signal using input from cable, over-the-air, satellite services, or other types of live TV services. And, the live TV application may display the video of the live TV signal on the display device 200 .

In some embodiments, a video-on-demand application may provide video from various storage sources. Unlike live TV applications, video-on-demand provides a display of video from certain storage sources. For example, video-on-demand can come from the server side of cloud storage, from local hard disk storage containing existing video programs.

In some embodiments, the media center application may provide various multimedia content playback applications. For example, a media center may provide services other than live TV or video-on-demand, where users can access various images or audio through a media center application.

In some embodiments, the application center may provide storage of various applications. An application can be a game, an application, or some other application that is related to a computer system or other device but can be run on a Smart TV. The application center can obtain these applications from various sources, store them in local storage, and then run them on the display device 200 .

In some embodiments, the display device itself includes a camera, ie the camera is built into the display device, as shown in Figure 6a.

In other embodiments, the display device itself does not include a camera, that is, the camera is connected to the controller 250 of the display device through the external device interface 240 on the display device, as shown in FIG. 6b .

In this embodiment of the present application, the camera, as a detector, may be built-in or externally connected to the display device. After the camera starts running, it can collect image data. The camera can be connected with the controller through the interface component, so as to send the collected image data to the controller for processing.

The camera application refers to the application that needs to access the camera, such as the "Magic Mirror" application, the "Hi See" application, the "K song" application, and so on. The camera application can start the camera through the camera control service (CameraService) in the system framework layer. For example, it sends a start command to the camera control service, and the camera control service controls the camera to turn on after receiving the start command. Since any camera application can activate the camera, it is easy for some applications to open the camera without the user's knowledge, resulting in user privacy leakage.

In order to solve this problem, the present application provides a display device. In the display device, the controller 250 is configured to: monitor whether an application activates the camera; when it is detected that an application activates the camera, determine whether the application is a foreground application or a first application. A predetermined application; if the application is a foreground application or the first predetermined application, the application is allowed to start the camera; if the application is not the foreground application or the first predetermined application, the application is refused to start the camera.

In some embodiments, the controller pre-generates a first application set, and the first application set includes application information corresponding to a first predetermined application, where the first predetermined application is an application that the user allows to start the camera in the background. When the controller detects that an application starts the camera, it obtains the application information corresponding to the application, determines whether the application information corresponding to the application is consistent with the application information of the foreground application, and determines whether the application information corresponding to the application is included in the first application set If the application information corresponding to the application is consistent with the application information of the foreground application, the application is a foreground application; if the application information corresponding to the application is included in the first application set, the application is the first predetermined application. The application information corresponding to the application may include a package name and an application name of the application.

In some embodiments, the controller generates a second application set in advance, and the second application set includes application information corresponding to a second predetermined application, where the second predetermined application is an application for which the user prohibits the camera from being started in the background. In the case that the application that starts the camera is not the first predetermined application or the foreground application, after the controller rejects the application to start the camera, it determines whether the application is the second predetermined application according to the application information of the application and the saved second application set. If the application is not the second predetermined application, a warning message for prompting the user is displayed; and a first control and a second control are displayed, where the first control is used to trigger setting the application as the first predetermined application, and the second control is displayed. The control is used to trigger setting the application as a second predetermined application; if a user operation on the first control is received, the application information corresponding to the application is added to the first application set; if the second control is received the user operation, the application information corresponding to the application is added to the second application set.

FIG. 7 is a user interface including warning information, a first control and a second control shown in some embodiments of the present application, that is, a user interface displayed in response to determining that the application that activates the camera is not the second predetermined application. As shown in Figure 7, the warning message displayed on the interface is "Currently xx application is using the camera in the background, please confirm whether you know; Background use will be allowed by default; if not allowed, please click Prohibit, and the application will be prohibited from being used in the background in the future to prevent user privacy; the first control displayed is the control named "Allow", and the second control is specifically A control named "Prohibited".

In some embodiments, all the first predetermined applications may be presented in a visual application list in the user interface, and the application names of the first predetermined applications are explicitly displayed in the list, so that the user can centrally view all the first predetermined applications application.

Fig. 8(a) is a visualization application list including a first predetermined application shown in some embodiments of this application. As shown in Fig. 8(a), the first predetermined application includes "Magic Mirror" application, "AI fitness" Apps and Video Calling apps that can turn on the camera in the background. The user can set a new first predetermined application by operating the "Add Application" control, and can delete the already set first predetermined application from the list by operating the "Delete Application" control.

In some embodiments, all the second predetermined applications may be presented in a visual application list in the user interface, and the application names of the second predetermined applications are explicitly displayed in the list, so that the user can centrally view all the second predetermined applications application.

Fig. 8(b) is a visual application list including a second predetermined application shown in some embodiments of this application. As shown in Fig. 8(b), the second predetermined application exemplarily includes a "cloud*listening" application, "Quick*" apps and "One*cast" apps, which are prohibited from opening the camera in the background. The user can set a new second predetermined application by operating the "Add Application" control, and can delete the already set second predetermined application from the list by operating the "Delete Application" control.

Exemplarily, when the "K song" application is running in the foreground, if it is detected that the "Magic Mirror" application starts the camera through the camera control service, it can be determined that the "Magic Mirror" application is the first predetermined set according to the saved first application set. application, therefore, according to the launch command sent by the "Magic Mirror" application, the camera is turned on. If it is detected that the "Weibo" application starts the camera through the camera control service, it can be determined that the "Weibo" application is not the first predetermined application according to the saved first application set, and the startup instruction sent by the "Weibo" application is ignored, and no Turn on the camera; according to the saved second application set, it can be determined that the "microblogging" application is not the second predetermined application, so a user interface including warning information, a first control and a second control is displayed, as shown in FIG. 7 . If it is detected that the "Cloud*Listening" application starts the camera through the camera control service, it can be determined that the "Cloud*Listening" application is not the first predetermined application according to the saved first application set, and the startup sent by the "Cloud*Listening" application is ignored. instruction, do not turn on the camera; according to the saved second application set, it can be determined that the "cloud*listening" application is the second predetermined application, so there is no need to display the user interface including the warning information, the first control and the second control.

In some embodiments, the list of visual applications presenting the first predetermined application and the list of visual applications presenting the second predetermined application may be contained in the same application list page, wherein the first predetermined application and the second predetermined application are marked as different status, so that the user can distinguish the first predetermined application, the second predetermined application and other applications according to the display status of the applications.

Fig. 8(c) is a visual application list showing all applications shown in some embodiments of the application. As shown in Fig. 8(c), the first predetermined application includes a "magic mirror" application, an "AI fitness" application and "Video calling" applications, these applications are marked as "allowed" status; the second predetermined applications include "cloud* listening" applications, "quick*" applications and "one* broadcast" applications, these applications are marked as "prohibited" status ; Other applications are marked as "not set", such as "Weibo" application, "Toutiao" application, "Encyclopedia" application, etc.

In some embodiments, the user can increase or decrease the first predetermined application and the second predetermined application in "Settings".

For example, the user can control the display device to display an interface including at least one application option through one or more preset operations, such as an interface including an application list, where the application list includes at least one application option; when displaying the user interface, the user can Through the first user operation, select the designated application option and set it as the first predetermined application, that is, add the application information corresponding to the designated application to the first application set, the user can select the designated application option through the second user operation, and set the application information corresponding to the designated application to the first application set. It is set as the second predetermined application, that is, the application information corresponding to the specified application is added to the second application set. The first user operation and the second user operation may be one user operation, or may be a series of user operations including multiple user operations.

9(a) to 9(c) are the application interface changing process shown in some embodiments of the present application. Among them, Fig. 9(a) is a user interface including all applications shown in some embodiments of the application. When the interface is displayed, the user can select the specified application option by moving the focus position, as shown in Fig. 9(a) Select application B; FIG. 9(b) is a user interface for setting application B shown in some embodiments of the present application, specifically, the user selects application B in FIG. 9(a) and displays it after determining interface, when the interface is displayed, the user can select the item "camera" for setting with the camera by moving the focus position; Another user interface, specifically the user interface displayed after the user selects the item "camera" in Figure 9(b) and confirms, when the interface is displayed, the user can select "Allow" or "Forbid" to apply B Set as the first predetermined application or the second predetermined application.

In some embodiments, the user may control the display device to display user interfaces of other applications except the first predetermined application and/or the second predetermined application through one or more preset operations; when displaying the user interface, the user may Through the first user operation, select the specified application and set it as the first predetermined application, that is, add the application information corresponding to the specified application to the first application set, the user can select the specified application through the second user operation, set it as the first application set. Determine as the second predetermined application, that is, add application information corresponding to the specified application to the second application set. In some embodiments, the controller runs a framework layer, and the framework layer includes a camera control service (Camera Service). When the application needs to start the camera, the application sends a start command to the camera control service, and the camera controller service receives the start command according to the start command. Control the camera to turn on.

In some embodiments, the camera control service monitors whether an application activates the camera according to whether it receives a camera activation instruction sent by the application. When receiving a camera startup instruction sent by an application, obtain application information of the application, such as the application package name or application name, and then determine whether the application is a foreground application according to the application information, and determine whether the application is a foreground application according to the saved first application set. for the first predetermined application. If the application is the first predetermined application or the foreground application, control the camera to turn on according to the startup instruction sent by the application; if the application is not the first predetermined application or the foreground application, the camera is not started according to the principle; and according to the saved second application The set judges whether the application is the second predetermined application; if the application is not the second predetermined application, the warning information is displayed, and the first control and the second control are displayed; if the application is the second predetermined application, the warning information is not displayed, The first and second controls are also not displayed.

According to the display device provided by the above embodiment, the present application also provides a method for controlling a camera. The method is applied to the display device provided by the above embodiment. As shown in FIG. 10 , the method may include:

Step 110, when it is detected that an application starts the camera, determine whether the application is a foreground application or a first predetermined application.

Step 120, if the application is the foreground application or the first predetermined application, allow the application to start the camera.

Step 130, if the application is not the foreground application or the first predetermined application, reject the application from starting the camera.

In some embodiments, after the application is rejected to start the camera, it further includes: judging whether the application is the second predetermined application; if the application is not the second predetermined application, displaying a warning message for prompting the user; and displaying a first control and a second control, where the first control is used to trigger setting of the application as a first predetermined application, and the second control is used to trigger setting of the application as a second predetermined application ; If a user operation on the first control is received, the application information corresponding to the application is added to the first application list; if a user operation on the second control is received, the application corresponding to the application is added. information is added to the second application list.

In some embodiments, determining whether the application is a foreground application or a first predetermined application includes: acquiring application information corresponding to the application; if the application information corresponding to the application is consistent with the application information of the foreground application, determining that the application corresponds to the application. The application is a foreground application; if the application information corresponding to the application is included in the first application set, it is determined that the application is the first predetermined application; the determining whether the displayed application is the second predetermined application includes: if the application The corresponding application information is included in the second application set, and it is determined that the application is the second predetermined application.

In some embodiments, the method further includes: in response to a preset user input, displaying a user interface including at least one application option; receiving a first user operation, and deleting the first application set or the second application set according to the first user operation Designated application information in the application set; or, receiving a second user operation, and adding the designated application information to the first application set or the second application set according to the second user operation.

In some embodiments, before determining whether the application is a foreground application or a first predetermined application when it is detected that the application starts the camera, the method further includes: monitoring whether the application starts the camera according to whether the camera control service receives an instruction for the application to start the camera Start the camera.

When the application that needs to turn on the camera is the foreground application or the first predetermined application, the camera can be turned on to perform corresponding image acquisition.

In some exemplary embodiments, in order to expand the acquisition range of the camera, the camera can be set as a camera with a rotatable angle. The angle at which the camera is located, including its tilt angle in the vertical direction or its angle in the horizontal direction, is related to the field of view of the camera.

The gimbal is a supporting device for installing and fixing the camera. It is divided into fixed gimbal and electric gimbal. Among them, the electric gimbal is suitable for large-scale scanning and shooting, and it can expand the field of view of the camera. The electric pan/tilt can be a horizontal rotating pan/tilt that can only rotate left and right, or an omnidirectional pan/tilt that can rotate left and right as well as up and down. Generally, two motors are installed in the omnidirectional pan/tilt head, which are used to drive the pan/tilt head to rotate in the horizontal direction and the vertical direction respectively, so as to change the angle of the camera. The camera installed on the electric PTZ is also called the PTZ camera. For a display device equipped with a PTZ camera, the display device controller is used to control the electric PTZ, so that the camera can shoot at multiple angles.

The limit angle that the camera can rotate in the horizontal direction and/or the vertical direction can be set by the user according to the needs. Exemplarily, the rotatable angle of the camera in the horizontal direction may range from 0° to 120°, where 0° and 120° are the corresponding limits of the two rotation directions (leftward and rightward) in the horizontal direction, respectively. Angle; the rotatable angle of the camera in the vertical direction can be from 0° to 180°, where 0° and 180° are respectively the limit angles corresponding to the two rotation directions (up and down) in the vertical direction.

Fig. 11a to Fig. 11f are schematic diagrams of the angle of the camera exemplarily shown in the application, wherein Fig. 11a exemplarily shows the state when the tilt angle of the camera in the vertical direction is 0°, Fig. 11b exemplarily shows The state when the tilt angle of the camera in the vertical direction is 90°, Fig. 11c exemplarily shows the state when the tilt angle of the camera in the vertical direction is 105°, and Fig. 11d exemplarily shows the state of the camera in the horizontal direction. The state when the horizontal angle is 0°, Fig. 11e exemplarily shows the state when the horizontal angle of the camera in the horizontal direction is 60°, and Fig. 11f exemplarily shows the state when the horizontal angle of the camera in the horizontal direction is 120° status.

In some embodiments, after the camera is started, the controller automatically controls the camera to rotate to a pre-saved initial angle. In other words, the initial angle of the camera refers to the target angle to which the camera rotates after it is activated.

Regarding the setting of the initial angle, the user can input a suitable angle according to his usage habits and common needs when the specific setting interface is displayed, or adjust the camera to a suitable angle according to a predetermined operation and save it as the initial angle, so that after the camera is started, it will automatically Rotate to the saved initial angle, otherwise, the camera will automatically rotate to the factory default initial angle after startup, that is, the preset initial angle.

In the process of setting the initial angle, since a specific setting interface needs to be displayed, the user needs to operate the display device to close the current application or exit the foreground, in order to open the setting application, and operate and display the aforementioned specific setting interface, thus affecting the user to continue to watch the playback content without Beneficial for user experience.

Fig. 12(a) is a user interface shown in some embodiments of the present application. As shown in Fig. 12(a), the display device displays a system home page. The user can input preset operations in the form of keys, voice, gestures, etc., to trigger the display of method setting options for camera control. In response to the input preset operation, the display device can display the method setting options for camera control while continuing to display the screen of the playback content, or display a plurality of settings including target options while continuing to display the screen of the playback content options, where the target option is the method setting option for camera control.

In some embodiments, the display device displays a user interface including a first content display area and a second content display area in response to an input preset operation, and continues to display the screen of the currently playing content in the first content display area, and in the first content display area A plurality of setting options including target options are displayed in the second content display area. In this user interface, the user can trigger the display angle adjustment layer by operating the target option.

In some embodiments, the first content display area overlaps with the second content display area, and the second content display area is located on an upper layer of the first content display area. The second content display area may be translucent (transparency greater than a preset value), so that the part of the first content display area covered by the second content display area is displayed through the second content display area.

Fig. 12(b) is a user interface shown in some embodiments of the present application, which is specifically a user interface displayed after receiving an input preset operation when the user interface shown in Fig. 12(a) is displayed. As shown in FIG. 12(b), the user interface includes a first content display area 810 and a second content display area 820, and the second content display area 820 covers the upper layer of the first content display area 810; The first content display area 810 continues to display the original size, while the second content display area 820 displays a plurality of setting options, namely "More Settings" 821, "Image Mode" 822, "Sound Mode" 823, "Camera Angle" Settings" 824 and "Wired Network" 825. In addition, a control 826 is also displayed in the second content display area 820, and the user can drag out the hidden setting options by operating the control 826 to display. In the user interface shown in FIG. 12(b), the second content display area 820 is translucent, and further, the content in the overlapping area of the first content display area and the second content display area can be displayed through the second content area is displayed.

Fig. 12(c) is a user interface shown in some embodiments of the present application. As shown in Fig. 12(c), a video is displayed and played in full screen on a display device. Fig. 12(d) is another user interface shown in some embodiments of the present application, which is specifically a user interface displayed after receiving an input preset operation when the user interface shown in Fig. 12(c) is displayed. The difference is that in FIG. 12( d ), the picture of the video continues to be displayed in the first content display area 810 .

In some embodiments, the first content display area and the second content display area do not overlap, and the screen of the currently playing content is displayed in the first content display area in accordance with the size of the first content display area.

Fig. 12(e) is another user interface shown in some embodiments of the present application, which is specifically a user interface displayed after receiving an input preset operation when the user interface shown in Fig. 12(c) is displayed. The difference is that in the user interface shown in FIG. 12(e), the first content display area 810 and the second content display area 820 do not overlap, and the screen of the currently playing video is displayed in the first content display area 810 in a reduced size. Continue to display.

In the above embodiment, the second content display area is displayed below the user interface, so that the main screen displayed in the first content display area will not be covered, and the user will not be affected to continue watching the playing content. It should be understood that the second content display area may also be displayed on the left, right or upper edge of the user interface, which is not limited in this application.

In some embodiments, when a user interface including a target option as shown in FIG. 12(b), FIG. 12(d) or FIG. 12(e) is displayed, if a user operation on the target option is received, the display device responds According to the user operation, the angle adjustment layer is displayed while continuing to display the screen of the playing content; when the angle adjustment layer is displayed, the input adjustment operation is received, and the camera angle is adjusted according to the adjustment operation.

In some embodiments, the display device displays a user interface including a first content display area and a third content display area in response to receiving a user operation on the target option, and continues to display the screen of the currently playing content in the first content display area , and the third content display area serves as an angle adjustment layer, which displays method operation information for camera control.

In some embodiments, the first content display area and the third content display area overlap, and the third content display area is located on an upper layer of the first content display area. The third content display area may be translucent (transparency greater than a preset value), so that the part of the first content display area covered by the third content display area is displayed through the third content display area.

Fig. 12(f) is a user interface shown in the embodiment of the present application, which is specifically a user interface displayed after receiving a user operation on the target option 824 when the user interface shown in Fig. 12(d) is displayed. As shown in FIG. 12(f), the user interface includes a first content display area 810 and a third content display area 830, the first content display area 810 overlaps with the third content display area 830, and the third content display area 830 covers the The upper layer of the first content display area 810; the screen of the currently playing content continues to be displayed in the original size in the first content display area 810, and the third content display area 830 displays the method operation information indicating how to control the camera, specifically “by direction”. key to adjust the angle", "press the OK key to save the angle", "press the menu key to restore the angle". When the user interface is displayed, the user can input an adjustment operation according to the displayed operation information to adjust the angle of the camera.

In an exemplary embodiment, the video playback in the first content display area will not be interrupted, and at the same time when the third content display area is called up, the camera is activated to display a small window on the upper layer of the first content display area or The image captured by the camera is presented in the third content display area, so that the user's instruction to adjust the angle can be viewed in the small window in real time, so that the camera capture angle desired by the user can be adjusted more accurately. The small window can be an angle adjustment feedback window, which is used for real-time feedback of the image data collected by the camera after the user sends out the angle adjustment instruction.

In some embodiments, the first content display area and the third content display area do not overlap, and the screen of the currently playing content is displayed in the first content display area in accordance with the size of the first content display area.

Fig. 12(g) is a user interface shown in some embodiments of the present application, which is specifically a user interface displayed after receiving an input preset operation when the user interface shown in Fig. 12(e) is displayed. The difference is that in the user interface shown in FIG. 12(g), the first content display area 810 and the third content display area 830 do not overlap, and the screen of the currently playing video is displayed in the first content display area 810 in a reduced size. Continue to display. When the user interface is displayed, the user can adjust the angle of the camera through adjustment operations.

In the above-described embodiment, the angle adjustment layer (the third content display area is displayed below the user interface, so that the main picture displayed in the first content display area will not be covered, and the user will not be affected to continue watching the playback content. It should be understood that, The angle adjustment layer (the third content display area may also be displayed on the left, right or upper edge of the user interface, which is not limited in this application.

For example, FIG. 12(h) is a user interface shown in some embodiments of the present application, which is a schematic diagram of a method for implementing a camera control method when a camera-related application such as a “magic mirror” application is used. As shown in Figure 12(h), the angle adjustment layer is displayed on the upper layer of the camera preview screen, and the method operation information used for camera control is displayed in the angle adjustment layer, indicating that it can be used to “adjust the angle by pressing the arrow keys” and “press the OK key to save” Angle", "Press the Menu key to reset the angle".

When the camera is turned on, when the angle adjustment layer is called up, the user can see the adjusted result feedback directly on the display according to the input angle adjustment command because the content captured by the camera has been displayed on the user interface. Therefore, there is no need to Then call up the angle adjustment feedback window to feedback image data.

For another example, FIG. 12( i ) is a user interface shown in some embodiments of the present application, which is a schematic diagram of a method for implementing a camera control method when a “video call” application is used. As shown in Figure 12(i), an angle adjustment layer is displayed on the upper layer of the video call interface.

In some embodiments, in the display of the user interface including the angle adjustment layer, the user can operate the designated keys on the control device to control the rotation of the pan/tilt, so as to adjust the angle at which the camera is located. For example, the user can input adjustment operations for controlling the left and right rotation of the PTZ by operating the "left" and "right" direction keys on the control device. "Down" and "Up" arrow keys to input adjustment operations for controlling the down and up rotation of the gimbal. After the display device receives the aforementioned adjustment operation for the camera control method, in response to the received adjustment operation, the motor in the gimbal is controlled to rotate to drive the gimbal to rotate in the target adjustment direction indicated by the adjustment operation.

In some embodiments, the user can operate the "left" and "right" direction keys on the control device. Since the user is located on the opposite side of the display device, when the user issues the "left" command, In fact, the controller needs to translate into a "right" command to control the camera to turn to the right. When the user issues a "right" command, in fact, the controller needs to convert to a "left" command to control the camera to rotate to the left.

In some embodiments, the display device controls the camera to rotate to a target angle in response to receiving the input adjustment operation, wherein the target angle is determined according to the target adjustment direction, the camera adjustment step, and the current camera angle, for example, assuming the current camera angle It is 30° (horizontal), 60° (vertical), the target adjustment direction is "right", and the preset adjustment step is 10°/time, then it can be determined that the target angle of this adjustment is 40°. If the determined target angle exceeds the adjustable limit angle, confirm that the limit angle is the target angle.

In some embodiments, during the process of adjusting the initial angle of the camera, the display device controller periodically obtains the current angle of the camera, and determines whether the current angle of the camera reaches the limit angle corresponding to the target adjustment direction; when the current angle of the camera reaches the target adjustment When the limit angle corresponding to the direction is reached, the adjustment of the angle of the camera is terminated. In some embodiments, when the difference between the current angle of the camera and the limit angle corresponding to the target adjustment direction is less than or equal to a preset value, it is considered that the current angle of the camera has reached the limit angle.

During specific implementation, the display device controller controls the rotation of the pan/tilt motor through the rotateAroundAxisTo() interface of the camera control service. When rotating, the camera application will obtain the current camera angle through the getRotation() interface. Once it reaches the critical value in a certain direction, it will prompt the user that the rotation has reached the limit angle, that is, it has rotated to the edge of the PTZ camera.

In some embodiments, the user may input an operation indicating saving the angle (referred to as the saving operation for short), and the display device, in response to the input saving operation, saves the current angle of the camera at the initial angle by calling the Record() interface of the camera control service. For example, the user can input a save operation by pressing the OK key on the remote control. It should be understood that the user may also input the save operation in other input forms, such as voice input, gesture input and so on.

In some embodiments, the user may input an operation indicating a reset angle (referred to as a reset operation for short), and the display device resets the saved initial angle to a preset initial angle in response to the input reset operation. For example, the user can enter a reset operation by pressing the menu key on the remote control. It should be understood that the user may also input the reset operation in other input forms, such as voice input, gesture input and so on.

According to the above-mentioned embodiment of the display device, the present application also provides a method for controlling a camera. The method is applied to the display device shown in FIG. 1-FIG. 12(i). As shown in FIG. 13 , the method may include the following steps:

Step 910, displaying a screen of playing content.

Step 920, in response to receiving the preset operation, while continuing to display the screen of the playing content, display the method target option for camera control.

In some embodiments, the displaying a method target option for camera control while continuing to display the screen of the playing content includes: displaying a user interface including a first content display area and a second content display area, wherein the The first content display area is used for continuing to display the picture of the playing content, and the second content display area is used for displaying a plurality of setting options including the target option.

In some embodiments, the first content display area and the second content display area overlap, and the second content display area is located on an upper layer of the first content display area. The first content display area and the second content display area may be in different layers, and the layer where the first content display area is located is in the lower layer of the layer in the second content display area.

In some embodiments, the transparency of the second content display area is greater than a preset value, so that the part of the first content display area covered by the second content display area is displayed through the second content display area .

In some embodiments, the first content display area and the second content display area do not overlap, and the screen of the playing content is adapted to the size of the second content display area in the first content display area displayed in.

In some embodiments, in response to receiving the operation on the target option, the method further comprises: revoking display of the target option.

Step 930, in response to receiving the operation on the target option, while continuing to display the screen of the playing content, display an angle adjustment layer.

In some embodiments, the displaying the angle adjustment layer while continuing to display the picture of the playing content includes: displaying a user interface including a first content display area and a third content display area, the first content displaying The third content display area is used as the angle adjustment layer to display the method operation information for camera control.

Step 940: When the angle adjustment layer is displayed, an input adjustment operation is received, and the camera angle is adjusted according to the adjustment operation.

In some embodiments, the adjusting the camera angle according to the adjustment operation includes: determining a target adjustment direction according to the adjustment operation; controlling the camera to rotate to a target angle, where the target angle is adjusted according to the target direction , the preset adjustment step, and the current angle of the camera.

It should be understood that the method for controlling the breathing light of the camera provided by the embodiment of the present application may include all the steps of configuring the controller of the display device of the present application. For relevant details, please refer to the description in the embodiment of the display device, which will not be repeated here.

In a specific implementation, the present invention also provides a computer storage medium, wherein the computer storage medium can store a program, and when the program is executed, it can include some or all of the steps in each embodiment of the camera control method provided by the present invention. The storage medium may be a magnetic disk, an optical disk, a read-only memory (English: read-only memory, abbreviated as: ROM) or a random access memory (English: random access memory, abbreviated as: RAM) and the like.

Those skilled in the art can clearly understand that the technology in the embodiments of the present invention can be implemented by means of software plus a necessary general hardware platform. Based on such understanding, the technical solutions in the embodiments of the present invention may be embodied in the form of software products in essence or the parts that make contributions to the prior art, and the computer software products may be stored in a storage medium, such as ROM/RAM , magnetic disk, optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of the present invention.

It is sufficient to refer to each other for the same and similar parts among the various embodiments in this specification. In particular, for the method embodiments, since they are basically similar to the display device embodiments, the description is relatively simple, and for related matters, please refer to the descriptions in the display device embodiments.

The embodiments of the present invention described above do not limit the protection scope of the present invention.

Claims

A display device, comprising:

A display, which is used to display the screen of the playback content;

Camera, used to collect image data;

Controller, configured as:

In response to receiving the preset operation, while continuing to display the screen of the playback content, display a method target option for camera control;

In response to receiving the operation on the target option, while continuing to display the picture of the playing content, display an angle adjustment layer;

When the angle adjustment layer is displayed, an input adjustment operation is received, and the camera angle is adjusted according to the adjustment operation.
The display device according to claim 1, wherein, while continuing to display the screen of the playing content, displaying a method target option for camera control, comprising:

Displaying a user interface including a first content display area and a second content display area, where the first content display area is used for continuing to display the screen of the playing content, and the second content display area is used for displaying the target option multiple setting options.
The display device according to claim 2, wherein the first content display area overlaps with the second content display area, and the second content display area is located on an upper layer of the first content display area.
The display device according to claim 3, wherein the transparency of the second content display area is greater than a preset value, so that a part of the first content display area covered by the second content display area is transparent The second content display area is displayed.
The display device according to claim 2, wherein the first content display area and the second content display area do not overlap, and the screen of the playing content is adapted to the size of the first content display area displayed in the first content display area.
The display device of claim 1, wherein in response to receiving the operation on the target option, further comprising: canceling the display of the target option.
The display device according to claim 1, wherein the displaying the angle adjustment layer while continuing to display the picture of the playing content, comprises:

Displaying a user interface including a first content display area and a third content display area, the first content display area is used for continuing to display the picture of the playing content, and the third content display area serves as the angle adjustment layer, wherein Displays method action information for camera control.
The display device according to claim 1, wherein the adjusting the camera angle according to the adjusting operation comprises:

Determine the target adjustment direction according to the adjustment operation;

The camera is controlled to rotate to a target angle, and the target angle is determined according to the target adjustment direction, the preset adjustment step, and the current angle of the camera.
A display device, comprising:

A display, which is used to display the screen of the playback content;

a controller and an external device interface, and the camera is connected to the controller through the external device interface;

Controller, configured as:

In response to receiving the preset operation, while continuing to display the screen of the playback content, display a method target option for camera control;

In response to receiving the operation on the target option, while continuing to display the picture of the playing content, display an angle adjustment layer;

When the angle adjustment layer is displayed, an input adjustment operation is received, and the camera angle is adjusted according to the adjustment operation.
A camera control method, characterized in that it is applied to a display device, where the display device includes the camera, or the display device includes an external device interface, and the camera is connected to the display device through the external device interface, The method includes:

Display the screen of the playback content;

In response to receiving the preset operation, while continuing to display the screen of the playback content, display a method target option for camera control;

In response to receiving the operation on the target option, while continuing to display the picture of the playing content, display an angle adjustment layer;

When the angle adjustment layer is displayed, an input adjustment operation is received, and the camera angle is adjusted according to the adjustment operation.