WO2021031608A1

WO2021031608A1 - Displaying method and device for locking state of camera physical switch

Info

Publication number: WO2021031608A1
Application number: PCT/CN2020/087803
Authority: WO
Inventors: 杨鲁明; 于文钦; 朱铄
Original assignee: 海信视像科技股份有限公司
Priority date: 2019-08-18
Filing date: 2020-04-29
Publication date: 2021-02-25
Also published as: CN112399220B; CN112399220A

Abstract

The present application provides a display device, comprising: a camera used for collecting image data and capable of lifting or lowering; a physical switch used for locking a lifting position of the camera; a display used for displaying a user interface and/or the image data; and a controller used for controlling the camera to lift by means of a lifting power apparatus in response to an instruction, input by a user, for instructing the camera to be turned on, and controlling the display to display a prompt message that the physical switch is in the locking state if it is detected that the camera is not lifted.

Description

Method for displaying locked state of camera physical switch and display device

This application claims the priority of a Chinese patent application filed with the Chinese Patent Office on August 18, 2019, the application number is 201910762189.0, and the invention title is "A method and device for determining the lock state of a TV camera physical switch", the entire content of which is incorporated by reference Incorporated in this application.

Technical field

The present application relates to the technical field of display devices, and in particular to a method for displaying the lock state of a camera physical switch and a display device.

Background technique

Currently, since display devices such as smart TVs can provide users with playback screens such as audio, video, and pictures, they have received extensive attention from users.

With the development of big data and artificial intelligence, users' demand for the functions of display devices is increasing day by day. For example, a user wants to watch a video program while having a video chat with at least one friend on a social platform. This requires the display device to display the video chat screen while displaying the video program screen; another example, the user wants to use education The app is used to watch educational content and video chat with at least one friend on the social platform. This requires the display device to display the video chat screen while displaying the educational application interface.

Therefore, there is an urgent need to provide a display device capable of realizing the above-mentioned functions, so as to present the user's screen on the display device.

Summary of the invention

Based on the above technical problems, the present application provides a method and display device for displaying the locked state of the physical switch of the camera.

The first aspect of the embodiments of the present application shows a display device, including:

The camera is configured to: collect image data, and be raised or lowered by the lifting power device;

A physical switch configured to: lock the lifting position of the camera;

A display configured to display a user interface, and/or the image data;

The controller is configured as:

In response to an instruction input by the user for instructing the camera to open, control the camera to rise through a lifting power device;

It is detected whether the camera is raised within a preset time, and if it is detected that the camera is not raised, the display is controlled to display a prompt message that the physical switch is in a locked state.

The second aspect of the embodiments of the present application shows a method for displaying the locked state of a camera physical switch, including:

Description of the drawings

In order to explain the embodiments of the present application or the implementation in the prior art more clearly, the following will briefly introduce the drawings that need to be used in the embodiments. Obviously, the drawings in the following description are only some of the present application. Embodiments, for those of ordinary skill in the art, without creative work, other drawings can be obtained from these drawings.

Fig. 1 exemplarily shows a schematic diagram of an operation scenario between a display device and a control device according to an embodiment;

FIG. 2 exemplarily shows a block diagram of the hardware configuration of the display device 200 according to the embodiment;

FIG. 3 exemplarily shows a hardware configuration block diagram of the control device 100 according to the embodiment;

FIG. 4 exemplarily shows a schematic diagram of the functional configuration of the display device 200 according to the embodiment;

FIG. 5a exemplarily shows a schematic diagram of software configuration in the display device 200 according to the embodiment;

FIG. 5b exemplarily shows a configuration diagram of an application program in the display device 200 according to the embodiment;

FIG. 6 exemplarily shows a schematic diagram of a user interface in the display device 200 according to the embodiment;

Fig. 7 is a front view of a camera according to an exemplary embodiment of the present application, which is completely raised from the back panel of the display device;

Fig. 8 is a top view of a physical switch according to an exemplary embodiment of the present application;

Fig. 9 is a schematic diagram showing an exemplary user interaction;

FIG. 10 is a flowchart of a method for displaying a locked state of a camera physical switch according to an optional embodiment;

Fig. 11 is a front view of a camera according to an exemplary embodiment of the present application partially raised from the back panel of the display device;

FIG. 12 is a photographed image when the camera according to an exemplary embodiment of the present application is partially raised from the back panel of the display device;

Fig. 13 is an example format of NV21 according to an exemplary embodiment of the present application.

detailed description

In order to make the purpose, implementation, and advantages of the exemplary embodiments of the present application clearer, the implementation manners in the exemplary embodiments of the present application will be described clearly and completely in conjunction with the accompanying drawings in the exemplary embodiments of the present application. Obviously The described exemplary embodiments are only a part of the embodiments of the present application, rather than all the embodiments.

Based on the exemplary embodiments shown in this application, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of this application. In addition, although the disclosure in this application is introduced in accordance with one or several exemplary examples, it should be understood that various aspects of these disclosures can also constitute a complete implementation separately.

It should be understood that the terms “first”, “second”, “third”, etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific sequence or Priority. It should be understood that the data used in this way can be interchanged under appropriate circumstances, for example, it can be implemented in an order other than those given in the illustration or description of the embodiments of the present application.

In addition, the terms "including" and "having" and any variations of them are intended to cover but not exclusively include. For example, a product or device including a series of components need not be limited to those clearly listed, but may include Other components that are not clearly listed or are inherent to these products or equipment.

The term "module" used in this application refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware or/and software code that can perform functions related to the element.

The term "remote control" used in this application refers to a component of an electronic device (such as the display device disclosed in this application), which can generally control the electronic device wirelessly within a short distance. In general, infrared and/or radio frequency (RF) signals and/or Bluetooth are used to connect to electronic devices, and may also include functional modules such as WiFi, wireless USB, Bluetooth, and motion sensors. For example, a handheld touch remote control uses a user interface in a touch screen to replace most of the physical built-in hard keys in general remote control devices.

The term "gesture" used in this application refers to a user's behavior through a change of hand shape or hand movement to express expected ideas, actions, goals, and/or results.

Fig. 1 exemplarily shows a schematic diagram of an operation scenario between a display device and a control device according to an embodiment. As shown in FIG. 1, the user can operate the display device 200 by controlling the device 100.

Wherein, the control device 100 may be a remote controller 100A, including infrared protocol communication or Bluetooth protocol communication, and other short-distance communication methods, etc., to control the display device 200 through wireless or other wired methods. The user can control the display device 200 by inputting user instructions through keys on the remote control, voice input, control panel input, etc. For example, the user can control the display device 200 by inputting corresponding control commands through the volume plus and minus keys, channel control keys, up/down/left/right movement keys, voice input keys, menu keys, and switch buttons on the remote control. Function.

The control device 100 may also be a smart device, such as a mobile terminal 100B, a tablet computer, a computer, a notebook computer, etc. For example, an application program running on a smart device is used to control the display device 200. The application can provide users with various controls through an intuitive user interface (UI) on the screen associated with the smart device.

For example, the mobile terminal 100B can install a software application with the display device 200, realize connection communication through a network communication protocol, and realize the purpose of one-to-one control operation and data communication. For example, the mobile terminal 100B can establish a control command protocol with the display device 200, synchronize the remote control keyboard to the mobile terminal 100B, and control the display device 200 by controlling the user interface on the mobile terminal 100B. The audio and video content displayed on the mobile terminal 100B can also be transmitted to the display device 200 to realize the synchronous display function.

As shown in FIG. 1, the display device 200 also performs data communication with the server 300 through a variety of communication methods. The display device 200 may be allowed to communicate through a local area network (LAN), a wireless local area network (WLAN), and other networks. The server 300 may provide various contents and interactions to the display device 200. For example, the display device 200 can receive software program updates or access a remotely stored digital media library by sending and receiving information and interacting with an electronic program guide (EPG). The server 300 may be a group or multiple groups, and may be one or more types of servers. The server 300 provides other network service content such as video on demand and advertising services.

The display device 200 may be a liquid crystal display, an OLED display, or a projection display device. The specific display device type, size, resolution, etc. are not limited. Those skilled in the art can understand that the display device 200 can be changed in performance and configuration as required.

In addition to providing the broadcast receiving TV function, the display device 200 may additionally provide a smart network TV function that provides a computer support function. Examples include Internet TV, display devices, Internet Protocol Television (IPTV) and so on.

As shown in Figure 1, the display device may be connected or provided with a camera, which is used to present the picture captured by the camera on the display interface of the display device or other display devices to realize interactive chats between users. Specifically, the picture captured by the camera can be displayed on the display device in full screen, half screen, or in any optional area.

As a connection method in an exemplary embodiment, the camera is connected to the display rear case through a connecting plate, and is fixedly installed on the upper middle of the display rear case. As an installable method, it can be fixedly installed on the display rear case. Any position can ensure that the image acquisition area is not blocked by the rear shell. For example, the image acquisition area and the display device have the same orientation.

As another connection method in an exemplary embodiment, the camera can be connected to the display rear shell through a connecting plate or other conceivable connectors. A lifting motor is installed on the connector. When the user wants to use the camera or When there is an application that needs to use the camera, it is raised above the display. When the camera is not needed, it can be embedded behind the back shell to protect the camera from damage.

As an embodiment, the camera used in this application may have 16 million pixels to achieve the purpose of ultra-high-definition display. In actual use, a camera with higher or lower than 16 million pixels can also be used.

When a camera is installed on the display device, the content displayed in different application scenarios of the display device can be merged in many different ways, so as to achieve functions that cannot be achieved by traditional display devices.

Exemplarily, the user can have a video chat with at least one other user while watching a video program. The presentation of the video program can be used as the background picture, and the video chat window is displayed on the background picture. Visually, you can call this function "watch and chat".

In an exemplary embodiment, in the scenario of "watching while chatting", while watching live video or online video, at least one video chat is performed across terminals.

In other embodiments, the user can video chat with at least one other user while entering the education application for learning. For example, students can realize remote interaction with teachers while learning content in educational applications. Visually, you can call this function "learning and chatting".

In other embodiments, when the user is playing a card game, a video chat is conducted with players entering the game. For example, when a player enters a game application to participate in a game, it can realize remote interaction with other players. Visually, you can call this function "watch and play".

In an exemplary embodiment, the game scene is integrated with the video picture, and the portrait in the video picture is cut out and displayed on the game picture, thereby improving user experience.

In an exemplary embodiment, in somatosensory games (such as ball games, boxing games, running games, dancing games, etc.), human body postures and movements are acquired through a camera, limb detection and tracking, and key point data of human bones Detection, and then fusion with the animation in the game, to achieve games such as sports, dance and other scenes.

In other embodiments, the user can interact with at least one other user in video and voice in the K song application. Visually, you can call this function "watch and sing". Preferably, when at least one user enters the application in the chat scene, multiple users can jointly complete the recording of a song.

In other embodiments, the user can turn on the camera locally to obtain pictures and videos. If it is vivid, this function can be called "mirror".

In other examples, more functions can be added or the aforementioned functions can be reduced. This application does not specifically limit the function of the display device.

FIG. 2 exemplarily shows a block diagram of the hardware configuration of the display device 200 according to an exemplary embodiment. As shown in FIG. 2, the display device 200 may include a tuner and demodulator 220, a communicator 230, a detector 240, an external device interface 250, a controller 210, a memory 290, a user input interface, a video processor 260-1, and audio Processor 260-2, display 280, audio output interface 272, power supply.

The tuner and demodulator 220, which receives broadcast and television signals through wired or wireless means, can perform modulation and demodulation processing such as amplification, mixing and resonance, and is used to demodulate the television channel selected by the user from multiple wireless or cable broadcast and television signals The audio and video signals carried in the frequency, and additional information (such as EPG data signals).

The tuner and demodulator 220 can be selected by the user and controlled by the controller 210 to respond to the TV channel frequency selected by the user and the TV signal carried by the frequency.

The tuner and demodulator 220 can receive signals in many ways according to different TV signal broadcasting systems, such as terrestrial broadcasting, cable broadcasting, satellite broadcasting, or Internet broadcasting; and according to different modulation types, it can be digital modulation or alternatively. Analog modulation method; and according to different types of received TV signals, analog and digital signals can be demodulated.

In some other exemplary embodiments, the tuner demodulator 220 may also be in an external device, such as an external set-top box. In this way, the set-top box outputs TV audio and video signals through modulation and demodulation, and inputs them to the display device 200 through the external device interface 250.

The communicator 230 is a component for communicating with external devices or external servers according to various communication protocol types. For example, the communicator 230 may include a WIFI module 231, a Bluetooth communication protocol module 232, a wired Ethernet communication protocol module 233 and other network communication protocol modules or near field communication protocol modules.

The display device 200 may establish a control signal and a data signal connection with an external control device or content providing device through the communicator 230. For example, the communicator may receive the control signal of the remote controller 100 according to the control of the controller.

The detector 240 is a component of the display device 200 for collecting signals from the external environment or interacting with the outside. The detector 240 may include a light receiver 242, a sensor used to collect the intensity of ambient light, which can adaptively display parameter changes by collecting ambient light, etc.; it may also include an image collector 241, such as a camera, camera, etc., which can be used to collect external Environmental scenes, as well as gestures used to collect user attributes or interact with users, can adaptively change display parameters, and can also recognize user gestures to achieve the function of interaction with users.

In some exemplary embodiments, the image collector 241 is a camera. When the camera is in an open state, it can collect image data around the display device and display the collected image data on the display. The user can trigger the instruction to take a photo by voice, gesture, or remote control, and obtain one frame of data from the camera. And, the user can trigger the instruction for recording through voice, gesture, or remote control, etc., to obtain multiple frames of data from the camera.

In some exemplary embodiments, the controller controls the raising or lowering of the camera through a lifting power device. For example, the lifting power device may be a motor.

In some other exemplary embodiments, the detector 240 may further include a temperature sensor. For example, by sensing the ambient temperature, the display device 200 may adaptively adjust the display color temperature of the image. Exemplarily, when the temperature is relatively high, the color temperature of the display device 200 can be adjusted to be relatively cool; when the temperature is relatively low, the color temperature of the display device 200 can be adjusted to be relatively warm.

In some other exemplary embodiments, the detector 240 may also include a sound collector, such as a microphone, which may be used to receive the user's voice, including the voice signal of the user's control instruction for controlling the display device 200, or to collect environmental sound for Recognizing the environmental scene type, the display device 200 can adapt to the environmental noise.

The external device interface 250 provides a component for the controller 210 to control data transmission between the display device 200 and other external devices. The external device interface can be connected to external devices such as set-top boxes, game devices, notebook computers, etc. in a wired/wireless manner, and can receive external devices such as video signals (such as moving images), audio signals (such as music), and additional information (such as EPG). ) And other data.

Among them, the external device interface 250 may include: a high-definition multimedia interface (HDMI) terminal 251, a composite video blanking synchronization (CVBS) terminal 252, an analog or digital component terminal 253, a universal serial bus (USB) terminal 254, red, green, and blue ( RGB) terminal (not shown in the figure) and any one or more.

The controller 210 controls the work of the display device 200 and responds to user operations by running various software control programs (such as an operating system and various application programs) stored on the memory 290.

As shown in FIG. 2, the controller 210 includes a random access memory RAM 213, a read only memory ROM 214, a graphics processor 216, a CPU processor 212, a communication interface 218, and a communication bus. Among them, RAM213 and ROM214, graphics processor 216, CPU processor 212, and communication interface 218 are connected by a bus.

ROM213, used to store various system startup instructions. For example, when the power-on signal is received, the power of the display device 200 starts to start, and the CPU processor 212 runs the system start-up instruction in the ROM, and copies the operating system stored in the memory 290 to the RAM 214 to start the start-up operating system. After the operating system is started up, the CPU processor 212 copies various application programs in the memory 290 to the RAM 214, and then starts to run and start various application programs.

The graphics processor 216 is used to generate various graphics objects, such as icons, operation menus, and user input instructions to display graphics. Including an arithmetic unit, which performs operations by receiving various interactive commands input by the user, and displays various objects according to display attributes. As well as including a renderer, various objects obtained based on the arithmetic unit are generated, and the rendering result is displayed on the display 280.

The CPU processor 212 is configured to execute operating system and application program instructions stored in the memory 290. And according to receiving various interactive instructions input from the outside, to execute various applications, data and content, so as to finally display and play various audio and video content.

In some exemplary embodiments, the CPU processor 212 may include multiple processors. The multiple processors may include one main processor and multiple or one sub-processors. The main processor is used to perform some operations of the display device 200 in the pre-power-on mode, and/or to display images in the normal mode. Multiple or one sub-processor, used to perform an operation in the standby mode and other states.

The communication interface may include the first interface 218-1 to the nth interface 218-n. These interfaces may be network interfaces connected to external devices via a network.

The controller 210 may control the overall operation of the display device 200. For example, in response to receiving a user command for selecting a UI object to be displayed on the display 280, the controller 210 may perform an operation related to the object selected by the user command.

Wherein, the object may be any one of the selectable objects, such as a hyperlink or an icon. Operations related to the selected object, for example: display operations connected to hyperlink pages, documents, images, etc., or perform operations corresponding to the icon. The user command for selecting the UI object may be a command input through various input devices (for example, a mouse, a keyboard, a touch pad, etc.) connected to the display device 200 or a voice command corresponding to the voice spoken by the user.

The memory 290 includes storing various software modules for driving and controlling the display device 200. For example, various software modules stored in the memory 290 include: a basic module, a detection module, a communication module, a display control module, a browser module, and various service modules.

Among them, the basic module is the underlying software module used for signal communication between various hardware in the display device 200 and sending processing and control signals to the upper module. The detection module is a management module used to collect various information from various sensors or user input interfaces, and perform digital-to-analog conversion and analysis management.

For example: the voice recognition module includes a voice analysis module and a voice command database module. The display control module is a module for controlling the display 280 to display image content, and can be used to play information such as multimedia image content and UI interfaces. The communication module is a module used for control and data communication with external devices. The browser module is a module used to perform data communication between browsing servers. The service module is a module used to provide various services and various applications.

At the same time, the memory 290 is also used to store and receive external data and user data, images of various items in various user interfaces, and visual effect diagrams of focus objects.

The user input interface is used to send a user's input signal to the controller 210, or to transmit a signal output from the controller to the user. Exemplarily, the control device (such as a mobile terminal or a remote control) may send input signals input by the user, such as a power switch signal, a channel selection signal, and a volume adjustment signal, to the user input interface, and then the user input interface forwards the input signal to the controller; Alternatively, the control device may receive output signals such as audio, video, or data output from the user input interface processed by the controller, and display the received output signal or output the received output signal as audio or vibration.

In some embodiments, the user may input a user command through a graphical user interface (GUI) displayed on the display 280, and the user input interface receives the user input command through the graphical user interface (GUI). Alternatively, the user can input a user command by inputting a specific sound or gesture, and the user input interface recognizes the sound or gesture through the sensor to receive the user input command.

The video processor 260-1 is used to receive video signals, and perform video data processing such as decompression, decoding, scaling, noise reduction, frame rate conversion, resolution conversion, and image synthesis according to the standard codec protocol of the input signal. The video signal displayed or played directly on the display 280.

Illustratively, the video processor 260-1 includes a demultiplexing module, a video decoding module, an image synthesis module, a frame rate conversion module, a display formatting module, and the like.

Among them, the demultiplexing module is used to demultiplex the input audio and video data stream. For example, if MPEG-2 is input, the demultiplexing module will demultiplex into a video signal and an audio signal.

The video decoding module is used to process the demultiplexed video signal, including decoding and scaling.

An image synthesis module, such as an image synthesizer, is used to superimpose and mix the GUI signal generated by the graphics generator with the zoomed video image according to user input or itself, to generate an image signal for display.

Frame rate conversion module, used to convert the frame rate of the input video, such as converting the frame rate of the input 24Hz, 25Hz, 30Hz, 60Hz video to the frame rate of 60Hz, 120Hz or 240Hz, where the input frame rate can be compared with the source The video stream is related, and the output frame rate can be related to the update rate of the display. The input has a usual format, such as frame insertion.

The display formatting module is used to change the signal output by the frame rate conversion module into a signal that conforms to a display format such as a display, such as format conversion of the signal output by the frame rate conversion module to output RGB data signals.

The display 280 is used to receive the image signal input from the video processor 260-1, display video content and images, and a menu control interface. The display 280 includes a display component for presenting a picture and a driving component for driving image display. The displayed video content can be from the video in the broadcast signal received by the tuner and demodulator 220, or from the video content input by the communicator or the external device interface. The display 220 simultaneously displays a user manipulation interface UI generated in the display device 200 and used to control the display device 200.

And, depending on the type of the display 280, it also includes a driving component for driving the display. Alternatively, if the display 280 is a projection display, it may also include a projection device and a projection screen.

The audio processor 260-2 is used to receive audio signals, and perform decompression and decoding according to the standard codec protocol of the input signal, as well as audio data processing such as noise reduction, digital-to-analog conversion, and amplification processing, and the result can be in the speaker 272 The audio signal to be played.

The audio output interface 270 is used to receive the audio signal output by the audio processor 260-2 under the control of the controller 210. The audio output interface may include a speaker 272 or output to an external audio output terminal 274 of a generator of an external device, such as : External audio terminal or headphone output terminal, etc.

In some other exemplary embodiments, the video processor 260-1 may include one or more chips. The audio processor 260-2 may also include one or more chips.

And, in some other exemplary embodiments, the video processor 260-1 and the audio processor 260-2 may be separate chips, or they may be integrated with the controller 210 in one or more chips.

The power supply is used to provide power supply support for the display device 200 with power input from an external power supply under the control of the controller 210. The power supply may include a built-in power supply circuit installed inside the display device 200, or may be a power supply installed outside the display device 200, such as a power interface that provides an external power supply in the display device 200.

Fig. 3 exemplarily shows a configuration block diagram of the control device 100 according to an exemplary embodiment. As shown in FIG. 3, the control device 100 includes a controller 110, a communicator 130, a user input/output interface 140, a memory 190, and a power supply 180.

FIG. 4 exemplarily shows a schematic diagram of the functional configuration of the display device 200 according to an exemplary embodiment. As shown in FIG. 4, the memory 290 is used to store an operating system, application programs, content, user data, etc., and executes the system operation of driving the display device 200 and responds to various operations of the user under the control of the controller 210. The memory 290 may include volatile and/or non-volatile memory.

The memory 290 is specifically used to store the operating program that drives the controller 210 in the display device 200, and store various application programs built in the display device 200, various application programs downloaded by the user from an external device, and various application-related programs. Graphical user interface, and various objects related to the graphical user interface, user data information, and various internal data supporting applications. The memory 290 is used to store system software such as an operating system (OS) kernel, middleware, and applications, and to store input video data and audio data, and other user data.

The memory 290 is specifically used to store driver programs and related data such as the video processor 260-1 and the audio processor 260-2, the display 280, the communication interface 230, the tuner and demodulator 220, the detector 240, and the input/output interface.

In some embodiments, the memory 290 may store software and/or programs. The software programs used to represent an operating system (OS) include, for example, a kernel, middleware, application programming interface (API), and/or application programs. Exemplarily, the kernel may control or manage system resources, or functions implemented by other programs (such as the middleware, API, or application program), and the kernel may provide interfaces to allow middleware and APIs, or applications to access the controller , To achieve control or management of system resources.

For example, the memory 290 includes a broadcast receiving module 2901, a channel control module 2902, a volume control module 2903, an image control module 2904, a display control module 2905, an audio control module 2906, an external command recognition module 2907, a communication control module 2908, and an optical receiver Module 2909, power control module 2910, operating system 2911, and other application programs 2912, browser module, etc. The controller 210 executes various software programs in the memory 290, such as: broadcast and television signal reception and demodulation function, TV channel selection control function, volume selection control function, image control function, display control function, audio control function, external command Recognition function, communication control function, optical signal receiving function, power control function, software control platform supporting various functions, browser function and other applications.

Fig. 5a exemplarily shows a configuration block diagram of the software system in the display device 200 according to an exemplary embodiment.

As shown in Figure 5a, the operating system 2911 includes operating software for processing various basic system services and for implementing hardware-related tasks, and acts as a medium for data processing between application programs and hardware components.

In some embodiments, part of the operating system kernel may include a series of software to manage the hardware resources of the display device and provide services for other programs or software codes.

In some other embodiments, part of the operating system kernel may include one or more device drivers, and the device drivers may be a set of software codes in the operating system to help operate or control devices or hardware associated with the display device. The drive may contain code to manipulate video, audio, and/or other multimedia components. Examples include displays, cameras, Flash, WiFi, and audio drivers.

Among them, the accessibility module 2911-1 is used to modify or access the application program, so as to realize the accessibility of the application program and the operability of its display content.

The communication module 2911-2 is used to connect to other peripherals via related communication interfaces and communication networks.

The user interface module 2911-3 is used to provide objects that display the user interface for access by various applications, and can realize user operability.

The control application 2911-4 is used to control process management, including runtime applications.

The event transmission system 2914 can be implemented in the operating system 2911 or in the application 2912. In some embodiments, it is implemented in the operating system 2911 on the one hand, and implemented in the application program 2912 at the same time, for monitoring various user input events, and responding to the recognition results of various events or sub-events according to various events. And implement one or more sets of pre-defined operation procedures.

Among them, the event monitoring module 2914-1 is used to monitor input events or sub-events of the user input interface.

The event recognition module 2914-1 is used to input the definitions of various events to various user input interfaces, recognize various events or sub-events, and transmit them to the processing to execute the corresponding one or more groups of processing programs .

Among them, the event or sub-event refers to the input detected by one or more sensors in the display device 200 and the input of an external control device (such as the control device 100). Such as: various sub-events of voice input, gesture input sub-events of gesture recognition, and sub-events of remote control button command input of control devices. For example, one or more sub-events in the remote control include multiple forms, including but not limited to one or a combination of pressing up/down/left/right/, confirming keys, and pressing keys. And the operations of non-physical buttons, such as moving, pressing, and releasing.

The interface layout management module 2913, which directly or indirectly receives various user input events or sub-events monitored by the event transmission system 2914, is used to update the layout of the user interface, including but not limited to the position of each control or sub-control in the interface, and the container The size, position, level, etc. of the interface are related to various execution operations.

As shown in FIG. 5b, the application layer 2912 includes various application programs that can be executed on the display device 200. The application program may include but is not limited to one or more application programs, such as: live TV application, video on demand application, media center application, application center, game application, etc.

Live TV applications can provide live TV through different sources. For example, a live TV application may use input from cable TV, wireless broadcasting, satellite services, or other types of live TV services to provide TV signals. And, the live TV application can display the video of the live TV signal on the display device 200.

Video-on-demand applications can provide videos from different storage sources. Unlike live TV applications, VOD provides video display from certain storage sources. For example, the video on demand can come from the server side of cloud storage, and from the local hard disk storage that contains the stored video programs.

Media center applications can provide various multimedia content playback applications. For example, the media center can provide services that are different from live TV or video on demand, and users can access various images or audio through the media center application.

Application center, can provide storage of various applications. The application program may be a game, an application program, or some other application program that is related to a computer system or other device but can be run on a display device. The application center can obtain these applications from different sources, store them in the local storage, and then run on the display device 200.

Fig. 6 exemplarily shows a schematic diagram of a user interface in the display device 200 according to an exemplary embodiment. As shown in Fig. 6, the user interface includes multiple view display areas, for example, a first view display area 201 and a play screen 202, where the play screen includes one or more different items laid out. And the user interface also includes a selector indicating that the item is selected, and the position of the selector can be moved through user input to change the selection of different items.

It should be noted that multiple view display areas can present display screens of different levels. For example, the first view display area can present system-level item content (eg, current attributes, etc.), and the second view display area can present application-level item content (eg, webpage video, VOD display, application screen, etc.).

In an exemplary embodiment, the presentation of different view display areas has different priorities, and the display priorities of the view display areas are different between view display areas with different priorities. For example, the priority of the system layer is higher than that of the application layer. When the user uses the acquisition selector and screen switching in the application layer, the screen display in the view display area of the system layer is not blocked; and the application layer is enabled according to the user's choice When the size and position of the view display area change, the size and position of the view display area of the system layer will not be affected.

The same level of display screen can also be presented. At this time, the selector can switch between the display area of the first view and the display area of the second view, and when the size and position of the display area of the first view change, the second view The size and position of the display area can be changed at any time.

In order to increase the social function of the display device, this application exemplarily shows a social TV, which adopts a self-contained camera to expand the use of the social TV. Generally, the camera is hidden in the back panel of the social TV and has an automatic lifting function. When the camera 1 is used, the camera 1 is raised, as shown in Figure 7; when the camera 1 is not used, the camera 1 is automatically lowered.

In order to protect user privacy and prevent the camera from being raised in the case of misoperation to take pictures of the surrounding environment of the social TV, an independent physical switch 2 that is not controlled by the main control board of the social TV is installed on the social TV, as shown in Figure 8. The physical switch 2 can lock the camera, and the camera 1 cannot be raised after being locked. The setting of the physical switch 2 can prevent the camera from shooting the surrounding environment of social TV without the user's will due to misoperation. The physical switch with anti-privacy function is set on the social TV, which can ensure that the camera will not take pictures of the surrounding environment under misoperation. However, the social TV cannot know the physical switch state, including the locked state or the open state. At this time, when the physical switch is in the locked state, the camera cannot capture normal images.

In order to solve the technical problems existing in the prior art, the first aspect of the embodiments of the present application shows a display device, and the display device includes:

The camera is configured to collect image data, and to be raised or lowered by the lifting power device. The camera may be the image collector 241 in FIG. 2.

The physical switch is configured to lock the lifting position of the camera. When the physical switch is in the locked state, the camera cannot be raised by the lifting power device. When the physical switch is on, the camera can be raised by the lifting power device , So that the camera can collect normal image data.

A display configured as: a user interface, and/or the display image data;

The controller is configured to: in response to an instruction input by a user for instructing the camera to turn on, control the camera to rise through a lifting power device;

Detect whether the camera is raised within a preset time, and if it is detected that the camera is not raised, control the display to display a prompt message indicating that the physical switch is in the locked state. For example, the physical switch is in the locked state. The prompt message is shown in Figure 9.

The display is configured to display a prompt message that the physical switch is in a locked state.

In the embodiment of this application, when the user inputs the camera opening instruction, if the camera does not rise, at this time, the user needs to be reminded how to make the camera rise normally, as shown in Figure 9, the physical switch in Figure 9 is in a locked state The reminder message includes a reminder text and a reminder operation schematic diagram. Specifically, the reminder text indicates that the camera may be locked or blocked, and it is recommended to unlock it as shown. The reminding operation diagram is an operation method image of how to turn on the physical switch. In this way, the user can be prompted to turn on the physical switch.

In some embodiments, the controller is configured to perform the following steps to detect whether the camera is up within a preset time, as shown in FIG. 10, including: S100. Preset in the preview image acquired by the camera Time frame image;

In this embodiment of the application, the display device adopts the Android system, which provides a callback interface for the camera, which can return the preview image taken by the camera.

It should be noted that if the physical switch is turned on, it will take a certain time for the camera to fully rise from the back panel of the display device. During the raising process, part of the camera will be blocked by the frame of the display device, as shown in Figure 11, the camera in Figure 11 Partially occluded, the dotted line is the position where the camera is fully raised, so the upper part of the image taken by the camera will appear as a normal image taken by the upper part, and the lower part is an abnormal image taken due to the frame occlusion, as shown in Figure 12. . As the camera continues to rise, the images captured by the camera appear normal images from top to bottom. When the camera is fully raised, all the images captured by the camera are normal images, and the normal images refer to real images of the surrounding environment of the display device. If the physical switch is in the locked state, the camera cannot be raised, and the images captured by the camera at this time are all abnormal images.

Therefore, when the physical switch is in the locked state, the image captured by the camera in the embodiment of the present application has all the characteristics of abnormal images, and the frame images in the preview image are analyzed to determine the state of the physical switch. If the physical switch is in the locked state, the camera can only capture frame images that are all abnormal images. If the physical switch is in the on state, the camera can capture frame images that are part of normal images or all normal images.

In order to save the time for determining the state of the physical switch, the embodiment of the present application uses the frame image of the preset time in the preview image to analyze and determine the state of the physical switch, instead of using the frame image captured after the camera is fully raised. It takes about 1.5s for the camera to be fully raised from the social TV. The preset time can be set to 0.5s for the frame image taken when the camera part is not blocked by the frame, which can save the time for determining the physical switch state.

However, the frame image is not the first frame image in the preview image captured by the camera after receiving the camera opening instruction. The reason is that the camera cannot be raised after receiving the camera opening instruction, and the first frame image may be all wrong. Normal images have no analytical value.

S200: If the frame image is a locked image, it is determined that the physical switch is in a locked state. The locked image refers to an image acquired by the camera in the locked state of the physical switch, and the physical switch in the locked state indicates that the camera is not raised. It should be noted that the frame image taken by the camera when the physical switch is in the locked state is different from the frame image taken by the camera when the physical switch is on, because when the physical switch is on, the camera can rise and the captured image may be Some are normal images and some are abnormal images. When the physical switch is locked, the camera cannot be raised and the captured images are all abnormal images.

In an exemplary embodiment, the controller is configured to perform the following steps if the frame image is a locked image, determining that the physical switch is in the locked state:

Get the color value of each pixel in the frame image. Specifically, the color value of each pixel is obtained according to the resolution of the frame image.

In the embodiment of the present application, before obtaining the color value of each pixel in the frame image, the format of the frame image may be converted into a preset format.

The preview image is several frames of images saved in NV21 encoding format. Specifically, the default format of the image collected by the camera in the Android system is NV21 encoding. NV21 belongs to the YUV420 format. Every four Ys share a set of uv. For example, in Figure 12, Y1, Y2, Y9, and Y10 share a set of V1, U1. NV21 is a two-plane (two plane) mode, namely Y And UV is divided into two Planes, UV is interleaved storage. Store all Y first, then UV interleave storage, store V first, then U in NV21 encoding format.

But the frame image of NV21 encoding format picks up the color value process of the pixel, because the color value of the pixel of the frame image of NV21 encoding format is composed of Y and UV stored separately, and then the color value of the pixel is obtained by obtaining the stored in Obtaining Y and UV at different positions increases the difficulty of obtaining pixel color values. Since acquiring the frame image in the NV21 encoding format is not conducive to acquiring the color value of the pixel, the embodiment of the application converts the acquired frame image in the NV21 encoding format into an easy-to-read bitmap (bitmap) image with color values, and the pixels of the bitmap image All are assigned a specific position and color value, and the color value of each pixel is combined by RGB.

The specific method for converting frame images in NV21 encoding format into bitmap images with easy-to-read RGB color values is as follows: According to the YuvImage function interface class provided by the Android system, input the image byte array, image width and height information in NV21 encoding format The frame image in NV21 encoding format is converted into a byte stream in JPEG format, and then the obtained byte stream can be decoded into a Bitmap image.

According to the color value of each pixel, the number of locked pixels in each row of the frame image is determined.

If the ratio of the number of locked pixels to the number of pixels in each row of the image is greater than the preset ratio, it is determined that the physical switch is in the locked state.

In the embodiment of the present application, the number of image lock pixels in each row is counted. If the ratio of the number of image lock pixels in each row to the number of pixels is greater than the preset proportion, the preset proportion may be 90%, then the physical switch is locked status.

Obtain the color value of each pixel of the first line of image in the frame image;

If the ratio of the number of locked pixels to the number of pixels is greater than the preset ratio, obtain the color value of each pixel of the next line of the image in the frame image, and repeat the step of determining the number of locked pixels according to the color value of each pixel, Until the ratio of the number of locked pixels to the number of pixels in the last line of the frame image is greater than the preset ratio, it is determined that the physical switch is in the locked state.

It should be noted that the number of locked pixels of the first line of image is determined. If the ratio of the number of locked pixels of the first line of image to the number of pixels of the first line of image is greater than the preset proportion, the preset proportion may be 90%, This indicates that the camera is not raised, or it may be caused by the fact that the real image of the surrounding image of the display device taken by the camera is similar to black after the camera is raised normally. Specifically, referring to FIG. 12 again, the peripheral image of the display device displayed above the frame image is a dark scene, so the ratio of the number of locked pixels to the number of pixels may be greater than the preset ratio. In order to accurately determine whether the frame image is a locked image, obtain the color value of each pixel of the second line of the frame image, perform the step of determining the number of locked pixels according to the color value of each pixel, and determine the locked image of the second line of image The ratio of the number of pixels in the second row of images to the preset ratio. If the ratio of the number of locked images in the second row to the number of pixels is greater than the preset ratio, continue to determine that the number of locked images in the third row of images accounts for the first The ratio of the number of pixels in the three rows of images until it is determined whether the ratio of the locked image to the number of pixels in the last row of the frame image is greater than the preset ratio. If the ratio of the locked image to the number of pixels in the last row of image is greater than the preset ratio, The physical switch is locked.

In an exemplary embodiment, if the ratio of the number of locked pixels to the number of pixels is less than or equal to the preset ratio, it is determined that the frame image is not a locked image, and the color value of each pixel of the next line of image is not acquired.

Specifically, when the preset proportion is 90%, if the ratio of the number of locked pixels to the number of pixels is less than or equal to 90%, it is determined that the frame image is not a locked image.

In an exemplary embodiment, the controller is configured to perform the determination of the number of locked pixels in each row of the frame image according to the color value of each pixel according to the following steps:

Obtained image deviation; specifically, in order to obtain a better display effect on the display device, the original image obtained by the camera will be corrected and image quality processed, so the obtained preview image is different from the original image obtained by the actual camera . For example, if the camera is locked and not up, the image captured by the camera is close to black (RGB value is (0, 0, 0)), and after the image quality processing, the RGB value of the frame image of the obtained preview image and black exist A certain deviation. When the format of the frame image is an RGB value, the image deviation includes Sr (red deviation), Sg (green deviation) and Sb (blue deviation).

In addition, the step of obtaining the image deviation can be completed before the step of subtracting the color value of the pixel from the image deviation and taking the absolute value as described in the embodiment of the present application.

In an exemplary embodiment, the image deviation is determined according to a PQ image quality adjustment parameter.

Specifically, the preview image is an image obtained after ISP processing. ISP is a hardware processing module used to pre-process the original image acquired by the camera's photosensitive module to obtain a higher-quality preview image. The adjustment of ISP module parameters will affect the display effect of the preview image. PQ Tools implements online image quality adjustment of ISP through the network port or serial port. The PQ image quality adjustment parameter is a parameter for adjusting image quality in the PQ Tools tool.

Subtract the color value of each pixel from the image deviation, and take the absolute value to get the absolute value of each pixel;

Count the number of locked pixels in each row of the image. The locked pixels mean that the absolute value of the color is greater than the color threshold.

If the frame image is a locked image, repeat the step of determining whether the frame image is a locked image at a preset time interval until the number of repetitions reaches the preset number of times;

Count the number of times that the frame image is a locked image, and if the number of times is equal to the number of repetitions, confirm that the frame image is a locked image.

Specifically, in order to make the detection of the state of the physical switch more accurate, the step of determining whether the frame image is a locked image can be repeated multiple times. If the preset number of times is set to 3, when the frame image is determined to be a locked image for the first time, the step of determining whether the frame image is a locked image is repeated at intervals of a preset time. When the frame image is determined to be a locked image for the second time, The step of determining whether the frame image is a locked image is repeated at intervals of a preset time. After the frame image is determined to be a locked image for the third time, it is confirmed that the physical switch is in the locked state.

If after determining that the frame image is a locked image for the first time, the step of determining whether the frame image is a locked image is repeated at intervals of a preset time, and it is determined that the frame image is not a locked image, then the determination of the physical switch lock state is ended. By repeatedly determining whether the frame image is a lock image, the accuracy of determining the state of the physical switch in the embodiment of the present application can be ensured.

In an exemplary embodiment, the controller is further configured to perform the detection of whether the camera is up within a preset time according to the following steps: if the frame image is not a locked image, it is determined that the physical switch is in an open state, The physical switch is on to indicate that the camera is up.

Specifically, if the frame image is not a locked image, it means that the ratio of the number of locked pixels to the number of pixels in one row of the frame image is less than or equal to the preset ratio, indicating that the camera has captured a normal image, and thus it is determined that the physical switch is at Open state.

In an example, the controller is configured to detect whether the camera is raised within a preset time according to the following steps:

Obtain the 0.5s frame image in the preview data, and convert the format of the frame image to RGB format. Determine the image deviation according to the PQ image quality adjustment parameters, obtain the RGB value of each pixel in the first line of the frame image, subtract the RGB value of each pixel from the image deviation, and take the absolute value to obtain the absolute color value, if If the absolute value of the color is greater than the color threshold, the pixels are locked pixels, and the locked pixels of the first row of images are counted. If the ratio of the locked pixels to the number of pixels in the first row of images is greater than the preset ratio, continue to obtain the second row of the frame image The RGB value of each pixel of the image, according to the RGB value of each pixel, determines the number of locked pixels in the second row of images. If the ratio of the number of locked pixels to the number of pixels is greater than the preset ratio, get each of the third row of images The color value of the pixel until the ratio of the number of locked pixels to the number of pixels in the last row of the frame image is greater than the preset ratio, and it is determined that the frame image is a locked image. In order to increase the accuracy, when it is determined that the frame image is a locked image for the first time, it is repeated every 0.5s to determine whether the frame image is a locked image. If the result of the three detections is that the frame image is a locked image, then the physical switch is determined to be in the locked state . After detecting that the frame images are all locked images for 3 times, it is determined that the camera is not raised, and a prompt message indicating that the physical switch is locked is displayed on the display.

It can be seen from the above embodiments that the embodiment of the present application shows a method for displaying the physical switch lock state of a camera and a display device, the controller of which can detect whether the camera is raised within a preset time, and if it is detected that the camera is not raised, The user is prompted to turn on the physical switch through the prompt message that the physical switch is in the locked state displayed on the display.

After considering the specification and practicing the invention disclosed herein, those skilled in the art will easily think of other embodiments of the present application. This application is intended to cover any variations, uses, or adaptive changes of this application. These variations, uses, or adaptive changes follow the general principles of this application and include common knowledge or customary technical means in the technical field not disclosed in this application. . The description and embodiments are only regarded as exemplary, and the true scope and spirit of the application are pointed out by the following claims.

It should be understood that the present application is not limited to the precise structure that has been described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the application is only limited by the appended claims.

Claims

A display device, characterized by comprising:

The camera is configured to: collect image data, and be raised or lowered by the lifting power device;

A physical switch configured to: lock the lifting position of the camera;

A display configured to display a user interface, and/or the image data;

The controller is configured as:

In response to an instruction input by the user for instructing the camera to open, control the camera to rise through a lifting power device;

It is detected whether the camera is raised within a preset time, and if it is detected that the camera is not raised, the display is controlled to display a prompt message that the physical switch is in a locked state.
The display device according to claim 1, wherein the controller is configured to perform the detection of whether the camera is raised within a preset time according to the following steps:

Acquiring a preset time frame image in the preview image acquired by the camera;

If the frame image is a locked image, it is determined that the physical switch is in a locked state. The locked image refers to an image acquired by the camera in the locked state of the physical switch, and the physical switch in the locked state indicates that the camera is not up.
The display device according to claim 2, wherein the controller is configured to perform the following steps if the frame image is a locked image, determining that the physical switch is in a locked state:

Acquiring the color value of each pixel in the frame image;

Determine the number of locked pixels in each row of the frame image according to the color value of each pixel;

If the ratio of the number of locked pixels to the number of pixels in each row of the image is greater than the preset ratio, it is determined that the physical switch is in the locked state.
The display device according to claim 2, wherein the controller is configured to perform the following steps if the frame image is a locked image, determining that the physical switch is in a locked state:

Acquiring the color value of each pixel of the first line of image in the frame image;

Determine the number of locked pixels according to the color value of each pixel;

If the ratio of the number of locked pixels to the number of pixels is greater than the preset ratio, obtain the color value of each pixel of the next line of the image in the frame image, and repeat the step of determining the number of locked pixels according to the color value of each pixel, Until the ratio of the number of locked pixels to the number of pixels in the last line of the frame image is greater than the preset ratio, it is determined that the physical switch is in the locked state.
The display device according to claim 4, wherein the controller is further configured to perform the following steps if the frame image is a locked image, determining that the physical switch is in a locked state:

If the ratio of the number of locked pixels to the number of pixels is less than or equal to the preset ratio, it is determined that the frame image is not a locked image.
The display device according to any one of claims 3 or 4, wherein the controller is further configured to, before the step of obtaining the color value of each pixel in the frame image, further comprising: The format is converted to a preset format.
3. The display device according to claim 3, wherein the controller is configured to execute the determination of the number of locked pixels in each row of the frame image according to the color value of each pixel according to the following steps:

Obtain image deviation;

Subtract the color value of each pixel from the image deviation, and take the absolute value to obtain the absolute color value of each pixel;

Count the number of locked pixels in each row of the image. The locked pixels mean that the absolute value of the color is greater than the color threshold.
The display device according to claim 2, wherein the controller is configured to perform the following steps to determine that the physical switch is in the locked state if the frame image is a locked image:

If the frame image is a locked image, repeat the step of determining whether the frame image is a locked image at a preset time interval until the number of repetitions reaches the preset number of times;

Count the number of times that the frame image is a locked image, and if the number of times is equal to the number of repetitions, confirm that the frame image is a locked image.
3. The display device according to claim 2, wherein the controller is further configured to perform the detection of whether the camera is raised within a preset time according to the following steps:

If the frame image is not a lock image, it is determined that the physical switch is in an open state, and the physical switch is in an open state indicating that the camera is raised.
A method for displaying the locked state of a physical switch of a camera, which is characterized in that it comprises:

In response to an instruction input by the user for instructing the camera to open, control the camera to rise through a lifting power device;

It is detected whether the camera is raised within a preset time, and if it is detected that the camera is not raised, the display is controlled to display a prompt message that the physical switch is in a locked state.