WO2021218473A1

WO2021218473A1 - Display method and display device

Info

Publication number: WO2021218473A1
Application number: PCT/CN2021/081562
Authority: WO
Inventors: 王大勇; 于颜梅; 杨鲁明; 王卫明; 鲍姗娟; 陈验方
Original assignee: 海信视像科技股份有限公司
Priority date: 2020-04-27
Filing date: 2021-03-18
Publication date: 2021-11-04

Abstract

The present application provides a display method and a display device. When the display device displays a panoramic picture, a target person in front of the display device can be identified, and next, an initial face angle of the target person and a current face angle after a preset duration are separately obtained; an offset distance that the panoramic picture on the display device needs to move is calculated using the initial face angle and the current face angle; and finally, the panoramic picture is adjusted according to the offset distance to enable the content of the panoramic picture that has not been displayed before to be displayed on the display device.

Description

Display method and display device

This application requires the priority of being submitted to the Chinese Patent Office on April 27, 2020, with the application number 202010342885.9 and the application titled "A method and display device for dynamically adjusting controls"; the entire content of which is incorporated into this application by reference; this The application requires that it be submitted to the Chinese Patent Office on June 18, 2020, and the application number is 202010559804.0 and the priority of the application titled "Panoramic Picture Browsing Method and Display Device"; the entire content of which is incorporated into this application by reference.

Technical field

This application relates to the technical field of smart TVs, and in particular to a display method and display device.

Background technique

A panoramic picture is a picture displayed through a wide-angle representation method, and a panoramic picture can express more images of the surrounding environment. Generally, the panoramic picture cannot be completely displayed on the display device. When content other than the current display content of the panoramic picture needs to be browsed, the panoramic picture needs to be adjusted so that other contents of the panoramic picture can be adjusted to the display area of the display device.

At present, when browsing panoramic pictures on a display device, it is usually necessary to use the remote control to adjust the display content of the panoramic picture. For example, press the arrow keys on the remote control to move the panoramic picture in the corresponding direction on the screen of the display device. Display the content that was not displayed before in the opposite direction; or collect the operator’s gestures, and then adjust the displayed panoramic picture content according to the direction of the gesture, for example, the operator’s finger slides to the right on the display device screen, then the panoramic picture It will move to the right on the screen to display the content that was not displayed on the left side of the previous panoramic picture.

Summary of the invention

This application provides a display method and display device.

In the first aspect, this application provides a panoramic picture browsing method, including:

In the case that the display device displays a panoramic picture, identifying a target person in front of the display device, where the target person is used to indicate an operator who browses the panoramic picture in front of the display device;

Acquiring the initial facial angle and the current facial angle of the target person before and after the first preset duration;

Obtaining the offset distance of the panoramic picture by using the initial face angle and the current face angle;

Adjust the display content of the panoramic picture on the display device according to the offset distance.

In the second aspect, this application also provides a display device, including:

monitor;

Detector, used to collect the image in front of the display device;

The controller is configured as:

Description of the drawings

In order to explain the technical solution of the present application more clearly, the following will briefly introduce the drawings needed in the embodiments. Obviously, for those of ordinary skill in the art, without creative work, Other drawings can be obtained from these drawings.

FIG. 1 exemplarily shows a schematic diagram of an operation scene between a display device and a control device according to some embodiments;

FIG. 2 exemplarily shows a hardware configuration block diagram of a display device 200 according to some embodiments;

FIG. 3 exemplarily shows a block diagram of the hardware configuration of the control device 100 according to some embodiments;

FIG. 4 exemplarily shows a schematic diagram of software configuration in a display device 200 according to some embodiments;

FIG. 5 exemplarily shows a schematic diagram of the icon control interface display of the application program in the display device 200 according to some embodiments;

FIG. 6 is a schematic diagram of interaction between a display device 200 and an operator according to an embodiment of the application;

FIG. 7 is a flowchart of a panoramic picture browsing method shown in an embodiment of the application;

FIG. 8 is a schematic diagram of a detector 230 collecting images according to an embodiment of the application;

FIG. 9 is a schematic diagram of the angle of the aircraft shown in an embodiment of the application;

FIG. 10 is a schematic diagram of interaction between another display device 200 and an operator according to an embodiment of the application;

FIG. 11 is a schematic diagram of the initial face angle of the target person obtained by the controller 110 according to an embodiment of the application; FIG.

FIG. 12 is a schematic diagram of the current face angle of the target person obtained by the controller 110 according to an embodiment of the application; FIG.

FIG. 13 exemplarily shows a flowchart of a method for dynamically adjusting a control according to an embodiment;

FIG. 14 exemplarily shows a flowchart of a method for obtaining the initial position parameter of a target control according to an embodiment;

FIG. 15 exemplarily shows a schematic diagram of the reference coordinate system according to the embodiment;

FIG. 16 exemplarily shows a schematic diagram of environmental image data corresponding to the initial position in the embodiment;

FIG. 17 exemplarily shows a schematic diagram of environmental image data corresponding to the end position in the embodiment;

FIG. 18 exemplarily shows a schematic diagram of the position change of the center point of the face frame on the display interface during the movement of the user according to the embodiment;

FIG. 19 exemplarily shows a flowchart of a method for calculating the offset of a target control according to an embodiment;

FIG. 20 exemplarily shows a schematic diagram when the theoretical second distance is determined according to the embodiment;

FIG. 21 exemplarily shows the first schematic diagram when the position of the control is dynamically adjusted according to the embodiment;

Fig. 22 exemplarily shows a second schematic diagram when the position of the control is dynamically adjusted according to the embodiment.

Detailed ways

In order to make the purpose, implementation and advantages of the present application clearer, the exemplary implementations of the present application will be described clearly and completely with reference to the accompanying drawings in the exemplary embodiments of the present application. Obviously, the described exemplary embodiments It is only a part of the embodiments of the present application, but not all of the embodiments.

Based on the exemplary embodiments described in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of the appended claims of the present application. In addition, although the disclosure in this application is introduced in accordance with one or several exemplary examples, it should be understood that various aspects of these disclosures can also be individually constituted as a complete implementation.

It should be noted that the brief description of the terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of this application. Unless otherwise stated, these terms should be understood according to their ordinary and usual meanings.

The terms "first", "second", "third", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar or similar objects or entities, and do not necessarily mean to limit specific Order or precedence, unless otherwise indicated (Unless otherwise indicated). It should be understood that the terms used in this way can be interchanged under appropriate circumstances, for example, they can be implemented in an order other than those given in the illustrations or descriptions of the embodiments of the present application.

In addition, the terms "including" and "having" and any variations of them are intended to cover but not exclusively include. For example, a product or device including a series of components need not be limited to those clearly listed, but may include Other components not clearly listed or inherent to these products or equipment.

The term "module" used in this application refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware or/and software code that can perform functions related to the element.

The term "remote control" used in this application refers to a component of an electronic device (such as the display device disclosed in this application), which can generally control the electronic device wirelessly within a short distance. Generally, infrared and/or radio frequency (RF) signals and/or Bluetooth are used to connect to electronic devices, and may also include functional modules such as WiFi, wireless USB, Bluetooth, and motion sensors. For example, a handheld touch remote control uses a user interface in a touch screen to replace most of the physical built-in hard keys in general remote control devices.

The term "gesture" used in this application refers to a user's behavior through a change of hand shape or hand movement to express expected ideas, actions, goals, and/or results.

Fig. 1 exemplarily shows a schematic diagram of an operation scenario between a display device and a control device according to an embodiment. As shown in FIG. 1, the user can operate the display device 200 through the mobile terminal 300 and the control device 100.

In some embodiments, the control device 100 may be a remote controller, and the communication between the remote controller and the display device includes infrared protocol communication or Bluetooth protocol communication, and other short-range communication methods, and the display device 200 is controlled by wireless or other wired methods. The user can control the display device 200 by inputting user instructions through keys on the remote control, voice input, control panel input, etc. For example, the user can control the display device 200 by inputting corresponding control commands through the volume plus and minus keys, channel control keys, up/down/left/right movement keys, voice input keys, menu keys, and power on/off keys on the remote control. Function.

In some embodiments, mobile terminals, tablet computers, computers, notebook computers, and other smart devices can also be used to control the display device 200. For example, an application program running on a smart device is used to control the display device 200. The application can be configured to provide users with various controls in an intuitive user interface (UI) on the screen associated with the smart device.

In some embodiments, the mobile terminal 300 can install a software application with the display device 200, realize connection communication through a network communication protocol, and realize the purpose of one-to-one control operation and data communication. For example, the mobile terminal 300 can be used to establish a control command protocol with the display device 200, the remote control keyboard can be synchronized to the mobile terminal 300, and the function of controlling the display device 200 can be realized by controlling the user interface on the mobile terminal 300. It is also possible to transmit the audio and video content displayed on the mobile terminal 300 to the display device 200 to realize the synchronous display function.

As also shown in FIG. 1, the display device 200 also performs data communication with the server 400 through multiple communication methods. The display device 200 may be allowed to communicate through a local area network (LAN), a wireless local area network (WLAN), and other networks. The server 400 may provide various contents and interactions to the display device 200. For example, the display device 200 can receive software program updates or access a remotely stored digital media library by sending and receiving information and interacting with an electronic program guide (EPG). The server 400 may be one cluster or multiple clusters, and may include one or more types of servers. The server 400 provides other network service content such as video-on-demand and advertising services.

The display device 200 may be a liquid crystal display, an OLED display, or a projection display device. The specific display device type, size, resolution, etc. are not limited, and those skilled in the art can understand that the display device 200 can make some changes in performance and configuration as required.

In addition to providing broadcast receiving TV functions, the display device 200 may also additionally provide computer-supported functions of smart network TV, including but not limited to, network TV, smart TV, Internet Protocol TV (IPTV), and the like.

FIG. 2 exemplarily shows a block diagram of the hardware configuration of the display device 200 according to an exemplary embodiment.

In some embodiments, the display device 200 includes a controller 250, a tuner and demodulator 210, a communicator 220, a detector 230, an input/output interface 255, a display 275, an audio output interface 285, a memory 260, a power supply 290, At least one of the user interface 265 and the external device interface 240.

In some embodiments, the display 275 is used to receive the image signal output from the first processor, and display the components of the video content and images and the menu manipulation interface.

In some embodiments, the display 275 includes a display screen component for presenting images, and a driving component for driving image display.

In some embodiments, the displayed video content can be from broadcast television content, or it can be said that various broadcast signals can be received through wired or wireless communication protocols. Or, it can display various image content received from the network server side from the network communication protocol.

In some embodiments, the display 275 is used to present a user manipulation UI interface generated in the display device 200 and used to control the display device 200.

In some embodiments, depending on the type of the display 275, a driving component for driving the display is further included.

In some embodiments, the display 275 is a projection display, and may also include a projection device and a projection screen.

In some embodiments, the communicator 220 is a component for communicating with external devices or external servers according to various communication protocol types. For example, the communicator may include at least one of Wifi chip, Bluetooth communication protocol chip, wired Ethernet communication protocol chip or other network communication protocol chip or near field communication protocol chip, and infrared receiver.

In some embodiments, the display device 200 may establish control signal and data signal transmission and reception with the external control device 100 or the content providing device through the communicator 220.

In some embodiments, the user interface 265 may be used to receive infrared control signals of the control device 100 (such as an infrared remote control, etc.).

In some embodiments, the detector 230 is the display device 200 for collecting signals from the external environment or interacting with the outside.

In some embodiments, the detector 230 includes a light receiver, a sensor used to collect the intensity of ambient light, and can adaptively display parameter changes and the like by collecting ambient light.

In some embodiments, the detector 230 may also include an image collector, such as a camera, a camera, etc., which can be used to collect external environment scenes, and to collect attributes of the user or interact with the user gestures, and can adaptively change the display parameters. It can also recognize user gestures to realize the function of interacting with the user.

In some embodiments, the detector 230 may also include a temperature sensor or the like, for example, by sensing the ambient temperature.

In some embodiments, the display device 200 may adaptively adjust the display color temperature of the image. For example, when the temperature is relatively high, the display device 200 can be adjusted to display a colder image color temperature, or when the temperature is relatively low, the display device 200 can be adjusted to display a warmer image.

In some embodiments, the detector 230 may also be a sound collector or the like, such as a microphone, which may be used to receive the user's voice. Exemplarily, a voice signal including a control instruction for the user to control the display device 200, or collecting environmental sounds, is used to identify the type of environmental scene, so that the display device 200 can adapt to environmental noise.

In some embodiments, as shown in FIG. 2, the input/output interface 255 is configured to perform data transmission between the controller 250 and other external devices or other controllers 250. Such as receiving video signal data and audio signal data from external devices, or command instruction data.

In some embodiments, the external device interface 240 may include, but is not limited to, the following: any one or more of high-definition multimedia interface HDMI interface, analog or data high-definition component input interface, composite video input interface, USB input interface, RGB port, etc. interface. It is also possible that the above-mentioned multiple interfaces form a composite input/output interface.

In some embodiments, as shown in FIG. 2, the tuner and demodulator 210 is configured to receive broadcast and television signals through wired or wireless reception, and can perform modulation and demodulation processing such as amplification, mixing, and resonance, from multiple wireless channels. Or the audio and video signal is demodulated from the cable broadcast television signal, and the audio and video signal may include the television audio and video signal carried in the frequency of the television channel selected by the user, and the EPG data signal.

In some embodiments, the frequency point demodulated by the tuner and demodulator 210 is controlled by the controller 250, and the controller 250 can send a control signal according to the user's selection, so that the modem responds to the TV signal frequency selected by the user and modulates and demodulates the frequency. The TV signal carried by the frequency.

In some embodiments, broadcast television signals can be classified into terrestrial broadcast signals, cable broadcast signals, satellite broadcast signals, or Internet broadcast signals according to different television signal broadcast formats. Or it can be divided into digital modulation signal, analog modulation signal, etc. according to different modulation types. Or it can be divided into digital signal, analog signal, etc. according to different signal types.

In some embodiments, the controller 250 and the tuner and demodulator 210 may be located in different separate devices, that is, the tuner and demodulator 210 may also be in an external device of the main device where the controller 250 is located, such as an external set-top box. Wait. In this way, the set-top box outputs the TV audio and video signals modulated and demodulated of the received broadcast TV signals to the main device, and the main device receives the audio and video signals through the first input/output interface.

In some embodiments, the controller 250 controls the operation of the display device and responds to user operations through various software control programs stored in the memory. The controller 250 may control the overall operation of the display device 200. For example, in response to receiving a user command for selecting a UI object to be displayed on the display 275, the controller 250 may perform an operation related to the object selected by the user command.

In some embodiments, the object may be any one of selectable objects, such as a hyperlink or an icon. Operations related to the selected object, for example: display operations connected to hyperlink pages, documents, images, etc., or perform operations corresponding to the icon. The user command for selecting the UI object may be a command input through various input devices (for example, a mouse, a keyboard, a touch pad, etc.) connected to the display device 200 or a voice command corresponding to the voice spoken by the user.

As shown in FIG. 2, the controller 250 includes a random access memory 251 (Random Access Memory, RAM), a read-only memory 252 (Read-Only Memory, ROM), a video processor 270, an audio processor 280, and other processors 253. (For example: at least one of Graphics Processing Unit (GPU), Central Processing Unit (CPU), Communication Interface, and Communication Bus 256 (Bus). Among them, the communication bus Connect the various components.

In some embodiments, RAM 251 is used to store temporary data of the operating system or other running programs

In some embodiments, the ROM 252 is used to store various system startup instructions.

In some embodiments, the ROM 252 is used to store a basic input output system, which is called a basic input output system (Basic Input Output System, BIOS). It is used to complete the power-on self-check of the system, the initialization of each functional module in the system, the basic input/output driver of the system and the boot operating system.

In some embodiments, when the power-on signal is received, the power of the display device 200 starts to start, and the CPU runs the system startup instruction in the ROM 252 to copy the temporary data of the operating system stored in the memory to the RAM 251 to facilitate startup or operation operating system. After the operating system is started up, the CPU copies the temporary data of various application programs in the memory to RAM 251, and then it is convenient to start or run various application programs.

In some embodiments, the CPU processor 254 is configured to execute operating system and application program instructions stored in the memory. And according to receiving various interactive instructions input from the outside, to execute various applications, data and content, so as to finally display and play various audio and video content.

In some exemplary embodiments, the CPU processor 254 may include multiple processors. The multiple processors may include a main processor and one or more sub-processors. The main processor is used to perform some operations of the display device 200 in the pre-power-on mode, and/or to display images in the normal mode. One or more sub-processors, used for an operation in a state such as standby mode.

In some embodiments, the graphics processor 253 is used to generate various graphics objects, such as icons, operation menus, and user input instructions to display graphics. Including an arithmetic unit, which performs operations by receiving various interactive instructions input by the user, and displays various objects according to the display attributes. It also includes a renderer, which renders various objects obtained based on the arithmetic unit, and the rendered objects are used for display on the display.

In some embodiments, the video processor 270 is configured to receive an external video signal, and perform decompression, decoding, scaling, noise reduction, frame rate conversion, resolution conversion, image synthesis, etc. according to the standard codec protocol of the input signal. After video processing, a signal that can be directly displayed or played on the display device 200 can be obtained.

In some embodiments, the video processor 270 includes a demultiplexing module, a video decoding module, an image synthesis module, a frame rate conversion module, a display formatting module, and the like.

Among them, the demultiplexing module is used to demultiplex the input audio and video data stream. For example, if MPEG-2 is input, the demultiplexing module will demultiplex into a video signal and an audio signal.

The video decoding module is used to process the demultiplexed video signal, including decoding and scaling.

An image synthesis module, such as an image synthesizer, is used to superimpose and mix the GUI signal generated by the graphics generator with the zoomed video image according to the user input or itself, so as to generate an image signal for display.

The frame rate conversion module is used to convert the frame rate of the input video, such as converting a 60Hz frame rate to a 120Hz frame rate or a 240Hz frame rate, and the usual format is realized by inserting a frame.

The display formatting module is used to convert the video output signal after the received frame rate is converted, and change the signal to conform to the signal of the display format, such as outputting RGB data signals.

In some embodiments, the graphics processor 253 can be integrated with the video processor, or can be configured separately. The graphics signal output to the display can be processed when integrated, and different functions can be performed separately when configured separately. For example, GPU+FRC (Frame Rate Conversion)) architecture.

In some embodiments, the audio processor 280 is used to receive an external audio signal, and perform decompression and decoding, as well as processing such as noise reduction, digital-to-analog conversion, and amplification processing, according to the standard codec protocol of the input signal, to obtain The sound signal played in the speaker.

In some embodiments, the video processor 270 may include one or more chips. The audio processor may also include one or more chips.

In some embodiments, the video processor 270 and the audio processor 280 may be separate chips, or may be integrated in one or more chips together with the controller.

In some embodiments, the audio output receives the sound signal output by the audio processor 280 under the control of the controller 250, such as the speaker 286, and can output to an external device in addition to the speaker carried by the display device 200 itself. The external audio output terminal of the device, such as an external audio interface or earphone interface, may also include a short-distance communication module in the communication interface, for example, a Bluetooth module for sound output from a Bluetooth speaker.

The power supply 290, under the control of the controller 250, provides power supply support for the display device 200 with power input from an external power supply. The power supply 290 may include a built-in power supply circuit installed inside the display device 200, or may be an external power supply installed in the display device 200, and a power interface for providing an external power supply in the display device 200.

The user interface 265 is used to receive user input signals, and then send the received user input signals to the controller 250. The user input signal may be a remote control signal received through an infrared receiver, and various user control signals may be received through a network communication module.

In some embodiments, the user inputs user commands through the control device 100 or the mobile terminal 300, the user input interface is based on the user's input, and the display device 200 responds to the user's input through the controller 250.

In some embodiments, the user may input a user command on a graphical user interface (GUI) displayed on the display 275, and the user input interface receives the user input command through the graphical user interface (GUI). Alternatively, the user may input a user command by inputting a specific sound or gesture, and the user input interface recognizes the sound or gesture through the sensor to receive the user input command.

In some embodiments, the "user interface" is a medium interface for interaction and information exchange between an application or operating system and a user, and it realizes the conversion between the internal form of information and the form acceptable to the user. The commonly used form of the user interface is the Graphic User Interface (GUI), which refers to a user interface related to computer operations that is displayed in a graphical manner. It can be an icon, window, control and other interface elements displayed on the display screen of an electronic device. The control can include icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, Widgets, etc. Visual interface elements.

The memory 260 includes storing various software modules used to drive the display device 200. For example, various software modules stored in the first memory include: at least one of a basic module, a detection module, a communication module, a display control module, a browser module, and various service modules.

The basic module is used to communicate signals between various hardware in the display device 200 and send processing and control signals to the lower layer software module of the upper layer module. The detection module is a management module used to collect various information from various sensors or user input interfaces, and perform digital-to-analog conversion and analysis management.

For example, the voice recognition module includes a voice parsing module and a voice command database module. The display control module is a module for controlling the display to display image content, and can be used to play information such as multimedia image content and UI interface. The communication module is a module used for control and data communication with external devices. The browser module is a module used to perform data communication between browsing servers. The service module is used to provide various services and modules including various applications. At the same time, the memory 260 is also used to store and receive external data and user data, images of various items in various user interfaces, and visual effect diagrams of focus objects, etc.

Fig. 3 exemplarily shows a configuration block diagram of the control device 100 according to an exemplary embodiment. As shown in FIG. 3, the control device 100 includes a controller 110, a communication interface 130, a user input/output interface, a memory, and a power supply.

The control device 100 is configured to control the display device 200, and can receive input operation instructions from the user, and convert the operation instructions into instructions that the display device 200 can recognize and respond to, so as to serve as an intermediary between the user and the display device 200. For example, the user operates the channel addition and subtraction key on the control device 100, and the display device 200 responds to the channel addition and subtraction operation.

In some embodiments, the control device 100 may be a smart device. For example, the control device 100 can install various applications for controlling the display device 200 according to user requirements.

In some embodiments, as shown in FIG. 1, the mobile terminal 300 or other smart electronic device can perform a similar function to control the device 100 after installing an application that controls the display device 200. For example, the user can install various function keys or virtual buttons of the graphical user interface that can be provided on the mobile terminal 300 or other smart electronic devices by installing applications to realize the function of controlling the physical keys of the device 100.

The controller 110 includes a processor 112, a RAM 113 and a ROM 114, a communication interface 130, and a communication bus. The controller is used to control the operation and operation of the control device 100, as well as the communication and cooperation between internal components, and external and internal data processing functions.

The communication interface 130 realizes the communication of control signals and data signals with the display device 200 under the control of the controller 110. For example, the received user input signal is sent to the display device 200. The communication interface 130 may include at least one of other near field communication modules such as a WiFi chip 131, a Bluetooth module 132, and an NFC module 133.

The user input/output interface 140, wherein the input interface includes at least one of other input interfaces such as a microphone 141, a touch panel 142, a sensor 143, and a button 144. For example, the user can implement the user instruction input function through voice, touch, gesture, pressing and other actions. The input interface converts the received analog signal into a digital signal and the digital signal into a corresponding instruction signal, and sends it to the display device 200.

The output interface includes an interface for sending the received user instruction to the display device 200. In some embodiments, it may be an infrared interface or a radio frequency interface. For example, in the case of an infrared signal interface, the user input instruction needs to be converted into an infrared control signal according to the infrared control protocol, and then sent to the display device 200 via the infrared sending module. For another example, in the case of a radio frequency signal interface, a user input instruction needs to be converted into a digital signal, which is then modulated according to the radio frequency control signal modulation protocol, and then sent to the display device 200 by the radio frequency sending terminal.

In some embodiments, the control device 100 includes at least one of a communication interface 130 and an input/output interface 140. The control device 100 is configured with a communication interface 130, such as WiFi, Bluetooth, NFC, etc. modules, which can encode user input instructions to the display device 200 through the WiFi protocol, or the Bluetooth protocol, or the NFC protocol.

The memory 190 is used to store various operating programs, data, and applications for driving and controlling the control device 200 under the control of the controller. The memory 190 can store various control signal instructions input by the user.

The power supply 180 is used to provide operating power support for each element of the control device 100 under the control of the controller. Can battery and related control circuit.

In some embodiments, the system may include a kernel (Kernel), a command parser (shell), a file system, and an application program. The kernel, shell, and file system together form the basic operating system structure. They allow users to manage files, run programs, and use the system. After power on, the kernel starts, activates the kernel space, abstracts hardware, initializes hardware parameters, etc., runs and maintains virtual memory, scheduler, signals, and inter-process communication (IPC). After the kernel is started, the Shell and user applications are loaded. After the application is started, it is compiled into machine code to form a process.

Referring to Figure 4, in some embodiments, the system is divided into four layers, from top to bottom, respectively, the application (Applications) layer (referred to as the "application layer"), and the Application Framework layer (referred to as the "framework layer"). "), Android runtime and system library layer (referred to as "system runtime library layer"), and kernel layer.

In some embodiments, there is at least one application program running in the application layer. These applications may be Window programs, system setting programs, clock programs, camera applications, etc., which are included in the operating system; they may also be developed by third parties. The application programs developed by the author, such as Hi Jian program, K song program, magic mirror program, etc. In specific implementation, the application package in the application layer is not limited to the above examples, and may actually include other application packages, which is not limited in the embodiment of the present application.

The framework layer provides application programming interfaces (application programming interface, API) and programming frameworks for applications in the application layer. The application framework layer includes some predefined functions. The application framework layer is equivalent to a processing center, which decides to let applications in the application layer take actions. Through the API interface, the application can access the resources in the system and obtain the services of the system during execution.

As shown in Figure 4, the application framework layer in the embodiment of the present application includes managers (Managers), content providers (Content Provider), etc., where the manager includes at least one of the following modules: It interacts with all activities running in the system; Location Manager is used to provide system services or applications with access to system location services; Package Manager is used to retrieve current installations on the device Various information related to the application package; Notification Manager is used to control the display and clearing of notification messages; Window Manager is used to manage icons, windows, toolbars, and wallpapers on the user interface And desktop widgets.

In some embodiments, the activity manager is used to: manage the life cycle of each application and the usual navigation rollback functions, such as controlling the exit of the application (including switching the user interface currently displayed in the display window to the system desktop), opening , Back (including switching the user interface currently displayed in the display window to the upper level user interface of the currently displayed user interface), etc.

In some embodiments, the window manager is used to manage all window programs, such as obtaining the size of the display screen, determining whether there is a status bar, locking the screen, capturing the screen, and controlling changes in the display window (for example, shrinking the display window, dithering, or distorting the display window). Deformation display, etc.) and so on.

In some embodiments, the system runtime layer provides support for the upper layer, that is, the framework layer. When the framework layer is used, the Android operating system will run the C/C++ library included in the system runtime layer to implement functions to be implemented by the framework layer.

In some embodiments, the kernel layer is a layer between hardware and software. As shown in Figure 4, the kernel layer contains at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, touch Sensors, pressure sensors, etc.) etc.

In some embodiments, the kernel layer further includes a power drive module for power management.

In some embodiments, the software programs and/or modules corresponding to the software architecture in FIG. 4 are stored in the first memory or the second memory shown in FIG. 2 or FIG. 3.

In some embodiments, taking the magic mirror application (photographing application) as an example, when the remote control receiving device receives the remote control input operation, the corresponding hardware interrupt is sent to the kernel layer. The kernel layer processes the input operation into the original input event (including the value of the input operation, the time stamp of the input operation and other information). The original input events are stored in the kernel layer. The application framework layer obtains the original input event from the kernel layer, recognizes the control corresponding to the input event according to the current position of the focus, and regards the input operation as a confirmation operation. The control corresponding to the confirmation operation is the control of the magic mirror application icon. The mirror application calls the interface of the application framework layer, starts the magic mirror application, and then starts the camera driver by calling the kernel layer to realize the capture of still images or videos through the camera.

In some embodiments, for a display device with touch function, taking split-screen operation as an example, the display device receives input operations (such as split-screen operations) that the user acts on the display screen, and the kernel layer can generate corresponding input operations based on the input operations. Enter the event and report the event to the application framework layer. The activity manager of the application framework layer sets the window mode (such as multi-window mode) and the window position and size corresponding to the input operation. The window management of the application framework layer draws the window according to the settings of the activity manager, and then sends the drawn window data to the display driver of the kernel layer, and the display driver displays the corresponding application interface in different display areas of the display screen.

In some embodiments, as shown in FIG. 5, the application layer includes at least one application that can display corresponding icon controls on the display, such as: live TV application icon controls, video on demand application icon controls, media center applications Program icon controls, application center icon controls, game application icon controls, etc.

In some embodiments, the live TV application can provide live TV through different signal sources. For example, a live TV application may use input from cable TV, wireless broadcasting, satellite services, or other types of live TV services to provide TV signals. And, the live TV application can display the video of the live TV signal on the display device 200.

In some embodiments, video-on-demand applications can provide videos from different storage sources. Unlike live TV applications, VOD provides video display from certain storage sources. For example, video on demand can come from the server side of cloud storage, and from the local hard disk storage that contains stored video programs.

In some embodiments, the media center application can provide various multimedia content playback applications. For example, the media center can provide services that are different from live TV or video on demand, and users can access various images or audio through the media center application.

In some embodiments, the application center may provide for storing various application programs. The application program may be a game, an application program, or some other application program that is related to a computer system or other equipment but can be run on a smart TV. The application center can obtain these applications from different sources, store them in the local storage, and then run on the display device 200.

The display device 200 can display not only some ordinary pictures, but also panoramic pictures. A panoramic picture is a picture displayed through a wide-angle representation method, and a panoramic picture can express more images of the surrounding environment. Generally, the panoramic picture cannot be completely displayed on the display device 200. When content other than the currently displayed content of the panoramic picture needs to be browsed, the panoramic picture needs to be adjusted so that other contents of the panoramic picture can be adjusted to the display area of the display device 200.

At present, when browsing panoramic pictures on the display device 200, the operator usually needs to use the remote control to adjust the display content of the panoramic picture. Move in the corresponding direction to display the content that was not displayed before in the opposite direction; or the display device 200 collects the operator’s gesture, and then adjusts the displayed panoramic image content according to the direction of the gesture, for example, the operator’s finger is on the display device Swipe to the right on the 200 screen, then the panoramic picture will move to the right on the screen, and then display the content that was not displayed on the left side of the panoramic picture before.

However, whether it is the aforementioned remote control adjustment or gesture adjustment, all panoramic pictures are moved according to a fixed step preset in the display device 200. When the operator only wants to fine-tune the content of a certain panoramic picture, if it is fixed If the step size is too large, the distance the panoramic picture moves on the screen of the display device 200 will be too large, and it is difficult to meet the operator's requirement for fine adjustment of the panoramic picture.

In response to the above content, the present application provides a panoramic picture browsing method and display device, which can control the moving direction and distance of the panoramic picture on the display device 200 through the operator’s face angle change in front of the display device 200, and then operate The content of the panoramic picture that the user needs to see is adjusted to the screen of the display device 200.

In the display device 200, the display 275 can be used to display a panoramic picture, the detector 230 can be used to collect an image of a person in front of the display device 200, and the controller 110 can be used to identify the target person based on the image collected by the detector 230 and recognize The face angle information of the target person in front of the display device 200 can also calculate the offset distance of the panoramic picture and control the movement of the panoramic picture.

FIG. 6 is a schematic diagram of interaction between a display device 200 and an operator according to an embodiment of the application. As shown in FIG. 6, when browsing panoramic pictures using the panoramic picture browsing method provided by the embodiment of the present application, the operator needs to stand in front of the display device 200, and the operator controls the panoramic picture on the display device 200 by turning his head. Move to show what you want to watch.

Fig. 7 is a flowchart of a panoramic picture browsing method shown in an embodiment of the application. As shown in Figure 7, the panoramic picture browsing method specifically includes the following steps:

In step S101, when the display device 200 displays a panoramic picture, a target person in front of the display device 200 is identified, and the target person is used to indicate an operator who browses the panoramic picture in front of the display device 200.

The target person mentioned in this embodiment specifically refers to the actual operator. For example, multiple people in front of the display device 200 are viewing panoramic pictures at the same time. In order to avoid confusion caused by the joint control of multiple people, the display device 200 can only receive operations from one person. , Then this person can be the target person. In this embodiment, generally, the person in front of the display device is identified first by means of face recognition, and then it is specifically determined whether the person is the target person. The way to select the target person is not unique. For example, the person closest to the display device 200 can be selected as the target person. In this case, the pixel area of each face in front of the display device 200 needs to be compared. Generally, the closer to the display device 200, the pixels of the face The larger the area; or if a specific person is selected as the target person, it is necessary to compare whether there is a specific face among the faces in front of the display device 200.

Therefore, in some embodiments, the step of identifying the target person in front of the display device 200 may specifically include:

Step S201: Detect whether there is a preset person in front of the display device 200, and the preset person is used to represent an operator who browses a panoramic picture pre-stored in the display device.

The preset person mentioned here may represent the specific person referred to above. Generally, each display device 200 stores a preset person. The preset character can be set during initialization. When the display device 200 browses the panoramic picture for the first time, it can be recognized whether there is this preset character in front of the display device 200; in addition, the preset character can also be the previous panoramic picture browse. When the last saved target person is, when the display device 200 currently browses the panoramic picture again, the target person saved last time can be used as the preset person, and it is recognized whether the preset person exists before the display device 200.

In step S202, if it exists, it is determined that the preset person in front of the display device 200 is the target person.

Of course, in some cases, the operator who browses the panoramic picture may not be a preset character, and a target character needs to be re-determined in order to operate on the panoramic picture. Therefore, in some embodiments, the step of identifying the target person in front of the display device 200 may further include:

In step S301, in the case that there are multiple characters in front of the display device 200 and there is no preset character, the pixel area corresponding to the face of each character is calculated respectively.

In this embodiment, when a target person is determined among a plurality of persons, the person closest to the display device 200 is selected. Generally, the distance between a person and the display device 200 is determined by the pixel area of the person's face. The closer the person is to the display device 200, the larger the pixel area of the person's face. In the display device 200, when the detector 230 collects an image in front of the display device 200, the controller 110 can identify the face images belonging to the person, and then further calculate the pixel area corresponding to each face image.

In step S302, the person corresponding to the face with the largest pixel area is selected as the target person.

FIG. 8 is a schematic diagram of a detector 230 collecting images according to an embodiment of the application. In FIG. 8, there are two persons in the image in front of the display device 200 collected by the detector 230. The controller 110 can recognize that the person standing in the front has the largest face pixel area, and then determine that the person is the target person. The dashed frame in FIG. 8 is the face range of the target person recognized by the controller 110.

It is worth noting that if there is only one person in front of the display device 200, then it can be considered that this person is browsing the panoramic pictures. At this time, whether this person is a specific person or not, the controller 110 can regard this person as the target person. And then control the movement of the panoramic picture.

Step S102: Obtain the initial face angle and the current face angle of the target person before and after the first preset duration.

In this embodiment, the panoramic picture needs to be adjusted according to the change in the angle of the target person's face. The angle of the face has a certain change time, and this time is the first preset duration.

The controller 110 may obtain the face rotation angle of the target person in front of the display device 200 after recognizing the target person. This angle can refer to the three angles in the aerospace field. FIG. 9 is a schematic diagram of the aircraft angle shown in an embodiment of the application. As shown in FIG. And the yaw angle is achieved. In this embodiment, the concepts of roll angle, pitch angle, and yaw angle are also used to define the face angle of the target person, where the roll angle refers to the angle at which the face rotates, and the pitch angle refers to the up or down of the face. The yaw angle refers to the angle at which the face rotates in the horizontal direction.

If the target person needs to browse the panoramic picture, the initial face angle before and after the first preset duration and the current face angle are generally different. The initial face angle specifically includes the three angles of the face, which are the initial roll angle, the initial pitch angle, and the initial yaw angle; the current face angle also specifically includes the three angles of the face, which are the current roll angle, Current pitch angle and current yaw angle.

Then, in some embodiments, after identifying the target person, it is necessary to obtain the initial roll angle, initial pitch angle, and initial yaw angle of the target person in front of the display device 200, and then obtain again after the first preset time period has elapsed. The current roll angle, current pitch angle, and current yaw angle of the target person.

Step S103: Obtain the offset distance of the panoramic picture by using the initial face angle and the current face angle.

Generally, when the operator needs to manipulate the panoramic picture, the angle of the face will change after the first preset time period, and the current face angle will have a certain offset from the initial face angle. In this embodiment, it is Use this offset to obtain the offset distance corresponding to the panoramic picture.

FIG. 10 is a schematic diagram of another interaction between a display device 200 and an operator according to an embodiment of the application. As shown in FIG. 10, when the display device 200 displays a panoramic picture, it can only display a screen as large as the screen of the display 275. If the operator wants to browse the content on the right side of the panoramic picture that is not displayed on the screen, he can face the display device 200 and turn his face to the right. The display device 200 will calculate the direction of the panoramic picture according to the angle at which the operator turns the face. By the distance of the left offset, the panoramic picture is then controlled to shift to the left, and then more content on the right side is displayed on the display device 200.

In the actual operation scene, the operation of the operator to turn the face is not absolutely right or left, and it will have a certain offset downwards or downwards. Furthermore, the movement of the panoramic picture on the display 275 is not absolute to the left or right or up and down. The operator can also turn his face in a specific direction to browse content in a specific direction according to his own browsing needs. For example, the operator Rotate the face to the upper right direction, and the panoramic picture moves in the reverse direction to the lower left to display the content in the upper right direction. Therefore, in this embodiment, it is necessary to detect and calculate the three angles of the face rotation to obtain a more accurate offset distance of the panoramic picture.

Step S104: Adjust the display content of the panoramic picture on the display device 200 according to the offset distance.

Generally, the panoramic picture displayed on the display device 200 is a two-dimensional picture, and the direction in which the two-dimensional picture moves includes two directions, horizontal and vertical. Therefore, the calculated offset distance includes the offset distance of the panoramic picture in the horizontal direction and the offset distance in the vertical direction. The specific adjustment method is not limited to the direction of the panoramic picture movement being opposite to the direction of the operator's face rotation, making the direction of the panoramic picture movement the same as the direction of the operator's face rotation can also achieve the purpose of browsing the panoramic picture in this embodiment. .

In addition, the face of the target person identified in the above steps can be displayed on the screen of the display 275, so that the target person can always observe the angle of his face, and then appropriately determine the rotation angle based on the range of the panoramic picture movement. Adjust to achieve a proper adjustment of the panoramic picture, the specific display position can be set in the upper right corner of the screen, etc. For some other characters in front of the display device 200, after observing the target person on the screen, they will also know who the actual operator is, so that they will not be too close to the display device 200, so as not to affect the viewing of panoramic pictures.

It can be seen from the above content that the panoramic picture browsing method in the embodiment of the present application can realize the browsing of the panoramic picture by displaying the face angle change of the target person in front of the device, and the offset distance of the panoramic picture can be determined by the angle of the target person’s face rotation. The size is determined, so that the operator can browse the panoramic picture by himself according to his own needs, and there is no need to adjust the panoramic picture according to a fixed moving step.

In some embodiments, the step of obtaining the offset distance of the panoramic picture by using the initial face angle and the current face angle includes:

Step S401: Calculate the horizontal offset distance of the panoramic picture by using the initial roll angle, the current roll angle, the initial yaw angle and the current yaw angle.

In some embodiments, the following formula may be used to calculate the horizontal offset distance of the panoramic picture:

Among them, Offset _X represents the horizontal offset distance of the panoramic picture, Roll ₁ and Roll ₂ represent the initial roll angle and the current roll angle, Yaw ₁ and Yaw ₂ represent the initial yaw angle and the current yaw angle, and A1 represents the correction Coefficient, delta _X represents the minimum adjustment angle value in the horizontal direction, and Time represents the minimum adjustment offset time.

Step S402: Calculate the vertical offset distance of the panoramic picture by using the initial pitch angle, the current pitch angle, the initial roll angle and the current roll angle.

In some embodiments, the following formula may be used to calculate the vertical offset distance of the panoramic picture:

Among them, Offset _Y represents the vertical offset distance of the panoramic picture, Roll ₁ and Roll ₂ represent the initial roll angle and the current roll angle, Pitch ₁ and Pitch ₂ represent the initial pitch angle and the current pitch angle, and A2 represents the correction coefficient. delta _Y represents the minimum adjustment angle value in the vertical direction, and Time represents the minimum adjustment offset time.

In this embodiment, delta _X and delta _Y need to be determined according to the resolution of the panoramic picture. Generally, the larger the resolution of the panoramic picture, the larger _{the angle of delta X} and delta _Y ; Time needs to be determined according to the first preset duration , Generally, the longer the first preset duration, the longer the Time. The values of A1 and A2 are usually less than 0.5.

FIG. 11 is a schematic diagram of the initial face angle of the target person obtained by the controller 110 according to an embodiment of the application, and FIG. 12 is a schematic diagram of the current face angle of the target person obtained by the controller 110 according to an embodiment of the application. In FIG. 12, the content in the dashed frame represents the face range of the target person recognized by the controller 110. In Figure 11, the initial roll angle of the target person’s face Roll ₁ is -3.2°, the initial yaw angle Yaw ₁ is 4.2°, and the initial pitch angle Pitch ₁ is -20.9°; in Fig. 12, the current roll of the target person’s face The angle Roll ₂ is -0.4°, the current yaw angle Yaw ₂ is 20.5°, and the current pitch angle Pitch ₂ is -2.0°. Taking the _{values of delta X} and delta _Y as 0.5°, Time as 2S, and A1 and A2 as 0.25 and 0.15 respectively, the controller 110 can respectively calculate the horizontal offset distance of the panoramic image as:

Then calculate the vertical offset distance of the panoramic picture as:

It can be seen that when the face angle of the target person changes from (-3.2°, 4.2°, -20.9°) to (-0.4°, 20.5°, -2.0°), the offset distance of the panoramic image is (1.71875, 1.40875) ), the controller 110 can finally control the panoramic picture to move a corresponding distance in the horizontal direction and the vertical direction, thereby displaying more content of the panoramic picture.

In addition, in order to be able to clarify the moving direction and distance of the panoramic picture, in some embodiments, the controller 110 may also pre-determine the coordinate system on the screen of the display 275, and may establish a two-dimensional rectangular coordinate system with the four vertices of the screen as the coordinate origin. , You can also establish a two-dimensional rectangular coordinate system with the exact center of the screen as the coordinate origin, and then move the panoramic picture based on the established coordinate system.

In some embodiments, after identifying the target person in front of the display device 200, the following steps are further included:

In step S501, if the target person in front of the display device 200 is not recognized after a preset period of time, the pixel area corresponding to each human face currently existing in front of the display device 200 is recalculated.

In some cases, the operator who initially operated the panoramic picture browsing may quit the operation midway, but other operators still need to continue browsing. At this time, the detector 230 cannot recognize the original target person again, and a new target person needs to be determined again. The method for determining the new target person is as described in the foregoing, and the face pixel area of all the characters in front of the display device 200 can also be judged, and then the person closest to the display device 200 is selected as the target person.

In step S502, the person corresponding to the face with the largest pixel area is selected as the target person.

Generally, the closer the person is to the display device 200, the larger the pixel area of the person's face. In the display device 200, when the detector 230 collects an image in front of the display device 200, the controller 110 can identify the face images belonging to the person, and then further calculate the pixel area corresponding to each face image.

Step S503: Obtain the initial face angle of the target person.

Step S504: After a second preset time period, obtain the current face angle of the target person.

Compared with the first preset duration, the second preset duration in this embodiment is essentially a preset duration, but the time range of the second preset duration is larger than the time range of the first preset duration For example, if the first preset duration is T1, then the second preset duration may be 2T1. In actual scenes, the first preset duration T1 is usually set to 0.3 seconds.

In some embodiments, after obtaining the offset distance of the panoramic picture, in order to ensure the operating experience of the operator and ensure the adjustment effect of the panoramic picture, the panoramic picture may also be adjusted after waiting for a certain period of time. This length of time needs to ensure the efficiency of the panoramic picture adjustment and the adjustment effect of the panoramic picture. It is usually set to 2 seconds.

In some embodiments, the panoramic picture browsing method not only needs to recognize the target person and determine whether the target person has changed, but also determine whether the currently browsed panoramic picture has changed. If the panoramic picture is adjusted according to the offset distance, After the picture is displayed on the display device 200, the controller 110 detects that the panoramic picture currently displayed on the display 275 is not the previous one, and then the process of identifying the target person in the above embodiment needs to be performed again for the current new panoramic picture. , And then adjust the new panoramic picture; if after adjusting the display content of the panoramic picture on the display device 200 according to the offset distance, the controller 110 detects that the panoramic picture currently displayed on the display 275 is still the previous picture, then controls The device 110 can directly recognize the face angle change of the target person in front of the display device 200, so as to realize the next adjustment of the panoramic picture.

It can be seen from the above technical solutions that the embodiment of the present application provides a panoramic picture browsing method. When the display device 200 displays the panoramic picture, the target person in front of the display device 200 can be recognized, and then the initial face angle and the target person’s initial face angle and The current face angle after the preset duration is used to calculate the offset distance that the panoramic picture on the display device 200 needs to move by using the initial face angle and the current face angle. Finally, the panoramic picture is adjusted according to the offset distance so that the panoramic picture has not been previously The displayed content is displayed on the display device 200. The solution of the present application can realize the browsing of the panoramic picture through the change of the face angle of the target person in front of the display device 200, and the offset distance of the panoramic picture can be determined by the angle of the target person’s face rotation, and the operator can follow his own You need to browse the panoramic pictures by yourself, no need to adjust the panoramic pictures according to the fixed moving step.

The present application also provides a display device 200, including: a display 275; a detector 230 for collecting an image in front of the display device 200; and a controller 110 configured to: recognize when the display device 200 displays a panoramic picture Display the target person in front of the device 200, where the target person is used to indicate the operator who browses the panoramic picture in front of the display device 200; obtain the initial face angle and the current face angle of the target person before and after the first preset duration; The offset distance of the panoramic picture is obtained by using the initial face angle and the current face angle; and the display content of the panoramic picture on the display device 200 is adjusted according to the offset distance.

In some embodiments, the controller 110 is further configured to detect whether there is a preset person in front of the display device 200, and the preset person is used to represent the operator who browses the panoramic picture pre-stored in the display device 200; if there is , It is determined that the preset person in front of the display device 200 is the target person.

In some embodiments, the controller 110 is further configured to: when multiple characters exist in front of the display device 200 and there are no preset characters, respectively calculate the pixel area corresponding to the face of each character; select The person corresponding to the face with the largest pixel area is the target person.

In some embodiments, the controller 110 is further configured to: obtain the initial roll angle, the initial pitch angle, and the initial yaw angle of the face of the target person in front of the display device 200; after the first preset time period has elapsed, obtain the display device 200 The current roll angle, current pitch angle, and current yaw angle of the front target person's face.

In some embodiments, the controller 110 is further configured to calculate the horizontal offset distance of the panoramic picture using the initial roll angle, the current roll angle, the initial yaw angle, and the current yaw angle; The pitch angle, the initial roll angle and the current roll angle are used to calculate the vertical offset distance of the panoramic picture.

In some embodiments, the controller 110 is further configured to calculate the horizontal offset distance of the panoramic picture by using the following formula:

In some embodiments, the controller 110 is further configured to calculate the vertical offset distance of the panoramic picture by using the following formula:

In some embodiments, the controller 110 is further configured to: if the target person in front of the display device 200 is not recognized after a preset period of time, recalculate the pixel area corresponding to each face currently existing in the display device 200; The person corresponding to the face with the largest pixel area is selected as the target person; the initial face angle of the target person is obtained; after a second preset time period, the current face angle of the target person is obtained.

In order to realize that when the user uses the display device at a far distance, the user can still see the display content of the control clearly, the display device provided by the embodiment of the present invention can control the control when the user moves the position while using the display device. Adjust the position according to the user's movement. For example, if the user moves to the left in front of the display device, the control follows the user's movement and moves the corresponding position to the left in the display interface, so that the perspective of the user and the control remains unchanged, thereby ensuring the user Regardless of the position of the display device, the display content of the control can be clearly seen from a certain angle of view.

Specifically, a display device provided by an embodiment of the present invention includes a controller, and a display and a camera respectively communicating with the controller. The camera is configured to collect environmental image data. The environmental image data is used to characterize the user's position parameters relative to the display. The camera sends the collected environmental image data to the controller, and the controller can obtain the user's position parameters; the position parameters include user and The vertical distance between the displays and the position of the user's face frame on the display interface when the center point of the user's face frame falls vertically on the display; the face frame is where the camera falls on the user's face when the camera captures the image of the user in front of the display device For the calibration frame, the center point of the face frame can be the center position of the face frame or the center position between the two pupils of the user. The display is configured to present a display interface, and a target control is displayed in the display interface. The target control can be a notification, a pop-up frame, or a floating window.

In order to make the display device have the function of dynamically adjusting the position of the control, the control key that realizes the function of dynamically adjusting the position of the control can be configured in the controller. When the control on the display page needs to be controlled by the display device to adjust the position following the movement of the user, it can be turned on in advance The control key enables the display device to dynamically adjust the position of the control. If the control key is not turned on, the control is displayed normally, and when the user moves, the control does not adjust the position according to the user's movement.

FIG. 13 exemplarily shows a flowchart of a method for dynamically adjusting a control according to an embodiment. Referring to FIG. 13, in the display device provided by the embodiment of the present invention, when realizing the dynamic adjustment of the control, the controller realizes the control based on the face recognition and distance detection algorithm. Specifically, the controller is configured to perform the following steps:

S1. During the process of the user moving from the initial position to the end position, obtain the initial position parameter of the target control, and receive the environmental image data corresponding to the initial position and the end position collected by the camera.

After the user turns on the control key, the display device has the function of dynamically adjusting the position of the control, and the controller obtains the environmental image data collected by the camera in real time. If the position of the user using the display device changes, the position before the position change is taken as the initial position of the user, and the position after the position change is taken as the end position of the user.

Since the camera collects the environmental image data in front of the display device in real time, the environmental image data corresponding to the user's initial position and the environmental image data corresponding to the end position can be obtained. The environmental image data corresponding to different positions can represent different relative distances between the user and the display and different positions on the display interface when the center point of the user's face frame falls vertically on the display.

In order to adjust the position of the control, the user's face must be included in the environmental image data collected by the camera. In some embodiments, the controller recognizes the number of human faces on the environmental image data, and when only one human face is recognized, continues to execute the subsequent method of dynamically adjusting the control.

Specifically, the controller is further configured to: receive the environmental image data collected by the camera; recognize the number of faces in the environmental image data; when the number of faces in the environmental image data is 1, execute the acquisition of user initial position parameters and user end Positional parameter steps.

When only one user uses the display device, if the method of dynamically adjusting the control is executed to adjust the position of the control, and the position of the control is controlled according to a user's position parameter, the accuracy can be improved. If the number of faces is more than one, the controller can control the normal display of the target control without executing the method of dynamically adjusting the control.

In other embodiments, if there are multiple user faces in the same environmental image data, the controller can also choose one of the users as the target follower, and the target follower is used as the control control to adjust the position. in accordance with.

FIG. 14 exemplarily shows a flowchart of a method for obtaining the initial position parameter of a target control according to an embodiment; FIG. 15 exemplarily shows a schematic diagram of a reference coordinate system according to an embodiment. In order to adjust the position of the target control, the initial position of the target needs to be determined first. See Figure 14. The controller is executed to obtain the initial position parameters of the target control, and is further configured to perform the following steps:

S121. Use the upper left corner of the display interface as the origin of the coordinates, the direction from the left to the right of the display interface as the positive direction of the X axis, and the direction from the upper side to the lower side of the display interface as the positive direction of the Y axis to establish a reference coordinate system .

In order to accurately determine the initial position parameters of the target control, in this embodiment, a reference coordinate system can be established in the display interface. Referring to Figure 15, the coordinate origin O of the reference coordinate system is set at the upper left corner of the display interface, the positive direction of the X axis is the direction from the left to the right of the display interface, and the positive direction of the Y axis is the direction from the top to the bottom of the display interface. direction.

S122: Obtain the number of pixels at the origin of the coordinates and the number of horizontal pixels and the number of vertical pixels at the control center point of the target control.

The initial position parameter of the target control can be represented by coordinate values, and the horizontal and vertical coordinate values can be calculated according to the pixel points of the control center point of the target control.

To this end, according to the system properties of the display device, the controller can obtain the resolution of the current display device, and then can determine the number of pixels at the origin of the coordinate and the number of pixels at the control center point of the target control. Since the coordinate origin is located on the leftmost side of the display interface, the number of pixels at which the coordinate origin can be determined equivalently is 0.

In order to accurately represent the initial position parameter of the target control, in this embodiment, the coordinate position of the control center point M of the target control is used to represent. To this end, the number of horizontal pixels and the number of vertical pixels of the control center point M of the target control are respectively obtained. The number of horizontal pixels refers to the number of pixels contained in the X-axis direction between the control center point M of the target control and the coordinate origin O. , The number of vertical pixels refers to the number of pixels contained in the Y-axis direction between the control center point M of the target control and the coordinate origin O.

S123: Calculate the difference in the number of horizontal pixels between the number of pixels at the origin of the coordinates and the number of horizontal pixels at the center of the control, and the difference between the number of pixels at the origin of the coordinates and the number of vertical pixels at the center of the control.

The number of pixels of the coordinate origin O is 0, and the coordinates of the corresponding coordinate origin are (0, 0). The number of horizontal pixels of the control center point is P ₁ , the number of vertical pixels of the control center point is P ₂ , and the pixel coordinates of the corresponding control center point are (P ₁ , P ₂ ).

The difference in the number of horizontal pixels M ₁ =the number of horizontal pixels at the center of the control-the number of pixels at the origin of the coordinate=P ₁ -0=P ₁ .

Longitudinal number of pixels difference between the number of pixels M longitudinal center point of control = ₂ - the number of pixels to the coordinate origin = P _₂ -0 = P _2.

S124. According to the difference in the number of horizontal pixels, the difference in the number of vertical pixels, and the length of each pixel, calculate the horizontal initial distance and the vertical initial distance between the control center point of the target control and the coordinate origin, and calculate the horizontal initial distance and the vertical initial distance , The number of horizontal and vertical pixels at the center of the control is used as the initial position parameter of the target control.

For a fixed-resolution display device, the number of pixels included in its display interface is certain, that is, one resolution corresponds to a set of pixel numbers. Therefore, the difference between two adjacent pixels can be obtained, that is, the length value of each pixel. If the pixel is square, the length and width of the pixel are the same. By multiplying the difference in the number of pixels by the length of each pixel, the corresponding distance can be obtained.

The horizontal initial distance L ₁ =the difference in the number of horizontal pixels×the length value of the pixel point=M ₁ ×n, the horizontal initial distance between the control center point of the target control and the coordinate origin can be determined.

Longitudinal initial distance L ₂ = difference in number of longitudinal pixels × length value of pixel point = M ₂ × n, the initial longitudinal distance between the control center point of the target control and the coordinate origin can be determined.

The coordinates of the control center point of the target control are determined by the horizontal initial distance and the vertical initial distance, that is, the coordinates of the control center point are (L ₁ , L ₂ ); the pixels of the control center point of the target control are determined by the number of horizontal and vertical pixels Point coordinates, that is, the pixel coordinates of the center point of the control are (P ₁ , P ₂ ). The coordinates of the control center point and the pixel coordinates of the control center point are used as the initial position parameters of the target control.

For example, taking a 60-inch display device as an example, the length of its long side is fixed at about 135.5cm. For a commonly used display interface with a resolution of 1080P, the corresponding pixel difference (the length of the pixel) is about 0.00071m. If the pixel coordinate of the control center point of the target control is (720P, 480P), then the horizontal initial distance L ₁ =720×0.00071=0.112m and the vertical initial distance L ₂ =480×0.00071=0.3408m can be determined, that is, determine the target The coordinates of the control center point of the control are (0.5112m, 0.3408m).

S2. Acquire the user initial position parameter carried in the environmental image data corresponding to the initial position, and the user end position parameter carried in the environmental image data corresponding to the end position.

Since the environmental image data collected by the camera can represent the vertical distance between the user and the display and the position of the user’s face frame on the display interface when the center point of the user’s face frame falls vertically on the display, therefore, the controller can be based on the acquired environment The user's initial position parameter and the user's end position parameter are directly retrieved from the image data.

FIG. 16 exemplarily shows a schematic diagram of the environmental image data corresponding to the initial position in the embodiment; FIG. 17 exemplarily shows a schematic diagram of the environmental image data corresponding to the end position in the embodiment.

Referring to FIG. 16, when the user is in the initial position in front of the display device, the controller can directly obtain the vertical distance (relative distance) between the user and the display from the corresponding environmental image data, for example, 1.70 m as shown in FIG. 16. Referring to Fig. 17, when the user moves from the initial position to the end position, the controller can directly obtain the vertical distance (relative distance) between the user and the display from the environmental image data corresponding to the end position, for example, 2.24 shown in Fig. 17 m.

FIG. 18 exemplarily shows a schematic diagram of the position change of the center point of the face frame on the display interface during the movement of the user according to the embodiment. Referring to Figure 18, when the user is at the initial position A, when the center point of the user's face frame falls vertically on the display, the position on the display interface is point X, and the AX line is perpendicular to the display interface. When the user moves to the end position B, when the center point of the user's face frame falls vertically on the display, the position on the display interface is point N, and the BN connection is perpendicular to the display interface.

For this reason, when the user is at the initial position A, the line AX is the vertical distance (relative distance) between the user and the display, and point X is when the center point of the user's face frame falls vertically on the display. Position, therefore, the user's initial position parameter can be determined by connecting AX and point X. When the user moves to the end position B, the line BN is the vertical distance (relative distance) between the user and the display, and point N is the position on the display interface when the center point of the user's face frame falls vertically on the display. Therefore, connecting BN and point N can determine the user end position parameter.

S3. Calculate the offset of the target control based on the user's initial position parameter and the user's end position parameter, where the offset is used to characterize the movement parameter of the target control when adjusting the position.

In order to ensure that the user’s viewing angle when viewing the display device remains the same, that is, the user’s viewing angle when viewing the display content of the target control remains the same, no matter where the user is, he can clearly view the display content of the target control, the display provided in this embodiment The device needs to control the target control to follow the user's movement to adjust the position when the user moves the position.

Therefore, in order to accurately determine the parameters of the target control when adjusting the position, the offset of the target control can be calculated according to the user's initial position parameter and the user's end position parameter, and the position parameter that the target control needs to move is determined by the offset.

The user’s initial position parameters include the user’s initial relative distance to the display and initial position parameters. The user’s end position parameters include the user’s end relative distance to the display and the end position parameters. The user’s corresponding position parameters refer to the parameters of the center point of the face frame. . The initial position parameter refers to the position on the display interface when the center point of the user's face frame falls vertically on the display when the user is in the initial position. The end position parameter refers to the position on the display interface where the center point of the user's face frame falls vertically on the display when the user moves to the end position.

Fig. 19 exemplarily shows a flow chart of the method for calculating the offset of the target control according to the embodiment. Referring to Figure 19, the controller is further configured to perform the following steps in the process of calculating the offset of the target control based on the user's initial position parameter and the user's end position parameter:

S31. Calculate the first distance between the initial position parameter corresponding to the user and the initial position parameter of the target control, and the second distance between the end position parameter corresponding to the user and the initial position parameter of the target control, and the position corresponding to the target control The parameter refers to the parameter of the center point of the control.

The first distance refers to the planar distance between the position X on the display interface and the control center point M of the target control when the center point of the user's face frame falls vertically on the display when the user is at the initial position A; The distance refers to the planar distance between the position N on the display interface and the control center point M of the target control when the center point of the user's face frame falls vertically on the display when the user moves to the end position B.

Taking the user's translation from the initial position to the end position as an example, the first distance (XM line) represents the horizontal plane distance along the display interface, and the second distance (NM line) represents the horizontal plane distance along the display interface. In this embodiment, the first distance and the second distance can be calculated by the pixel point difference between the center point of the face frame and the control center point of the target control.

Specifically, when the controller executes the calculation of the first distance between the initial position parameter corresponding to the user and the initial position parameter of the target control, it is further configured to:

Step 311: Obtain the number of pixels at the center point of the face frame when the user is at the initial position and the number of pixels at the control center point of the target control.

Step 312: Calculate the difference between the number of pixels at the center of the face frame and the number of pixels at the center of the control.

Step 313: Calculate the first distance between the initial position parameter corresponding to the user and the initial position parameter of the target control according to the difference in the number of pixels and the length value of each pixel.

The number of pixels at the center point of the face frame when the user is at the initial position can be obtained by the controller from the environmental image data corresponding to the initial position, and the number of pixels at the control center point of the target control can be obtained according to the system properties. Both pixel points can read the corresponding pixel point coordinates in the reference coordinate system.

Taking the user's translation from the initial position to the end position as an example, the number of pixels at the center point of the face frame does not change in the Y-axis direction. Therefore, the number of pixels at the center point of the face frame corresponding to the initial position and the control center point The number of pixels, that is, the difference in the number of pixels is calculated based on the number of horizontal pixels at the center of the face frame and the number of horizontal pixels at the center of the control. For the specific calculation method of the pixel point difference and the first distance, please refer to the content of steps S121 to S124 provided in the foregoing embodiment, which will not be repeated here.

For example, when the user is at the initial position A, the number of pixels at the center of the face frame X is 480P, and the number of pixels at the center of the control of the target control is 720P. Therefore, the difference in the number of pixels is determined to be 720P-480P=240P.

The length value of each pixel 0.00071m, to determine a first distance S ₁ is at 240 × 0.00071 = 0.1704m.

The controller performs the calculation of the second distance between the end position parameter corresponding to the user and the initial position parameter of the target control, which is further configured to:

Step 321: Obtain the number of pixels at the center point of the face frame when the user is at the end position and the number of pixels at the center point of the control of the target control.

Step 322: Calculate the difference between the number of pixels at the center of the face frame and the number of pixels at the center of the control.

Step 323: Calculate the second distance between the end position parameter corresponding to the user and the initial position parameter of the target control according to the difference in the number of pixels and the length value of each pixel.

Taking the user's translation from the initial position to the end position as an example, the number of pixels at the center point of the face frame has not changed in the Y-axis direction. Therefore, the number of pixels at the center point of the face frame corresponding to the end position and the control center point The number of pixels, that is, the difference in the number of pixels is calculated based on the number of horizontal pixels at the center of the face frame and the number of horizontal pixels at the center of the control. The specific calculation method of the pixel point difference and the second distance can refer to the content of steps S121 to S124 provided in the foregoing embodiment, which will not be repeated here.

For example, when the user is at the end position B, the number of pixels at the center point of the face frame X is 360P, and the number of pixels at the center point of the control of the target control is 720P. Therefore, the difference in the number of pixels is determined to be 720P-360P=360P.

According to the length value of each pixel point of 0.00071m, the second distance S ₂ can be determined to be 360×0.00071=0.2556m.

S32. Calculate the theoretical second distance when the user moves to the end position based on the initial relative distance, the end relative distance, and the first distance. The theoretical second distance is used to characterize the distance between the end position corresponding to the user and the end position of the target control. Theoretical distance.

Since the second distance from the control center point of the target control after the user moves the position cannot guarantee that when the user is at the end position, the user’s viewing angle of the target control is the same as the viewing angle of the target control when the user is at the initial position. Therefore, it is necessary to change the target control’s To adjust the position, it is necessary to determine the theoretical second distance required for the user to view the target control with the same angle of view when the user moves to the end position.

In this embodiment, the controller calculates the theoretical second distance when the user moves to the end position based on the initial relative distance, the end relative distance, and the first distance as follows:

S ₂ '=BN·S ₁ /AX;

In the formula, S ₂ ′ is the theoretical second distance, S ₁ is the first distance, AX is the initial relative distance, and BN is the ending relative distance.

FIG. 20 exemplarily shows a schematic diagram when the theoretical second distance is determined according to the embodiment. Refer to Figures 18 and 13, in order to ensure that the user's viewing angle of the target control is consistent during the movement, the line between the center point of the face frame and the center point of the control must be the same as the line between the center point of the face and the display interface. , That is, α and β are the same, α is the angle between the line between the center point of the face frame and the center point of the control and the line between the center point of the face and the display interface when the user is in the initial position, that is, the AM line and AX The angle of the connection; β is the angle formed by the connection between the center of the face frame and the center of the control and the connection between the center of the face and the display interface when the user moves to the end position, that is, the BM connection and BN The angle of the connection.

In order to make α=β, tan(α)=tan(β), that is, S ₁ /AX=S ₂ '/BN, then the theoretical second distance S ₂ '=BN·S ₁ /AX can be calculated.

S33. Calculate the distance difference between the theoretical second distance and the second distance to obtain the offset of the target control.

The theoretical second distance is the theoretical distance between the end position corresponding to the user and the end position M'of the target control. Therefore, the offset Offset of the target control is obtained according to the distance difference between the theoretical second distance and the second distance.

Offset=S ₂ -S ₂ '.

The offset can realize the distance from the control center point M of the target control to the point M'.

S4. Based on the initial position parameter of the target control and the offset of the target control, the end position parameter of the target control is obtained, and the target control is moved to the position corresponding to the end position parameter.

According to the initial position parameter of the target control and the offset of the target control, it is possible to determine the end position of the target control after the position needs to be adjusted, and realize the position adjustment of the target control according to the end position parameter.

FIG. 21 exemplarily shows the first schematic diagram when the position of the control is dynamically adjusted according to the embodiment. Referring to Figure 21, point M is the initial position parameter of the target control, and point M'is the end position parameter of the target control. Move the target control from point M to point M'to realize the position adjustment of the target control when the user moves from the initial position A to the end position B. At this time, the end position parameter of the target control = initial position parameter-offset.

The foregoing embodiment is based on a situation in which the position adjustment of the target control is realized when the user moves from the initial position A to the end position B. In practical applications, the user may also move in the vertical direction during the movement, that is, the user changes from a standing state to a sitting state. At this time, in the vertical direction, the user also has a position in the Y-axis direction. Change.

In order to adapt to the situation where the user changes both in the X-axis direction and the Y-axis direction, the display device provided by the embodiment of the present invention needs to determine the horizontal offset and the vertical offset when determining the offset of the target control. For example, when the user changes from a state of standing directly in front of the display device to sitting on a chair at the rear left, the target control needs to be controlled to move from the initial position to the lower left corner.

At this time, the user initial position parameter includes a horizontal initial position parameter and a vertical initial position parameter, and the user end position parameter includes a horizontal end position parameter and a vertical end position parameter. The horizontal initial position parameter includes the horizontal initial relative distance and the horizontal initial position parameter of the user in the initial position relative to the display, and the vertical initial position parameter includes the vertical initial relative distance and the vertical initial position parameter of the user in the initial position relative to the display. The horizontal end position parameter includes the horizontal initial relative distance and the horizontal initial position parameter of the user at the end position relative to the display, and the vertical end position parameter includes the vertical initial relative distance and the vertical initial position parameter of the user at the end position relative to the display.

The vertical relative distance (including the initial position and the end position) refers to the corresponding distance when the center point of the face frame moves along the Y axis, that is, the height of the center point of the face frame when the user is standing and the center point of the face frame when the user sits down The height difference. The vertical position parameter (including the initial position and the end position) refers to the position on the display interface when the center point of the user's face frame falls vertically on the display when the user moves vertically to the end position.

Specifically, the controller calculates the offset of the target control based on the user's initial position parameter and the user's end position parameter, and is further configured to:

Step 701: Calculate the lateral offset of the target control based on the lateral initial position parameter and the lateral end position parameter.

Step 702: Calculate the longitudinal offset of the target control based on the longitudinal initial position parameter and the longitudinal end position parameter.

When calculating the horizontal offset and vertical offset of the target control, you can refer to all the content described in step S3 provided in the foregoing embodiment for calculation, which can realize the calculation of the target from the horizontal initial position parameter and the horizontal end position parameter The horizontal offset of the control, and the vertical offset of the target control are calculated from the vertical initial position parameter and the vertical end position parameter. The specific calculation process is not repeated here.

After determining the horizontal offset and the vertical offset of the target control, the final position parameter of the target control after the adjustment of the target control can be determined according to the initial position parameter of the target control. In this embodiment, the initial position parameter of the target control includes a horizontal initial position parameter and a vertical initial position parameter. Therefore, the determined end position parameter of the target control also includes the horizontal end position parameter and the vertical end position parameter of the target control.

Specifically, the controller obtains the end position parameter of the target control based on the initial position parameter of the target control and the offset of the target control, and is further configured to:

Step 801: Calculate the horizontal end position parameter of the target control according to the horizontal initial position parameter and the horizontal offset of the target control.

Step 802: Calculate the longitudinal end position parameter of the target control according to the longitudinal initial position parameter and the longitudinal offset of the target control.

According to the horizontal initial position parameter and horizontal offset of the target control, you can determine the horizontal end position of the target control after adjusting the position, and according to the vertical initial position parameter and vertical offset of the target control, you can determine the target control needs The longitudinal end position after adjusting the position. According to the horizontal termination position parameter and the vertical termination position parameter, the position adjustment of the target control is realized, so that the target control can adjust the position following the movement of the user.

Fig. 22 exemplarily shows a second schematic diagram when the position of the control is dynamically adjusted according to the embodiment. Referring to Figure 22, when the user changes in both the X-axis direction and the Y-axis direction, for example, when the user changes from standing in front of the display device to sitting on a chair behind the left, the target control needs to be controlled by The initial position moves to the lower left corner.

At this time, the horizontal end position parameter of the target control = the horizontal initial position parameter-the horizontal offset, and the vertical end position parameter of the target control = the vertical initial position parameter + the vertical offset.

It can be seen from the foregoing embodiments that, in the display device provided by the embodiment of the present invention, when the user moves from the initial position to the end position, the controller receives the environmental image data corresponding to the initial position collected by the camera and the corresponding end position. Environmental image data to obtain user initial position parameters and user end position parameters; calculate the offset of the target control according to the user initial position parameters and user end position parameters; obtain based on the initial position parameters of the target control and the offset of the target control The end position parameter of the target control, move the target control to the position corresponding to the end position parameter. It can be seen that the display device provided by the embodiment of the present invention can adjust the position of the target control following the movement of the user, so that the user can watch the target control in any direction within the visual range of the camera of the display device, thereby ensuring that the user Both can clearly see the display content of the control and improve the user's subjective visual experience.

FIG. 13 exemplarily shows a flowchart of a method for dynamically adjusting a control according to an embodiment. This application also provides a method for dynamically adjusting controls, which is executed by a controller in a display device, and the method includes the following steps:

S1. During the process of the user moving from the initial position to the end position, obtain the initial position parameter of the target control, and receive the environmental image data corresponding to the initial position and the end position collected by the camera;

S2. Acquire the user initial position parameter carried in the environmental image data corresponding to the initial position, and the user end position parameter carried in the environmental image data corresponding to the end position;

S3. Calculate an offset of the target control based on the user initial position parameter and the user end position parameter, where the offset is used to characterize the movement parameter of the target control when adjusting the position;

S4. Obtain the end position parameter of the target control based on the initial position parameter of the target control and the offset of the target control, and move the target control to a position corresponding to the end position parameter.

In some embodiments of the present application, the calculating the offset of the target control based on the user initial position parameter and the user end position parameter includes:

The user's initial position parameter includes the user's initial relative distance to the display and initial position parameters, the user end position parameter includes the user's end relative distance to the display and end position parameters, and the user's corresponding position parameter refers to a person Parameters of the center point of the face frame;

Calculate the first distance between the initial position parameter corresponding to the user and the initial position parameter of the target control, and the second distance between the end position parameter corresponding to the user and the initial position parameter of the target control, the target The position parameter corresponding to the control refers to the parameter of the center point of the control;

Based on the initial relative distance, the end relative distance, and the first distance, calculate the theoretical second distance when the user moves to the end position, and the theoretical second distance is used to characterize the end position corresponding to the user and the termination of the target control The theoretical distance between locations;

The distance difference between the theoretical second distance and the second distance is calculated to obtain the offset of the target control.

In some embodiments of the present application, the calculating the first distance between the initial position parameter corresponding to the user and the initial position parameter of the target control includes:

Obtaining the number of pixels at the center point of the face frame when the user is at the initial position and the number of pixels at the center point of the control of the target control;

Calculating the difference between the number of pixels at the center point of the face frame and the number of pixels at the center point of the control;

According to the difference in the number of pixels and the length value of each pixel, the first distance between the initial position parameter corresponding to the user and the initial position parameter of the target control is calculated.

In some embodiments of the present application, the calculating the second distance between the end position parameter corresponding to the user and the initial position parameter of the target control includes:

Obtaining the number of pixels at the center point of the face frame when the user is at the end position and the number of pixels at the control center point of the target control;

According to the difference in the number of pixels and the length value of each pixel, the second distance between the end position parameter corresponding to the user and the initial position parameter of the target control is calculated.

In some embodiments of the present application, the calculation of the theoretical second distance when the user moves to the end position based on the initial relative distance, the end relative distance, and the first distance includes:

According to the formula S ₂ '=BN·S ₁ /AX, calculate the theoretical second distance when the user moves to the end position;

In some embodiments of the present application, the obtaining the initial position parameter of the target control includes:

Taking the upper left corner of the display interface as the origin of coordinates, the direction from the left to the right of the display interface as the positive X-axis, and the direction from the upper side to the lower side of the display interface as the positive Y-axis, Establish a reference coordinate system;

Acquiring the number of pixels of the coordinate origin and the number of horizontal pixels and the number of vertical pixels of the control center point of the target control;

Calculating the difference in the number of horizontal pixels between the number of pixels at the origin of the coordinates and the number of horizontal pixels at the center of the control, and the difference between the number of pixels at the origin of the coordinates and the number of vertical pixels at the center of the control;

According to the difference in the number of horizontal pixels, the difference in the number of vertical pixels, and the length value of each pixel, the horizontal initial distance and the vertical initial distance between the control center point of the target control and the coordinate origin are calculated, and the horizontal initial distance and the vertical initial distance are calculated. The initial distance, the number of horizontal pixels and the number of vertical pixels of the control center point are used as the initial position parameters of the target control.

The user initial position parameter includes a horizontal initial position parameter and a vertical initial position parameter, and the user end position parameter includes a horizontal end position parameter and a vertical end position parameter;

Calculating the lateral offset of the target control based on the lateral initial position parameter and the lateral end position parameter;

Based on the longitudinal initial position parameter and the longitudinal end position parameter, the longitudinal offset of the target control is calculated.

In some embodiments of the present application, the obtaining the end position parameter of the target control based on the initial position parameter of the target control and the offset of the target control includes:

The initial position parameter of the target control includes a horizontal initial position parameter and a vertical initial position parameter;

Calculating the lateral end position parameter of the target control according to the lateral initial position parameter and the lateral offset of the target control;

According to the longitudinal initial position parameter and the longitudinal offset of the target control, the longitudinal end position parameter of the target control is calculated.

In some embodiments of this application, it further includes:

Receiving environmental image data collected by the camera;

Identifying the number of human faces in the environmental image data;

When the number of human faces in the environmental image data is 1, the step of acquiring the user's initial position parameter and the user's end position parameter is performed.

In specific implementation, the present invention also provides a computer storage medium, wherein the computer storage medium may store a program, and the program may include some or all of the steps in each embodiment of the method for dynamically adjusting controls provided by the present invention when the program is executed. . The storage medium may be a magnetic disk, an optical disc, a read-only memory (English: read-only memory, abbreviated as ROM) or a random access memory (English: random access memory, abbreviated as: RAM), etc.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the application, not to limit them; although the application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: It is still possible to modify the technical solutions described in the foregoing embodiments, or equivalently replace some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the technical solutions of the embodiments of the present application. Scope.

For the convenience of explanation, the above description has been made in conjunction with specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or to limit the implementation to the specific form disclosed above. According to the above teaching, various modifications and variations can be obtained. The selection and description of the foregoing embodiments are for better explaining the principles and practical applications, so that those skilled in the art can better use the embodiments and various modified embodiments suitable for specific use considerations.

Claims

A display method, characterized in that it comprises:

In the case that the display device displays a panoramic picture, identifying a target person in front of the display device, where the target person is used to indicate an operator who browses the panoramic picture in front of the display device;

Acquiring the initial facial angle and the current facial angle of the target person before and after the first preset duration;

Obtaining the offset distance of the panoramic picture by using the initial face angle and the current face angle;

Adjust the display content of the panoramic picture on the display device according to the offset distance.
The method according to claim 1, wherein the step of identifying the target person in front of the display device comprises:

Detecting whether there is a preset person in front of the display device, the preset person being used to represent an operator who browses a panoramic picture pre-stored in the display device;

If it exists, it is determined that the preset person in front of the display device is the target person.
The method according to claim 2, wherein the step of identifying the target person in front of the display device further comprises:

In the case that there are multiple characters in front of the display device and there are no preset characters, respectively calculate the pixel area corresponding to the face of each character;

The person corresponding to the face with the largest pixel area is selected as the target person.
The method according to claim 1, wherein the step of obtaining the initial face angle and the current face angle of the target person before and after the first preset duration comprises:

Obtain the initial roll angle, initial pitch angle and initial yaw angle of the target person’s face in front of the display device;

After the first preset time period has elapsed, the current roll angle, the current pitch angle, and the current yaw angle of the face of the target person in front of the display device are acquired.
The method according to claim 4, wherein the step of obtaining the offset distance of the panoramic picture by using the initial face angle and the current face angle comprises:

Calculate the horizontal offset distance of the panoramic picture by using the initial roll angle, the current roll angle, the initial yaw angle and the current yaw angle;

The vertical offset distance of the panoramic picture is calculated by using the initial pitch angle, the current pitch angle, the initial roll angle and the current roll angle.
The method according to claim 5, wherein the following formula is used to calculate the horizontal offset distance of the panoramic picture:

Among them, Offset X represents the horizontal offset distance of the panoramic picture, Roll 1 and Roll 2 represent the initial roll angle and the current roll angle, Yaw 1 and Yaw 2 represent the initial yaw angle and the current yaw angle, and A1 represents the correction Coefficient, delta X represents the minimum adjustment angle value in the horizontal direction, and Time represents the minimum adjustment offset time.
The method according to claim 5, wherein the vertical offset distance of the panoramic picture is calculated using the following formula:

Among them, Offset Y represents the vertical offset distance of the panoramic picture, Roll 1 and Roll 2 represent the initial roll angle and the current roll angle, Pitch 1 and Pitch 2 represent the initial pitch angle and the current pitch angle, and A2 represents the correction coefficient. delta Y represents the minimum adjustment angle value in the vertical direction, and Time represents the minimum adjustment offset time.
The method according to claim 1, wherein after identifying the target person in front of the display device, the method further comprises:

If the target person in front of the display device is not recognized after the preset time period, recalculate the pixel area corresponding to each face currently existing in front of the display device;

Select the person corresponding to the face with the largest pixel area as the target person;

Acquiring the initial face angle of the target person;

After the second preset duration, the current face angle of the target person is acquired.
The method according to any one of claims 1-8, wherein the face of the target person is displayed on the display screen of the display device.
A display device, characterized in that it comprises:

monitor;

Detector, used to collect the image in front of the display device;

The controller is configured as:

In the case that the display device displays a panoramic picture, identifying a target person in front of the display device, where the target person is used to indicate an operator who browses the panoramic picture in front of the display device;

Acquiring the initial facial angle and the current facial angle of the target person before and after the first preset duration;

Obtaining the offset distance of the panoramic picture by using the initial face angle and the current face angle;

Adjust the display content of the panoramic picture on the display device according to the offset distance.