WO2020248697A1 - 显示设备及视频通讯数据处理方法 - Google Patents

显示设备及视频通讯数据处理方法 Download PDF

Info

Publication number
WO2020248697A1
WO2020248697A1 PCT/CN2020/084717 CN2020084717W WO2020248697A1 WO 2020248697 A1 WO2020248697 A1 WO 2020248697A1 CN 2020084717 W CN2020084717 W CN 2020084717W WO 2020248697 A1 WO2020248697 A1 WO 2020248697A1
Authority
WO
WIPO (PCT)
Prior art keywords
target
image
local
display device
video
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2020/084717
Other languages
English (en)
French (fr)
Inventor
于颜梅
杨鲁明
李志强
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hisense Visual Technology Co Ltd
Original Assignee
Hisense Visual Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hisense Visual Technology Co Ltd filed Critical Hisense Visual Technology Co Ltd
Publication of WO2020248697A1 publication Critical patent/WO2020248697A1/zh
Priority to US17/332,713 priority Critical patent/US11917329B2/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/4104—Peripherals receiving signals from specially adapted client devices
    • H04N21/4126—The peripheral being portable, e.g. PDAs or mobile phones
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/14—Systems for two-way working
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/14—Systems for two-way working
    • H04N7/141—Systems for two-way working between two video terminals, e.g. videophone
    • H04N7/147—Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/422—Input-only peripherals, i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/422—Input-only peripherals, i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS]
    • H04N21/42204—User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/478—Supplemental services, e.g. displaying phone caller identification, shopping application
    • H04N21/4788—Supplemental services, e.g. displaying phone caller identification, shopping application communicating with other users, e.g. chatting
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/485—End-user interface for client configuration
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30242—Counting objects in image

Definitions

  • This application relates to the technical field of display devices, and in particular to a display device, a video communication data processing method, a video communication method, and a display method.
  • This application provides a display device, a video communication data processing method, a video communication method, and a display method to solve the above problems.
  • this application provides a display device, including:
  • Display used to present the user interface
  • Image collector used to collect the local global image
  • a local preview window is presented in the user interface, and the local video picture is displayed in the local preview window according to the global image, wherein, when there is a moving target in the global image, the local video picture is all The partial image corresponding to the target stable area of the moving target;
  • local video data is generated according to the local video screen displayed in the local preview window and sent to the opposite terminal device.
  • this application also provides a display device, including:
  • Display used to present the user interface
  • the opposite end video picture is displayed in the opposite end display window according to the opposite end video data, wherein, when there is a moving target in the opposite end video picture, the opposite end video picture is the target stable of the moving target The screen corresponding to the area.
  • this application also provides a display device, including:
  • Display used to present the user interface
  • Image collector used to collect the local global image
  • a local preview window is presented in the user interface, and the local video picture is displayed in the local preview window according to the local global image, wherein when there is a moving target in the global image, the local video picture Is a partial image corresponding to the target stable area of the moving target.
  • this application also provides a video communication data processing method, which is applied to a display device, and the method includes:
  • this application also provides a video communication method, which includes:
  • the first display device In response to the establishment of a video communication connection with the second display device, the first display device receives the opposite end video data sent by the second display device;
  • the opposite end video picture is displayed in the opposite end display window according to the opposite end video data, wherein, when there is a moving target in the opposite end video picture, the opposite end video picture is the target stable of the moving target The screen corresponding to the area.
  • the present application also provides a display method applied to a display device, the display device having an image collector, and the method includes:
  • a local preview window is presented in the user interface, and the local video picture is displayed in the local preview window according to the local global image, wherein when there is a moving target in the global image, the local video picture Is a partial image corresponding to the target stable area of the moving target.
  • Fig. 1 exemplarily shows a schematic diagram of an operation scene between a display device and a control device
  • FIG. 2 exemplarily shows a hardware configuration block diagram of the control device 100
  • FIG. 3 exemplarily shows a hardware configuration block diagram of the display device 200
  • FIG. 4 exemplarily shows a block diagram of the hardware architecture of the display device 200 according to FIG. 3;
  • FIG. 5 exemplarily shows a schematic diagram of the functional configuration of the display device 200
  • Fig. 6a exemplarily shows a schematic diagram of the software configuration in the display device 200
  • FIG. 6b exemplarily shows a configuration diagram of an application program in the display device 200
  • FIG. 7 exemplarily shows a schematic diagram of the user interface of the display device 200
  • Fig. 8a is a scenario in which a user interacts with a display device through the control apparatus 100A according to an exemplary embodiment of the application;
  • Fig. 8b is another scenario in which a user interacts with a display device through the control device 100A according to an exemplary embodiment of the application;
  • Figure 9a is a schematic diagram of an application scenario of the technical solution of this application.
  • Figure 9b is a schematic diagram of another application scenario of the technical solution of this application.
  • Figure 10a shows the video interface presented by the local device during a two-person video chat
  • Figure 10b shows another video interface presented by the local device during a two-person video chat
  • Figure 10c shows a video interface presented by the local device during a four-person video chat
  • Figure 10d shows another video interface presented by the local device when two people are in a video chat
  • FIG. 11 is a flowchart of an embodiment of a method for processing video communication data based on a display device according to the present application
  • Fig. 12 is a schematic diagram of interaction between display devices participating in video communication according to an exemplary embodiment of this application.
  • the display device provided in this application may be a display device with multiple chip architectures, such as the display device with a dual-chip (dual hardware system) architecture shown in Figures 3 to 5 of this application, or a non-dual-chip architecture
  • the display device of this application is not limited.
  • various external device interfaces are usually provided on the display device to facilitate the connection of different peripheral devices or cables to realize corresponding functions.
  • a high-definition camera is connected to the interface of the display device, if the hardware system of the display device does not have the hardware interface of the high-pixel camera that receives the source code, then the data received by the camera cannot be presented to the display of the display device. On the screen.
  • module used in the various embodiments of this application can refer to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or a combination of hardware or/and software code that can execute related to the component Function.
  • remote control used in the various embodiments of this application refers to a component of an electronic device (such as the display device disclosed in this application), which can generally control the electronic device wirelessly within a short distance.
  • This component can generally use infrared and/or radio frequency (RF) signals and/or Bluetooth to connect to electronic devices, and can also include functional modules such as WiFi, wireless USB, Bluetooth, and motion sensors.
  • RF radio frequency
  • a handheld touch remote control uses a user interface in a touch screen to replace most of the physical built-in hard keys in general remote control devices.
  • gesture used in the embodiments of the present application refers to a user's behavior through a change of hand shape or hand movement to express expected ideas, actions, goals, and/or results.
  • the term "hardware system” used in the various embodiments of this application may refer to a computing device composed of mechanical, optical, electrical, and magnetic devices such as integrated circuit (IC) and printed circuit board (PCB). , Control, storage, input and output functions.
  • IC integrated circuit
  • PCB printed circuit board
  • Control storage, input and output functions.
  • the hardware system is usually also referred to as a motherboard or chip.
  • Fig. 1 exemplarily shows a schematic diagram of an operation scenario between a display device and a control device according to an embodiment. As shown in FIG. 1, the user can operate the display device 200 by controlling the device 100.
  • the control device 100 may be a remote controller 100A, which can communicate with the display device 200 through infrared protocol communication, Bluetooth protocol communication, ZigBee protocol communication or other short-distance communication methods for wireless or other communication
  • the display device 200 is controlled in a wired manner.
  • the user can control the display device 200 by inputting user instructions through keys on the remote control, voice input, control panel input, etc.
  • the user can control the display device 200 by inputting corresponding control commands through the volume plus and minus keys, channel control keys, up/down/left/right movement keys, voice input keys, menu keys, and switch buttons on the remote control. Function.
  • the control device 100 can also be a smart device, such as a mobile terminal 100B, a tablet computer, a computer, a notebook computer, etc., which can be connected through a local area network (LAN, Wide Area Network), a wide area network (WAN, Wide Area Network), and a wireless local area network ((WLAN) , Wireless Local Area Network) or other networks communicate with the display device 200, and realize the control of the display device 200 through an application program corresponding to the display device 200.
  • LAN Local area network
  • WAN Wide Area Network
  • WLAN wireless local area network
  • the application can provide users with various controls through an intuitive user interface (UI, User Interface) on the screen associated with the smart device.
  • UI User Interface
  • User interface is a medium interface for interaction and information exchange between applications or operating systems and users. It realizes the conversion between the internal form of information and the form acceptable to users.
  • the commonly used form of the user interface is a graphical user interface (graphic user interface, GUI), which refers to a user interface related to computer operations that is displayed in a graphical manner. It can be an icon, window, control and other interface elements displayed on the display screen of an electronic device.
  • the control can include icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, Widgets, etc. Visual interface elements.
  • both the mobile terminal 100B and the display device 200 can be installed with software applications, so that the connection and communication between the two can be realized through a network communication protocol, thereby realizing one-to-one control operation and data communication.
  • the mobile terminal 100B can establish a control command protocol with the display device 200, synchronize the remote control keyboard to the mobile terminal 100B, and control the display device 200 by controlling the user interface of the mobile terminal 100B; or the mobile terminal 100B
  • the audio and video content displayed on the screen is transmitted to the display device 200 to realize the synchronous display function.
  • the display device 200 can also communicate with the server 300 through multiple communication methods.
  • the display device 200 may be allowed to communicate with the server 300 via a local area network, a wireless local area network, or other networks.
  • the server 300 may provide various contents and interactions to the display device 200.
  • the display device 200 transmits and receives information, interacts with an Electronic Program Guide (EPG, Electronic Program Guide), receives software program updates, or accesses a remotely stored digital media library.
  • EPG Electronic Program Guide
  • the server 300 may be a group or multiple groups, and may be one or more types of servers.
  • the server 300 provides other network service content such as video on demand and advertising services.
  • the display device 200 may be a liquid crystal display, an OLED (Organic Light Emitting Diode) display, a projection display device, or a smart TV.
  • OLED Organic Light Emitting Diode
  • the specific display device type, size, resolution, etc. are not limited, and those skilled in the art can understand that the display device 200 can make some changes in performance and configuration as required.
  • the display device 200 may additionally provide a smart network TV function that provides a computer support function. Examples include Internet TV, Smart TV, Internet Protocol TV (IPTV) and so on.
  • IPTV Internet Protocol TV
  • the display device may be connected or provided with a camera to present the picture captured by the camera on the display interface of the display device or other display devices, so as to realize interactive chat between users.
  • the picture captured by the camera can be displayed on the display device in full screen, half screen, or in any optional area.
  • the camera is connected to the monitor rear shell through a connecting plate, and is fixedly installed on the upper middle of the monitor rear shell.
  • a connecting plate As an installable method, it can be fixedly installed at any position of the monitor rear shell to ensure its It is sufficient that the image capture area is not blocked by the rear shell, for example, the image capture area and the display device have the same orientation.
  • the camera can be connected to the display rear shell through a connecting plate or other conceivable connectors.
  • the connector is equipped with a lifting motor.
  • the user wants to use the camera or has an application to use the camera
  • it can be raised above the display.
  • the camera is not needed, it can be embedded behind the back shell to protect the camera from damage.
  • the camera used in this application may have 16 million pixels to achieve the purpose of ultra-high-definition display. In actual use, a camera with higher or lower than 16 million pixels can also be used.
  • the content displayed in different application scenarios of the display device can be merged in many different ways, so as to achieve functions that cannot be achieved by traditional display devices.
  • the user can video chat with at least one other user while watching a video program.
  • the presentation of the video program can be used as the background picture, and the video chat window is displayed on the background picture.
  • At least one video chat is performed across terminals.
  • the user can video chat with at least one other user while entering the education application for learning.
  • students can realize remote interaction with teachers while learning content in educational applications. Visually, you can call this function "learning and chatting”.
  • a video chat is conducted with players entering the game.
  • players For example, when a player enters a game application to participate in a game, it can realize remote interaction with other players. Visually, you can call this function "watch and play".
  • the game scene is integrated with the video picture, and the portrait in the video picture is cut out and displayed on the game picture to improve user experience.
  • somatosensory games such as ball games, boxing games, running games, dancing games, etc.
  • human body postures and movements are acquired through the camera, body detection and tracking, and the detection of human bone key points data, and then the game Animations are integrated to realize games such as sports and dance scenes.
  • the user can interact with at least one other user in video and voice in the K song application.
  • multiple users can jointly complete the recording of a song.
  • the user can turn on the camera locally to obtain pictures and videos. If it is vivid, this function can be called "mirror”.
  • Fig. 2 exemplarily shows a configuration block diagram of the control device 100 according to an exemplary embodiment.
  • the control device 100 includes a controller 110, a communicator 130, a user input/output interface 140, a memory 190, and a power supply module 180.
  • the control device 100 is configured to control the display device 200, and can receive user input operation instructions, and convert the operation instructions into instructions that can be recognized and responded to by the display device 200, and serve as an interactive intermediary between the user and the display device 200 effect.
  • the user operates the channel addition and subtraction keys on the control device 100, and the display device 200 responds to the channel addition and subtraction operations.
  • control device 100 may be a smart device.
  • control device 100 can install various applications for controlling the display device 200 according to user requirements.
  • the mobile terminal 100B or other smart electronic devices can perform similar functions to the control device 100 after installing an application for controlling the display device 200.
  • the user can install various function keys or virtual buttons of the graphical user interface that can be provided on the mobile terminal 100B or other smart electronic devices by installing applications to realize the function of the physical keys of the control device 100.
  • the controller 110 includes a processor 112, RAM 113 and ROM 114, a communication interface, and a communication bus.
  • the controller 110 is used to control the operation and operation of the control device 100, as well as the communication and cooperation between internal components, and external and internal data processing functions.
  • the communicator 130 realizes communication of control signals and data signals with the display device 200 under the control of the controller 110. For example, the received user input signal is sent to the display device 200.
  • the communicator 130 may include at least one of communication modules such as a WIFI module 131, a Bluetooth module 132, and an NFC module 133.
  • the user input/output interface 140 wherein the input interface includes at least one of input interfaces such as a microphone 141, a touch panel 142, a sensor 143, and a button 144.
  • input interfaces such as a microphone 141, a touch panel 142, a sensor 143, and a button 144.
  • the user can implement the user instruction input function through voice, touch, gesture, pressing and other actions.
  • the input interface converts the received analog signal into a digital signal and the digital signal into a corresponding instruction signal, which is sent to the display device 200.
  • the output interface includes an interface for sending the received user instruction to the display device 200.
  • it may be an infrared interface or a radio frequency interface.
  • the user input instruction needs to be converted into an infrared control signal according to the infrared control protocol, and sent to the display device 200 via the infrared sending module.
  • a radio frequency signal interface a user input instruction needs to be converted into a digital signal, which is then modulated according to the radio frequency control signal modulation protocol, and then sent to the display device 200 by the radio frequency transmitting terminal.
  • control device 100 includes at least one of a communicator 130 and an output interface.
  • the control device 100 is equipped with a communicator 130, such as WIFI, Bluetooth, NFC and other modules, which can encode user input commands through the WIFI protocol, or the Bluetooth protocol, or the NFC protocol, and send them to the display device 200.
  • a communicator 130 such as WIFI, Bluetooth, NFC and other modules, which can encode user input commands through the WIFI protocol, or the Bluetooth protocol, or the NFC protocol, and send them to the display device 200.
  • the memory 190 is used to store various operating programs, data and applications for driving and controlling the control device 100 under the control of the controller 110.
  • the memory 190 can store various control signal instructions input by the user.
  • the power supply module 180 is used to provide operating power support for each element of the control device 100 under the control of the controller 110. Can battery and related control circuit.
  • FIGS. 3 to 5 a block diagram of the hardware configuration of the hardware system in the display device 200 adopting a dual chip is given.
  • the mechanism relationship of the hardware system can be shown in Figure 3.
  • one hardware system in the dual hardware system architecture is referred to as the first hardware system or A system, A chip, and the other hardware system is referred to as the second hardware system or N system, N chip.
  • the A chip includes the controller and various interfaces of the A chip
  • the N chip includes the controller and various interfaces of the N chip.
  • An independent operating system may be installed in the A chip and the N chip, so that there are two independent but interrelated subsystems in the display device 200.
  • the A chip may also be called the first chip, and the first controller and the N chip may also be called the second chip and the second controller.
  • the A chip and the N chip can realize connection, communication and power supply through multiple different types of interfaces.
  • the interface type of the interface between the A chip and the N chip may include general-purpose input/output (GPIO), USB interface, HDMI interface, UART interface, etc.
  • GPIO general-purpose input/output
  • USB interface USB interface
  • HDMI interface HDMI interface
  • UART interface UART interface
  • One or more of these interfaces can be used between the A chip and the N chip for communication or power transmission.
  • the N chip can be powered by an external power source, while the A chip can be powered by the N chip instead of the external power source.
  • the A chip may also include interfaces for connecting other devices or components, such as the MIPI interface for connecting to a camera (Camera) shown in FIG. 3, a Bluetooth interface, etc.
  • the N chip can also include a VBY interface for connecting to the display screen TCON (Timer Control Register), which is used to connect a power amplifier (Amplifier, AMP) and a speaker (Speaker). ) I2S interface; and IR/Key interface, USB interface, Wifi interface, Bluetooth interface, HDMI interface, Tuner interface, etc.
  • TCON Timer Control Register
  • AMP power amplifier
  • Speaker speaker
  • I2S interface I2S interface
  • IR/Key interface IR/Key interface
  • USB interface USB interface
  • Wifi interface Wireless Fidelity
  • Bluetooth interface HDMI interface
  • Tuner interface etc.
  • FIG. 4 is only some exemplary descriptions of the dual hardware system architecture of the present application, and does not represent a limitation of the present application. In practical applications, both hardware systems can contain more or less hardware or interfaces as required.
  • FIG. 4 exemplarily shows a hardware architecture block diagram of the display device 200 according to FIG. 3.
  • the hardware system of the display device 200 may include an A chip and an N chip, and modules connected to the A chip or the N chip through various interfaces.
  • the N chip may include a tuner and demodulator 220, a communicator 230, an external device interface 250, a controller 210, a memory 290, a user input interface, a video processor 260-1, an audio processor 260-2, a display 280, and an audio output interface 270. Power supply module. In other embodiments, the N chip may also include more or fewer modules.
  • the tuner and demodulator 220 is used to perform modulation and demodulation processing such as amplifying, mixing, and resonating broadcast television signals received through wired or wireless methods, thereby demodulating the user’s information from multiple wireless or cable broadcast television signals. Select the audio and video signals carried in the frequency of the TV channel, and additional information (such as EPG data signals).
  • the signal path of the tuner and demodulator 220 can be various, such as: terrestrial broadcasting, cable broadcasting, satellite broadcasting or Internet broadcasting; and according to different modulation types, the signal adjustment method can be digitally modulated The method may also be an analog modulation method; and according to different types of received television signals, the tuner demodulator 220 may demodulate analog signals and/or digital signals.
  • the tuner and demodulator 220 is also used to respond to the TV channel frequency selected by the user and the TV signal carried by the frequency according to the user's selection and control by the controller 210.
  • the tuner demodulator 220 may also be in an external device, such as an external set-top box.
  • the set-top box outputs TV audio and video signals through modulation and demodulation, and inputs them to the display device 200 through the external device interface 250.
  • the communicator 230 is a component for communicating with external devices or external servers according to various communication protocol types.
  • the communicator 230 may include a WIFI module 231, a Bluetooth communication protocol module 232, a wired Ethernet communication protocol module 233, and an infrared communication protocol module and other network communication protocol modules or near field communication protocol modules.
  • the display device 200 may establish a control signal and a data signal connection with an external control device or content providing device through the communicator 230.
  • the communicator may receive the control signal of the remote controller 100A according to the control of the controller.
  • the external device interface 250 is a component that provides data transmission between the N chip controller 210 and the A chip and other external devices.
  • the external device interface can be connected to external devices such as set-top boxes, game devices, notebook computers, etc. in a wired/wireless manner, and can receive external devices such as video signals (such as moving images), audio signals (such as music), and additional information (such as EPG). ) And other data.
  • the external device interface 250 may include: a high-definition multimedia interface (HDMI) terminal 251, a composite video blanking synchronization (CVBS) terminal 252, an analog or digital component terminal 253, a universal serial bus (USB) terminal 254, red, green, and blue ( RGB) terminal (not shown in the figure) and any one or more.
  • HDMI high-definition multimedia interface
  • CVBS composite video blanking synchronization
  • USB universal serial bus
  • RGB red, green, and blue
  • the controller 210 controls the work of the display device 200 and responds to user operations by running various software control programs (such as an operating system and/or various application programs) stored on the memory 290.
  • various software control programs such as an operating system and/or various application programs
  • the controller 210 includes a read-only memory RAM 214, a random access memory ROM 213, a graphics processor 216, a CPU processor 212, a communication interface 218, and a communication bus.
  • RAM 214 and ROM 213, graphics processor 216, CPU processor 212, and communication interface 218 are connected by a bus.
  • the graphics processor 216 is used to generate various graphics objects, such as icons, operation menus, and user input instructions to display graphics. Including an arithmetic unit, which performs operations by receiving various interactive commands input by the user, and displays various objects according to display attributes. As well as including a renderer, various objects obtained based on the arithmetic unit are generated, and the rendering result is displayed on the display 280.
  • the CPU processor 212 is configured to execute operating system and application program instructions stored in the memory 290. And according to receiving various interactive instructions input from the outside, to execute various applications, data and content, so as to finally display and play various audio and video content.
  • the CPU processor 212 may include multiple processors.
  • the multiple processors may include one main processor and multiple or one sub-processors.
  • the main processor is used to perform some operations of the display device 200 in the pre-power-on mode, and/or to display images in the normal mode.
  • the communication interface may include the first interface 218-1 to the nth interface 218-n. These interfaces may be network interfaces connected to external devices via a network.
  • the controller 210 may control the overall operation of the display device 200. For example, in response to receiving a user command for selecting a UI object to be displayed on the display 280, the controller 210 may perform an operation related to the object selected by the user command.
  • the object may be any one of the selectable objects, such as a hyperlink or an icon.
  • Operations related to the selected object for example: display operations connected to hyperlink pages, documents, images, etc., or perform operations corresponding to the icon.
  • the user command for selecting the UI object may be a command input through various input devices (for example, a mouse, a keyboard, a touch pad, etc.) connected to the display device 200 or a voice command corresponding to the voice spoken by the user.
  • the memory 290 includes storing various software modules for driving and controlling the display device 200.
  • various software modules stored in the memory 290 include: a basic module, a detection module, a communication module, a display control module, a browser module, and various service modules.
  • the basic module is the underlying software module used for signal communication between various hardware in the display device 200 and sending processing and control signals to the upper module.
  • the detection module is a management module used to collect various information from various sensors or user input interfaces, and perform digital-to-analog conversion and analysis management.
  • the voice recognition module includes a voice analysis module and a voice command database module.
  • the display control module is a module for controlling the display 280 to display image content, and can be used to play information such as multimedia image content and UI interfaces.
  • the communication module is a module used for control and data communication with external devices.
  • the browser module is a module used to perform data communication between browsing servers.
  • the service module is a module used to provide various services and various applications.
  • the memory 290 is also used to store and receive external data and user data, images of various items in various user interfaces, and visual effect diagrams of focus objects.
  • the user input interface is used to send a user's input signal to the controller 210, or to transmit a signal output from the controller to the user.
  • the control device (such as a mobile terminal or a remote control) may send input signals input by the user, such as a power switch signal, a channel selection signal, and a volume adjustment signal, to the user input interface, and then the user input interface forwards the input signal to the controller;
  • the control device may receive output signals such as audio, video, or data output from the user input interface processed by the controller, and display the received output signal or output the received output signal as audio or vibration.
  • the user may input a user command through a graphical user interface (GUI) displayed on the display 280, and the user input interface receives the user input command through the graphical user interface (GUI).
  • GUI graphical user interface
  • the user can input a user command by inputting a specific sound or gesture, and the user input interface recognizes the sound or gesture through the sensor to receive the user input command.
  • the video processor 260-1 is used to receive video signals, and perform video data processing such as decompression, decoding, scaling, noise reduction, frame rate conversion, resolution conversion, and image synthesis according to the standard codec protocol of the input signal.
  • the video signal displayed or played directly on the display 280.
  • the video processor 260-1 includes a demultiplexing module, a video decoding module, an image synthesis module, a frame rate conversion module, a display formatting module, and the like.
  • the demultiplexing module is used to demultiplex the input audio and video data stream. For example, if MPEG-2 is input, the demultiplexing module will demultiplex into a video signal and an audio signal.
  • the video decoding module is used to process the demultiplexed video signal, including decoding and scaling.
  • An image synthesis module such as an image synthesizer, is used to superimpose and mix the GUI signal generated by the graphics generator with the zoomed video image according to user input or itself to generate an image signal for display.
  • Frame rate conversion module used to convert the frame rate of the input video, such as converting the frame rate of the input 24Hz, 25Hz, 30Hz, 60Hz video to the frame rate of 60Hz, 120Hz or 240Hz, where the input frame rate can be compared with the source
  • the video stream is related, and the output frame rate can be related to the update rate of the display.
  • the input has a usual format, such as frame insertion.
  • the display formatting module is used to change the signal output by the frame rate conversion module into a signal that conforms to a display format such as a display, such as format conversion of the signal output by the frame rate conversion module to output RGB data signals.
  • the display 280 is used to receive the image signal input from the video processor 260-1, display video content and images, and a menu control interface.
  • the display 280 includes a display component for presenting a picture and a driving component for driving image display.
  • the displayed video content can be from the video in the broadcast signal received by the tuner and demodulator 220, or from the video content input by the communicator or the interface of an external device.
  • the display 220 simultaneously displays a user manipulation interface UI generated in the display device 200 and used to control the display device 200.
  • the display 280 it also includes a driving component for driving the display.
  • the display 280 is a projection display, it may also include a projection device and a projection screen.
  • the audio processor 260-2 is used to receive audio signals, and perform decompression and decoding according to the standard codec protocol of the input signal, as well as audio data processing such as noise reduction, digital-to-analog conversion, and amplification processing, and the result can be in the speaker 272 The audio signal to be played.
  • the audio output interface 270 is used to receive the audio signal output by the audio processor 260-2 under the control of the controller 210.
  • the audio output interface may include a speaker 272 or output to an external audio output terminal 274 of a generator of an external device, such as : External audio terminal or headphone output terminal, etc.
  • the video processor 260-1 may include one or more chips.
  • the audio processor 260-2 may also include one or more chips.
  • the video processor 260-1 and the audio processor 260-2 may be separate chips, or they may be integrated with the controller 210 in one or more chips.
  • the power supply module is used to provide power supply support for the display device 200 with power input from an external power supply under the control of the controller 210.
  • the power supply module may include a built-in power supply circuit installed inside the display device 200, or may be a power supply installed outside the display device 200, such as a power interface for providing an external power supply in the display device 200.
  • the A chip may include a controller 310, a communicator 330, a detector 340, and a memory 390. In some embodiments, it may also include a user input interface, a video processor, an audio processor, a display, and an audio output interface. In some embodiments, there may also be a power supply module that independently supplies power to the A chip.
  • the communicator 330 is a component for communicating with external devices or external servers according to various communication protocol types.
  • the communicator 330 may include a WIFI module 331, a Bluetooth communication protocol module 332, a wired Ethernet communication protocol module 333, and an infrared communication protocol module and other network communication protocol modules or near field communication protocol modules.
  • the communicator 330 of the A chip and the communicator 230 of the N chip also interact with each other.
  • the WiFi module 231 of the N chip is used to connect to an external network and generate network communication with an external server and the like.
  • the WiFi module 331 of the A chip is used to connect to the WiFi module 231 of the N chip, and does not directly connect to an external network or the like. Therefore, for the user, a display device as in the above embodiment can externally display a WiFi account.
  • the detector 340 is a component used by the chip of the display device A to collect signals from the external environment or interact with the outside.
  • the detector 340 may include a light receiver 342, a sensor used to collect the intensity of ambient light, which can adaptively display parameter changes by collecting ambient light, etc.; it may also include an image collector 341, such as a camera, a camera, etc., which can be used to collect external Environmental scenes, as well as gestures used to collect user attributes or interact with users, can adaptively change display parameters, and can also recognize user gestures to achieve the function of interaction with users.
  • the external device interface 350 provides components for data transmission between the controller 310 and the N chip or other external devices.
  • the external device interface can be connected to external devices such as set-top boxes, game devices, notebook computers, etc., in a wired/wireless manner.
  • the controller 310 controls the work of the display device 200 and responds to user operations by running various software control programs (such as installed third-party applications, etc.) stored on the memory 390 and interacting with the N chip.
  • various software control programs such as installed third-party applications, etc.
  • the controller 310 includes a read-only memory ROM 313, a random access memory RAM 314, a graphics processor 316, a CPU processor 312, a communication interface 318, and a communication bus.
  • the ROM 313 and the RAM 314, the graphics processor 316, the CPU processor 312, and the communication interface 318 are connected by a bus.
  • the CPU processor 312 runs the system startup instruction in the ROM, and copies the operating system stored in the memory 390 to the RAM 314 to start the startup operating system. After the operating system is started up, the CPU processor 312 copies various application programs in the memory 390 to the RAM 314, and then starts to run and start various application programs.
  • the CPU processor 312 is used to execute operating system and application instructions stored in the memory 390, communicate with the N chip, transmit and interact with signals, data, instructions, etc., and execute according to various interactive instructions received from external inputs Various applications, data and content, in order to finally display and play various audio and video content.
  • the communication interface may include the first interface 318-1 to the nth interface 318-n. These interfaces may be network interfaces connected to external devices via a network, or network interfaces connected to the N chip via a network.
  • the controller 310 may control the overall operation of the display device 200. For example, in response to receiving a user command for selecting a UI object to be displayed on the display 280, the controller 210 may perform an operation related to the object selected by the user command.
  • the graphics processor 316 is used to generate various graphics objects, such as icons, operation menus, and user input instructions to display graphics. Including an arithmetic unit, which performs operations by receiving various interactive commands input by the user, and displays various objects according to display attributes. As well as including a renderer, various objects obtained based on the arithmetic unit are generated, and the rendering result is displayed on the display 280.
  • Both the graphics processor 316 of the A chip and the graphics processor 216 of the N chip can generate various graphics objects. Differentily, if application 1 is installed on the A chip and application 2 is installed on the N chip, when the user is in the interface of the application 1 and the user inputs instructions in the application 1, the A chip graphics processor 316 generates a graphic object. When the user is on the interface of Application 2 and performs the user-input instructions in Application 2, the graphics processor 216 of the N chip generates the graphics object.
  • Fig. 5 is a schematic diagram of a functional configuration of a display device exemplarily shown according to some embodiments of the application.
  • the memory 390 of the A chip and the memory 290 of the N chip are respectively used to store the operating system, application programs, content and user data, etc., under the control of the controller 310 of the A chip and the controller 210 of the N chip. , Drive the system operation of the display device 200 and respond to various operations of the user.
  • the memory 390 of the A chip and the memory 290 of the N chip may include volatile and/or nonvolatile memory.
  • the memory 290 is specifically used to store the operating program that drives the controller 210 in the display device 200, and store various application programs built in the display device 200, and various application programs downloaded by the user from an external device, and application programs.
  • the memory 290 is used to store system software such as an operating system (OS) kernel, middleware, and applications, and to store input video data and audio data, and other user data.
  • OS operating system
  • the memory 290 is specifically used to store driver programs and related data such as the video processor 260-1 and the audio processor 260-2, the display 280, the communication interface 230, the tuner and demodulator 220, and the input/output interface.
  • the memory 290 may store software and/or programs.
  • the software programs used to represent an operating system (OS) include, for example, a kernel, middleware, application programming interface (API), and/or application programs.
  • OS operating system
  • the kernel may control or manage system resources, or functions implemented by other programs (such as the middleware, API, or application program), and the kernel may provide interfaces to allow middleware and APIs, or applications to access the controller , In order to achieve control or management of system resources.
  • the memory 290 includes a broadcast receiving module 2901, a channel control module 2902, a volume control module 2903, an image control module 2904, a display control module 2905, an audio control module 2906, an external command recognition module 2907, a communication control module 2908, and power control Module 2910, operating system 2911, and other application programs 2912, interface layout management module 2913, event transmission system 2914, browser module and so on.
  • the controller 210 executes various software programs in the memory 290, such as: broadcast and television signal reception and demodulation function, TV channel selection control function, volume selection control function, image control function, display control function, audio control function, external command Various functions such as identification function, communication control function, optical signal receiving function, power control function, software control platform supporting various functions, and browser function.
  • the memory 390 includes storing various software modules for driving and controlling the display device 200.
  • various software modules stored in the memory 390 include: a basic module, a detection module, a communication module, a display control module, a browser module, and various service modules. Since the functions of the memory 390 and the memory 290 are relatively similar, please refer to the memory 290 for related parts, and will not be repeated here.
  • the memory 390 includes an image control module 3904, an audio control module 2906, an external command recognition module 3907, a communication control module 3908, an optical receiving module 3909, an operating system 3911, and other application programs 3912, a browser module, and so on.
  • the controller 210 executes various software programs in the memory 290, such as: image control function, display control function, audio control function, external command recognition function, communication control function, light signal receiving function, power control function, support for various Functional software control platform, and various functions such as browser functions.
  • the external command recognition module 2907 of the N chip and the external command recognition module 3907 of the A chip can recognize different commands.
  • the external command recognition module 3907 of the A chip may include a graphic recognition module 3907-1.
  • the graphic recognition module 3907-1 stores a graphic database, and the camera receives external commands. In order to control the display device, the corresponding relationship is made with the instructions in the graphics database.
  • the voice receiving device and the remote controller are connected to the N chip, the external command recognition module 2907 of the N chip may include a voice recognition module 2907-2.
  • the voice recognition module 2907-2 stores a voice database, and the voice receiving device receives The external voice commands or time correspond to the commands in the voice database to control the display device.
  • a control device 100 such as a remote controller is connected to the N chip, and the key command recognition module interacts with the control device 100.
  • the controller of the display device is an operating system at the software level, and the built-in applications may be the same as those in the above-mentioned dual-chip architecture display device. Also set up all the above interfaces.
  • Fig. 6a exemplarily shows a configuration block diagram of the software system in the display device 200 in some embodiments.
  • the operating system 2911 includes operating software for processing various basic system services and for implementing hardware-related tasks, acting as a medium for data processing between application programs and hardware components.
  • part of the operating system kernel may include a series of software to manage the hardware resources of the display device and provide services for other programs or software codes.
  • part of the operating system kernel may include one or more device drivers, and the device drivers may be a set of software codes in the operating system to help operate or control the devices or hardware associated with the display device.
  • the drive may contain code to manipulate video, audio, and/or other multimedia components. Examples include displays, cameras, Flash, WiFi, and audio drivers.
  • the operating system 2911 may specifically include: an accessibility module 2911-1, a communication module 2911-2, a user interface module 2911-3, and a control application 2911-4.
  • the operating system 2911 may further include a camera scheduling module 2911-5, a camera driving module 2911-6, and a camera switch module 2911-7.
  • the accessibility module 2911-1 is used to modify or access the application program, so as to realize the accessibility of the application program and the operability of its display content.
  • the communication module 2911-2 is used to connect to other peripherals via related communication interfaces and communication networks.
  • the user interface module 2911-3 is used to provide objects that display the user interface for access by various applications, and can realize user operability.
  • the control application 2911-4 is used to control process management and switch foreground applications, including runtime applications.
  • the camera scheduling module 2911-5 is used to control the camera to turn on or off, and to raise or lower the camera.
  • the camera driving module 2911-6 which is used to drive the motor mechanically connected with the camera to raise or lower the camera under the control of the camera scheduling module 2911-5;
  • the camera switch module 2911-7 is used to turn on the camera under the control of the camera scheduling module 2911-5, even if it enters the on state, or turn off the camera, even if it enters the off state.
  • the event transmission system 2914 may be implemented in the operating system 2911 or in the application program 2912. In some embodiments, it is implemented in the operating system 2911 on the one hand, and implemented in the application program 2912 at the same time, for monitoring various user input events, and responding to the recognition results of various events or sub-events according to various events. And implement one or more sets of pre-defined operation procedures.
  • the event transmission system 2914 may include an event monitoring module 2914-1 and an event identification module 2914-2.
  • the event monitoring module 2914-1 is used to monitor input events or sub-events of the user input interface.
  • Event recognition module 2914-2 used to input the definition of various events to various user input interfaces, recognize various events or sub-events, and transmit them to the processing to execute the corresponding one or more groups of processing programs .
  • the event or sub-event refers to the input detected by one or more sensors in the display device 200 and the input of an external control device (such as the control device 100, etc.).
  • an external control device such as the control device 100, etc.
  • various sub-events of voice input, gesture input sub-events of gesture recognition, and sub-events of remote control button command input of control devices include multiple forms, including but not limited to one or a combination of pressing up/down/left/right/, confirming keys, and pressing keys.
  • non-physical buttons such as moving, pressing, and releasing.
  • the interface layout management module 2913 which directly or indirectly receives various user input events or sub-events monitored by the event transmission system 2914, is used to update the layout of the user interface, including but not limited to the position of each control or sub-control in the interface, and the container
  • the size, position, level, etc. of the interface are related to various execution operations.
  • the application layer of the display device includes various applications that can be executed on the display device 200.
  • the application layer 2912 of the N chip may include, but is not limited to, one or more applications, such as video-on-demand applications, application centers, and game applications.
  • the application layer 3912 of the A chip may include, but is not limited to, one or more applications, such as a live TV application, a media center application, and so on. It should be noted that the application programs contained on the A chip and the N chip are determined according to the operating system and other designs. The present invention does not need to specifically limit and divide the application programs contained on the A chip and the N chip.
  • Live TV applications can provide live TV through different sources.
  • a live TV application may use input from cable TV, wireless broadcasting, satellite services, or other types of live TV services to provide TV signals.
  • the live TV application can display the video of the live TV signal on the display device 200.
  • Video-on-demand applications can provide videos from different storage sources. Unlike live TV applications, VOD provides video display from certain storage sources. For example, the video on demand can come from the server side of cloud storage, and from the local hard disk storage that contains the stored video programs.
  • Media center applications can provide various multimedia content playback applications.
  • the media center can provide services that are different from live TV or video on demand, and users can access various images or audio through the media center application.
  • Application center can provide storage of various applications.
  • the application program may be a game, an application program, or some other application program that is related to a computer system or other device but can be run on a display device.
  • the application center can obtain these applications from different sources, store them in the local storage, and then run on the display device 200.
  • the A chip and the N chip may be respectively installed with independent operating systems, there are two independent but related sub-systems in the display device 200.
  • both the A chip and the N chip can be independently installed with Android and various APPs, so that each chip can realize a certain function, and the A chip and the N chip can cooperate to realize a certain function.
  • FIG. 7 exemplarily shows a schematic diagram of a user interface in the display device 200 according to an exemplary embodiment.
  • the user interface includes multiple view display areas, for example, a first view display area 201 and a play screen 202, where the play screen includes layout of one or more different items.
  • the user interface also includes a selector indicating that the item is selected, and the position of the selector can be moved through user input to change the selection of different items.
  • multiple view display areas can present display screens of different levels.
  • the first view display area can present video chat item content
  • the second view display area can present application layer item content (eg, webpage video, VOD display, application program screen, etc.).
  • the presentation of different view display areas has priority differences, and the display priority of the view display areas is different between view display areas with different priorities.
  • the priority of the system layer is higher than that of the application layer.
  • the screen display in the view display area of the system layer is not blocked; and the application layer is enabled according to the user's choice
  • the size and position of the view display area change the size and position of the view display area of the system layer will not be affected.
  • the same level of display screen can also be presented.
  • the selector can switch between the display area of the first view and the display area of the second view, and when the size and position of the display area of the first view change, the second view The size and position of the display area can be changed at any time.
  • any area in FIG. 7 may display a picture obtained by a camera.
  • Items refer to visual objects displayed in each view display area of the user interface of the display device 200 to represent corresponding content such as icons, thumbnails, and video clips.
  • the item can represent the image content or video clip of a movie, TV series, audio content of music, application program, or other user access content history information.
  • "items” may display image thumbnails.
  • the project when the project is a movie or TV series, the project can be displayed as a poster of the movie or TV series. If the item is music, the poster of the music album can be displayed.
  • the project when the project is an application, it can be displayed as an icon of the application, or a screenshot of the content of the application captured when the application was executed most recently. If the project is the user's access history, it can be displayed as a screenshot of the content during the most recent execution.
  • "Projects" can be displayed as video clips. For example, the project is a dynamic picture of a video clip of a trailer of a TV or TV series.
  • the item may indicate an interface or a set display of interfaces through which the display device 200 is connected to an external device, or may indicate the name of an external device connected to the display device, or the like.
  • Each item can have the same size or different sizes. In some implementation examples, the size of the item can be changed.
  • “Selector” is used to indicate that any item has been selected, such as cursor or focus object.
  • the cursor is moved on the display device 200 to select or control items.
  • the movement of the focus object displayed in the display device 200 can be selected to select the control item, and one or more items can be selected or controlled.
  • the user can use the arrow keys on the control device 100 to control the movement of the focus object between items to select and control items.
  • Fig. 8a is a scenario in which a user interacts with a display device through the control device 100A according to an exemplary embodiment of the application.
  • the display device presents the operation interface of the application center.
  • the application center includes a plurality of application programs, at least one of which is a social application, such as QQ, WeChat, SNS, etc.
  • the user chooses to start a social application.
  • Fig. 8b is another scenario in which a user interacts with a display device through the control device 100A according to an exemplary embodiment of this application.
  • the display device presents an operation interface corresponding to the social application selected by the user in the scene shown in Fig. 8a.
  • this operation interface the user can select one or more friends’ accounts in the address book to initiate a video to chat with.
  • the social application When the user selects a friend’s account and initiates a video chat, the social application will generate an instruction for video communication with the one or more friends, and the communication module of the display device will respond to the instruction to send one or more pairs of users who log in to the corresponding account.
  • the end device sends a video communication request.
  • the image collector of the display device responds to the instruction and collects the image data of the scene where the local user is located.
  • the video processor of the display device will encode the collected image data and send it to the peer through the communication module. equipment.
  • the opposite device When the communication link between the local device and the opposite device is successfully connected, the opposite device will receive the video data stream sent by the local device. The opposite device decodes the video data stream to obtain the corresponding image and present it on its display screen.
  • the local device will also receive the video data stream sent by the peer device, and display it on the display screen after decoding.
  • Figure 9a is a schematic diagram of an application scenario of the technical solution of this application.
  • the local display device 200-1 and a counterpart display device 200-2 conduct a video call by establishing a communication connection.
  • a video window is presented on the screen of the local display device for presenting a video picture (ie, the opposite video picture), and a preview window of the local video picture is also presented.
  • the above-mentioned video window used to present the opposite end video picture is called the opposite end display window, and the preview window used to present the local video picture is called the local preview window.
  • Figure 9b is a schematic diagram of another application scenario of the technical solution of this application.
  • the local display device 200-1 and the five peer display devices 200-2, 200-3, 200-4, 200-5, and 200-6 establish a communication connection to make a video call.
  • five video windows are presented on the screen of the local display device, which are used to present 5 channels of video images.
  • a local video image preview window is also presented.
  • the user due to the fixed position of the display device in the room, the user’s comfortable viewing distance is relatively long. Therefore, when two or more people video chat, in the global image collected by the opposite device, the opposite user especially The face of the end user only occupies a small part of the entire image, so that the peer user and its face presented by the local device to the local user are very small. In addition, the viewing distance between the local user and the display device causes the local The user and the face of the opposite user actually seen by the user are smaller. In the case of multi-person chat, multiple video windows are presented on the local screen at the same time, causing the target in each video window to be further reduced, which seriously affects the user's video chat experience. In addition, when the peer user moves to a farther location, the above-mentioned problems will become more serious.
  • FIG. 10a shows a video interface presented by the local device during a two-person video chat, and the video interface includes the opposite user and the scene in which it is located.
  • FIG. 10b shows another video interface presented by the local device during a two-person video chat, and the video interface includes the opposite user and the scene in which it is located.
  • FIGS. 10a and 10b By comparing FIGS. 10a and 10b, it can be seen that because the opposite end user has moved to a position farther from the opposite end image collector, the area of the person shown in Fig. 10b, especially the face of the person, is small.
  • Figure 10c shows the video interface and the video interface presented by the local device during a four-person video chat.
  • the video interface includes windows 101, 102, and 103 corresponding to three peer users, and one user To preview the preview video 104 of the local screen. It can be seen that, compared with 10a, due to the simultaneous presentation of multiple video windows on the local screen, the target in each video window is further reduced, which seriously affects the user's video chat experience.
  • the controller of the display device in response to the instruction of video communication with at least one peer device, collects the global image of the local end through an image collector (such as a camera); presents it in the user interface
  • the local preview window displays the local video screen in the local preview window according to the global image.
  • the local video screen is the local image corresponding to the target stable area of the moving target;
  • local video data is generated based on the local video screen displayed in the local preview window and sent to the opposite device.
  • the controller of the opposite device receives the video data sent by the display device in response to the establishment of the video communication connection with the above-mentioned display device; presents the opposite display window in the user interface;
  • the video data displays the opposite end video picture in the opposite end display window, where, when there is a moving target in the opposite end video picture, the opposite end video picture is the picture corresponding to the target stable area of the moving target.
  • the display device processes the global image according to the target stable area containing the moving target to obtain a partial image, and uses the partial image to perform video communication with the peer device, so it can achieve real-time tracking of the moving target during the video communication process. It mainly presents the effect of the target stable area, so that even if the local user is far away from the image collector of the display device, or the video window on the local screen is small, the moving target, that is, the user's face, can be clearly presented, thereby improving The user’s video communication experience.
  • the user when the user uses the "mirror" function, the user activates the image collector by operating the control device. After the image collector is started, it starts to collect local global images; the display device controller responds to the activation of the image collector, Obtain the local global image collected by the image collector, present the local preview window in the user interface, and display the local video screen in the local preview window according to the local global image.
  • the local video when there is a moving target in the global image, the local video The picture is a partial image corresponding to the target stable area of the moving target.
  • the display device processes the global image according to the target stable area containing the moving target to obtain a partial image, and presents the partial image in the local preview window.
  • the user uses the "mirror” function It can also achieve the effect of real-time tracking of moving targets and mainly showing the stable area of the target, so that even if the local user is far away from the image collector of the display device, or the video window on the local screen is small, the display can be clearly displayed
  • the movement target that is, the user’s face, will be enhanced to enhance the user’s "mirror" experience
  • the partial image obtained by the display device according to the global image is only used for local display, and it is not necessary to use it. It is sent to the peer device.
  • the specific implementation process of the partial image obtained by the display device according to the global image can refer to the following method embodiment.
  • FIG. 11 is a flowchart of an embodiment of a video communication data processing method of this application.
  • the method is applied to a display device, or in other words, the execution subject of the method is a controller of the display device, such as the first controller or the first controller shown in FIG. 4 Two controllers, but not limited to this.
  • the method may include:
  • Step 111 The display device responds to the instruction of video communication with at least one peer device, and collects a global image of the scene where the moving target is located through the image collector.
  • the local user operates on an application with a video communication function, and selects one or more friends from the address book to initiate a video communication.
  • the application will respond to the user operation and generate an instruction. Instruct the local device to send a video communication request to the opposite device corresponding to the friend account selected by the local user to conduct video communication with the opposite device.
  • the peer user first operates on an application with a video communication function, and selects one or more friends from the address book that at least includes the local user to initiate a video communication, and the peer application
  • an instruction is generated, which instructs the peer device to send a video communication request to one or more peer devices including at least the local display device, and the local user operates on the local application to receive the request
  • the local application generates another instruction in response to the user's operation, and the instruction instructs the local display device to receive the video request sent by the peer device to perform video communication with the peer device.
  • the local display device has an image collector, such as a camera as shown in FIG. 1.
  • the peer device also has an image capture function. For example, if the peer device is a portable terminal device such as a mobile phone, the image will be captured through the built-in camera of this type of terminal. If the peer device is a home display device such as a smart TV, the The home display device is at least an image collector 341 as shown in FIG. 4.
  • video communication Since the purpose of video communication is to provide real-time video images of the peer user and the scene in which the user at both ends of the video is conducted, if the user at one end cannot see the peer user clearly in the video screen due to the foregoing problems, it will affect The user experience of video communication even loses the meaning of video communication.
  • the moving target described in this application is a user participating in a video communication that can move at any time, and at the local end, it is a local user.
  • the movement target is the face or head of the user participating in the video communication.
  • the scene where the moving target is located is the scene corresponding to the collection range of the image collector, and the global image is the video image of the scene where the moving target is collected by the image collector.
  • Fig. 10a shows a global image collected by the image collector at the first moment
  • Fig. 10b shows a global image collected by the image collector at the second moment. If the user’s face is set as the moving target in advance, it can be seen by comparison that the moving target has moved in the scene where it is located. Therefore, the moving target shown in Figure 10a has a larger size and the moving target shown in Figure 10b has a smaller size. .
  • Step 112 Obtain the global image.
  • the image collector is connected to the first chip, and the global image data collected by the image collector is obtained by the first chip, and the first chip or the first chip
  • the video processor connected to the chip executes the steps provided in the method of the present application, that is, the execution subject of the steps in the method embodiment of the present application may be the first chip or the video processor connected to the first chip.
  • the first chip since there are multiple communication interfaces or serial ports between the first chip and the second chip, after the first chip obtains the global image data collected by the image collector, the first chip performs data processing on the data. After partial processing or no processing, the data is sent to the second chip, and the second chip or the video processor connected to the second chip executes the steps provided in the method of this application, that is, the method described in the method embodiment of this application
  • the execution subject of the steps may be the second chip or the video processor connected to the second chip.
  • the image collector can be connected to the second chip, and the second chip obtains the global image data collected by the image collector, and the second chip or the video processor connected to the second chip executes this Steps provided in the application method embodiment.
  • the data can also be sent to the first chip for processing, which will not be repeated here.
  • the Android system there are generally two ways to obtain camera data based on the Android system.
  • One is to set a preview SurfaceTexture for the camera and obtain the camera texture through a callback, and the other is to obtain buffer data in a specified format through the PreviewCallback callback of the camera.
  • the camera data is generally rendered on the screen through GLSurfaceView or SurfaceView.
  • the PreviewCallback callback and SurfaceTexture texture are used to obtain camera data and GLSurfaceView rendering is used for browsing.
  • the SurfaceTexture texture is used to obtain camera data and GLSurfaceView is used to render for browsing.
  • Step 113 Detect a moving target in the global image, and determine a target stable area containing the moving target.
  • a person's face is preset as a moving target
  • detecting the moving target in the global image is to detect the face area in the global image.
  • a deep learning neural network model such as a convolutional neural network model
  • the use of a trained neural network model to detect the target region in the image belongs to the prior art and is well known to those skilled in the art, so the details are not repeated in the embodiment of the present application.
  • the target stable area refers to a stable area where the moving target is located in the global image.
  • the moving target has a high probability of appearing in the target stable area.
  • N frames of global images correspond to a target stable area, and N is a preset number of frames.
  • the moving target in each frame of the global image is first detected, and the target position is located.
  • the target position is the position of the center point of the area where the moving target is located in the global image.
  • the area where the target location is located is area A
  • the target location is the center point of area A.
  • the quantitative parameter used to characterize the target position is the coordinate of the center point of the area where the moving target is located, such as (x, y).
  • Figure 10d shows a global image presented by the local device during a two-person video chat.
  • the global image includes the facial regions of two people.
  • the two facial regions are two moving targets.
  • the area where the target is located is the area B shown in the figure, and the target position is also the center point of the area B.
  • this application pre-establishes the two-dimensional coordinate system where the global image is located according to the texture of the camera of the display device, and uses the two-dimensional coordinate system as a reference standard to discuss any position or position relationship described in this application.
  • the area where the moving target is located is a part or several parts of the global image, which is a collection of countless points, the center point of the area where the moving target is located is taken as the target position.
  • the values of the center point coordinates described in this application are all relative values adapted to the zoom ratio, not absolute values. value.
  • the N target positions corresponding to the N frames of global images are traversed, and when the i-th target position is traversed, the i-th target position is matched with the first to i-1th target positions respectively, i ⁇ (1,N]; if the i-th target position matches at least one of the 1st to i-1th target positions, add 1 to the current frequency of the matched two target positions.
  • the current frequency of the i-th target position and the first target position is increased by 1
  • the current frequency of the i-th target position is increased by 2
  • the current frequency of the first and i-5th target positions are respectively increased by 1.
  • the initial frequency of each target position is 1.
  • the current frequency of the i-th target position is 1.
  • the current frequencies of the first to i-th target positions may be as shown in Table 1 below.
  • Target position number Current frequency 1 1 2 3 ... ... i-1 2 i 3 i+1 1 (initial frequency) ... ...
  • the matching of the i-th target position with the 1st to i-1th target positions respectively may include:
  • the reference distance corresponding to the i-th target position is calculated according to the i-th target position and the preset reference point.
  • the preset reference point can be any one of the four vertices of the global image.
  • a vertex of the global image is the coordinate origin, and the coordinate origin can be preset as the reference point.
  • the coordinates of the target position (the coordinates of the center point of the area where the moving target is located) and the preset reference point coordinates are substituted into the two-point distance calculation formula, and the reference distance corresponding to the target position is calculated.
  • the difference between the reference distance corresponding to the i-th target position and the reference distance corresponding to the first to i-1th target positions is calculated respectively.
  • the reference distance corresponding to the first to the i-1th target position is the data that has been calculated and saved.
  • i-1 difference data corresponding to the first to i-1th target positions will be obtained.
  • each difference value belongs to the preset range. Specifically, if a certain difference value belongs to the preset range, it is determined that the corresponding two target positions are matched, otherwise it is not matched.
  • the preset range can be set to (-20px, 20px), that is, if the absolute difference between the reference distance corresponding to the i-th target position and the reference distance corresponding to another target position is 20 pixels Within, it is determined that the two target positions match, which means that the two target positions are similar.
  • the matching the i-th target position with the 1st to i-1th target positions respectively may include:
  • the coordinates of the two target positions are substituted into the two-point distance calculation formula, and the true distance between the two target positions is calculated.
  • i-1 real distance data corresponding to the first to i-1th target positions will be obtained.
  • each real distance belongs to the preset range. Specifically, if a certain real distance belongs to the preset range, it is determined that the corresponding two target positions are matched, otherwise it is not matched.
  • step 113 by detecting the moving target in each frame of the global image, the target position is located, and N consecutive frames of global images are used as a statistical object group, and each of the N target positions corresponding to the consecutive N frames of global images is counted.
  • the area corresponding to the position with the largest frequency is the target stable area of the N frames of global images. It can be seen that since the area corresponding to the target position with the largest frequency is the target stable area of the N frames of global image, the target stable area determined by the method provided in step 113 is for the N frames of global image, the moving target is located A stable area of, or in other words, for the N frames of global images, the moving target has a high probability of appearing in the target stable area.
  • the ratio of the target stable area determined in the embodiment of the present application is preferably consistent with the resolution ratio of the global image.
  • Step 114 Process the global image according to the target stable area to obtain a partial image.
  • step 114 the N frames of global images corresponding to the target stable region are used as a processing target group, and N frames of global images in the group are batch processed. Specifically, firstly, a partial image corresponding to the target stable region is obtained from each frame of the global image; then, the partial image is enlarged according to a preset rule.
  • a cropping frame is preset according to the edge of the target stable area, and the image area outside the cropping frame is cropped according to the cropping frame to obtain the partial image corresponding to the target stable area.
  • the crop frame is moved from the target stable area before the change to the target stable area after the change, so as to change the target stable area outside the crop frame.
  • the image area is cropped.
  • a possible preset zooming rule is as follows:
  • this application also provides another possible preset amplification rule, which is different from the above amplification rule:
  • Step 115 Use the partial image to perform video communication with the peer device.
  • the partial image data is encoded to generate a video data stream and send it to the peer device.
  • the method of the present application may also include
  • Step 106 Present the partial image in the preview window of the display device.
  • the local device previews the partial image or the video data stream corresponding to the partial image is presented in the scene after being decoded at the opposite end, if the size of the partial image is larger than the screen size or the window size, the original The end device or the opposite end device performs corresponding adaptive processing.
  • an embodiment of the present application provides a video communication data processing method, the method includes: responding to an instruction for video communication with at least one peer device, collecting a global image of the scene where the moving target is located through an image collector, Acquire the collected global image; then detect the moving target in the global image, and determine the target stable area containing the moving target; then process the global image according to the target stable area to obtain a partial image; finally use The partial image performs video communication with the peer device.
  • the method of the present application processes the global image according to the target stable area containing the moving target to obtain the partial image, and uses the partial image to perform video communication with the peer device, it can achieve real-time tracking of the moving target and mainly present the target during the video communication process
  • the effect of stabilizing the area so that even if the local user is far away from the image collector of the display device, or the video window on the local screen is small, the moving target, that is, the user's face, can be clearly presented, thereby improving the user's video Communication experience.
  • FIG. 12 is a schematic diagram of interaction between display devices participating in video communication according to an exemplary embodiment of this application. As shown in Figure 12, the method may include:
  • step 121 the first display device executes the methods described in steps 111-114 to obtain first partial image data
  • step 122 the first display device executes the methods described in steps 111-114 to obtain second partial image data
  • step 123 the first display device generates a first video data stream according to the first partial image data, and sends the first video data stream to the second display device;
  • the second display device generates a second video data stream according to the second partial image data, and sends the second video data stream to the first display device;
  • step 125 the first display device receives and decodes the second video data stream to obtain a second partial image
  • step 126 the first display device presents the second partial image on the display
  • step 127 the second display device receives and decodes the first video data stream to obtain the first partial image
  • step 1208 the second display device presents the first partial image on the display.
  • the method of the present application processes the global image according to the target stable area containing the moving target to obtain the partial image, and uses the partial image to perform video communication with the peer device, it can achieve real-time tracking of the moving target and mainly present the target during the video communication process
  • the effect of stabilizing the area so that even if the local user is far away from the image collector of the display device, or the video window on the local screen is small, the moving target, that is, the user's face, can be clearly presented, thereby improving the user's video Communication experience.
  • the present invention also provides a computer storage medium, wherein the computer storage medium may store a program, and the program may include some or all of the steps in each embodiment of the method provided by the present invention when the program is executed.
  • the storage medium may be a magnetic disk, an optical disc, a read-only memory (English: read-only memory, abbreviated as: ROM) or a random access memory (English: random access memory, abbreviated as: RAM), etc.
  • the technology in the embodiments of the present invention can be implemented by means of software plus a necessary general hardware platform.
  • the technical solutions in the embodiments of the present invention can be embodied in the form of software products, which can be stored in a storage medium, such as ROM/RAM. , Magnetic disks, optical disks, etc., including a number of instructions to enable a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments of the present invention.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Human Computer Interaction (AREA)
  • General Engineering & Computer Science (AREA)
  • Controls And Circuits For Display Device (AREA)

Abstract

本申请提供了显示设备及视频通讯数据处理方法，控制器响应于与至少一个对端设备视频通讯的指令，通过图像采集器采集本端的全局图像；在用户界面中呈现本地预览窗口，根据全局图像在本地预览窗口中显示本端视频画面，其中，在全局图像中存在运动目标时，本端视频画面为运动目标的目标稳定区域对应的局部图像。

Description

显示设备及视频通讯数据处理方法
本申请要求在2019年6月10日提交中国专利局、申请日为201910496921.4、申请名称为“基于显示设备的视频通讯数据处理方法、装置及显示设备”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及显示设备技术领域，尤其涉及一种显示设备、视频通讯数据处理方法、视频通讯方法和显示方法。
背景技术
当前，由于如智能电视的显示设备可以为用户提供诸如音频、视频、图片等的播放画面，受到用户的广泛关注。
随着大数据与人工智能的发展，用户对显示设备的功能需求与日俱增。例如，用户想要播放显示画面的同时，呈现多路视频聊天画面；或者，当用户在游戏场景中，实时显示参与者是真实画面；或者，用户在教育类应用程序中，在学习当前画面内容的同时，实时与家长/老师进行远程音视频互动等。
因此，亟待提供一种能够实现上述功能的显示设备。
发明内容
本申请提供了一种显示设备、视频通讯数据处理方法、视频通讯方法和显示方法以解决上述问题。
第一方面，本申请提供了一种显示设备，包括：
显示器，用于呈现用户界面；
图像采集器，用于采集本端的全局图像；
控制器，用于：
响应于与至少一个对端设备进行视频通讯的指令，通过所述图像采集器采集本端的全局图像；
在所述用户界面中呈现本地预览窗口，根据所述全局图像在所述本地预览窗口中显示本端视频画面，其中，在所述全局图像中存在运动目标时，所述本端视频画面为所述运动目标的目标稳定区域对应的局部图像；
响应于与所述对端设备之间视频通讯连接的建立，根据所述本地预览窗口中显示的本端视频画面生成本端视频数据，并发送给所述对端设备。
第二方面，本申请还提供一种显示设备，包括：
显示器，用于呈现用户界面；
控制器，用于：
响应于与所述对端设备之间视频通讯连接的建立，接收对端显示设备发送的对端视频数据；
在用户界面中呈现对端显示窗口；
根据所述对端视频数据在所述对端显示窗口中显示对端视频画面，其中，在所述对端视频画面中存在运动目标时，所述对端视频画面为所述运动目标的目标稳定区域对应的画面。
第三方面，本申请还提供一种显示设备，包括：
显示器，用于呈现用户界面；
图像采集器，用于采集本端的全局图像；
控制器，用于：
响应于所述图像采集器的启动，获取所述图像采集器采集的本端的全局图像；
在所述用户界面中呈现本地预览窗口，根据所述本端的全局图像在所述本地预览窗口中显示本端视频画面，其中，在所述全局图像中存在运动目标时，所述本端视频画面为所述运动目标的目标稳定区域对应的局部图像。
第四方面，本申请还提供一种视频通讯数据处理方法，应用于显示设备，所述方法包括：
响应与至少一个对端设备进行视频通讯的指令，通过图像采集器采集运动目标所在场景的全局图像；
获取所述全局图像；
检测所述全局图像中的运动目标，确定包含所述运动目标的目标稳定区域；
根据所述目标稳定区域对所述全局图像进行处理，得到局部图像；
使用所述局部图像与所述对端设备进行视频通讯。
第五方面，本申请还提供一种视频通讯方法，所述方法包括：
第一显示设备响应于与第二显示设备之间视频通讯连接的建立，接收所述第二显示设备发送的对端视频数据；
在用户界面中呈现对端显示窗口；
根据所述对端视频数据在所述对端显示窗口中显示对端视频画面，其中，在所述对端视频画面中存在运动目标时，所述对端视频画面为所述运动目标的目标稳定区域对应的画面。
第六方面，本申请还提供一种显示方法，应用于显示设备，所述显示设备具有 图像采集器，所述方法包括：
响应于所述图像采集器的启动，获取所述图像采集器采集的本端的全局图像；
在所述用户界面中呈现本地预览窗口，根据所述本端的全局图像在所述本地预览窗口中显示本端视频画面，其中，在所述全局图像中存在运动目标时，所述本端视频画面为所述运动目标的目标稳定区域对应的局部图像。
附图说明
为了更清楚地说明本申请的技术方案，下面将对实施例中所需要使用的附图作简单地介绍，显而易见地，对于本领域普通技术人员而言，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。
图1中示例性示出了显示设备与控制装置之间操作场景的示意图；
图2中示例性示出了控制装置100的硬件配置框图；
图3中示例性示出了显示设备200的硬件配置框图；
图4中示例性示出了根据图3显示设备200的硬件架构框图；
图5中示例性示出了显示设备200的功能配置示意图；
图6a中示例性示出了显示设备200中软件配置示意图；
图6b中示例性示出了显示设备200中应用程序的配置示意图；
图7中示例性示出了显示设备200用户界面的示意图；
图8a为本申请根据一示例性实施例示出的用户通过控制装置100A与显示设备进行交互的一种场景；
图8b为本申请根据一示例性实施例示出的用户通过控制装置100A与显示设备进行交互的另一种场景；
图9a为本申请技术方案的一种应用场景示意图；
图9b为本申请技术方案的另一种应用场景示意图；
图10a示出了两人视频聊天时本端设备所呈现的视频界面；
图10b示出了两人视频聊天时本端设备所呈现的另一视频界面；
图10c示出了四人视频聊天时本端设备所呈现的视频界面；
图10d示出了两人视频聊天时本端设备所呈现的又一视频界面；
图11为本申请基于显示设备的视频通讯数据处理方法的一个实施例流程图；
图12为本申请根据一示例性实施例示出的参与视频通讯的显示设备间的交互示意图。
具体实施方式
为使本申请示例性实施例的目的、技术方案和优点更加清楚，下面将结合本申请示例性实施例中的附图，对本申请示例性实施例中的技术方案进行清楚、完整地描述，显然，所描述的示例性实施例仅是本申请一部分实施例，而不是全部的实施例。
为了使本技术领域的人员更好地理解本申请中的技术方案，下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都应当属于本申请保护的范围。
本申请提供的显示设备，可以是具有多个芯片架构的显示设备，如本申请图3 至图5示出的具有双芯片(双硬件系统)架构的显示设备，也可以是具有非双芯片架构的显示设备，本申请不予限定。
为便于用户使用，显示设备上通常会设置各种外部装置接口，以便于连接不同的外设设备或线缆以实现相应的功能。而在显示设备的接口上连接有高清晰度的摄像头时，如果显示设备的硬件系统没有接收源码的高像素摄像头的硬件接口，那么就会导致无法将摄像头接收到的数据呈现到显示设备的显示屏上。
下面首先结合附图对本申请所涉及的概念进行说明。在此需要指出的是，以下对各个概念的说明，仅为了使本申请的内容更加容易理解，并不表示对本申请保护范围的限定。
本申请各实施例中使用的术语“模块”，可以是指任何已知或后来开发的硬件、软件、固件、人工智能、模糊逻辑或硬件或/和软件代码的组合，能够执行与该元件相关的功能。
本申请各实施例中使用的术语“遥控器”，是指电子设备(如本申请中公开的显示设备)的一个组件，该组件通常可在较短的距离范围内无线控制电子设备。该组件一般可以使用红外线和/或射频(RF)信号和/或蓝牙与电子设备连接，也可以包括WiFi、无线USB、蓝牙、动作传感器等功能模块。例如：手持式触摸遥控器，是以触摸屏中用户界面取代一般遥控装置中的大部分物理内置硬键。
本申请各实施例中使用的术语“手势”，是指用户通过一种手型的变化或手部运动等动作，用于表达预期想法、动作、目的/或结果的用户行为。
本申请各实施例中使用的术语“硬件系统”，可以是指由集成电路(Integrated Circuit，IC)、印刷电路板(Printed circuit board，PCB)等机械、光、电、磁器件构成的具有计算、控制、存储、输入和输出功能的实体部件。在本申请各个实施例 中，硬件系统通常也会被称为主板(motherboard)或芯片。
图1中示例性示出了根据实施例中显示设备与控制装置之间操作场景的示意图。如图1所示，用户可通过控制装置100来操作显示设备200。
其中，控制装置100可以是遥控器100A，其可与显示设备200之间通过红外协议通信、蓝牙协议通信、紫蜂(ZigBee)协议通信或其他短距离通信方式进行通信，用于通过无线或其他有线方式来控制显示设备200。用户可以通过遥控器上按键、语音输入、控制面板输入等输入用户指令，来控制显示设备200。如：用户可以通过遥控器上音量加减键、频道控制键、上/下/左/右的移动按键、语音输入按键、菜单键、开关机按键等输入相应控制指令，来实现控制显示设备200的功能。
控制装置100也可以是智能设备，如移动终端100B、平板电脑、计算机、笔记本电脑等，其可以通过本地网(LAN，Local Area Network)、广域网(WAN，Wide Area Network)、无线局域网((WLAN，Wireless Local Area Network)或其他网络与显示设备200之间通信，并通过与显示设备200相应的应用程序实现对显示设备200的控制。例如，使用在智能设备上运行的应用程序控制显示设备200。该应用程序可以在与智能设备关联的屏幕上通过直观的用户界面(UI，User Interface)为用户提供各种控制。
“用户界面”，是应用程序或操作系统与用户之间进行交互和信息交换的介质接口，它实现信息的内部形式与用户可以接受形式之间的转换。用户界面常用的表现形式是图形用户界面(graphicuserinterface，GUI)，是指采用图形方式显示的与计算机操作相关的用户界面。它可以是在电子设备的显示屏中显示的一个图标、窗口、控件等界面元素，其中控件可以包括图标、按钮、菜单、选项卡、文本框、对话框、状态栏、导航栏、Widget等可视的界面元素。
示例的，移动终端100B与显示设备200均可安装软件应用，从而可通过网络通信协议实现二者之间的连接通信，进而实现一对一控制操作的和数据通信的目的。如：可以使移动终端100B与显示设备200建立控制指令协议，将遥控控制键盘同步到移动终端100B上，通过控制移动终端100B上用户界面，实现控制显示设备200的功能；也可以将移动终端100B上显示的音视频内容传输到显示设备200上，实现同步显示功能。
如图1所示，显示设备200还可与服务器300通过多种通信方式进行数据通信。在本申请各个实施例中，可允许显示设备200通过局域网、无线局域网或其他网络与服务器300进行通信连接。服务器300可以向显示设备200提供各种内容和互动。
示例的，显示设备200通过发送和接收信息，以及电子节目指南(EPG，Electronic Program Guide)互动，接收软件程序更新，或访问远程储存的数字媒体库。服务器300可以是一组，也可以是多组，可以是一类或多类服务器。通过服务器300提供视频点播和广告服务等其他网络服务内容。
显示设备200，可以是液晶显示器、OLED(Organic Light Emitting Diode)显示器、投影显示设备、智能电视。具体显示设备类型，尺寸大小和分辨率等不作限定，本领技术人员可以理解的是，显示设备200可以根据需要做性能和配置上的一些改变。
显示设备200除了提供广播接收电视功能之外，还可以附加提供计算机支持功能的智能网络电视功能。示例的包括，网络电视、智能电视、互联网协议电视(IPTV)等。
如图1所述，显示设备上可以连接或设置有摄像头，用于将摄像头拍摄到的画 面呈现在本显示设备或其他显示设备的显示界面上，以实现用户之间的交互聊天。具体的，摄像头拍摄到的画面可在显示设备上全屏显示、半屏显示、或者显示任意可选区域。
作为一种可选的连接方式，摄像头通过连接板与显示器后壳连接，固定安装在显示器后壳的上侧中部，作为可安装的方式，可以固定安装在显示器后壳的任意位置，能保证其图像采集区域不被后壳遮挡即可，例如，图像采集区域与显示设备的显示朝向相同。
作为另一种可选的连接方式，摄像头通过连接板或者其他可想到的连接器可升降的与显示后壳连接，连接器上安装有升降马达，当用户要使用摄像头或者有应用程序要使用摄像头时，再升出显示器之上，当不需要使用摄像头时，其可内嵌到后壳之后，以达到保护摄像头免受损坏。
作为一种实施例，本申请所采用的摄像头可以为1600万像素，以达到超高清显示目的。在实际使用中，也可采用比1600万像素更高或更低的摄像头。
当显示设备上安装有摄像头以后，显示设备不同应用场景所显示的内容可得到多种不同方式的融合，从而达到传统显示设备无法实现的功能。
示例性的，用户可以在边观看视频节目的同时，与至少一位其他用户进行视频聊天。视频节目的呈现可作为背景画面，视频聊天的窗口显示在背景画面之上。形象的，可以称该功能为“边看边聊”。
可选的，在“边看边聊”的场景中，在观看直播视频或网络视频的同时，跨终端的进行至少一路的视频聊天。
另一些实施例中，用户可以在边进入教育应用学习的同时，与至少一位其他用户进行视频聊天。例如，学生在学习教育应用程序中内容的同时，可实现与老师的 远程互动。形象的，可以称该功能为“边学边聊”。
另一些实施例中，用户在玩纸牌游戏时，与进入游戏的玩家进行视频聊天。例如，玩家在进入游戏应用参与游戏时，可实现与其他玩家的远程互动。形象的，可以称该功能为“边看边玩”。
可选的，游戏场景与视频画面进行融合，将视频画面中人像进行抠图，显示在游戏画面中，提升用户体验。
可选的，在体感类游戏中(如打球类、拳击类、跑步类、跳舞类等)，通过摄像头获取人体姿势和动作，肢体检测和追踪、人体骨骼关键点数据的检测，再与游戏中动画进行融合，实现如体育、舞蹈等场景的游戏。
另一些实施例中，用户可以在K歌应用中，与至少一位其他用户进行视频和语音的交互。形象的，可以称该功能为“边看边唱”。优选的，当至少一位用户在聊天场景进入该应用时，可多个用户共同完成一首歌的录制。
另一些实施例中，用户可在本地打开摄像头获取图片和视频，形象的，可以称该功能为“照镜子”。
在另一些示例中，还可以再增加更多功能或减少上述功能。本申请对该显示设备的功能不作具体限定。
图2中示例性示出了根据示例性实施例中控制装置100的配置框图。如图2所示，控制装置100包括控制器110、通信器130、用户输入/输出接口140、存储器190、供电模块180。
控制装置100被配置为可控制所述显示设备200，以及可接收用户的输入操作指令，且将操作指令转换为显示设备200可识别和响应的指令，起到用户与显示设备200之间交互中介作用。如：用户通过操作控制装置100上频道加减键，显示设 备200响应频道加减的操作。
在一些实施例中，控制装置100可是一种智能设备。如：控制装置100可根据用户需求安装控制显示设备200的各种应用。
在一些实施例中，如图1所示，移动终端100B或其他智能电子设备，可在安装操控显示设备200的应用之后，起到控制装置100类似功能。如：用户可以通过安装应用，在移动终端100B或其他智能电子设备上可提供的图形用户界面的各种功能键或虚拟按钮，以实现控制装置100实体按键的功能。
控制器110包括处理器112、RAM113和ROM114、通信接口以及通信总线。控制器110用于控制控制装置100的运行和操作，以及内部各部件之间通信协作以及外部和内部的数据处理功能。
通信器130在控制器110的控制下，实现与显示设备200之间控制信号和数据信号的通信。如：将接收到的用户输入信号发送至显示设备200上。通信器130可包括WIFI模块131、蓝牙模块132、NFC模块133等通信模块中至少一种。
用户输入/输出接口140，其中，输入接口包括麦克风141、触摸板142、传感器143、按键144等输入接口中至少一者。如：用户可以通过语音、触摸、手势、按压等动作实现用户指令输入功能，输入接口通过将接收的模拟信号转换为数字信号，以及数字信号转换为相应指令信号，发送至显示设备200。
输出接口包括将接收的用户指令发送至显示设备200的接口。在一些实施例中，可以是红外接口，也可以是射频接口。如：红外信号接口时，需要将用户输入指令按照红外控制协议转化为红外控制信号，经红外发送模块进行发送至显示设备200。再如：射频信号接口时，需将用户输入指令转化为数字信号，然后按照射频控制信号调制协议进行调制后，由射频发送端子发送至显示设备200。
在一些实施例中，控制装置100包括通信器130和输出接口中至少一者。控制装置100中配置通信器130，如：WIFI、蓝牙、NFC等模块，可将用户输入指令通过WIFI协议、或蓝牙协议、或NFC协议编码，发送至显示设备200。
存储器190，用于在控制器110的控制下存储驱动和控制控制装置100的各种运行程序、数据和应用。存储器190，可以存储用户输入的各类控制信号指令。
供电模块180，用于在控制器110的控制下为控制装置100各元件提供运行电力支持。可以电池及相关控制电路。
在一些实施例中，如图3-图5中所示，给出了采用双芯片的显示设备200中硬件系统的硬件配置框图。
在采用双硬件系统架构时，硬件系统的机构关系可以图3所示。为便于表述以下将双硬件系统架构中的一个硬件系统称为第一硬件系统或A系统、A芯片，并将另一个硬件系统称为第二硬件系统或N系统、N芯片。A芯片包含A芯片的控制器及各类接口，N芯片则包含N芯片的控制器及各类接口。A芯片及N芯片中可以各自安装有独立的操作系统，从而使显示设备200中存在两个在独立但又存在相互关联的子系统。
在一些实施例中，A芯片也可被称为第一芯片，第一控制器，N芯片也可以被称为第二芯片，第二控制器。
如图3所示，A芯片与N芯片之间可以通过多个不同类型的接口实现连接、通信及供电。A芯片与N芯片之间接口的接口类型可以包括通用输入输出接口(General-purpose input/output，GPIO)、USB接口、HDMI接口、UART接口等。A芯片与N芯片之间可以使用这些接口中的一个或多个进行通信或电力传输。例如图3所示，在双硬件系统架构下，可以由外接的电源(power)为N芯片供电，而 A芯片则可以不由外接电源，而由N芯片供电。
除用于与N芯片进行连接的接口之外，A芯片还可以包含用于连接其他设备或组件的接口，例如图3中所示的用于连接摄像头(Camera)的MIPI接口，蓝牙接口等。
类似的，除用于与N芯片进行连接的接口之外，N芯片还可以包含用于连接显示屏TCON(Timer Control Register)的VBY接口，用于连接功率放大器(Amplifier，AMP)及扬声器(Speaker)的i2S接口；以及IR/Key接口，USB接口，Wifi接口，蓝牙接口，HDMI接口，Tuner接口等。
下面结合图4对双芯片架构进行进一步的说明。需要说明的是图4仅仅是对本申请双硬件系统架构的一些示例性性说明，并不表示对本申请的限定。在实际应用中，两个硬件系统均可根据需要包含更多或更少的硬件或接口。
图4中示例性示出了根据图3显示设备200的硬件架构框图。如图4所示，显示设备200的硬件系统可以包括A芯片和N芯片，以及通过各类接口与A芯片或N芯片相连接的模块。
N芯片可以包括调谐解调器220、通信器230、外部装置接口250、控制器210、存储器290、用户输入接口、视频处理器260-1、音频处理器260-2、显示器280、音频输出接口270、供电模块。在其他实施例中N芯片也可以包括更多或更少的模块。
其中，调谐解调器220，用于对通过有线或无线方式接收广播电视信号，进行放大、混频和谐振等调制解调处理，从而从多个无线或有线广播电视信号中解调出用户所选择电视频道的频率中所携带的音视频信号，以及附加信息(例如EPG数据信号)。根据电视信号广播制式不同，调谐解调器220的信号途径可以有很多种， 诸如：地面广播、有线广播、卫星广播或互联网广播等；以及根据调制类型不同，所述信号的调整方式可以数字调制方式，也可以模拟调制方式；以及根据接收电视信号种类不同，调谐解调器220可以解调模拟信号和/或数字信号。
调谐解调器220，还用于根据用户选择，以及由控制器210控制，响应用户选择的电视频道频率以及该频率所携带的电视信号。
在其他一些示例性实施例中，调谐解调器220也可在外置设备中，如外置机顶盒等。这样，机顶盒通过调制解调后输出电视音视频信号，经过外部装置接口250输入至显示设备200中。
通信器230是用于根据各种通信协议类型与外部设备或外部服务器进行通信的组件。例如：通信器230可以包括WIFI模块231，蓝牙通信协议模块232，有线以太网通信协议模块233，及红外通信协议模块等其他网络通信协议模块或近场通信协议模块。
显示设备200可以通过通信器230与外部控制设备或内容提供设备之间建立控制信号和数据信号的连接。例如，通信器可根据控制器的控制接收遥控器100A的控制信号。
外部装置接口250，是提供N芯片控制器210和A芯片及外部其他设备间数据传输的组件。外部装置接口可按照有线/无线方式与诸如机顶盒、游戏装置、笔记本电脑等的外部设备连接，可接收外部设备的诸如视频信号(例如运动图像)、音频信号(例如音乐)、附加信息(例如EPG)等数据。
其中，外部装置接口250可以包括：高清多媒体接口(HDMI)端子251、复合视频消隐同步(CVBS)端子252、模拟或数字分量端子253、通用串行总线(USB)端子254、红绿蓝(RGB)端子(图中未示出)等任一个或多个。本申请不对外部 装置接口的数量和类型进行限制。
控制器210，通过运行存储在存储器290上的各种软件控制程序(如操作系统和/或各种应用程序)，来控制显示设备200的工作和响应用户的操作。
如图4所示，控制器210包括只读存储器RAM214、随机存取存储器ROM213、图形处理器216、CPU处理器212、通信接口218、以及通信总线。其中，RAM214和ROM213以及图形处理器216、CPU处理器212、通信接口218通过总线相连接。
ROM213，用于存储各种系统启动的指令。如在收到开机信号时，显示设备200电源开始启动，CPU处理器212运行ROM中系统启动指令，将存储在存储器290的操作系统拷贝至RAM214中，以开始运行启动操作系统。当操作系统启动完成后，CPU处理器212再将存储器290中各种应用程序拷贝至RAM214中,然后，开始运行启动各种应用程序。
图形处理器216，用于产生各种图形对象，如：图标、操作菜单、以及用户输入指令显示图形等。包括运算器，通过接收用户输入各种交互指令进行运算，根据显示属性显示各种对象。以及包括渲染器，产生基于运算器得到的各种对象，进行渲染的结果显示在显示器280上。
CPU处理器212，用于执行存储在存储器290中操作系统和应用程序指令。以及根据接收外部输入的各种交互指令，来执行各种应用程序、数据和内容，以便最终显示和播放各种音视频内容。
在一些示例性实施例中，CPU处理器212，可以包括多个处理器。所述多个处理器中可包括一个主处理器以及多个或一个子处理器。主处理器，用于在预加电模式中执行显示设备200一些操作，和/或在正常模式下显示画面的操作。多个或一 个子处理器，用于执行在待机模式等状态下的一种操作。
通信接口，可包括第一接口218-1到第n接口218-n。这些接口可以是经由网络被连接到外部设备的网络接口。
控制器210可以控制显示设备200的整体操作。例如：响应于接收到用于选择在显示器280上显示UI对象的用户命令，控制器210便可以执行与由用户命令选择的对象有关的操作。
其中，所述对象可以是可选对象中的任何一个，例如超链接或图标。与所选择的对象有关操作，例如：显示连接到超链接页面、文档、图像等操作，或者执行与图标相对应程序的操作。用于选择UI对象用户命令，可以是通过连接到显示设备200的各种输入装置(例如，鼠标、键盘、触摸板等)输入命令或者与由用户说出语音相对应的语音命令。
存储器290，包括存储用于驱动和控制显示设备200的各种软件模块。如：存储器290中存储的各种软件模块，包括：基础模块、检测模块、通信模块、显示控制模块、浏览器模块、和各种服务模块等。
其中，基础模块是用于显示设备200中各个硬件之间信号通信、并向上层模块发送处理和控制信号的底层软件模块。检测模块是用于从各种传感器或用户输入接口中收集各种信息，并进行数模转换以及分析管理的管理模块。
例如：语音识别模块中包括语音解析模块和语音指令数据库模块。显示控制模块是用于控制显示器280进行显示图像内容的模块，可以用于播放多媒体图像内容和UI界面等信息。通信模块，是用于与外部设备之间进行控制和数据通信的模块。浏览器模块，是用于执行浏览服务器之间数据通信的模块。服务模块，是用于提供各种服务以及各类应用程序在内的模块。
同时，存储器290还用于存储接收外部数据和用户数据、各种用户界面中各个项目的图像以及焦点对象的视觉效果图等。
用户输入接口，用于将用户的输入信号发送给控制器210，或者，将从控制器输出的信号传送给用户。示例性的，控制装置(例如移动终端或遥控器)可将用户输入的诸如电源开关信号、频道选择信号、音量调节信号等输入信号发送至用户输入接口，再由用户输入接口转送至控制器；或者，控制装置可接收经控制器处理从用户输入接口输出的音频、视频或数据等输出信号，并且显示接收的输出信号或将接收的输出信号输出为音频或振动形式。
在一些实施例中，用户可在显示器280上显示的图形用户界面(GUI)输入用户命令，则用户输入接口通过图形用户界面(GUI)接收用户输入命令。或者，用户可通过输入特定的声音或手势进行输入用户命令，则用户输入接口通过传感器识别出声音或手势，来接收用户输入命令。
视频处理器260-1，用于接收视频信号，根据输入信号的标准编解码协议，进行解压缩、解码、缩放、降噪、帧率转换、分辨率转换、图像合成等视频数据处理，可得到直接在显示器280上显示或播放的视频信号。
示例的，视频处理器260-1，包括解复用模块、视频解码模块、图像合成模块、帧率转换模块、显示格式化模块等。
其中，解复用模块，用于对输入音视频数据流进行解复用处理，如输入MPEG-2,则解复用模块进行解复用成视频信号和音频信号等。
视频解码模块，用于对解复用后的视频信号进行处理，包括解码和缩放处理等。
图像合成模块，如图像合成器，其用于将图形生成器根据用户输入或自身生成的GUI信号，与缩放处理后视频图像进行叠加混合处理，以生成可供显示的图像信 号。
帧率转换模块，用于对输入视频的帧率进行转换，如将输入的24Hz、25Hz、30Hz、60Hz视频的帧率转换为60Hz、120Hz或240Hz的帧率，其中，输入帧率可以与源视频流有关，输出帧率可以与显示器的更新率有关。输入有通常的格式采用如插帧方式实现。
显示格式化模块，用于将帧率转换模块输出的信号，改变为符合诸如显示器显示格式的信号，如将帧率转换模块输出的信号进行格式转换以输出RGB数据信号。
显示器280，用于接收源自视频处理器260-1输入的图像信号，进行显示视频内容和图像以及菜单操控界面。显示器280包括用于呈现画面的显示器组件以及驱动图像显示的驱动组件。显示视频内容，可以来自调谐解调器220接收的广播信号中的视频，也可以来自通信器或外部设备接口输入的视频内容。显示器220，同时显示显示设备200中产生且用于控制显示设备200的用户操控界面UI。
以及，根据显示器280类型不同，还包括用于驱动显示的驱动组件。或者，倘若显示器280为一种投影显示器，还可以包括一种投影装置和投影屏幕。
音频处理器260-2，用于接收音频信号，根据输入信号的标准编解码协议，进行解压缩和解码，以及降噪、数模转换、和放大处理等音频数据处理，得到可以在扬声器272中播放的音频信号。
音频输出接口270，用于在控制器210的控制下接收音频处理器260-2输出的音频信号，音频输出接口可包括扬声器272，或输出至外接设备的发生装置的外接音响输出端子274，如：外接音响端子或耳机输出端子等。
在其他一些示例性实施例中，视频处理器260-1可以包括一个或多个芯片组成。音频处理器260-2，也可以包括一个或多个芯片组成。
以及，在其他一些示例性实施例中，视频处理器260-1和音频处理器260-2，可以为单独的芯片，也可以与控制器210一起集成在一个或多个芯片中。
供电模块，用于在控制器210控制下，将外部电源输入的电力为显示设备200提供电源供电支持。供电模块可以包括安装显示设备200内部的内置电源电路，也可以是安装在显示设备200外部的电源，如在显示设备200中提供外接电源的电源接口。
与N芯片相类似，如图4所示，A芯片可以包括控制器310、通信器330、检测器340、存储器390。在某些实施例中还可以包括用户输入接口、视频处理器、音频处理器、显示器、音频输出接口。在某些实施例中，也可以存在独立为A芯片供电的供电模块。
通信器330是用于根据各种通信协议类型与外部设备或外部服务器进行通信的组件。例如：通信器330可以包括WIFI模块331，蓝牙通信协议模块332，有线以太网通信协议模块333，及红外通信协议模块等其他网络通信协议模块或近场通信协议模块。
A芯片的通信器330和N芯片的通信器230也有相互交互。例如，N芯片的WiFi模块231用于连接外部网络，与外部服务器等产生网络通信。A芯片的WiFi模块331用于连接至N芯片的WiFi模块231，而不与外界网络等产生直接连接。因此，对于用户而言，一个如上述实施例中的显示设备至对外显示一个WiFi账号。
检测器340，是显示设备A芯片用于采集外部环境或与外部交互的信号的组件。检测器340可以包括光接收器342，用于采集环境光线强度的传感器，可以通过采集环境光来自适应显示参数变化等；还可以包括图像采集器341，如相机、摄像头等，可以用于采集外部环境场景，以及用于采集用户的属性或与用户交互手势，可 以自适应变化显示参数，也可以识别用户手势，以实现与用户之间互动的功能。
外部装置接口350，提供控制器310与N芯片或外部其他设备间数据传输的组件。外部装置接口可按照有线/无线方式与诸如机顶盒、游戏装置、笔记本电脑等的外部设备连接。
控制器310，通过运行存储在存储器390上的各种软件控制程序(如用安装的第三方应用等)，以及与N芯片的交互，来控制显示设备200的工作和响应用户的操作。
如图4所示，控制器310包括只读存储器ROM313、随机存取存储器RAM314、图形处理器316、CPU处理器312、通信接口318、以及通信总线。其中，ROM313和RAM314以及图形处理器316、CPU处理器312、通信接口318通过总线相连接。
ROM313，用于存储各种系统启动的指令。CPU处理器312运行ROM中系统启动指令，将存储在存储器390的操作系统拷贝至RAM314中，以开始运行启动操作系统。当操作系统启动完成后，CPU处理器312再将存储器390中各种应用程序拷贝至RAM314中,然后，开始运行启动各种应用程序。
CPU处理器312，用于执行存储在存储器390中操作系统和应用程序指令,和与N芯片进行通信、信号、数据、指令等传输与交互，以及根据接收外部输入的各种交互指令，来执行各种应用程序、数据和内容，以便最终显示和播放各种音视频内容。
通信接口，可包括第一接口318-1到第n接口318-n。这些接口可以是经由网络被连接到外部设备的网络接口，也可以是经由网络被连接到N芯片的网络接口。
控制器310可以控制显示设备200的整体操作。例如：响应于接收到用于选 择在显示器280上显示UI对象的用户命令，控制器210便可以执行与由用户命令选择的对象有关的操作。
图形处理器316，用于产生各种图形对象，如：图标、操作菜单、以及用户输入指令显示图形等。包括运算器，通过接收用户输入各种交互指令进行运算，根据显示属性显示各种对象。以及包括渲染器，产生基于运算器得到的各种对象，进行渲染的结果显示在显示器280上。
A芯片的图形处理器316与N芯片的图形处理器216均能产生各种图形对象。区别性的，若应用1安装于A芯片，应用2安装在N芯片，当用户在应用1的界面，且在应用1内进行用户输入的指令时，由A芯片图形处理器316产生图形对象。当用户在应用2的界面，且在应用2内进行用户输入的指令时，由N芯片的图形处理器216产生图形对象。
图5为本申请根据一些实施例示例性示出的显示设备的功能配置示意图。
如图5所示，A芯片的存储器390和N芯片的存储器290分别用于存储操作系统、应用程序、内容和用户数据等，在A芯片的控制器310和N芯片的控制器210的控制下，驱动显示设备200的系统运行以及响应用户的各种操作。A芯片的存储器390和N芯片的存储器290可以包括易失性和/或非易失性存储器。
对于N芯片，存储器290，具体用于存储驱动显示设备200中控制器210的运行程序，以及存储显示设备200内置各种应用程序，以及用户从外部设备下载的各种应用程序、以及与应用程序相关的各种图形用户界面，以及与图形用户界面相关的各种对象，用户数据信息，以及各种支持应用程序的内部数据。存储器290用于存储操作系统(OS)内核、中间件和应用等系统软件，以及存储输入的视频数据和音频数据、及其他用户数据。
存储器290，具体用于存储视频处理器260-1和音频处理器260-2、显示器280、通信接口230、调谐解调器220、输入/输出接口等驱动程序和相关数据。
在一些实施例中，存储器290可以存储软件和/或程序，用于表示操作系统(OS)的软件程序包括，例如：内核、中间件、应用编程接口(API)和/或应用程序。示例性的，内核可控制或管理系统资源，或其它程序所实施的功能(如所述中间件、API或应用程序)，以及内核可以提供接口，以允许中间件和API，或应用访问控制器，以实现控制或管理系统资源。
示例的，存储器290，包括广播接收模块2901、频道控制模块2902、音量控制模块2903、图像控制模块2904、显示控制模块2905、音频控制模块2906、外部指令识别模块2907、通信控制模块2908、电力控制模块2910、操作系统2911、以及其他应用程序2912、界面布局管理模块2913、事件传输系统2914以及浏览器模块等等。控制器210通过运行存储器290中各种软件程序，来执行诸如：广播电视信号接收解调功能、电视频道选择控制功能、音量选择控制功能、图像控制功能、显示控制功能、音频控制功能、外部指令识别功能、通信控制功能、光信号接收功能、电力控制功能、支持各种功能的软件操控平台、以及浏览器功能等各类功能。存储器390，包括存储用于驱动和控制显示设备200的各种软件模块。如：存储器390中存储的各种软件模块，包括：基础模块、检测模块、通信模块、显示控制模块、浏览器模块、和各种服务模块等。由于存储器390与存储器290的功能比较相似，相关之处参见存储器290即可，在此就不再赘述。
示例的，存储器390，包括图像控制模块3904、音频控制模块2906、外部指令识别模块3907、通信控制模块3908、光接收模块3909、操作系统3911、以及其他应用程序3912、浏览器模块等等。控制器210通过运行存储器290中各种软 件程序，来执行诸如：图像控制功能、显示控制功能、音频控制功能、外部指令识别功能、通信控制功能、光信号接收功能、电力控制功能、支持各种功能的软件操控平台、以及浏览器功能等各类功能。区别性的，N芯片的外部指令识别模块2907和A芯片的外部指令识别模块3907可识别不同的指令。
示例性的，由于摄像头等图像接收设备与A芯片连接，因此，A芯片的外部指令识别模块3907可包括图形识别模块3907-1，图形识别模块3907-1内存储有图形数据库，摄像头接收到外界的图形指令时，与图形数据库中的指令进行对应关系，以对显示设备作出指令控制。而由于语音接收设备以及遥控器与N芯片连接，因此，N芯片的外部指令识别模块2907可包括语音识别模块2907-2，语音识别模块2907-2内存储有语音数据库，语音接收设备等接收到外界的语音指令或时，与语音数据库中的指令进行对应关系，以对显示设备作出指令控制。同样的，遥控器等控制装置100与N芯片连接，由按键指令识别模块与控制装置100进行指令交互。
在一些实施例中，可以没有第一芯片第二芯片之分，显示设备具有的控制器在软件层面为一个操作系统，内置的应用可与上述双芯片架构显示设备中的应用相同。同样设置上述所有接口。
图6a中示例性示出了一些实施例中显示设备200中软件系统的配置框图。
对N芯片，如图6a中所示，操作系统2911，包括用于处理各种基础系统服务和用于实施硬件相关任务的执行操作软件，充当应用程序和硬件组件之间完成数据处理的媒介。
一些实施例中，部分操作系统内核可以包含一系列软件，用以管理显示设备硬件资源，并为其他程序或软件代码提供服务。
其他一些实施例中，部分操作系统内核可包含一个或多个设备驱动器，设备驱 动器可以是操作系统中的一组软件代码，帮助操作或控制显示设备关联的设备或硬件。驱动器可以包含操作视频、音频和/或其他多媒体组件的代码。示例的，包括显示器、摄像头、Flash、WiFi和音频驱动器。
如图6a所示，在一些实施例中，操作系统2911具体可以包括：可访问性模块2911-1、通信模块2911-2、用户界面模块2911-3和控制应用程序2911-4。
在一些实施例中，操作系统2911还可以包括：摄像头调度模块2911-5、摄像头驱动模块2911-6和摄像头开关模块2911-7。
其中，可访问性模块2911-1，用于修改或访问应用程序，以实现应用程序的可访问性和对其显示内容的可操作性。
通信模块2911-2，用于经由相关通信接口和通信网络与其他外设的连接。
用户界面模块2911-3，用于提供显示用户界面的对象，以供各应用程序访问，可实现用户可操作性。
控制应用程序2911-4，用于控制进程管理、切换前台应用，包括运行时间应用程序等。
摄像头调度模块2911-5，用于控制摄像头开启或者关闭，以及升起或者降落。
和摄像头驱动模块2911-6，用于在摄像头调度模块2911-5的控制下，驱动与摄像头机械连接的马达以使摄像头升起或者降落；
摄像头开关模块2911-7，用于在摄像头调度模块2911-5的控制下，开启摄像头，即使其进入开启状态，或者关闭摄像头，即使其进入关闭状态。
如图6a所示，在一些实施例中，事件传输系统2914，可在操作系统2911内或应用程序2912中实现。一些实施例中，一方面在在操作系统2911内实现，同时在应用程序2912中实现，用于监听各种用户输入事件，将根据各种事件指代响应 各类事件或子事件的识别结果，而实施一组或多组预定义的操作的处理程序。
具体的，事件传输系统2914可以包括事件监听模块2914-1和事件识别模块2914-2。其中，事件监听模块2914-1，用于监听用户输入接口输入事件或子事件。
事件识别模块2914-2，用于对各种用户输入接口输入各类事件的定义，识别出各种事件或子事件，且将其传输给处理用以执行其相应一组或多组的处理程序。
需要说明的是，事件或子事件，是指显示设备200中一个或多个传感器检测的输入，以及外界控制设备(如控制装置100等)的输入。如：语音输入各种子事件，手势识别的手势输入子事件，以及控制装置的遥控按键指令输入的子事件等。示例的，遥控器中一个或多个子事件包括多种形式，包括但不限于按键按上/下/左右/、确定键、按键按住等中一个或组合。以及非实体按键的操作，如移动、按住、释放等操作。
界面布局管理模块2913，直接或间接接收来自于事件传输系统2914监听到各用户输入事件或子事件，用于更新用户界面的布局，包括但不限于界面中各控件或子控件的位置，以及容器的大小或位置、层级等与界面布局相关各种执行操作。
由于A芯片的操作系统3911与N芯片的操作系统2911的功能比较相似，相关之处参见操作系统2911即可，在此就不再赘述。
如图6b中所示，显示设备的应用程序层包含可在显示设备200执行的各种应用程序。
N芯片的应用程序层2912可包含但不限于一个或多个应用程序，如：视频点播应用程序、应用程序中心、游戏应用等。A芯片的应用程序层3912可包含但不限于一个或多个应用程序，如：直播电视应用程序、媒体中心应用程序等。需要说明的是，A芯片和N芯片上分别包含什么应用程序是根据操作系统和其他设计确定 的，本发明无需对A芯片和N芯片上所包含的应用程序做具体的限定和划分。
直播电视应用程序，可以通过不同的信号源提供直播电视。例如，直播电视应用程可以使用来自有线电视、无线广播、卫星服务或其他类型的直播电视服务的输入提供电视信号。以及，直播电视应用程序可在显示设备200上显示直播电视信号的视频。
视频点播应用程序，可以提供来自不同存储源的视频。不同于直播电视应用程序，视频点播提供来自某些存储源的视频显示。例如，视频点播可以来自云存储的服务器端、来自包含已存视频节目的本地硬盘储存器。
媒体中心应用程序，可以提供各种多媒体内容播放的应用程序。例如，媒体中心，可以为不同于直播电视或视频点播，用户可通过媒体中心应用程序访问各种图像或音频所提供服务。
应用程序中心，可以提供储存各种应用程序。应用程序可以是一种游戏、应用程序，或某些和计算机系统或其他设备相关但可以在显示设备中运行的其他应用程序。应用程序中心可从不同来源获得这些应用程序，将它们储存在本地储存器中，然后在显示设备200上可运行。
在一些实施例中，由于A芯片及N芯片中可能分别安装有独立的操作系统，从而使显示设备200中存在两个在独立但又存在相互关联的子系统。例如，A芯片和N均可以独立安装有安卓(Android)及各类APP，使得每个芯片均可以实现一定的功能，并且使A芯片和N芯片协同实现某项功能。
图7中示例性示出了根据示例性实施例中显示设备200中用户界面的示意图。如图7所示，用户界面包括多个视图显示区，示例的，第一视图显示区201和播放画面202，其中，播放画面包括布局一个或多个不同项目。以及用户界面中还包括 指示项目被选择的选择器，可通过用户输入而移动选择器的位置，以改变选择不同的项目。
需要说明的是，多个视图显示区可以呈现不同层级的显示画面。如，第一视图显示区可呈现视频聊天项目内容，第二视图显示区可呈现应用层项目内容(如，网页视频、VOD展示、应用程序画面等)。
可选的，不同视图显示区的呈现存在优先级区别，优先级不同的视图显示区之间，视图显示区的显示优先级不同。如，系统层的优先级高于应用层的优先级，当用户在应用层使用获取选择器和画面切换时，不遮挡系统层的视图显示区的画面展示；以及，根据用户的选择使应用层的视图显示区的大小和位置发生变化时，系统层的视图显示区的大小和位置不受影响。
也可以呈现相同层级的显示画面，此时，选择器可以在第一视图显示区和第二视图显示区之间做切换，以及当第一视图显示区的大小和位置发生变化时，第二视图显示区的大小和位置可随及发生改变。
在一些实施例中，图7中的任意一个区域可以显示摄像头获取的画面。
“项目”是指在显示设备200中用户界面的各视图显示区中显示以表示，诸如图标、缩略图、视频剪辑等对应内容的视觉对象。例如：项目可以表示电影、电视剧的图像内容或视频剪辑、音乐的音频内容、应用程序，或其他用户访问内容历史信息。
一些实施例中，“项目”可显示图像缩略图。如：当项目为电影或电视剧时，项目可显示为电影或电视剧的海报。如项目为音乐时，可显示音乐专辑的海报。如项目为应用程序时，可显示为应用程序的图标，或当应用程序被执行最近执行时捕捉到应用程序的内容截图。如项目为用户访问历史时，可显示为最近执行过程中内 容截图。“项目”可显示为视频剪辑。如：项目为电视或电视剧的预告片的视频剪辑动态画面。
此外，项目可以表示显示设备200与外接设备连接的接口或接口集合显示，或可表示连接至显示设备的外部设备名称等。如：信号源输入接口集合、或HDMI接口、USB接口、PC端子接口等。各个项目可具有相同尺寸，也可以具有不相同的尺寸。在一些实施示例中，项目的尺寸可以被改变。
“选择器”用于指示其中任意项目已被选择，如:光标或焦点对象。根据用户通过控制装置100上输入，控制在显示设备200上光标移动来选择或控制项目。可根据用户通过控制装置100的输入，可使显示设备200中显示焦点对象的移动来选择控制项目，可选择或控制其中一个或多个项目。如：用户可通过控制装置100上方向键控制焦点对象在项目之间的移动来选择和控制项目。
图8a为本申请根据一示例性实施例示出的用户通过控制装置100A与显示设备进行交互的一种场景。如图8a所示，显示设备呈现应用中心的操作界面，应用中心包括多个应用程序，其中至少一个应用为社交应用，如QQ、微信、SNS等等。在图8a所示场景中，用户选择启动一个社交应用。
图8b为本申请根据一示例性实施例示出的用户通过控制装置100A与显示设备进行交互的另一种场景。如图8b所示，显示设备呈现用户在图8a所示场景中所选择的社交应用对应的操作界面，在该操作界面中，用户可以在通讯录中选择一个或多个好友的账户，发起视频聊天。
当用户选定一个好友的账户，发起视频聊天时，社交应用将生成与该一个或多个好友进行视频通讯的指令，显示设备的通信模块则响应该指令向登陆相应账号的 一个或多个对端设备发送视频通讯请求。与此同时，显示设备的图像采集器则响应该指令，采集本端用户所在场景的图像数据，另外，显示设备的视频处理器将对采集的图像数据进行编码，并通过通信模块发送给对端设备。当本端设备与对端设备的通信链路连接成功时，对端设备将接收到本端设备发送的视频数据流。对端设备将视频数据流解码，得到相应的图像，并呈现在其显示屏幕上。与此同时，本端设备也将接收到对端设备发送的视频数据流，经解码处理后，呈现在显示屏幕上。
图9a为本申请技术方案的一种应用场景示意图。如图9a所示，本端显示设备200-1与一个对端显示设备200-2通过建立通信连接进行视频通话。在该场景中，本端显示设备屏幕上呈现一个视频窗口，用于呈现一路视频画面(即对端视频画面)，此外还呈现一个本端视频画面预览窗口。
在一些实施例中，上述用于呈现对端视频画面的视频窗口被称为对端显示窗口，用于呈现本端视频画面的预览窗口被称为本地预览窗口。
图9b为本申请技术方案的另一种应用场景示意图。如图9b所示，本端显示设备200-1与5个对端显示设备200-2、200-3、200-4、200-5和200-6通过建立通信连接进行视频通话。在该场景中，本端显示设备屏幕上呈现5个视频窗口，用于呈现5路视频画面，此外还呈现一个本端视频画面预览窗口。
基于以上应用场景，由于显示设备在房间中的位置比较固定，用户舒适的观看距离较远，因此当两人或多人视频聊天时，对端设备采集的全局图像中，对端用户尤其是对端用户的面部只占整幅图像的一小部分，从而使本端设备呈现给本端用户的对端用户及其面部很小，加之本端用户与显示设备较远的观看距离，导致本端用户实际观看到的对端用户及其面部更小。若为多人聊天，多个视频窗口在本端屏幕上的同时呈现，导致每一视频窗口中的目标都被进一步缩小，严重影响用户的视频 聊天体验。另外，当对端用户移动到更远的位置时，上述问题则将更加严重。
示例性地，图10a示出了两人视频聊天时，本端设备所呈现的视频界面，该视频界面中包括对端用户及其所在场景。
示例性地，图10b示出了两人视频聊天时，本端设备所呈现的另一视频界面，该视频界面中包括对端用户及其所在场景。
通过对比图10a和图10b可知，由于对端用户移动到了距离对端图像采集器更远的位置，因此图10b中呈现的人物尤其是人物面部的区域面积很小。
示例性地，图10c示出了四人视频聊天时，本端设备所呈现的视频界面，视频界面，该视频界面中包括分别对应三个对端用户的窗口分别为101,102和103，和一用于预览本端画面的预览视104。可见，与10a相比，由于多个视频窗口在本端屏幕上的同时呈现，导致每一视频窗口中的目标都被进一步缩小，严重影响用户的视频聊天体验。
为了解决上述问题，在一些实施例中，显示设备的控制器，响应于与至少一个对端设备进行视频通讯的指令，通过图像采集器(如摄像头)采集本端的全局图像；在用户界面中呈现本地预览窗口，根据全局图像在本地预览窗口中显示本端视频画面，其中，在全局图像中存在运动目标时，本端视频画面为运动目标的目标稳定区域对应的局部图像；响应于与对端设备之间视频通讯连接的建立，根据本地预览窗口中显示的本端视频画面生成本端视频数据，并发送给对端设备。
在对端设备侧，对端设备的控制器，响应于与其上述显示设备之间视频通讯连接的建立，接收显示设备发送的视频数据；在用户界面中呈现对端显示窗口；根据显示设备发送的视频数据在对端显示窗口中显示对端视频画面，其中，在对端视频画面中存在运动目标时，对端视频画面为所述运动目标的目标稳定区域对应的画面。
在该示例中，显示设备根据包含运动目标的目标稳定区域对全局图像进行处理得到局部图像，并使用局部图像与对端设备进行视频通讯，因此能够在视频通讯过程中到达到实时追踪运动目标并主要呈现目标稳定区域的效果，从而即使本端用户距离显示设备的图像采集器较远，或者，本端屏幕上的视频窗口较小，也能清晰地呈现出运动目标，即用户面部，进而提升用户的视频通讯体验。
在一些实施例中，在用户使用“照镜子”功能时，用户通过操作控制装置启动图像采集器，图像采集器启动后开始采集本端的全局图像；显示设备控制器响应于图像采集器的启动，获取图像采集器采集的本端的全局图像，在用户界面中呈现本地预览窗口，根据本端的全局图像在本地预览窗口中显示本端视频画面，其中，在全局图像中存在运动目标时，本端视频画面为运动目标的目标稳定区域对应的局部图像。
在用户使用“照镜子”功能的场景中，显示设备根据包含运动目标的目标稳定区域对全局图像进行处理得到局部图像，在本地预览窗口中呈现局部图像，这样，在用户使用“照镜子”功能时，也可以达到实时追踪运动目标并主要呈现目标稳定区域的效果，从而即使本端用户距离显示设备的图像采集器较远，或者，本端屏幕上的视频窗口较小，也能清晰地呈现出运动目标，即用户面部，进而提升用户的“照镜子”体验。
需要说明的是，与前述视频通讯的实施情形相比，在前述关于“照镜子”功能的实施情形中，显示设备根据全局图像得到的局部图像仅用于在本地进行展示，而无需将其用于发送给对端设备。对于任意实施情形中，显示设备根据全局图像得到的局部图像的具体实现过程可以参考下述方法实施例。
以下结合本申请提供的视频通讯数据处理方法，对上述示例的具体实现过程，尤其是显示设备根据全局图像得到局部图像的过程，予以详细说明。图11为本申请视频通讯数据处理方法的一个实施例流程图，该方法应用于显示设备，或者说，方法的执行主体为显示设备的控制器，例如图4示出的第一控制器或者第二控制器，但不限于此。如图11所示，该方法可以包括：
步骤111，显示设备响应与至少一个对端设备进行视频通讯的指令，通过图像采集器采集运动目标所在场景的全局图像。
在一种可能的场景中，本端用户在一具有视频通讯功能的应用程序上操作，从通讯录中选择一个或多个好友发起视频通讯，应用程序将响应用户操作，生成一个指令，该指令指示本端设备向本端用户所选择的好友账户对应的对端设备发送视频通讯请求，以与对端设备进行视频通讯。
在另一种可能的场景中，首先由对端用户在一具有视频通讯功能的应用程序上操作，从通讯录中选择至少包括本端用户的一个或多个好友发起视频通讯，对端应用程序将响应用户操作，生成一个指令，该指令指示对端设备向至少包括本端显示设备的一个或多个对端设备发送视频通讯请求，本端用户在本端应用程序上操作，接收该请求，本端应用程序响应用户操作生成另一指令，该指令指示本端显示设备接收对端设备发送的视频请求，以与对端设备进行视频通讯。
本实施例中，本端显示设备具有图像采集器，如图1中示出的摄像头。另外，对端设备也具有图像采集功能，例如，如果对端设备是手机等便携式终端设备，则通过此类终端内置的摄像头采集图像，如果对端设备是如智能电视等家用显示设备，则该家用显示设备至少如图4所示的图像采集器341。
由于视频通讯的目的是为进行视频的两端用户分别为提供对端用户及其所在 场景的实时视频画面，因此如果因为出现前述问题导致一端用户无法在视频画面中看清对端用户，则影响用户对视频通讯的体验，甚至失去视频通讯的意义。
由于用户在画面中具有可移动的特性，因此本申请所述的运动目标即可以随时移动的参与视频通讯的用户，在本端，即为本端用户。在更为具体的实现方案中，运动目标为参与视频通讯的用户的面部或头部。
而运动目标所在场景即图像采集器的采集范围对应的场景，全局图像即图像采集器采集的运动目标所在场景的视频图像。
示例性地，图10a示出了图像采集器在第一时刻采集的全局图像，图10b示出了图像采集器在第二时刻采集的全局图像。如果预先将用户的面部设为运动目标，通过对比可知，由于运动目标在其所在场景中发生了位置移动，因此图10a示出的运动目标尺寸较大，图10b示出的运动目标尺寸较小。
步骤112，获取所述全局图像。
参阅图4，在一种可能的实现方式中，在显示设备中，图像采集器与第一芯片连接，由第一芯片获取图像采集器采集的全局图像数据，并由第一芯片或与第一芯片连接的视频处理器执行本申请方法提供的步骤，也就是说，本申请方法实施例所述步骤的执行主体可以是第一芯片或者与第一芯片连接的视频处理器。
在另一种可能的实现方式中，由于第一芯片和第二芯片之间具有多个通信接口或串口，因此由第一芯片获取图像采集器采集的全局图像数据后，第一芯片对数据做部分处理后或者不做处理，再将数据发送给第二芯片，由第二芯片或与第二芯片连接的视频处理器执行本申请方法提供的步骤，也就是说，本申请方法实施例所述步骤的执行主体可以是第二芯片或者与第二芯片连接的视频处理器。
当然，在另外可能的实现方式中，图像采集器可以与第二芯片连接，由第二芯 片获取图像采集器采集的全局图像数据，并由第二芯片或与第二芯片连接视频处理器执行本申请方法实施例提供的步骤。同理，也可将数据发送给第一芯片进行处理，此处不再赘述。
具体实现时，基于Android系统的摄像头数据获取一般有两种方式，一种是为摄像头设置预览的SurfaceTexture，通过回调获得摄像头纹理，另一种是通过摄像头的PreviewCallback回调获取指定格式的buffer数据。而摄像头数据一般是通过GLSurfaceView或者SurfaceView渲染到屏幕上。基于此，在本申请实施例的一种具体实现方式中，使用PreviewCallback回调和SurfaceTexture纹理获取摄像头数据并使用GLSurfaceView渲染进行浏览。在另一种具体实现方式中，使用SurfaceTexture纹理获取摄像头数据并使用GLSurfaceView渲染进行浏览。
步骤113，检测所述全局图像中的运动目标，确定包含所述运动目标的目标稳定区域。
具体应用中，如果预先设定人的面部为运动目标，那么检测全局图像中的运动目标，即是检测全局图像中的面部区域。具体实现时，可以通过调用深度学习的神经网络模型，如卷积神经网络模型对全局图像中的面部区域进行检测。利用训练好的神经网络模型检测图像中的目标区域属于已有技术，且为本领域技术人员所熟知，因此本申请实施例不再赘述。
需要说明的是，一帧全局图像中可能存在一个运动目标，即一个人的面部，也可能存在多个运动目标，即多个人的面部，还可能不存在运动目标，这种情况可以理解为用户不在图像采集器的采集范围内。
目标稳定区域是指全局图像中运动目标所在的一个稳定区域，或者说，在连续N帧全局图像中，运动目标大概率出现在目标稳定区域内。其中，N帧全局图像对 应一个目标稳定区域，N为一预设帧数。
具体的，在步骤113中，首先检测每帧全局图像中的运动目标，定位目标位置。其中，目标位置为所述全局图像中所述运动目标所在区域的中心点位置。例如，在图10a所示的全局图像中，目标位置所在区域为区域A，目标位置为区域A的中心点。其中，用于表征目标位置的定量参数为运动目标所在区域的中心点位置坐标，如(x，y)。
需要说明的是，由于全局图像中的运动目标可能为1个，也可能为多个，因此上述运动目标所在区域应当为全局图像中全部的运动目标所在的区域。其中当存在多个运动目标时，需要遍历每个运动目标，并根据位置最靠上、位置最靠下、位置最靠左以及位置最靠右的运动目标，确定相应的运动目标所在区域，以保证每个运动目标均在确定的区域内。例如，图10d示出了两人视频聊天时本端设备所呈现的一全局图像，在该全局图像中，包括两个人的面部区域，该两个面部区域即为两个运动目标，相应的运动目标所在区域如图中示出的区域B，目标位置也即区域B的中心点。
具体实现时，本申请根据显示设备摄像头的纹理，预先建立全局图像所处的二维坐标系，并以该二维坐标系为参照标准来讨论本申请所述的任何位置或位置关系。另外，由于运动目标所在区域是全局图像的一部分或几部分区域，是无数个点的集合，因此，取运动目标所在区域的中心点位置作为目标位置。另外，由于不同缩放比例下，全局图像、运动目标所在区域以及目标稳定区域的实际尺寸会发生变化，因此本申请所述的中心点坐标的值均是适应于缩放比例的相对值，而非绝对值。
在检测每帧全局图像中的运动目标，定位目标位置之后，统计连续N帧全局图像对应的N个目标位置中每一位置的频数，以频数最大的位置对应的区域为所述N 帧全局图像的目标稳定区域。
具体的，遍历所述N帧全局图像对应的N个目标位置，当遍历到第i个目标位置时，将第i个目标位置分别与第1至第i-1个目标位置进行匹配，i∈(1,N]；如果第i个目标位置与第1至第i-1个目标位置中的至少一个匹配，将相匹配两个目标位置的当前频数加1。
例如，如果第i个目标位置与第1个目标位置匹配，则将第i个目标位置和第1个目标位置的当前频数加1，如果第i个目标位置与第1个目标位置和第i-5个目标位置均匹配，则将第i个目标位置的当前频数加2，将第1个和第i-5个目标位置的当前频数分别加1。
需要说明的是，每一目标位置的初始频数为1。或者说，在将第i个目标位置分别与第1至第i-1个目标位置进行匹配之前，第i个目标位置的当前频数为1。
示例性地，当遍历到第i个目标位置时，第1个至第i个目标位置的当前频数可能如下表1所示。
表1
目标位置序号 当前频数
1 1
2 3
… …
i-1 2
i 3
i+1 1(初始频数)
… …
作为一种可能的匹配方式，所述将第i个目标位置分别与第1至第i-1个目标位置进行匹配，可以包括：
首先，根据第i个目标位置和预设参考点计算第i个目标位置对应的参考距离。
其中预设参考点可以为全局图像的四个顶点中的任意一个。为了便于计算，使全局图像的一个顶点为坐标原点，进而可以将坐标原点预设为参考点。具体计算时，将目标位置的坐标(运动目标所在区域中心点坐标)与和预设参考点坐标代入到两点距离计算公式中，计算得到该目标位置对应的参考距离。
其次，分别计算第i个目标位置对应的参考距离与第1至第i-1个目标位置对应的参考距离的差值。其中，第1至第i-1个目标位置对应的参考距离为已经计算得到并保存的数据。在本步骤中，将得到分别与第1至第i-1个目标位置对应的i-1个差值数据。
最后，通过判断每个差值是否属于预设范围，判定相应的两个目标位置是否匹配。具体的，如果某一差值属于预设范围，则判定相应的两个目标位置是匹配的，否则非匹配。具体应用中，该预设范围可以设为(-20px,20px)，也就是说，如果第i个目标位置对应的参考距离与另一个目标位置对应的参考距离的绝对差值在20个像素点以内，即判定这两个目标位置匹配，也就说明，这两个目标位置相近。
作为另一种可能的实现方式，所述将第i个目标位置分别与第1至第i-1个目标位置进行匹配，可以包括：
首先，分别计算第i个目标位置与第1至第i-1个目标位置之间的真实距离。
具体计算时，将两个目标位置的坐标(运动目标所在区域中心点坐标)代入到两点距离计算公式中，计算得到该两个目标位置之间的真实距离。在本步骤中，将得到分别与第1至第i-1个目标位置对应的i-1个真实距离数据。
然后，通过判断每个真实距离是否属于预设范围，判定相应的两个目标位置是否匹配。具体的，如果某一真实距离属于预设范围，则判定相应的两个目标位置是匹配的，否则非匹配。
在步骤113中，通过检测每帧所述全局图像中的运动目标，定位目标位置，并以连续N帧全局图像为一个统计对象组，统计该连续N帧全局图像对应的N个目标位置中每一位置的频数，以频数最大的位置对应的区域为所述N帧全局图像的目标稳定区域。可见，由于以频数最大的目标位置对应的区域为该N帧全局图像的目标稳定区域，因此通过步骤113提供的方式所确定的目标稳定区域，是对于该N帧全局图像而言，运动目标所在的一个稳定区域，或者说，对于该N帧全局图像而言，运动目标大概率出现在该目标稳定区域内。
需要说明的是，为了保证局部图像的画质和呈现效果，本申请实施例所确定的目标稳定区域的比例优选与全局图像的分辨率比例保持一致。
步骤114，根据所述目标稳定区域对所述全局图像进行处理，得到局部图像。
在步骤114中，以该目标稳定区域所对应的N帧全局图像为一个处理对象组，对该组中N帧全局图像进行批处理。具体的，首先从每帧全局图像中获取所述目标稳定区域对应的局部图像；然后按照预设规则对所述局部图像进行放大处理。
在获取目标稳定区域对应的局部图像的具体实现方式中，根据目标稳定区域的边缘预置一个裁剪框，根据裁剪框对裁剪框以外的图像区域进行裁剪，得到目标稳定区域对应的局部图像。需要说明的是，对于相邻两个处理对象组，即相邻的两组连续N帧全局图像，如第1-N帧和第N+1至第2N帧，如果在后的处理对象组对应的目标稳定区域相比于在前的处理对象组对应的目标稳定区域发生变化，则将裁剪框从变化前的目标稳定区域等步长移动到变化后的目标稳定区域，以将裁剪框以 外的图像区域裁剪掉。
在按照预设规则放大局部图像的具体实现方式中，一种可能的预设放大规则如下：
如果局部图像的宽W 局部小于全局图像的宽度的四分之一1/4W 全局，或者，局部图像的高H 局部小于全局图像的高度的四分之一1/4H 全局，则将局部图像放大到全局图像尺寸的一半，即令W 局部＝1/2W 全局，H 局部＝1/2H 全局。
如果局部图像的宽W 局部大于全局图像的宽度的四分之一1/4W 全局且小于全局图像的宽度的二分之一1/2W 全局，或者，局部图像的高H 局部大于全局图像的高度的四分之一1/4H 全局且小于全局图像的高度的二分之一1/2H 全局，则将局部图像放大到原来的二倍，即令W 局部＝2W 局部，H 局部＝2H 局部。
如果局部图像的宽W 局部大于全局图像的宽度的二分之一1/2W 全局，或者，局部图像的高H 局部大于全局图像的高度的二分之一1/2H 全局，则将局部图像放大到全局图像的尺寸，即令W 局部＝W 全局，H 局部＝H 全局。
另外，本申请还提供另一种可能的预设放大规则，与上述放大规则不同的是：
如果局部图像的宽W 局部大于全局图像的宽度的四分之一1/4W 全局且小于全局图像的宽度的二分之一1/2W 全局，或者，局部图像的高H 局部大于全局图像的高度的四分之一1/4H 全局且小于全局图像的高度的二分之一1/2H 全局，则将局部图像放大到原来的1.5倍，即令W 局部＝1.5W 局部，H 局部＝1.5H 局部。
步骤115，使用所述局部图像与所述对端设备进行视频通讯。
具体实现时，对局部图像数据进行编码，生成视频数据流，并发送给对端设备。
在另外的实施例中，本申请方法还可以包括
步骤106，在所述显示设备的预览视窗内呈现所述局部图像。例如图10c示出 的预览视窗104。
需要说明的是，无论是本端设备预览所述局部图像的场景，还是局部图像相应的视频数据流在对端经解码后呈现场景中，如果局部图像的尺寸大于屏幕尺寸或者窗口尺寸，则本端设备或对端设备进行相应的自适应处理。
由以上技术方案可知，本申请实施例提供一种视频通讯数据处理方法，所述方法包括：响应与至少一个对端设备进行视频通讯的指令，通过图像采集器采集运动目标所在场景的全局图像，获取采集的全局图像；然后检测所述全局图像中的运动目标，并确定包含所述运动目标的目标稳定区域；再根据所述目标稳定区域对所述全局图像进行处理，得到局部图像；最后使用所述局部图像与所述对端设备进行视频通讯。
由于本申请方法根据包含运动目标的目标稳定区域对全局图像进行处理得到局部图像，并使用局部图像与对端设备进行视频通讯，因此能够在视频通讯过程中到达到实时追踪运动目标并主要呈现目标稳定区域的效果，从而即使本端用户距离显示设备的图像采集器较远，或者，本端屏幕上的视频窗口较小，也能清晰地呈现出运动目标，即用户面部，进而提升用户的视频通讯体验。
根据本申请提供的视频通讯数据处理方法实施例，本申请还提供一种视频通讯方法，图12为本申请根据一示例性实施例示出的参与视频通讯的显示设备间的交互示意图。如图12所示，该方法可以包括：
在步骤121中，第一显示设备执行步骤111-114所述的方法，获得第一局部图像数据；
与此同时，在步骤122中，第一显示设备执行步骤111-114所述的方法，获得第二局部图像数据；
在步骤123中，第一显示设备根据第一局部图像数据生成第一视频数据流，并将第一视频数据流发送给第二显示设备；
与此同时，在步骤124中，第二显示设备根据第二局部图像数据生成第二视频数据流，并将第二视频数据流发送给第一显示设备；
在步骤125中，第一显示设备接收第二视频数据流并解码，得到第二局部图像；
在步骤126中，第一显示设备在显示器上呈现第二局部图像；
与此同时，在步骤127中，第二显示设备接收第一视频数据流并解码，得到第一局部图像；
在步骤128中，第二显示设备在显示器上呈现第一局部图像。
由于本申请方法根据包含运动目标的目标稳定区域对全局图像进行处理得到局部图像，并使用局部图像与对端设备进行视频通讯，因此能够在视频通讯过程中到达到实时追踪运动目标并主要呈现目标稳定区域的效果，从而即使本端用户距离显示设备的图像采集器较远，或者，本端屏幕上的视频窗口较小，也能清晰地呈现出运动目标，即用户面部，进而提升用户的视频通讯体验。
基于本申请中示出的示例性实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。此外，虽然本申请中公开内容按照示范性一个或几个实例来介绍，但应理解，可以就这些公开内容的各个方面也可以单独构成一个完整技术方案。
具体实现中，本发明还提供一种计算机存储介质，其中，该计算机存储介质可存储有程序，该程序执行时可包括本发明提供的方法的各实施例中的部分或全部步骤。所述的存储介质可为磁碟、光盘、只读存储记忆体(英文：read-only memory，简称：ROM)或随机存储记忆体(英文：random access memory，简称：RAM) 等。
本领域的技术人员可以清楚地了解到本发明实施例中的技术可借助软件加必需的通用硬件平台的方式来实现。基于这样的理解，本发明实施例中的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品可以存储在存储介质中，如ROM/RAM、磁碟、光盘等，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本发明各个实施例或者实施例的某些部分所述的方法。
本说明书中各个实施例之间相同相似的部分互相参见即可。尤其，对于装置和设备实施例而言，由于其基本相似于方法实施例，所以描述的比较简单，相关之处参见方法实施例中的说明即可。

Claims (18)

  1. 一种显示设备，其特征在于，包括：
    显示器，用于呈现用户界面；
    图像采集器，用于采集本端的全局图像；
    控制器，用于：
    响应于与至少一个对端设备进行视频通讯的指令，通过所述图像采集器采集本端的全局图像；
    在所述用户界面中呈现本地预览窗口，根据所述全局图像在所述本地预览窗口中显示本端视频画面，其中，在所述全局图像中存在运动目标时，所述本端视频画面为所述运动目标的目标稳定区域对应的局部图像；
    响应于与所述对端设备之间视频通讯连接的建立，根据所述本地预览窗口中显示的本端视频画面生成本端视频数据，并发送给所述对端设备。
  2. 根据权利要求1所述的显示设备，其特征在于，所述控制器还用于：
    响应于与所述对端设备之间视频通讯连接的建立，接收对端设备发送的对端视频数据；
    在所述用户界面中呈现本地预览窗口的同时，呈现对端显示窗口；
    根据所述对端视频数据在所述对端显示窗口中显示对端视频画面。
  3. 根据权利要求1或2所述的显示设备，其特征在于，所述根据所述全局图像在所述本地预览窗口中显示本端视频画面，包括：
    检测所述全局图像中的运动目标，确定包含所述运动目标的目标稳定区域；
    根据所述目标稳定区域对所述全局图像进行处理，得到所述目标稳定区域对应的局部图像；
    将所述局部图像作为本地视频画面显示在所述本地预览窗口中。
  4. 根据权利要求3所述的显示设备，其特征在于，所述检测全局图像中的运动目标，确定包含所述运动目标的目标稳定区域，包括：
    检测每帧所述全局图像中的运动目标，定位目标位置，所述目标位置为所述全局图像中所述运动目标所在区域的中心点位置；
    统计连续N帧全局图像对应的N个目标位置中每一位置的频数，以频数最大的位置对应的区域为所述N帧全局图像的目标稳定区域。
  5. 根据权利要求4所述的显示设备，其特征在于，所述统计连续N帧全局图像对应的N个目标位置中每一位置的频数，包括：
    遍历所述N帧全局图像对应的N个目标位置，当遍历到第i个所述目标位置时，将所述第i个目标位置分别与第1至第i-1个目标位置进行匹配，i∈(1,N]；
    如果所述第i个目标位置与所述第1至第i-1个目标位置中的至少一个匹配，将相匹配两个目标位置的当前频数加1。
  6. 根据权利要求5所述的显示设备，其特征在于，所述将第i个目标位置分别与第1至第i-1个目标位置进行匹配，包括：
    根据所述第i个目标位置和预设参考点计算第i个目标位置对应的参考距离；
    分别计算所述第i个目标位置对应的参考距离与第1至第i-1个目标位置对应的参考距离的差值；
    通过判断每个所述差值是否属于预设范围，判定相应的两个目标位置是否匹配。
  7. 根据权利要求4所述的显示设备，其特征在于，所述根据目标稳定区域对所述全局图像进行处理，得到局部图像，包括：
    从所述全局图像中获取所述目标稳定区域对应的局部图像；
    按照预设规则对所述局部图像进行放大处理。
  8. 一种显示设备，其特征在于，包括：
    显示器，用于呈现用户界面；
    控制器，用于：
    响应于与所述对端设备之间视频通讯连接的建立，接收对端显示设备发送的对端视频数据；
    在用户界面中呈现对端显示窗口；
    根据所述对端视频数据在所述对端显示窗口中显示对端视频画面，其中，在所述对端视频画面中存在运动目标时，所述对端视频画面为所述运动目标的目标稳定区域对应的画面。
  9. 一种显示设备，其特征在于，包括：
    显示器，用于呈现用户界面；
    图像采集器，用于采集本端的全局图像；
    控制器，用于：
    响应于所述图像采集器的启动，获取所述图像采集器采集的本端的全局图像；
    在所述用户界面中呈现本地预览窗口，根据所述本端的全局图像在所述本地预览窗口中显示本端视频画面，其中，在所述全局图像中存在运动目标时，所述本端视频画面为所述运动目标的目标稳定区域对应的局部图像。
  10. 一种视频通讯数据处理方法，应用于显示设备，其特征在于，所述方法包括：
    响应与至少一个对端设备进行视频通讯的指令，通过图像采集器采集运动目标所在场景的全局图像；
    获取所述全局图像；
    检测所述全局图像中的运动目标，确定包含所述运动目标的目标稳定区域；
    根据所述目标稳定区域对所述全局图像进行处理，得到局部图像；
    使用所述局部图像与所述对端设备进行视频通讯。
  11. 根据权利要求10所述的方法，其特征在于，所述方法还包括：
    在显示器上呈现本地预览窗口；
    在本地预览窗口中显示所述局部图像。
  12. 根据权利要求10所述的方法，其特征在于，所述与至少一个对端设备进行视频通讯的指令包括：
    显示设备应用程序响应用户操作生成的用于指示与至少一个对端设备进行视频通讯的指令；
    显示设备应用程序响应用户操作生成的用于指示接受视频通讯请求的指令，所述视频通讯请求由对端设备发送。
  13. 根据权利要求10所述的方法，其特征在于，所述检测全局图像中的运动目标，确定包含所述运动目标的目标稳定区域，包括：
    检测每帧所述全局图像中的运动目标，定位目标位置，所述目标位置为所述全局图像中所述运动目标所在区域的中心点位置；
    统计连续N帧全局图像对应的N个目标位置中每一位置的频数，以频数最大的位置对应的区域为所述N帧全局图像的目标稳定区域。
  14. 根据权利要求13所述的方法，其特征在于，所述统计连续N帧全局图 像对应的N个目标位置中每一位置的频数，包括：
    遍历所述N帧全局图像对应的N个目标位置，当遍历到第i个所述目标位置时，将所述第i个目标位置分别与第1至第i-1个目标位置进行匹配，i∈(1,N]；
    如果所述第i个目标位置与所述第1至第i-1个目标位置中的至少一个匹配，将相匹配两个目标位置的当前频数加1。
  15. 根据权利要求14所述的方法，其特征在于，所述将第i个目标位置分别与第1至第i-1个目标位置进行匹配，包括：
    根据所述第i个目标位置和预设参考点计算第i个目标位置对应的参考距离；
    分别计算所述第i个目标位置对应的参考距离与第1至第i-1个目标位置对应的参考距离的差值；
    通过判断每个所述差值是否属于预设范围，判定相应的两个目标位置是否匹配。
  16. 根据权利要求13所述的方法，其特征在于，所述根据目标稳定区域对所述全局图像进行处理，得到局部图像，包括：
    从所述全局图像中获取所述目标稳定区域对应的局部图像；
    按照预设规则对所述局部图像进行放大处理。
  17. 一种视频通讯方法，其特征在于，所述方法包括：
    第一显示设备响应于与第二显示设备之间视频通讯连接的建立，接收所述第二显示设备发送的对端视频数据；
    在用户界面中呈现对端显示窗口；
    根据所述对端视频数据在所述对端显示窗口中显示对端视频画面，其中，在所述对端视频画面中存在运动目标时，所述对端视频画面为所述运动目标的 目标稳定区域对应的画面。
  18. 一种显示方法，其特征在于，应用于显示设备，所述显示设备具有图像采集器，所述方法包括：响应于所述图像采集器的启动，获取所述图像采集器采集的本端的全局图像；
    在所述用户界面中呈现本地预览窗口，根据所述本端的全局图像在所述本地预览窗口中显示本端视频画面，其中，在所述全局图像中存在运动目标时，所述本端视频画面为所述运动目标的目标稳定区域对应的局部图像。
PCT/CN2020/084717 2019-06-10 2020-04-14 显示设备及视频通讯数据处理方法 Ceased WO2020248697A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US17/332,713 US11917329B2 (en) 2019-06-10 2021-05-27 Display device and video communication data processing method

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910496921.4 2019-06-10
CN201910496921 2019-06-10

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/332,713 Continuation US11917329B2 (en) 2019-06-10 2021-05-27 Display device and video communication data processing method

Publications (1)

Publication Number Publication Date
WO2020248697A1 true WO2020248697A1 (zh) 2020-12-17

Family

ID=73657910

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/084717 Ceased WO2020248697A1 (zh) 2019-06-10 2020-04-14 显示设备及视频通讯数据处理方法

Country Status (3)

Country Link
US (1) US11917329B2 (zh)
CN (2) CN112073770B (zh)
WO (1) WO2020248697A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115700462A (zh) * 2021-07-29 2023-02-07 荣耀终端有限公司 快捷方式显示系统、方法及相关装置

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113011259A (zh) * 2021-02-09 2021-06-22 苏州臻迪智能科技有限公司 电子设备的操作方法
CN115081384B (zh) * 2022-07-21 2023-05-23 北京云枢创新软件技术有限公司 一种基于eda的数据动态可视化系统
CN115277810B (zh) * 2022-08-03 2024-02-09 北京仁光科技有限公司 信号源并行交互系统
CN117579769B (zh) * 2023-11-30 2024-10-22 郑州迪维勒普科技有限公司 一种基于5g的轻量级应急通信方法与系统

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102025965A (zh) * 2010-12-07 2011-04-20 华为终端有限公司 视频通话方法及可视电话
US20140055552A1 (en) * 2012-08-27 2014-02-27 Samsung Electronics Co., Ltd. Mobile device and method for messenger-based video call service
CN105787884A (zh) * 2014-12-18 2016-07-20 联想(北京)有限公司 一种图像处理方法及电子设备
CN105979194A (zh) * 2016-05-26 2016-09-28 努比亚技术有限公司 一种视频图像处理装置和方法
CN106792092A (zh) * 2016-12-19 2017-05-31 广州虎牙信息科技有限公司 直播视频流分镜显示控制方法及其相应的装置
CN107317993A (zh) * 2017-08-08 2017-11-03 维沃移动通信有限公司 一种视频通话方法及移动终端
CN107396165A (zh) * 2016-05-16 2017-11-24 杭州海康威视数字技术股份有限公司 一种视频播放方法及装置

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8174556B1 (en) * 1997-10-01 2012-05-08 8X8, Inc. Videoconferencing arrangement having multi-purpose digital still camera
KR100469727B1 (ko) * 2003-03-07 2005-02-02 삼성전자주식회사 통화자의 얼굴을 화면 가운데에 위치시킬 수 있는화상통신용 통신단말기 및 방법
CN101951493A (zh) * 2010-09-25 2011-01-19 中兴通讯股份有限公司 移动终端及其视频通话中对远端图像局部放大方法
CN104754182B (zh) * 2015-03-19 2016-01-20 长安大学 基于自适应运动滤波的高分辨航拍视频稳相方法
US10055895B2 (en) * 2016-01-29 2018-08-21 Snap Inc. Local augmented reality persistent sticker objects
CN106791571B (zh) * 2017-01-09 2020-05-19 宇龙计算机通信科技(深圳)有限公司 一种用于双屏显示终端的视频显示方法及装置

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102025965A (zh) * 2010-12-07 2011-04-20 华为终端有限公司 视频通话方法及可视电话
US20140055552A1 (en) * 2012-08-27 2014-02-27 Samsung Electronics Co., Ltd. Mobile device and method for messenger-based video call service
CN105787884A (zh) * 2014-12-18 2016-07-20 联想(北京)有限公司 一种图像处理方法及电子设备
CN107396165A (zh) * 2016-05-16 2017-11-24 杭州海康威视数字技术股份有限公司 一种视频播放方法及装置
CN105979194A (zh) * 2016-05-26 2016-09-28 努比亚技术有限公司 一种视频图像处理装置和方法
CN106792092A (zh) * 2016-12-19 2017-05-31 广州虎牙信息科技有限公司 直播视频流分镜显示控制方法及其相应的装置
CN107317993A (zh) * 2017-08-08 2017-11-03 维沃移动通信有限公司 一种视频通话方法及移动终端

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115700462A (zh) * 2021-07-29 2023-02-07 荣耀终端有限公司 快捷方式显示系统、方法及相关装置

Also Published As

Publication number Publication date
CN112073770B (zh) 2022-12-09
CN116260999A (zh) 2023-06-13
US20210289165A1 (en) 2021-09-16
US11917329B2 (en) 2024-02-27
CN112073770A (zh) 2020-12-11

Similar Documents

Publication Publication Date Title
CN110611787B (zh) 一种显示器及图像处理方法
CN112068741B (zh) 显示设备及显示设备蓝牙开关状态的显示方法
CN112073798B (zh) 一种数据传输方法及设备
CN112073770B (zh) 显示设备及视频通讯数据处理方法
WO2021031623A1 (zh) 显示设备、分享文件的方法和服务器
CN112463267B (zh) 在显示设备屏幕上呈现屏保信息的方法及显示设备
WO2020248829A1 (zh) 音视频处理方法及显示设备
WO2021189358A1 (zh) 显示设备和音量调节方法
WO2021031598A1 (zh) 视频聊天窗口位置的自适应调整方法及显示设备
CN112788378B (zh) 显示设备与内容显示方法
WO2020248654A1 (zh) 显示设备及应用共同显示的方法
CN112783380A (zh) 显示设备和方法
CN113938633B (zh) 一种视频通话处理方法及显示设备
CN112786036B (zh) 显示设备与内容显示方法
CN112073666B (zh) 一种显示设备的电源控制方法及显示设备
CN113448529B (zh) 显示设备和音量调节方法
CN112073789B (zh) 一种声音处理法及显示设备
CN112073777B (zh) 一种语音交互方法及显示设备
WO2020248788A1 (zh) 一种语音控制方法及显示设备
CN112399225B (zh) 放映厅业务管理方法及显示设备
WO2020248886A1 (zh) 图像处理方法及显示设备
CN111385631A (zh) 一种显示设备、通信方法及存储介质
CN112788381B (zh) 显示设备及显示方法
CN113497966B (zh) 一种双屏显示设备
WO2020248790A1 (zh) 语音控制方法及显示设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20821734

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20821734

Country of ref document: EP

Kind code of ref document: A1