WO2025123762A1 - 一种显示设备及设备控制方法 - Google Patents

一种显示设备及设备控制方法 Download PDF

Info

Publication number
WO2025123762A1
WO2025123762A1 PCT/CN2024/113670 CN2024113670W WO2025123762A1 WO 2025123762 A1 WO2025123762 A1 WO 2025123762A1 CN 2024113670 W CN2024113670 W CN 2024113670W WO 2025123762 A1 WO2025123762 A1 WO 2025123762A1
Authority
WO
WIPO (PCT)
Prior art keywords
content
hdmi
node
virtual
display device
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
PCT/CN2024/113670
Other languages
English (en)
French (fr)
Inventor
崔文华
于宁
周晓磊
张继栋
于清晓
王之奎
宋虎
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hisense Visual Technology Co Ltd
Original Assignee
Hisense Visual Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hisense Visual Technology Co Ltd filed Critical Hisense Visual Technology Co Ltd
Priority to CN202480038768.XA priority Critical patent/CN121312122A/zh
Publication of WO2025123762A1 publication Critical patent/WO2025123762A1/zh
Anticipated expiration legal-status Critical
Pending legal-status Critical Current

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/436—Interfacing a local distribution network, e.g. communicating with another STB or one or more peripheral devices inside the home
    • H04N21/4363—Adapting the video stream to a specific local network, e.g. a Bluetooth® network
    • H04N21/43632—Adapting the video stream to a specific local network, e.g. a Bluetooth® network involving a wired protocol, e.g. IEEE 1394
    • H04N21/43635—HDMI
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/426—Internal components of the client ; Characteristics thereof
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/426—Internal components of the client ; Characteristics thereof
    • H04N21/42607—Internal components of the client ; Characteristics thereof for processing the incoming bitstream
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
    • H04N21/4307—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen
    • H04N21/43076—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen of the same content streams on multiple devices, e.g. when family members are watching the same movie on different devices
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/436—Interfacing a local distribution network, e.g. communicating with another STB or one or more peripheral devices inside the home
    • H04N21/4363—Adapting the video stream to a specific local network, e.g. a Bluetooth® network
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/14—Systems for two-way working
    • H04N7/15—Conference systems

Definitions

  • the present disclosure relates to the technical field of display devices, and in particular to a display device and a device control method.
  • High Definition Multimedia Interface is a special digital interface suitable for image transmission, which can transmit audio signals and image signals at the same time.
  • HDMI has been standardized
  • Consumer Electronics Control is a complete single-bus protocol generated by HDMI standardization.
  • the sending device can use CEC signals to control external devices connected to the HDMI port, where the controlled external device is called a consumer electronics control device (CEC device for short).
  • CEC devices can transmit HDMI content to the sending device through the HDMI port, and display HDMI content on the sending device.
  • the sending device cannot share the displayed HDMI content to the receiving device, which leads to the failure of content sharing in the remote video conference.
  • an embodiment of the present disclosure provides a display device, including: a display configured to display a first HDMI content transmitted by a consumer electronics control device CEC via an input port HDMIIN of a high-definition multimedia interface HDMI;
  • a user input interface is configured to receive instructions from a user; a communication device is configured to communicate with an external device according to a predetermined protocol; a memory is configured to store computer instructions and data associated with a display device; at least one processor is connected to the display, the user input interface, the communication device and the memory, and is configured to execute computer instructions so that the display device executes: in response to a first content sharing request sent by a first application, constructing the HDMIIN into a virtual device node so that the first HDMI content is stored in the virtual device node, wherein the first content sharing request is used to request that the first HDMI content be shared to a first device; adding a sharing interface corresponding to the virtual device node in a hardware abstraction layer HAL, so that the first application obtains the first HDMI content from the virtual device node by calling the sharing interface, and sharing the first HDMI content to the first device.
  • an embodiment of the present disclosure provides a device control method, including: receiving a first HDMI content transmitted by a consumer electronics control device CEC through an input port HDMIIN of a high-definition multimedia interface HDMI; in response to a first content sharing request sent by a first application, constructing the HDMIIN into a virtual device node so that the first HDMI content is stored in the virtual device node, wherein the first content sharing request is used to request that the first HDMI content be shared to a first device; adding a shared interface corresponding to the virtual device node in a hardware abstraction layer HAL, so that the first application obtains the first HDMI content from the virtual device node by calling the shared interface, and shares the first HDMI content to the first device.
  • FIG1 is a schematic diagram of an operation scenario between a display device and a control device according to some embodiments
  • FIG2 is a block diagram of a hardware configuration of a control device 100 according to some embodiments.
  • FIG3 is a block diagram of a hardware configuration of a display device 200 according to some embodiments.
  • FIG. 4 is a diagram of software configuration in the display device 200 according to some embodiments.
  • FIG5 is a schematic diagram of a remote video conference scenario according to some embodiments.
  • FIG6 is a schematic diagram of another remote video conference scenario according to some embodiments.
  • FIG7 is a schematic diagram of a framework of an HDMI content sharing system according to some embodiments.
  • FIG8 is a schematic diagram of a flow chart of a device control method executed by a display device 200 according to some embodiments.
  • FIG9 is a schematic diagram of a user interface of a display device 200 according to some embodiments.
  • FIG10 is a schematic diagram of an overall design framework of an HDMI content sharing example according to some embodiments.
  • FIG11 is a schematic diagram of a process of simulating HDMI as a virtual camera according to some embodiments.
  • FIG12 is a schematic diagram of a flow process of HDMI content to an Android application layer according to some embodiments.
  • FIG13 is a schematic diagram of a user interface of another display device 200 according to some embodiments.
  • FIG14 is a schematic diagram of a process for determining whether a virtual camera is available according to some embodiments.
  • FIG15 is a schematic diagram of a logic flow of updating a Camera list according to some embodiments.
  • FIG16 is a schematic diagram of yet another remote video conferencing scenario according to some embodiments.
  • Fig. 1 is a schematic diagram of an operation scenario between a display device and a control device according to some embodiments. As shown in Fig. 1 , a user can operate a display device 200 through a mobile terminal 300 and a control device 100 .
  • control device 100 may be a remote controller, and the communication between the remote controller and the display device includes infrared protocol communication or Bluetooth protocol communication, and other short-range communication methods, and the display device 200 is controlled wirelessly or wired.
  • the user may control the display device 200 by inputting user commands through buttons on the remote controller, voice input, control panel input, etc.
  • a smart device 300 (such as a mobile terminal, a tablet computer, a computer, a laptop computer, etc.) may also be used to control the display device 200.
  • the display device 200 is controlled using an application running on the smart device.
  • the display device may not use the above-mentioned smart device or control device to receive instructions, but may receive user control through touch or gestures.
  • the display device 200 can also be controlled in a manner other than the control device 100 and the smart device 300.
  • the user's voice command control can be directly received through a module for obtaining voice commands configured inside the display device 200, or the user's voice command control can be received through a voice control device set outside the display device 200.
  • the display device 200 can also communicate data with the server 400.
  • the display device 200 can be allowed to communicate through a local area network (LAN), a wireless local area network (WLAN) and other networks.
  • the server 400 can provide various content and interactions to the display device 200.
  • the server 400 can be a cluster or multiple clusters, and can include one or more types of servers.
  • FIG2 is a block diagram of a hardware configuration of a control device 100 according to some embodiments.
  • the control device 100 includes a processor 110, a communication interface 130, a user input/output interface 140, a memory, and a power supply. It can receive the user's input operation instructions, and convert the operation instructions into instructions that can be recognized and responded to by the display device 200, playing the role of an interactive intermediary between the user and the display device 200.
  • FIG 3 is a hardware configuration block diagram of the display device 200 according to some embodiments.
  • the display device 200 may include at least one of a tuner and demodulator 210, a communication device 220, a detector 230, an external device interface 240, at least one processor 250, a display 260, an audio output interface 270, a memory, a power supply, and a user input interface 280.
  • At least one processor 250 may include a video processor, an audio processor, a graphics processor, RAM, ROM, and first to nth interfaces for input/output.
  • the display 260 includes a display screen component for presenting a picture, and a driving component for driving an image display, a component for receiving an image signal output from the processor, and a component for displaying video content, image content, and a menu control interface, and a user control UI interface.
  • the display 260 may be a liquid crystal display, an OLED display, and a projection display, and may also be a projection device and a projection screen.
  • the communication device 220 is a component that can be used to communicate with external devices or servers according to various communication protocol types.
  • the communication device can include at least one of a Wifi module, a Bluetooth module, a wired Ethernet module, other network communication protocol chips or near field communication protocol chips, and an infrared receiver.
  • the display device 200 can establish the transmission and reception of control signals and data signals with the external control device 100 or the server 400 through the communication device 220.
  • the user input interface 280 may be used to receive a control signal from the control device 100 (eg, an infrared remote controller, etc.).
  • the control device 100 eg, an infrared remote controller, etc.
  • the detector 230 can be used to collect signals from the external environment or the external interaction.
  • the detector 230 includes a light receiver, a sensor for collecting the intensity of ambient light; or, the detector 230 includes an image collector, such as a camera, which can be used to collect external environment scenes, user attributes or user interaction gestures; or, the detector 230 includes a sound collector, such as a microphone, etc., for receiving external sounds.
  • the external device interface 240 may include, but is not limited to, any one or more of the following interfaces: a high-definition multimedia interface (HDMI), an analog or digital high-definition component input interface (component), a composite video input interface (CVBS), a USB input interface (USB), an RGB port, etc. It may also be a composite input/output interface formed by the above multiple interfaces.
  • HDMI high-definition multimedia interface
  • component analog or digital high-definition component input interface
  • CVBS composite video input interface
  • USB USB input interface
  • RGB port an RGB port
  • the tuner-demodulator 210 may receive broadcast television signals via wired or wireless reception, and demodulate audio and video signals, such as EPG data signals, from a plurality of wireless or wired broadcast television signals.
  • At least one processor 250 and the tuner-demodulator 210 may be located in different separate devices, that is, the tuner-demodulator 210 may also be in an external device of the main device where the at least one processor 250 is located, such as an external set-top box.
  • At least one processor 250 can control the operation of the display device and respond to user operations through various software control programs stored in the memory. At least one processor 250 can control the overall operation of the display device 200, for example, in response to receiving a user command for selecting a UI object to be displayed on the display 260, at least one processor 250 can perform operations related to the object selected by the user command.
  • At least one processor 250 may also include a central processing unit (CPU), a video processor, an audio processor, a graphics processing unit (GPU), RAM (Random Access Memory, RAM), ROM (Read-Only Memory, ROM), a first interface to an nth interface for input/output, a communication bus (Bus), etc.
  • CPU central processing unit
  • video processor video processor
  • audio processor audio processor
  • graphics processing unit GPU
  • RAM Random Access Memory
  • ROM Read-Only Memory
  • ROM Read-Only Memory
  • the user may input a user command through a graphical user interface (GUI) displayed on the display 260, and the user input interface receives the user input command through the graphical user interface (GUI).
  • GUI graphical user interface
  • the user may input a user command through a specific sound or gesture, and the user input interface recognizes the sound or gesture through a sensor to receive the user input command.
  • GUI Graphical User Interface
  • the display device system can be divided into three layers, namely the application layer, the middleware layer and the hardware layer from top to bottom.
  • the application layer mainly includes commonly used applications on the TV and the application framework. Among them, commonly used applications are mainly applications developed based on the browser, such as HTML5 APPs; and native applications (Native APPs).
  • the Application Framework is a complete program model that has all the basic functions required by standard application software, such as file access, data exchange, etc., as well as the user interfaces of these functions (toolbars, status bars, menus, dialog boxes).
  • Native apps can support online or offline, message push or local resource access.
  • the middleware layer includes various TV protocols, multimedia protocols, system components and other middleware.
  • the middleware can use the basic services (functions) provided by the system software to connect various parts of the application system or different applications on the network, and can achieve the purpose of resource sharing and function sharing.
  • the hardware layer mainly includes the interface of the Hardware Abstraction Layer (HAL) (HAL interface for short), hardware and drivers.
  • HAL interface is a unified interface for TV chips to connect, and the specific logic is implemented by each chip.
  • Drivers mainly include: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver.
  • CEC Consumer Electronics Control
  • EDID Extended Display Identification Data
  • EDID Extended Display Identification Data
  • the built-in EDID method means that the EDID data is written in the program. When the smart TV is powered on by DC, the EDID data is written into the Electrically Erasable Programmable Read-Only Memory (EEPROM) chip. When it is used, the EDID data of the smart TV is obtained from the EEPROM chip.
  • the external EDID method means that the EDID data is directly written into the external EEPROM chip, and is directly read by the external device through the display data channel without the control of the smart TV main program.
  • HDMI High Definition Multimedia Interface
  • a high-definition multimedia interface is a digital video/audio interface technology, a dedicated digital interface suitable for image transmission, which can transmit audio and image signals at the same time, with a maximum data transmission speed of 48Gbps (version 2.1), and no digital/analog or analog/digital conversion is required before signal transmission.
  • the HDMI interface has hot-swappable function.
  • HDCP High-bandwidth Digital Content Protection
  • HDCP technology processes each pixel in the image, making the image irregular and unrecognizable. Only after the synchronization between the sender and the receiver can the reverse process be performed to restore the data. The system will confirm the connection every 2 seconds, and synchronize the identification code between the sender and the receiver every 128 frames to ensure the synchronization of the connection.
  • HAL Hardware Abstraction Layer
  • HAL Hardware Abstraction Layer
  • HAL is an interface layer between the operating system kernel and the hardware circuit, and its purpose is to abstract the hardware. It hides the hardware interface details of a specific platform and provides a virtual hardware platform for the operating system, making it hardware-independent and portable on multiple platforms. From the perspective of software and hardware testing, the testing of software and hardware can be completed based on the hardware abstraction layer, making it possible to conduct software and hardware testing in parallel.
  • Remote video conferencing is actually a process of content sharing between two devices.
  • Content sharing can be divided into four stages: information collection, information processing, information transmission, and information restoration.
  • Information collection can be completed using the audio and video capture card of the sending device.
  • Information processing and information transmission rely on the remote video conferencing server.
  • the remote video conferencing server determines the processing method and forwarding logic of the audio and video signals according to the current video conferencing mode, and finally sends the processed audio and video data to each receiving device. After receiving the transmitted audio and video data from the remote video conferencing server, the receiving device decodes the transmitted audio and video data and restores the image and sound to the receiving device for playback.
  • the remote video conferencing server is actually the server corresponding to the remote video conferencing application.
  • the user can use the remote video conferencing application on the display device 200-1 to initiate a remote video conference to the display device 200-2.
  • the display device 200-2 also needs to install and run the same remote video conferencing application.
  • the display device 200-1 can share its collected audio and video content with the display device 200-2, and the display device 200-2 can also share its collected audio and video content with the display device 200-1.
  • the display device 200-1 and the display device 200-2 can both serve as the sending device and the receiving device in the remote video conference.
  • the display device 200-1 and the display device 200-2 can share the content normally.
  • the sending device uses the HDMI port to receive the HDMI content transmitted by the CEC device, and needs to share the HDMI content with the receiving device, since there is no mechanism for sharing HDMI content in the relevant technology of the Android system, the sending device cannot share the displayed HDMI content to the receiving device, which leads to the failure of content sharing in the remote video conference.
  • display device 200-1 is connected to a CEC device, and the CEC device transmits HDMI content to display device 200-1.
  • the CEC device transmits HDMI content to display device 200-1.
  • display device 200-1 since there is no mechanism for sharing HDMI content in the relevant technology of the Android system, display device 200-1 cannot share the HDMI content it is displaying with display device 200-2, which will cause the remote video conference content sharing between display device 200-1 and display device 200-2 to fail.
  • the HDMI content displayed on display device 200-1 will not be displayed on display device 200-2, and even a black screen will appear on display device 200-2.
  • FIG7 is a diagram of an HDMI content sharing system framework according to some embodiments, and the content sharing system framework shown in FIG7 includes a display device 200-1 and a display device 200-2.
  • the display device 200-1 is connected to the CEC device 300 through an HDMI port, and the CEC device 300 transmits the first HDMI content to the display device 200-1 through an HDMI IN (i.e., an HDMI input port).
  • HDMI IN i.e., an HDMI input port
  • the display device 200-1 receives the first HDMI content transmitted by the CEC device 300, the first HDMI content can be displayed on the display.
  • FIG8 is a flow chart of a device control method provided by an embodiment of the present disclosure executed by a display device 200. As shown in FIG8, the following steps are included:
  • Step S800 receiving first HDMI content
  • the first HDMI content is transmitted by the CEC device through HDMIIN.
  • the display device 200 receives the first HDMI After the content is received, the first HDMI content is output to the display for display.
  • Step S801 In response to a first content sharing request sent by a first application, construct the HDMIIN into a virtual device node so that the first HDMI content is stored in the virtual device node, wherein the first content sharing request is used to request sharing the first HDMI content to a first device.
  • the first application may be a remote video conferencing application.
  • the first application needs to be run on display device 200-1 and display device 200-2 (ie, the first device).
  • Display device 200-1 and display device 200-2 use the first application to conduct a remote video conference, ie, to share content.
  • the user interface shown in FIG9 is a user interface of the first application
  • the user interface of the first application shown in FIG9 includes a screen display area and a function operation area.
  • the screen display area is used to display the first HDMI content transmitted from the CEC device 300 to the display device 200-1
  • the function operation area is used to display different button controls that can realize the sharing function: "HDMI sharing” button control, "desktop sharing” button control, "whiteboard sharing” button control, and different button controls are used to realize different sharing functions.
  • the "HDMI sharing” button control is used to realize HDMI content sharing
  • other button controls are used to realize ordinary content sharing.
  • the first content sharing request can be input by clicking the "HDMI sharing" button control, that is, the first content sharing request is used to request to share the first HDMI content to the display device 200-2. If the user only needs to share ordinary content to the display device 200-2, other content sharing requests can be input by clicking other button controls. In other words, only after receiving the first content sharing request input by the user, the process of sharing the HDMI content to the display device 200-2 can be performed.
  • HDMIIN can be constructed into a virtual device node.
  • HDMIIN is first registered as V4L2_device to generate a corresponding vedio virtual device node.
  • V4L2 Video for linux two
  • V4L2 is a set of driver frameworks for video devices in the Linux kernel, which provides a unified interface specification for video device driver development and application layer.
  • Devices registered using the V4L2 device driver framework will generate corresponding device node files in the Linux system /dev/ directory.
  • the name of the device node is usually videoX (X standard a digital number: /dev/videox), and each videoX device file represents a video device.
  • the application configures and uses virtual devices by performing I/O (Input/Output) operations on the videoX device file.
  • the device node corresponding to the video device is /dev/videoX, where X is a digital number, usually starting from 0. Calling the open() function to open the device node can get the file descriptor fd. After opening the virtual device, you need to query the properties of the virtual device to determine whether the device is a video acquisition device. Through ioctl(), you will get a struct v4l2_capability type data.
  • the struct v4l2_capability data structure describes some properties of the virtual device, including driver (driver name), card (device name), bus_info (bus name), version (version information), capabilities (device capabilities), etc.
  • the capabilities field describes the capabilities of the device, so you can determine whether the device is a camera device by judging whether the capabilities field contains V4L2_CAP_VIDEO_CAPTURE.
  • HDMIIN may be first registered as V4L2_device, thereby generating a virtual camera node corresponding to HDMIIN.
  • the data stream of the virtual camera node is the first HDMI content.
  • the first HDMI content may be stored in the virtual device node.
  • the premise for constructing HDMIIN into a virtual camera node is that the first HDMI content only includes video content, that is, only the video content needs to be shared between the display device 200-1 and the display device 200-2, and the constructed virtual camera node is only used as a virtual camera. Therefore, only video content can be obtained from the virtual camera node, but audio content cannot be obtained.
  • Step S802 adding a shared interface corresponding to the virtual device node in the hardware abstraction layer HAL.
  • HDMIIN After constructing HDMIIN into a virtual device node, it is also necessary to add a shared interface corresponding to the virtual device node in HAL.
  • the HAL is Camera HAL, that is, support for the constructed virtual camera node is added in Camera HAL, that is, a shared interface for the virtual camera node is added in Camera HAL.
  • the first application can open the virtual camera node in Camera HAL.
  • the first application can read the first HDMI content from the virtual camera node, and then the first application can share the first HDMI content read from the virtual camera node with display device 200-2, thereby enabling display device 200-1 to share the HDMI content with display device 200-2.
  • FIG10 shows the overall design framework of the above HDMI content sharing example, including HDMI port, VDIN module (hardware processing module), Camera HAL, GE2D (2D graphics acceleration engine) module, Surfacefliger (the core of the graphical user interface, which exists in the form of system services and is responsible for mixing and outputting all App's graphic data to FrameBuffer in Z Order) and APP.
  • VDIN and GE2D are HW Modules (hardware modules), Surfacefliger and APP are SW Modules (software modules), and Video output and Preview are DMA Buffers (direct memory buffers).
  • the VDIN module converts the HDMI content (such as decoding), and then inputs the Video output of the camera HAL and then passes through the GE2D module for graphics performance processing.
  • the data processed by graphics performance is returned to the preview plug-in of the camera HAL.
  • the mixing processing of Surfacefliger it is output to the APP, which displays and shares the content.
  • FIG. 11 shows a process of simulating HDMI as a virtual camera in the above HDMI content sharing example, including the following steps:
  • Step S1101 registering a V4L2 device
  • Step S1102 when an HDMI RX signal input is detected, the input HDMI RX signal is converted into stream file data through the hardware processing module VDIIN;
  • Step S1103 Obtain the stream file data converted from the HDMI RX signal through the vdin_v4l2_isr interface, and import the stream file data into the V4L2 device.
  • the input HDMI RX signal is converted into stream file data through the hardware processing VDIIN module.
  • the stream file data converted from the HDMI RX signal is obtained through the vdin_v4l2_isr interface.
  • the data is imported into the registered V4L2 device through vdin_v4l2_if_isr.
  • the device node of the virtual camera obtained through the above process can be called by Android's camera HAL.
  • FIG. 12 shows the flow process of HDMI content to the Android application layer in the above HDMI content sharing example, including the following steps:
  • Step S1201 The first application Android APP sends a call instruction to the server CameraService;
  • Step S1202 the server CameraService calls the hardware abstraction layer Camera HAL to obtain data
  • Step S1203 The first application Android APP sends a display surface Surface or a buffer buffer to the hardware abstraction layer Camera HAL;
  • Step S1204 the high-definition multimedia interface HDMI writes the data sent by the CEC device into the V4L2 virtual device;
  • Step S1205 The hardware abstraction layer Camera HAL uses the V4L2 protocol to obtain data from the virtual device V4L2;
  • Step S1206 The hardware abstraction layer Camera HAL decodes the data
  • Step S1207 the hardware abstraction layer Camera HAL sends the decoded data to the first application Android APP;
  • Android APP can call Camera HAL through CameraService, and Camera HAL opens the real vedioX node, which corresponds to the V4L2 device, i.e., the virtual camera device.
  • Android APP can also pass its display surface Surface, or the buffer where the user receives data, to Camera HAL. Then, after Camera HAL opens the V4L2 virtual device, it uses the V4L2 protocol to continuously obtain data from the V4L2 virtual device. This is because HDMI continuously writes data to the V4L2 virtual device through HDMIIN, and Camera HAL can continuously obtain data written by the CEC device from the V4L2 virtual device.
  • V4L2 virtual device There are two ways for Camera HAL to read data from V4L2 virtual device: one is read mode, which directly reads the data written by V4L2 virtual device through read() system call; the other is streaming mode.
  • VIDIOC_QUERYCAP instruction to query the properties of the device and get a struct v4l2_capability type data, in which the capabilities field records the capabilities of the device.
  • V4L2_CAP_READWRITE it means that the device supports read I/O mode to read data
  • V4L2_CAP_STREAMING it means that the device supports streaming I/O mode
  • streaming I/O mode it is necessary to apply for frame buffer from the device and map the frame buffer to the application process address space.
  • the data written by the CEC device needs to be actually stored in the physical memory.
  • the data written by the CEC device can be mapped from the physical memory to the virtual memory corresponding to the V4L2 virtual device.
  • the physical memory can be mapped to the user control through the vedioX node to obtain the virtual address of the virtual memory corresponding to the V4L2 virtual device, and the virtual address points to the physical memory. Therefore, the Camera HAL reads the first HDMI content from the virtual device node, and can read the first HDMI content from the physical memory based on the virtual address.
  • the Camera HAL After the Camera HAL reads the data from the V4L2 virtual device, it starts to decode the data. Finally, the Camera HAL sends each frame of successfully decoded data to the surface of the Android APP for display on the screen of the display device 200-1. At the same time, after sending the decoded data to the buffer, the Android APP processes the data by itself, which includes displaying it on the screen or sharing it with other devices.
  • HDMIIN is constructed into a virtual device node so that the first HDMI content is stored in the virtual device node.
  • a shared interface corresponding to the virtual device node is added to the HAL.
  • the first application can obtain the first HDMI content from the virtual device node by calling the shared interface, and then share the first HDMI content to the display device 200-1.
  • the display device 200-2 can also share the HDMI content to the display device 200-2, thereby avoiding the failure of content sharing during the remote video conference.
  • the first HDMI content is not HDCP encrypted when it is transmitted from the CEC device 300 to the display device 200-1, it is only necessary to directly store the first HDMI content to the virtual device node. If the first HDMI content is HDCP encrypted when it is transmitted from the CEC device 300 to the display device 200-1, it is necessary to perform HDCP decryption on the first HDMI content, and then store the first HDMI content after HDCP decryption to the virtual device node. In this way, the first application obtains the first HDMI content after HDCP decryption from the virtual device node by calling the sharing interface, and shares the decrypted first HDMI content to the display device 200-2.
  • the embodiment of the present disclosure can not only establish a sharing mechanism for sharing HDMI content to other devices through a video conferencing application, but also for HDMI content encrypted by HDCP, after HDCP decryption in the current device, the decrypted HDMI content can be shared with other devices, further avoiding the failure of displaying HDMI content on other devices when sharing HDMI content.
  • the display device 200-1 may also include an image collector, such as a built-in camera or a USB camera. Therefore, it is possible to determine whether to use a real camera or a virtual camera according to the usage scenario.
  • the user interface shown in Figure 13 is a user interface of the first application, including a "real camera” area and a "virtual camera” area, wherein the "real camera” area includes a built-in camera and a USB camera, and the "virtual camera” area includes HDMI sharing 1 and HDMI sharing 2. If the user needs to share HDMI content, the first content sharing request can be entered by clicking the "HDMI sharing 1" button control or the "HDMI sharing 2" button control, and then the first HDMI content is shared to the display device 200-2 according to the first content sharing request. If the user does not need to share HDMI content, the second content sharing request can be entered by clicking the button control in the "real camera” area, and then the second image content collected by the real camera is shared to the display device 200-2 according to the second content sharing request.
  • an image collector such as
  • preview images of different cameras can be displayed in the screen display area, and the user can determine the camera to be selected by viewing the preview image.
  • CEC device 1 and CEC device 2 correspond to HDMI share 1 and HDMI share 2, respectively.
  • CEC device 1 can input high-definition movie resources to display device 200-1, and CEC device 2 can only input movie resources of ordinary picture quality. If a selection instruction input by the user by clicking "HDMI share 1" is received, the high-definition movie video image can be previewed in the screen display area; if a selection instruction input by the user by clicking "HDMI share 2" is received, the movie video image of ordinary picture quality can be previewed in the screen display area. Users can share different HDMI content input by different CEC devices to display device 200-2 by clicking different camera button controls according to their needs.
  • preview images can also be displayed in the image display area for users to judge and select.
  • the built-in camera and the USB camera shoot images at different angles.
  • users can be prompted to see the different shooting angles corresponding to different cameras.
  • the HDMI content sharing process is performed. If it is determined that the first content sharing request does not carry the predetermined parameter, the HDMI content sharing is not performed.
  • Step S1401 define and add different camera categories
  • different camera categories may be defined in Camera HAL and Camera Service, and the categories may include built-in camera, USB camera, and HDMI virtual camera (Vcam0).
  • Step S1402 Update the properties of the virtual camera according to the upper layer application call.
  • the attribute of Vcam0_id is set to 0. If the upper layer application cannot use the virtual camera, the attribute of Vcam0_id is set to -1. And the corresponding virtual camera ID (HDMIcameraid) is updated, for example, HDMIcameraid is set to camera0 or camera1.
  • Step S1403 determining whether the virtual camera is available
  • step S1404 determines whether the virtual camera is available according to the attribute setting of Vcam0_id. If it is determined that the virtual camera is available, step S1404 is performed; if it is determined that the virtual camera is not available, step S1405 is performed.
  • Step S1404 if the virtual camera is available, open the virtual camera.
  • the virtual camera corresponding to the virtual camera ID can be determined according to HDMI camera aid, and then Then read the HDMI data from the virtual camera
  • Step S1405 if the virtual camera is not available, determine whether there is an available real camera
  • Step S1406 if the real camera is available, open the real camera;
  • the real camera may be a built-in camera or a USB camera.
  • Step S1407 If the real camera is not available, output a prompt indicating that there is no camera.
  • a first list can be configured in the display device 200-1, and the first list can record all currently available virtual device nodes.
  • the application can search for available virtual device nodes from the first list through the HAL.
  • the constructed virtual device node can be added to the first list, so that the HAL can find the virtual device node corresponding to HDMIIN from the first list. If the CEC device is unplugged from the display device 200-1, the virtual device node corresponding to HDMIIN can be deleted from the first list according to the unplug signal. In this way, the HAL can no longer find the virtual device node corresponding to HDMIIN from the first list, and it is also impossible to open the virtual device node corresponding to HDMIIN.
  • Step S1501 Camera HAL process starts, monitors the creation and deletion of /dev/videoX node.
  • HDMIIN is connected, proceed to step S1501, when HDMIIN is unplugged, proceed to step S1502;
  • Step S1502 if the creation of the /dev/videoX node is detected, a Camera object corresponding to the node is generated;
  • Step S1503 add the Camera object to the maintained Camera list, update the list and notify the server Camera Service to synchronize the update;
  • the Camera HAL may add the Camera object to the maintained Camera list (i.e., the first list), update the IDs of all Cameras in the list, notify the upper-layer Camera Service to synchronously update the list and the CameraID ID, and notify the upper-layer Camera Service to synchronously update the list and the CameraID;
  • Step S1504 If the deletion of the /dev/videoX node is detected, the Camera object corresponding to the node in the Camera list is removed, the list is updated, and the server Camera Service is notified to update synchronously;
  • the Camera HAL can delete the Camera object in the list, update the IDs of all Cameras in the list, and notify the upper-layer Camera Service to synchronously update the list and CameraID.
  • Android can dynamically perceive the addition and deletion of Camera objects based on the connection and removal of HDMIIN.
  • CEC device 300-1 is connected to display device 200-1 through HDMI port 1
  • CEC device 300-2 is connected to display device 200-1 through HDMI port 2.
  • Both CEC device 300-1 and CEC device 300-2 input HDMI content to display device 200-1.
  • a virtual device node /dev/video1 can be constructed for the HDMIIN corresponding to CEC device 300-1
  • a virtual device node /dev/video1 can be constructed for the HDMIIN corresponding to CEC device 300-2.
  • the Camera objects corresponding to the virtual device nodes respectively: the Camera object corresponding to /dev/video1 can be Camera1, and the Camera object corresponding to /dev/video2 can be Camera2.
  • HAL can be from Camera Camera1 and Camera2 are found in the list at the same time.
  • HAL can choose to open different virtual device nodes to read different HDMI content according to the selection instruction input by the user. For example, in response to the selection instruction input by the user to select the HDMI content input by the CEC device 300-1, HAL chooses to open the virtual device node / dev / video1, and then can continuously read the HDMI content from the virtual device node / dev / video1 node.
  • the HDMI content input by the CEC device 300-1 can be shared to the display device 200-2.
  • HAL chooses to open the virtual device node / dev / video2, and then can continuously read the HDMI content from the virtual device node / dev / video2 node.
  • the HDMI content input by the CEC device 300-2 can be shared to the display device 200-2.
  • the first HDMI content to be shared to the display device 200-2 only includes video content
  • the first application can obtain the video content from the virtual camera node by calling the sharing interface in the Camera HAL, and share the video content with the display device 200-2.
  • HDMIIN is constructed into a virtual camera node and a virtual microphone node representing the virtual device node. Then, the video content in the first HDMI content is stored in the virtual camera node, and the audio content in the first HDMI content is stored in the virtual microphone node. And it is necessary to add a shared interface corresponding to the virtual camera node in the Camera HAL, and it is necessary to add a shared interface corresponding to the virtual microphone node in the Audio HAL.
  • the first application can obtain video content from the virtual camera node by calling the shared interface in the Camera HAL, and can obtain audio content from the virtual microphone node by calling the shared interface in the Audio HAL. Then the first application can share the video content and audio content with the display device 200-2.
  • camera application A and camera application B can be installed in display device 200 at the same time, wherein camera application A is an application that only uses a real camera, for example, camera application A can be a photo taking application, and camera application B is an application that can use a virtual camera and a real camera at the same time, for example, camera application B can be a remote video conferencing application.
  • camera application A When camera application A is running, camera application A will send a call request for calling a real camera, and camera application A can directly obtain image data from the real camera.
  • camera application B When camera application B is running, camera application B will simultaneously send a call request for calling a real camera and a call request for calling a virtual camera.
  • the device node /dev/video0 of the real camera corresponds to the call request for calling the real camera.
  • the virtual device node /dev/videoX of the virtual camera corresponds to the call request for calling the virtual camera.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

本公开提供了一种显示设备及设备控制方法。该方法中，在消费电子控制设备CEC通过输入端口HDMI IN向显示设备传输第一HDMI内容时，通过将HDMI IN构建成虚拟设备节点，以将第一HDMI内容存储至该虚拟设备节点。然后在硬件抽象层HAL中增加该虚拟设备节点对应的共享接口，以使第一应用可以通过调用共享接口的方式，从虚拟设备节点中获取第一HDMI内容，然后将第一HDMI内容共享至第一设备。这样，在显示设备上显示HDMI内容时，显示设备也可以将HDMI内容共享至第一设备，避免远程视频会议过程中内容共享失败。

Description

一种显示设备及设备控制方法
相关申请的交叉引用
本公开要求在2023年12月15日提交、申请号为202311735183.7的中国申请的优先权，其全部内容通过引用结合在本公开中。
技术领域
本公开涉及显示设备技术领域，尤其涉及一种显示设备及设备控制方法。
背景技术
高清晰度多媒体接口(High Definition Multimedia Interface，HDMI)是一种适合影像传输的专用型数字化接口，能够同时传送音频信号和图像信号。目前HDMI已经被标准化，消费电子控制(Consumer Electronics Control，CEC)为HDMI标准化产生的一套完成的单总线协议。发送端设备可以利用CEC信号控制HDMI端口上连接的外接设备，其中该被控制的外接设备称为消费电子控制设备(简称CEC设备)。CEC设备可通过HDMI端口向发送端设备传输HDMI内容，并且在发送端设备上显示HDMI内容。
在远程视频会议过程中，如果发送端设备上显示的内容为HDMI内容，由于安卓系统(Android)中不存在共享HDMI内容的机制，因此发送端设备无法将显示的HDMI内容共享至接收端设备，进而导致远程视频会议中内容共享失败。
发明内容
第一方面，本公开实施例提供了一种显示设备，包括：显示器，被配置为显示消费电子控制设备CEC通过高清晰度多媒体接口HDMI的输入端口HDMIIN传输的第一HDMI内容；
用户输入接口，被配置接收来自用户的指令；通信装置，被配置为根据预定协议与外部设备通信；存储器，被配置为保存计算机指令和与显示设备关联的数据；至少一个处理器，与所述显示器，用户输入接口，通信装置和存储器连接，被配置执行计算机指令以使得所述显示设备执行：响应于第一应用发送的第一内容共享请求，将所述HDMIIN构建成虚拟设备节点，以使所述第一HDMI内容存储至所述虚拟设备节点，其中，所述第一内容共享请求用于请求将所述第一HDMI内容共享至第一设备；在硬件抽象层HAL中增加所述虚拟设备节点对应的共享接口，以使所述第一应用通过调用所述共享接口从所述虚拟设备节点中获取所述第一HDMI内容，以及，将所述第一HDMI内容共享至所述第一设备。
第二方面，本公开实施例提供了一种设备控制方法，包括：接收消费电子控制设备CEC通过高清晰度多媒体接口HDMI的输入端口HDMIIN传输的第一HDMI内容；响应于第一应用发送的第一内容共享请求，将所述HDMIIN构建成虚拟设备节点，以使将所述第一HDMI内容存储至所述虚拟设备节点，其中，所述第一内容共享请求用于请求将所述第一HDMI内容共享至第一设备；在硬件抽象层HAL中增加所述虚拟设备节点对应的共享接口，以使所述第一应用通过调用所述共享接口从所述虚拟设备节点中获取所述第一HDMI内容，以及，将所述第一HDMI内容共享至所述第一设备。
附图说明
图1为根据一些实施例的显示设备与控制设备之间操作场景的示意图；
图2为根据一些实施例的控制装置100的硬件配置框图；
图3为根据一些实施例的显示设备200的硬件配置框图；
图4为根据一些实施例的显示设备200中软件配置图；
图5为根据一些实施例的远程视频会议场景示意图；
图6为根据一些实施例的又一种远程视频会议场景示意图；
图7为根据一些实施例的HDMI内容共享系统框架示意图；
图8为根据一些实施例的显示设备200执行设备控制方法流程示意图；
图9为根据一些实施例的显示设备200用户界面示意图；
图10为根据一些实施例的HDMI内容共享示例的整体设计框架示意图；
图11为根据一些实施例的将HDMI模拟为虚拟摄像头的流程示意图；
图12为根据一些实施例的HDMI内容到Android应用层的流转过程示意图；
图13为根据一些实施例的又一种显示设备200用户界面示意图；
图14为根据一些实施例的确定虚拟摄像头是否可用的流程示意图；
图15为根据一些实施例的更新Camera列表的逻辑流程示意图；
图16为根据一些实施例的又一种远程视频会议场景示意图。
具体实施方式
图1为根据一些实施例的显示设备与控制设备之间操作场景的示意图。如图1中示出，用户可通过移动终端300和控制设备100操作显示设备200。
在一些实施例中，控制装置100可以是遥控器，遥控器和显示设备的通信包括红外协议通信或蓝牙协议通信，及其他短距离通信方式，通过无线或有线方式来控制显示设备200。用户可以通过遥控器上按键、语音输入、控制面板输入等输入用户指令，来控制显示设备200。
在一些实施例中，也可以使用智能设备300(如移动终端、平板电脑、计算机、笔记本电脑等)以控制显示设备200。例如，使用在智能设备上运行的应用程序控制显示设备200。
在一些实施例中，显示设备可以不使用上述的智能设备或控制设备接收指令，而是通过触摸或者手势等接收用户的控制。
在一些实施例中，显示设备200还可以采用除了控制装置100和智能设备300之外的方式进行控制，例如，可以通过显示设备200设备内部配置的获取语音指令的模块直接接收用户的语音指令控制，也可以通过显示设备200设备外部设置的语音控制设备来接收用户的语音指令控制。
在一些实施例中，显示设备200还可以与服务器400进行数据通信。可允许显示设备200通过局域网(LAN)、无线局域网(WLAN)和其他网络进行通信连接。服务器400可以向显示设备200提供各种内容和互动。服务器400可以是一个集群，也可以是多个集群，可以包括一类或多类服务器。
图2为根据一些实施例的控制装置100的硬件配置框图。如图2所示，控制装置100包括处理器110、通信接口130、用户输入/输出接口140、存储器、供电电源。控制装置100 可接收用户的输入操作指令，且将操作指令转换为显示设备200可识别和响应的指令，起用用户与显示设备200之间交互中介作用。
图3为根据一些实施例的显示设备200的硬件配置框图，如图3所示，显示设备200可包括调谐解调器210、通信装置220、检测器230、外部装置接口240、至少一个处理器250、显示器260、音频输出接口270、存储器、供电电源、用户输入接口280中的至少一种。
在一些实施例中，至少一个处理器250可包括视频处理器，音频处理器，图形处理器，RAM，ROM，用于输入/输出的第一接口至第n接口。显示器260包括用于呈现画面的显示屏组件，以及驱动图像显示的驱动组件，用于接收源自处理器输出的图像信号，进行显示视频内容、图像内容以及菜单操控界面的组件以及用户操控UI界面。显示器260可为液晶显示器、OLED显示器、以及投影显示器，还可以为一种投影装置和投影屏幕。
通信装置220可用于根据各种通信协议类型与外部设备或服务器进行通信的组件。例如：通信装置可以包括Wifi模块，蓝牙模块，有线以太网模块等其他网络通信协议芯片或近场通信协议芯片，以及红外接收器中的至少一种。显示设备200可以通过通信装置220与外部控制设备100或服务器400建立控制信号和数据信号的发送和接收。
用户输入接口280可用于接收控制装置100(如：红外遥控器等)的控制信号。
检测器230可用于采集外部环境或与外部交互的信号。例如，检测器230包括光接收器，用于采集环境光线强度的传感器；或者，检测器230包括图像采集器，如摄像头，可以用于采集外部环境场景、用户的属性或用户交互手势，再或者，检测器230包括声音采集器，如麦克风等，用于接收外部声音。
外部装置接口240可以包括但不限于如下：高清晰度多媒体接口(HDMI)、模拟或数据高清分量输入接口(分量)、复合视频输入接口(CVBS)、USB输入接口(USB)、RGB端口等任一个或多个接口。也可以是上述多个接口形成的复合性的输入/输出接口。
调谐解调器210可通过有线或无线接收方式接收广播电视信号，以及从多个无线或有线广播电视信号中解调出音视频信号，如以及EPG数据信号。
在一些实施例中，至少一个处理器250可以和调谐解调器210位于不同的分体设备中，即调谐解调器210也可在至少一个处理器250所在的主体设备的外置设备中，如外置机顶盒等。
在一些实施例中，至少一个处理器250可以通过存储在存储器上中各种软件控制程序，来控制显示设备的工作和响应用户的操作。至少一个处理器250可控制显示设备200的整体操作，例如：响应于接收到用于选择在显示器260上显示UI对象的用户命令，至少一个处理器250便可以执行与由用户命令选择的对象有关的操作。
在一些实施例中，至少一个处理器250还可包括中央处理器(Central Processing Unit，CPU)，视频处理器，音频处理器，图形处理器(Graphics Processing Unit，GPU)，RAM(Random AccessMemory，RAM)，ROM(Read-Only Memory,ROM)，用于输入/输出的第一接口至第n接口，通信总线(Bus)等中的至少一种。
用户可在显示器260上显示的图形用户界面(GUI)输入用户命令，则用户输入接口通过图形用户界面(GUI)接收用户输入命令。或者，用户可通过输入特定的声音或手势进行输入用户命令，则用户输入接口通过传感器识别出声音或手势，来接收用户输入命令。
“用户界面”，是应用程序或操作系统与用户之间进行交互和信息交换的介质接口，它 实现信息的内部形式与用户可以接受形式之间的转换。用户界面常用的表现形式是图形用户界面(Graphic User Interface，GUI)，是指采用图形方式显示的与计算机操作相关的用户界面。它可以是在电子设备的显示屏中显示的一个图标、窗口、控件等界面元素，其中控件可以包括图标、按钮、菜单、选项卡、文本框、对话框、状态栏、导航栏、Widget等可视的界面元素。
如图4所示，显示设备的系统可分为三层，从上至下分别为应用层、中间件层和硬件层。应用层主要包含电视上的常用应用，以及应用框架(Application Framework)，其中，常用应用主要是基于浏览器Browser开发的应用，例如：HTML5APPs；以及原生应用(Native APPs)。
应用框架(Application Framework)是一个完整的程序模型，具备标准应用软件所需的一切基本功能，例如：文件存取、资料交换...，以及这些功能的使用接口(工具栏、状态列、菜单、对话框)。
原生应用(Native APPs)可以支持在线或离线，消息推送或本地资源访问。
中间件层包括各种电视协议、多媒体协议以及系统组件等中间件。中间件可以使用系统软件所提供的基础服务(功能)，衔接网络上应用系统的各个部分或不同的应用，能够达到资源共享、功能共享的目的。
硬件层主要包括硬件抽象层(Hardware Abstraction Layer，HAL)的接口(简称HAL接口)、硬件以及驱动，HAL 接口为电视芯片对接的统一接口，具体逻辑由各个芯片来实现。驱动主要包含：音频驱动、显示驱动、蓝牙驱动、摄像头驱动、WIFI驱动、USB驱动、HDMI驱动、传感器驱动(如指纹传感器，温度传感器，压力传感器等)、以及电源驱动等。
为了清楚的阐述本公开的技术方案，首先对本公开设计的名词进行解释说明：
CEC(Consumer Electronics Control)，消费类电子控制，它允许终端用户使用一个遥控器控制多个支持CEC的HD(High Definition，高解析度)设备，从而无需使用多个遥控器来控制电子设备，如：电视机(TV)、机顶盒和便携式HD设备。
EDID(Extended Display Identification Data)，扩展显示识别数据，是视频标准组织VESA为PC显示器设置的优化显示格式而设计的数据规范。EDID包含显示设备的基本参数，如制造厂商、产品名称、最大行场频、可支持的分辨率等，是实现即插即用功能背后的数据。EDID的存储方式包括内置和外置两种方式。EDID内置方式是指将EDID数据写在程序中，智能电视直流开机的时候将EDID数据写入电可擦可编程只读存储器(Electrically Erasable Programmable Read-Only Memory，EEPROM)芯片中，使用的时候去EEPROM芯片中获取智能电视的EDID数据。EDID外置方式是指直接将EDID数据写在外置的EEPROM芯片中，通过显示数据通道直接被外接设备读取，而不需经过智能电视主程序的控制。
HDMI(High Definition Multimedia Interface)，高清晰度多媒体接口，是一种数字化视频/音频接口技术，适合影像传输的专用型数字化接口，可同时传送音频和影像信号，最高数据传输速度为48Gbps(2.1版)，同时无需在信号传送前进行数/模或者模/数转换。HDMI接口具有热插拔功能。
HDCP(High-bandwidth Digital Content Protection)，高带宽数字内容保护技术，HDCP技术会对图像中每个像素进行处理，使得图像画面变得毫无规律、无法识别，只有确认同步后的发送端和接收端才可能进行逆向处理，完成数据的还原。在解密过程中，HDCP系 统会每2秒钟进行一次连接确认，同时每128帧画面进行一次发送端和接收端同步识别码，确保连接的同步。
HAL(Hardware Abstraction Layer，硬件抽象层)，硬件抽象层是位于操作系统内核与硬件电路之间的接口层，其目的在于将硬件抽象化。它隐藏了特定平台的硬件接口细节，为操作系统提供虚拟硬件平台，使其具有硬件无关性，可在多种平台上进行移植。从软硬件测试的角度来看，软硬件的测试工作都可分别基于硬件抽象层来完成，使得软硬件测试工作的并行进行成为可能。
远程视频会议实际为两台设备进行内容共享的过程，内容共享可以分为四个阶段：信息采集、信息处理、信息传输和信息还原。信息采集可以利用发送端设备的音视频采集卡完成。信息处理和信息传输依赖远端视频会议服务器。由远端视频会议服务器根据当前视频会议的模式确定对音视频信号的处理方式和转发逻辑，最后将处理后的音视频数据再发送到每一个接收端设备。接收端设备从远端视频会议服务器接收传输的音视频数据后，将传输过来的音视频数据进行解码，将图像和声音还原到接收端设备上进行播放。
远端视频会议服务器实际上为远程视频会议应用对应的服务器。如图5所示的远程视频会议场景，用户可以利用显示设备200-1上的远程视频会议应用，向显示设备200-2发起远程视频会议，这里显示设备200-2也需要安装并运行相同的远程视频会议应用。这图5所示的远程视频会议场景中，显示设备200-1可以将其采集的音视频内容共享给显示设备200-2，显示设备200-2也可以将其采集的音视频内容共享给显示设备200-1，也就是说显示设备200-1和显示设备200-2均可以既可以作为远程视频会议中的发送端设备，也可以作为接收端设备。
如果需要共享的音视频内容为普通内容，则显示设备200-1和显示设备200-2之间可以正常的共享内容，但是如果发送端设备利用HDMI端口接收CEC设备传输的HDMI内容，并且需要将HDMI内容共享给接收端设备，由于Android系统的相关技术中不存在共享HDMI内容的机制，因此发送端设备无法将显示的HDMI内容共享至接收端设备，进而导致远程视频会议中内容共享失败。
例如，如图6所示的应用场景中，显示设备200-1接入有CEC设备，并且CEC设备向显示设备200-1传输HDMI内容，此时由于Android系统的相关技术中不存在共享HDMI内容的机制，显示设备200-1不可将其正在显示的HDMI内容共享至显示设备200-2，这样将会导致显示设备200-1和显示设备200-2之间的远程视频会议内容共享失败。在显示设备200-2上不会显示显示设备200-1上显示的HDMI内容，甚至在显示设备200-2上会出现黑屏的显示情况。
图7为根据一些实施例的HDMI内容共享系统框架图，图7所示的内容共享系统框架中包括显示设备200-1和显示设备200-2。其中，显示设备200-1通过HDMI端口接入CEC设备300，并且CEC设备300通过HDMIIN(即HDMI输入端口)向显示设备200-1传输第一HDMI内容。显示设备200-1接收到CEC设备300传输的第一HDMI内容之后，可以在显示器上显示第一HDMI内容。
图8为显示设备200执行本公开实施例提供的一种设备控制方法的流程图。如图8所示，包括以下步骤：
步骤S800：接收第一HDMI内容；
第一HDMI内容是CEC设备通过HDMIIN传输的。显示设备200接收该第一HDMI 内容后，将第一HDMI内容输出到显示器中显示。
步骤S801，响应于第一应用发送的第一内容共享请求，将所述HDMIIN构建成虚拟设备节点，以使将所述第一HDMI内容存储至所述虚拟设备节点，其中，所述第一内容共享请求用于请求将所述第一HDMI内容共享至第一设备。
第一应用可以为远程视频会议应用，在显示设备200-1和显示设备200-2(即第一设备)上需要运行第一应用，显示设备200-1和显示设备200-2利用第一应用进行远程视频会议，即进行内容共享。
图9所示的用户界面为第一应用的用户界面，在图9所示的第一应用的用户界面中包括画面显示区域和功能操作区域。其中，画面显示区域用于显示CEC设备300传输至显示设备200-1的第一HDMI内容，功能操作区域用于展示可实现共享功能的不同按钮控件：“HDMI共享”按钮控件、“桌面共享”按钮控件、“白板共享”按钮控件，不同的按钮控件用于实现不同的共享功能。例如，“HDMI共享”按钮控件用于实现HDMI内容共享，其他按钮控件用于实现普通内容共享。如果用户需要共享HDMI内容时，可以通过点击“HDMI共享”按钮控件输入第一内容共享请求，也就是说第一内容共享请求用于请求将第一HDMI内容共享至显示设备200-2。如果用户只需要将普通内容共享至显示设备200-2，可以通过点击其他按钮控件输入其他内容共享请求。也就是说，只有接收到用户输入的第一内容共享请求，才可以进行将HDMI内容共享至显示设备200-2的过程。
响应于第一应用发送的第一内容共享请求后，可以将HDMIIN构建成虚拟设备节点。本公开实施例中将HDMIIN首先注册为V4L2_device，从而生成对应的vedio虚拟设备节点。V4L2(Video for linux two)是Linux内核中视频类设备的一套驱动框架，为视频类设备驱动开发和应用层提供了一套统一的接口规范。使用V4L2设备驱动框架注册的设备会在Linux系统/dev/目录下生成对应的设备节点文件，设备节点的名称通常为videoX(X标准一个数字编号:/dev/videox)，每一个videoX设备文件就代表一个视频类设备。应用程序通过对videoX设备文件进行I/O(Input/Output，输入/输出)操作来配置、使用虚拟设备。
视频类设备对应的设备节点为/dev/videoX，X为数字编号，通常从0开始，调用open()函数打开设备节点，可以得到文件描述符fd。打开虚拟设备后，需要查询虚拟设备的属性，确定该设备是否是一个视频采集类设备。通过ioctl()将获取到一个struct v4l2_capability类型数据，struct v4l2_capability数据结构描述了虚拟设备的一些属性，这些属性包括driver(驱动的名字)、card(设备的名字)、bus_info(总线的名字)、version(版本信息)、capabilities(设备拥有的能力)等。其中capabilities字段描述了设备拥有的能力，因此可以通过判断capabilities字段是否包含V4L2_CAP_VIDEO_CAPTURE，来确定设备是都为摄像头设备。
实施时，可将HDMIIN首先注册为V4L2_device，从而生成一个HDMIIN对应的虚拟摄像头节点，该虚拟摄像头节点的数据流即为第一HDMI内容，具体实现上可以是将第一HDMI内容存储至该虚拟设备节点。
需要说明的是，将HDMIIN构建成虚拟摄像头节点的前提是，第一HDMI内容只包括视频内容，也就是说只需要在显示设备200-1和显示设备200-2之间共享视频内容，构建的虚拟摄像头节点只作为虚拟摄像头使用，因此从虚拟摄像头节点中只能够获取到视频内容，而不能够获取到音频内容。
步骤S802，在硬件抽象层HAL中增加所述虚拟设备节点对应的共享接口。
在将HDMIIN构建成虚拟设备节点之后，还需要在HAL中增加虚拟设备节点对应的共享接口。如果是将HDMIIN构建成虚拟摄像头节点，则HAL为Camera HAL，即在Camera HAL中增加对构建的虚拟摄像头节点的支持，也就是说在Camera HAL增加针对虚拟摄像头节点的共享接口。然后第一应用可以在Camera HAL中打开该虚拟摄像头节点。第一应用在Camera HAL中打开虚拟摄像头节点之后，第一应用即可从虚拟摄像头节点中读取到第一HDMI内容，然后第一应用可以将从虚拟摄像头节点中读取到的第一HDMI内容共享给显示设备200-2，从而实现显示设备200-1将HDMI内容共享至显示设备200-2。
图10示出了上述HDMI内容共享示例的整体设计框架，包括HDMI端口、VDIN模块(硬件处理模块)、Camera HAL、GE2D(2D图形加速引擎)模块、Surfacefliger(图形用户界面的核心，以系统服务的形式存在，负责将所有App的图形数据按照Z Order顺序混合并输出到FrameBuffer)以及APP。其中VDIN和GE2D为HW Module(硬件模块)，Surfacefliger和APP为SW Module(软件模块)，Video output和Preview为DMA Buffer(直接存储器缓存区)。从HDMI端口输入的HDMI内容之后，由VDIN模块将HDMI内容进行转化(例如解码)，然后输入camera HAL的Video output之后再经过GE2D模块进行图形性能处理。将经过图形性能处理的数据再返回至camera HAL的preview(预览)插件。最后经过Surfacefliger的混合处理后输出至APP，由APP进行显示和内容共享。
图11示出了上述HDMI内容共享示例中将HDMI模拟为虚拟摄像头的过程，包括以下步骤：
步骤S1101：注册V4L2设备；
步骤S1102：在检测到有HDMI RX信号输入时，通过硬件处理模块VDIIN，将输入的HDMI RX信号转化为流文件数据；
步骤S1103：通过vdin_v4l2_isr接口获取由HDMI RX信号转化而来的流文件数据，并将流文件数据导入V4L2设备。
实施时可首先注册一个虚拟设备V4L2，并生成一个虚拟camera的设备节点(虚拟相机节点)，例如将该节点定义为vedio70。在检测到有HDMI RX(receive)信号输入时，通过硬件处理VDIIN模块，将输入的HDMI RX信号转化为流文件数据。然后在注册的V4L2设备中，通过vdin_v4l2_isr接口获取由HDMI RX信号转化而来的流文件数据。通过vdin_v4l2_if_isr将数据导入到已注册的V4L2设备。测试V4L2设备节点中的数据与HDMI RX信号转化而来的流文件数据相同，由此完成将HDMIIN转化为虚拟camera的设备节点，通过上述流程得到的虚拟camera的设备节点可由Android的camera HAL调用。
图12示出了上述HDMI内容共享示例中HDMI内容到Android应用层的流转过程，包括以下步骤：
步骤S1201：第一应用Android APP向服务器CameraService发送调用指令；
步骤S1202：服务器CameraService调用硬件抽象层Camera HAL获取数据；
步骤S1203：第一应用Android APP向硬件抽象层Camera HAL发送显示面Surface或缓冲器buffer；
步骤S1204：高清晰度多媒体接口HDMI将CEC设备发送的数据写入V4L2虚拟设备；
步骤S1205：硬件抽象层Camera HAL采用V4L2协议从虚拟设备V4L2中获取数据；
步骤S1206：硬件抽象层Camera HAL对数据进行解码；
步骤S1207：硬件抽象层Camera HAL将解码数据发送给第一应用Android APP；
在一些实施例中，Android APP可以通过CameraService调用Camera HAL，由Camera HAL打开真正的vedioX节点，vedioX节点对应V4L2设备。即虚拟摄像头设备。
此外，Android APP还可以将其显示面Surface，或者用户接收数据的缓冲器buffer传递给Camera HAL。然后Camera HAL打开V4L2虚拟设备之后，使用V4L2协议从V4L2虚拟设备中不断地获取数据。这是由于HDMI通过HDMIIN不断的将数据写入V4L2虚拟设备，此时Camera HAL可以从V4L2虚拟设备不断的获取到CEC设备写入的数据。
Camera HAL从V4L2虚拟设备读取数据的方式有两种：一种是read方式，直接通过read()系统调用读取V4L2虚拟设备写入的数据；另一种是streaming方式。使用VIDIOC_QUERYCAP指令查询设备的属性、得到一个struct v4l2_capability类型数据，其中capabilities字段记录了设备拥有的能力，当该字段包含V4L2_CAP_READWRITE时，表示设备支持read I/O方式读取数据；当该字段包含V4L2_CAP_STREAMING时，表示设备支持streaming I/O方式；使用streaming I/O方式，需要向设备申请帧缓冲，并将帧缓冲映射到应用程序进程地址空间中。
需要说明的是，由于创建的只是HDMIIN对应的虚拟设备节点，CEC设备写入的数据需要实际的存储至物理内存中。可以将CEC设备写入的数据从物理内存中映射到V4L2虚拟设备对应的虚拟内存中。具体的，可以通过vedioX节点将物理内存映射到用户控件，从而得到V4L2虚拟设备对应的虚拟内存的虚拟地址，而虚拟地址指向物理内存。因此Camera HAL从虚拟设备节点读取第一HDMI内容，可以是基于该虚拟地址从物理内存中读取第一HDMI内容。
Camera HAL从V4L2虚拟设备中读取到数据之后，开始进行数据解码。最后Camera HAL将解码成功的每一帧数据送入Android APP的surface用于在显示设备200-1的屏幕上显示。同时将解码后的数据送入buffer之后，由Android APP自行对数据进行处理，这里的处理包括在屏幕上显示或者共享给其他设备。
本公开实施例提供的一种设备控制方法中，CEC设备300通过HDMIIN向显示设备200-1传输第一HDMI内容时，如果响应于请求将第一HDMI内容共享至显示设备200-2，将HDMIIN构建成虚拟设备节点，以使将第一HDMI内容存储至该虚拟设备节点。然后在HAL中增加该虚拟设备节点对应的共享接口。第一应用可以通过调用共享接口的方式，从虚拟设备节点中获取第一HDMI内容，然后将第一HDMI内容共享至显示设备200-1。这样，在显示设备200-2上显示HDMI内容时，显示设备200-2也可以将HDMI内容共享至显示设备200-2，避免远程视频会议过程中内容共享失败。
在一些实施例中，如果第一HDMI内容在从CEC设备300传输至显示设备200-1时未经过HDCP加密，则只需要直接将第一HDMI内容存储至虚拟设备节点。如果第一HDMI内容在从CEC设备300传输至显示设备200-1时经过HDCP加密，则需要对第一HDMI内容进行HDCP解密后，然后将HDCP解密后的第一HDMI内容存储至虚拟设备节点。这样第一应用通过调用共享接口从虚拟设备节点获取HDCP解密后的第一HDMI内容，并且将解密就的第一HDMI内容共享至显示设备200-2。因此本公开实施例不仅可以建立一套将HDMI内容通过视频会议应用共享给其他设备的共享机制，并且针对经过HDCP加密的HDMI内容，也可以在当前设备中进行HDCP解密后，将解密后的HDMI内容共享给其他设备，更进一步的避免在共享HDMI内容时在其他设备上显示HDMI内容失败的情况发生。
在一些实施例中，显示设备200-1还可以包括图像采集器，例如自带摄像头或USB摄像头。因此可以根据使用场景判断使用真实的摄像头或者使用虚拟摄像头。图13所示的用户界面为第一应用的用户界面中包括“真实摄像头”区域和“虚拟摄像头”区域，其中“真实摄像头”区域又包括内置摄像头和USB摄像头，“虚拟摄像头”区域包括HDMI共享1和HDMI共享2。如果用户需要共享HDMI内容时，可以通过点击“HDMI共享1”按钮控件或“HDMI共享2”按钮控件输入第一内容共享请求，然后根据第一内容共享请求将第一HDMI内容共享至显示设备200-2。如果用户不需要共享HDMI内容，可以通过点击“真实摄像头”区域中的按钮控件输入第二内容共享请求，然后根据第二内容共享请求，将真实摄像头采集的第二图像内容共享至显示设备200-2。
在图13所示的应用场景中，可以在画面展示区域显示不同的摄像头的预览画面，用户可以通过查看预览画面，确定需要选择的摄像头。例如，CEC设备1和CEC设备2分别对应HDMI共享1和HDMI共享2，CEC设备1可以向显示设备200-1输入高清电影资源，CEC设备2只可以输入普通画质的电影资源。如果接收到用户通过点击“HDMI共享1”输入的选择指令，可以在画面展示区域预览高清电影视频画面；如果接收到用户通过点击“HDMI共享2”输入的选择指令，可以在画面展示区域预览普通画质的电影视频画面。用户可以根据自己需求通过点击不同的摄像头按钮控件，向显示设备200-2共享不同CEC设备输入的不同的HDMI内容。
针对真实摄像头也可以通过在画面展示区域显示预览画面以供用户判断选择。例如，内置摄像头和USB摄像头拍摄画面的角度不同，通过在画面展示区域显示不同角度的预览画面，可以提示用户不同摄像头对应的不同拍摄角度。
在一些实施例中，在执行响应于与第一应用发送的第一内容共享请求之前，还需要确定第一内容共享请求是否携带有预定参数，该预定参数表征当前场景允许将第一HDMI内容共享至显示设备200-2。只有在确定第一内容共享请求携带有预定参数基础上，才进行HDMI内容共享的过程。如果确定第一内容共享请求未携带预定参数，则不进行HDMI内容共享。
例如，在一些使用摄像头的应用中，有的摄像头引用不可以使用虚拟摄像头，因此需要在Camera HAL中增加图14所示的逻辑处理来开启摄像头，如图14所示，包括以下步骤：
步骤S1401，定义增加不同的摄像头类别；
在一些实施例中，可在Camera HAL和Camera Service中定义增加不同的摄像头类别，类别可包括内置摄像头、USB摄像头，以及HDMI虚拟摄像头(Vcam0)。
步骤S1402，根据上层应用调用，更新虚拟摄像头的属性。
如果上层应用可以使用虚拟摄像头，Vcam0_id的属性设置为0，如果上层应用不可以使用虚拟摄像头，Vcam0_id的属性设置为-1。并且更新对应的虚拟摄像头ID(HDMIcameraid)，例如HDMIcameraid设置为camera0或者camera1。
步骤S1403，判断虚拟摄像头是否可用；
上层应用打开摄像头时根据Vcam0_id的属性设置，判断虚拟摄像头是否可用，如果确定虚拟摄像头可用则进行步骤S1404，如果确定虚拟摄像头不可用则进行步骤S1405。
步骤S1404，若虚拟摄像头可用，则打开虚拟摄像头。
在一些实施例中，可根据HDMIcameraid确定该虚拟摄像头ID对应的虚拟摄像头，然 后从该虚拟摄像头中读取HDMI数据
步骤S1405，若虚拟摄像头不可用，则判断是否有可用的真实摄像头；
步骤S1406，若真实摄像头可用，则打开真实摄像头；
在一些实施例中，真实摄像头可为内置摄像头或者USB摄像头。
步骤S1407：若真实摄像头不可用，则输出表征无摄像头的提示。
在一些实施例中，在显示设备200-1中可以配置第一列表，第一列表可以记录当前可使用的所有的虚拟设备节点，在需要调用虚拟设备节点时，应用程序可以通过HAL从第一列表中查找可使用的虚拟设备节点。本公开实施例中，在执行响应于第一应用发送的第一内容共享请求，将HDMIIN构建成虚拟设备节点之后，可以将构建的虚拟设备节点添加至第一列表，这样HAL能够从第一列表中查找到HDMIIN对应的虚拟设备节点。如果CEC设备从显示设备200-1上拔出，根据拔出信号可以将HDMIIN对应的虚拟设备节点从第一列表中删除。这样HAL不再能够从第一列表中查找到HDMIIN对应的虚拟设备节点，也就不能够打开HDMIIN对应的虚拟设备节点。
上述过程的具体的逻辑实现如图15所示：
步骤S1501，Camera HAL进程启动，监听/dev/videoX节点的创建与删除。HDMIIN接入时，进行步骤S1501，HDMIIN拔出时，进行步骤S1502；
步骤S1502，若检测到/dev/videoX节点的创建，则生成与该节点对应的Camera对象；
步骤S1503，将Camera对象添加到维护的Camera列表中，更新列表并通知服务器Camera Service同步更新；
在一些实施例中，Camera HAL可将Camera对象添加到维护的Camera列表(即第一列表)中，更新列表中的所有Camera的ID，通知上层Camera Service同步更新列表和CameraID的ID，通知上层Camera Service同步更新列表和CameraID；
步骤S1504，若检测到/dev/videoX节点的删除，则将Camera列表中与该节点对应的Camera对象移除，更新列表并通知服务器Camera Service同步更新；
在一些实施例中，Camera HAL可将列表中的Camera对象删除，更新列表中的所有Camera的ID，通知上层Camera Service同步更新列表和CameraID。基于上述过程Android可以基于HDMIIN的接入和拔出动态感知Camera对象添加和删除。
需要说明的是，Camera列表中具备多个Camera对象。这种情况可以是，由于显示设备可以具有多个HDMI端口，多台CEC设备300通过HDMI端口接入显示设备200-1，并且多台CEC设备300均可向显示设备200-1输入HDMI内容。在多台CEC设备300接入显示设备200-1时，可以分别针对不同的HDMIIN构建虚拟设备节点。然后在HAL层中针对不同的虚拟设备节点添加共享接口。应用程序可以调用不同的共享接口从不同的虚拟设备节点中读取HDMI内容，从而实现可以将不同的HDMI内容共享给显示设备200-2。
例如，如图16所示的应用场景中，CEC设备300-1通过HDMI端口1接入显示设备200-1，同时CEC设备300-2通过HDMI端口2接入显示设备200-1。CEC设备300-1和CEC设备300-2均向显示设备200-1输入HDMI内容。为了方便用户切换传输线路，可以针对CEC设备300-1对应的HDMIIN构建虚拟设备节点/dev/video1，同时针对CEC设备300-2对应的HDMIIN构建虚拟设备节点/dev/video1。然后分别生成虚拟设备节点对应的Camera对象：/dev/video1对应的Camera对象可以是Camera1，/dev/video2对应的Camera对象可以是Camera2。将两个Camera对象同时添加至Camera列表。HAL可以从Camera 列表中同时查找到Camera1和Camera2。
这里，虽然有两台CEC设备接入显示设备200-1，但是通常用户会根据不同CEC设备接入的不同的HDMI端口选择接收哪台CEC设备输入的HDMI内容。HAL可以根据用户输入的选择指令，选择打开不同的虚拟设备节点从而读取不同的HDMI内容。例如，响应于用户输入的选择播放CEC设备300-1输入的HDMI内容的选择指令，HAL选择打开虚拟设备节点/dev/video1，然后可以从虚拟设备节点/dev/video1节点中不断的读取HDMI内容。从而使得可以将CEC设备300-1输入的HDMI内容共享至显示设备200-2。响应于用户输入的选择播放CEC设备300-2输入的HDMI内容的选择指令，HAL选择打开虚拟设备节点/dev/video2，然后可以从虚拟设备节点/dev/video2节点中不断的读取HDMI内容。从而使得可以将CEC设备300-2输入的HDMI内容共享至显示设备200-2。
在一些实施例中，如果需要共享至显示设备200-2的第一HDMI内容只包括视频内容，则响应于第一应用发送的第一内容共享请求，只需要将HDMIIN构建成表征虚拟设备节点的虚拟摄像头节点，然后将视频内容存储至虚拟摄像头节点。同样的只需要在Camera HAL中增加虚拟摄像头节点对应的共享接口。这样，第一应用可以通过调用Camera HAL中的共享接口，从虚拟摄像头节点中获取视频内容，并且将该视频内容共享给显示设备200-2。
如果需要共享至显示设备200-2的第一HDMI内容包括视频内容和音频内容，则响应于第一应用发送的第一内容共享请求，将HDMIIN构建成表征虚拟设备节点的虚拟摄像头节点和虚拟麦克风节点。然后，将第一HDMI内容中的视频内容存储至虚拟摄像头节点，将第一HDMI内容中的音频内容存储至虚拟麦克风节点。并且需要在Camera HAL中增加虚拟摄像头节点对应的共享接口，同时需要在Audio HAL中增加虚拟麦克风节点对应的共享接口。这样，第一应用可以通过调用Camera HAL中的共享接口，从虚拟摄像头节点中获取视频内容，同时可以通过调用Audio HAL中的共享接口，从虚拟麦克风节点中获取音频内容。然后第一应用可以将视频内容和音频内容共享给显示设备200-2。
在一些实施例中，显示设备200中可以同时安装摄像头应用A和摄像头应用B，其中摄像头应用A为只使用真实摄像头的应用，例如摄像头应用A可以为拍照应用，摄像头应用B为可同时使用虚拟摄像头和真实摄像头的应用，例如摄像头应用B可以为远程视频会议应用。当摄像头应用A运行时，摄像头应用A会发送用于调用真实摄像头的调用请求，摄像头应用A可以直接从真实摄像头获取图像数据。的那个摄像头应用B运行时，摄像头应用B会同时发送用于调用真实摄像头的调用请求，和用于调用虚拟摄像头的调用请求。接收到调用虚拟摄像头的调用请求时，需要构建对应的虚拟设备节点。因此此时显示设备200-1中共存在两个设备节点：真实摄像头的设备节点/dev/video0和虚拟摄像头的虚拟设备节点/dev/videoX。真实摄像头的设备节点/dev/video0和调用真实摄像头的调用请求对应。虚拟摄像头的虚拟设备节点/dev/videoX和调用虚拟摄像头的调用请求对应。

Claims (10)

  1. 一种显示设备，包括：
    显示器，被配置为显示消费电子控制设备CEC通过高清晰度多媒体接口HDMI的输入端口HDMIIN传输的第一HDMI内容；
    用户输入接口，被配置接收来自用户的指令；
    通信装置，被配置为根据预定协议与外部设备通信；
    存储器，被配置为保存计算机指令和与显示设备关联的数据；
    至少一个处理器，与所述显示器，用户输入接口，通信装置和存储器连接，被配置执行计算机指令以使得所述显示设备执行：
    响应于第一应用发送的第一内容共享请求，将所述HDMIIN构建成虚拟设备节点，以使所述第一HDMI内容存储至所述虚拟设备节点，其中，所述第一内容共享请求用于请求将所述第一HDMI内容共享至第一设备；
    在硬件抽象层HAL中增加所述虚拟设备节点对应的共享接口，以使所述第一应用通过调用所述共享接口从所述虚拟设备节点中获取所述第一HDMI内容，以及，将所述第一HDMI内容共享至所述第一设备。
  2. 根据权利要求1所述的显示设备，所述至少一个处理器，被进一步配置为执行计算机指令以使得所述显示设备执行：
    若从所述CEC设备传输至所述显示设备的所述第一HDMI内容存在高带宽数字内容保护技术HDCP加密，则在响应于所述第一应用发送的内容共享请求时，对所述第一HDMI内容进行HDCP解密；
    将解密后的所述第一HDMI内容存储至所述虚拟设备节点，以使所述第一应用通过调用所述共享接口从所述虚拟设备节点中获取HDCP解密后的所述第一HDMI内容，并将解密后的所述第一HDMI内容共享至所述第一设备。
  3. 根据权利要求1所述的显示设备，所述第一应用通过以下方式获取所述第一HDMI内容：
    调用所述共享接口，在HAL中打开所述虚拟设备节点之后从所述虚拟设备节点中读取所述第一HDMI内容；
    将所述第一应用对应的缓冲器传输至所述HAL，以使所述HAL将从所述虚拟设备节点中读取的所述第一HDMI内容存储至所述第一应用对应的缓冲器。
  4. 根据权利要求1所述的显示设备，所述至少一个处理器，被进一步配置为执行计算机指令以使得所述显示设备执行：
    构建所述虚拟设备节点之后，将所述虚拟设备节点添加至第一列表；其中，所述第一列表用于记录当前可使用的虚拟设备节点，所述HAL用于从所述第一列表中查找可使用的虚拟设备节点。
  5. 根据权利要求4所述的显示设备，所述至少一个处理器，被进一步配置为执行计算机指令以使得所述显示设备执行：
    若检测到所述CEC设备未接入所述显示设备，则从所述第一列表中删除所述虚拟设备节点。
  6. 根据权利要求1所述的显示设备，所述显示设备还包括图像采集器，所述图像采 集器被配置为采集第二图像内容，所述至少一个处理器，被进一步配置为执行计算机指令以使得所述显示设备执行：
    响应于所述第一应用发送的第二内容共享请求，将所述第二图像内容发送至所述第一应用，以使所述第一应用将所述第二图像内容共享至所述第一设备。
  7. 根据权利要求6所述的显示设备，所述至少一个处理器，被进一步配置为执行计算机指令以使得所述显示设备执行：
    响应于第一应用发送的第一内容共享请求之前，确定所述第一内容共享请求是否携带预定参数；其中，所述预定参数表征当前场景允许将所述第一HDMI内容共享至所述第一设备；
    若未携带所述预定参数，则将所述第二图像内容发送至所述第一应用，以使所述第一应用将所述第二图像内容共享至所述第一设备。
  8. 根据权利要求1所述的显示设备，所述至少一个处理器，具体被配置为执行计算机指令以使得所述显示设备通过下述执行将所述HDMIIN构建成虚拟设备节点：
    若检测到所述第一HDMI内容为视频内容，则将所述HDMIIN构建为表征所述虚拟设备节点的虚拟摄像头节点；
    所述至少一个处理器，具体被配置为执行计算机指令以使得所述显示设备通过下述执行在硬件抽象层HAL中增加所述虚拟设备节点对应的共享接口：
    在所述HAL中增加所述虚拟摄像头节点对应的共享接口，以将所述视频内容存储至所述虚拟摄像头节点，并使所述第一应用通过调用所述共享接口从所述虚拟设备节点中获取所述第一HDMI内容，以及，将所述第一HDMI内容共享至所述第一设备。
  9. 根据权利要求1所述的显示设备，所述至少一个处理器，具体被配置为执行计算机指令以使得所述显示设备通过下述执行将所述HDMIIN构建成虚拟设备节点：
    若检测到所述第一HDMI内容包括视频内容和音频内容，则将所述HDMIIN构建为标准所述虚拟设备节点的虚拟摄像头节点和虚拟麦克风节点，以将所述第一HDMI内容中的视频内容存储至所述虚拟摄像头节点，并将所述第一HDMI内容中的音频内容存储至所述虚拟麦克风节点；
    所述至少一个处理器，具体被配置为执行计算机指令以使得所述显示设备通过下述执行在硬件抽象层HAL中增加所述虚拟设备节点对应的共享接口：
    在所述HAL中增加所述虚拟摄像头节点对应的第一共享接口和所述虚拟麦克风节点对应的第二共享接口，以使所述第一应用通过调用所述第一共享接口从所述虚拟摄像头节点中获取所述第一HDMI内容中的视频内容，并使所述第一应用通过调用所述第二共享接口从所述虚拟麦克风节点中获取所述第一HDMI内容中的音频内容，以及，将所述第一HDMI内容中的视频内容和音频内容共享至所述第一设备。
  10. 一种设备控制方法，包括：
    接收消费电子控制设备CEC通过高清晰度多媒体接口HDMI的输入端口HDMIIN传输的第一HDMI内容；
    响应于第一应用发送的第一内容共享请求，将所述HDMIIN构建成虚拟设备节点，以使将所述第一HDMI内容存储至所述虚拟设备节点，其中，所述第一内容共享请求用于请求将所述第一HDMI内容共享至第一设备；
    在硬件抽象层HAL中增加所述虚拟设备节点对应的共享接口，以使所述第一应用通 过调用所述共享接口从所述虚拟设备节点中获取所述第一HDMI内容，以及，将所述第一HDMI内容共享至所述第一设备。
PCT/CN2024/113670 2023-12-15 2024-08-21 一种显示设备及设备控制方法 Pending WO2025123762A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202480038768.XA CN121312122A (zh) 2023-12-15 2024-08-21 一种显示设备及设备控制方法

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202311735183.7 2023-12-15
CN202311735183.7A CN120166250A (zh) 2023-12-15 2023-12-15 一种显示设备及hdmi内容共享方法

Publications (1)

Publication Number Publication Date
WO2025123762A1 true WO2025123762A1 (zh) 2025-06-19

Family

ID=96006458

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2024/113670 Pending WO2025123762A1 (zh) 2023-12-15 2024-08-21 一种显示设备及设备控制方法

Country Status (2)

Country Link
CN (2) CN120166250A (zh)
WO (1) WO2025123762A1 (zh)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114007035A (zh) * 2021-09-24 2022-02-01 深圳壹秘科技有限公司 会议数据分享系统及方法
CN114416292A (zh) * 2021-12-31 2022-04-29 北京字节跳动网络技术有限公司 定位设备串口的虚拟化方法、设备、装置、介质及产品
CN115185594A (zh) * 2022-09-06 2022-10-14 湖北芯擎科技有限公司 基于虚拟显示的数据交互方法、装置、电子设备及介质
US20230064996A1 (en) * 2021-08-31 2023-03-02 Citrix Systems, Inc. Sharing Virtual Environment Data

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20230064996A1 (en) * 2021-08-31 2023-03-02 Citrix Systems, Inc. Sharing Virtual Environment Data
CN114007035A (zh) * 2021-09-24 2022-02-01 深圳壹秘科技有限公司 会议数据分享系统及方法
CN114416292A (zh) * 2021-12-31 2022-04-29 北京字节跳动网络技术有限公司 定位设备串口的虚拟化方法、设备、装置、介质及产品
CN115185594A (zh) * 2022-09-06 2022-10-14 湖北芯擎科技有限公司 基于虚拟显示的数据交互方法、装置、电子设备及介质

Also Published As

Publication number Publication date
CN120166250A (zh) 2025-06-17
CN121312122A (zh) 2026-01-09

Similar Documents

Publication Publication Date Title
CN111954044B (zh) 通道切换方法和显示设备
CN112367543B (zh) 显示设备、移动终端、投屏方法及投屏系统
CN114286165B (zh) 一种显示设备、移动终端、投屏数据传输方法及系统
CN111752518A (zh) 一种显示设备投屏方法及显示设备
CN114297436A (zh) 一种显示设备及用户界面主题更新方法
CN111669621A (zh) 一种媒资数据下发方法、服务器及显示设备
CN111787377B (zh) 显示设备及投屏方法
WO2022105409A1 (zh) 故障诊断方法、终端设备及显示设备
CN112153447A (zh) 一种显示设备及音画同步控制方法
CN111683279A (zh) 显示设备及图像恢复方法
CN112486934A (zh) 一种文件同步方法及显示设备
CN114286137A (zh) 一种镜像投屏方法、显示设备及终端
CN113992963A (zh) 一种显示设备及投屏方法
CN114900386A (zh) 一种终端设备及数据接力方法
CN113518257A (zh) 多系统投屏处理方法与设备
CN111885665A (zh) 一种无线网络连接控制方法及显示设备
CN111954043A (zh) 一种信息栏显示方法及显示设备
CN111918132A (zh) 一种显示设备及多接口设备判断方法
CN112269668A (zh) 一种应用资源共享及显示设备
CN114286320A (zh) 一种显示设备、移动终端及蓝牙连接方法
CN112040309A (zh) 一种频道切换方法及显示设备
CN111818654A (zh) 一种信道接入方法及显示设备
CN115665468B (zh) 投屏方法、装置和电子设备
CN111818368B (zh) 管理显示设备权限的方法、移动终端以及服务器
CN116527971A (zh) 显示设备和音频播放方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 24902188

Country of ref document: EP

Kind code of ref document: A1