WO2022105344A1 - Projection picture correction method, terminal device and display device - Google Patents

Abstract

Disclosed are a projection picture correction method, a terminal device and a display device. The method comprises: in response to a second correction operation inputted by a user according to an operation guide page, establishing a communication connection with a display device, wherein the operation guide page is a page displayed by the display device after receiving a first correction operation and used for guiding the user to start a cooperated correction at the terminal device; acquiring a correction graphic card displayed by the display device, and calculating correction parameters according to the correction graphic card; and sending the correction parameters to the display device, so that the display device corrects a laser signal by using the correction parameters.

Description

Projection picture correction method, terminal device and display device

This application claims the priority of the Chinese patent application filed on November 23, 2020, with application number 202011322273.X and titled "A projection screen correction method, terminal device and display device", the entire contents of which are incorporated by reference in in this application.

technical field

The present application relates to the field of televisions, and in particular, to a projection image correction method, a terminal device and a display device.

Background technique

The imaging principle of laser TV is to separate the video signal into image signals of RGB three primary colors, respectively control the RGB three primary color semiconductor laser generators to emit lasers of corresponding intensity. Projected on the screen, so that the user can watch the projected picture. Laser TV has the advantages of eye protection, energy saving, high spectral purity and high cost performance, and has a good application prospect.

SUMMARY OF THE INVENTION

The present application provides a projection image correction method, a terminal device and a display device.

The terminal equipment provided in the first aspect includes:

a display for displaying the calibration page;

Image grabber, used to capture the calibration chart on the display device;

A communicator for communicating with the display device and the server;

A controller, configured to execute:

In response to the second correction operation input by the user according to the operation guide page, a communication connection is established with the display device; wherein the operation guide page is displayed by the display device after receiving the first correction operation and is used to guide the user to initiate cooperation on the terminal device Corrected pages;

Control the image collector to collect the calibration chart displayed by the display device, and calculate the calibration parameters according to the calibration chart;

The correction parameters are sent to the display device, so that the display device uses the correction parameters to correct the laser signal.

The display device provided by the second aspect includes:

A display for projecting and displaying operation guide pages and calibration charts;

A communicator for communicating with terminal equipment;

A controller, configured to execute:

In response to receiving the first correction operation, the display is controlled to display an operation guide page, and a broadcast packet is sent to the terminal device; the operation guide page is used to guide the user to start a coordinated correction on the terminal device; the broadcast packet is used to request the terminal device. establishing a communication connection, the broadcast packet contains identification information of the display device;

After establishing a communication connection with the terminal device, control the display to display the calibration chart;

The receiving terminal device corrects the laser signal according to the correction parameters calculated by the correction chart, and uses the correction parameters.

A third aspect provides a projection image correction method in a terminal device, the method further comprising:

In response to the second correction operation input by the user according to the operation guide page, a communication connection is established with the display device; wherein the operation guide page is displayed by the display device after receiving the first correction operation and is used to guide the user to initiate cooperation on the terminal device Corrected pages;

Collect the calibration chart displayed by the display device, and calculate the calibration parameters according to the calibration chart;

The correction parameters are sent to the display device, so that the display device uses the correction parameters to correct the laser signal.

A fourth aspect provides a projection image correction method in a display device, including:

In response to receiving the first correction operation, an operation guide page is displayed, and a broadcast packet is sent to the terminal device; the operation guide page is used to guide the user to initiate coordination correction on the terminal device; the broadcast packet is used to request the establishment of communication with the terminal device connection, the broadcast packet contains the identification information of the display device;

After establishing a communication connection with the terminal device, display a calibration chart;

The receiving terminal device corrects the laser signal according to the correction parameters calculated by the correction chart, and uses the correction parameters.

Description of drawings

FIG. 1 exemplarily shows a schematic diagram of an operation scene between the display device 200 and the control apparatus 100;

FIG. 2 exemplarily shows the hardware configuration block diagram of the display device 200 in FIG. 1;

FIG. 3 exemplarily shows the hardware configuration block diagram of the control device 100 in FIG. 1;

FIG. 4 exemplarily shows a schematic diagram of the software configuration in the display device 200 in FIG. 1;

FIG. 5 exemplarily shows a schematic diagram of the icon control interface display of the application program in the display device 200;

FIG. 6 exemplarily shows a schematic diagram of an existing display device displaying an operation guidance page;

FIG. 7 exemplarily shows a schematic diagram of an existing display device displaying a calibration chart;

FIG. 8 exemplarily shows a schematic diagram of a page jumped after scanning a two-dimensional code by an existing terminal device;

Fig. 9 exemplarily shows a schematic diagram when an existing terminal device collects a calibration chart;

FIG. 10 exemplarily shows a schematic diagram of the software architecture among the terminal device, display device and server of the present application;

Fig. 11 exemplarily shows the interaction logic diagram between the terminal device, the display device and the server;

FIG. 12 exemplarily shows a schematic diagram of the operation guidance page displayed by the display device in the present application;

FIG. 13 exemplarily shows a schematic interface diagram of a terminal device displaying a device list;

FIG. 14 exemplarily shows a schematic diagram of a terminal device displaying a correction page.

Detailed ways

In order to make the objectives, implementations and advantages of the present application clearer, the exemplary embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings in the exemplary embodiments of the present application. Obviously, the exemplary embodiments described It is only a part of the embodiments of the present application, but not all of the embodiments.

Based on the exemplary embodiments described in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the appended claims of this application. Furthermore, although the disclosures in this application have been presented in terms of illustrative example or instances, it should be understood that various aspects of this disclosure may also constitute a complete embodiment in isolation.

It should be noted that the brief description of the terms in the present application is only for the convenience of understanding the embodiments described below, rather than intended to limit the embodiments of the present application. Unless otherwise specified, these terms are to be understood according to their ordinary and ordinary meanings.

The terms "first", "second", "third", etc. in the description and claims of this application and the above drawings are used to distinguish similar or similar objects or entities, and are not necessarily meant to limit specific Order or precedence unless otherwise indicated. It should be understood that the terms of such accesses can be interchanged under appropriate circumstances, for example, can be implemented according to an order other than those presented in the illustrations or descriptions of the embodiments of the present application.

Furthermore, the terms "comprising" and "having" and any variations thereof, are intended to cover but not exclusively include, for example, a product or device incorporating a series of components is not necessarily limited to those explicitly listed, but may include No other components are expressly listed or inherent to these products or devices.

The term "remote control", as accessed in this application, refers to a component of an electronic device, such as the display device disclosed in this application, that can wirelessly control the electronic device, usually over a short distance. Generally access infrared and/or radio frequency (RF) signals and/or Bluetooth to connect with electronic devices, and may also include functional modules such as WiFi, wireless USB, Bluetooth, and motion sensors. For example, a hand-held touch remote control replaces most of the physical built-in hard keys in a general remote control device with a user interface in a touch screen.

The term "gesture" as used in this application refers to a user's behavior through an action such as a change in hand shape or hand movement, which is used to express an intended thought, action, purpose/or result.

FIG. 1 exemplarily shows a schematic diagram of an operation scenario between a display device and a control apparatus according to an embodiment. As shown in FIG. 1 , the user can operate the display device 200 through the terminal device 300 and the control device 100 .

In some embodiments, the control device 100 may be a remote control, and the communication between the remote control and the display device includes infrared protocol communication or Bluetooth protocol communication, and other short-distance communication methods, etc., and the display device 200 is controlled by wireless or other wired methods. . The wireless mode may be a direct connection or a non-direct connection, and may be routed or not routed. The user can control the display device 200 by inputting user instructions through keys on the remote control, voice input, control panel input, and the like. For example, the user can control the display device 200 by inputting corresponding control commands through the volume up/down key, channel control key, up/down/left/right movement keys, voice input key, menu key, power-on/off key, etc. on the remote control. function.

In some embodiments, mobile terminals, tablet computers, computers, notebook computers, and other smart terminal devices may also be accessed to control the display device 200 . For example, accessing an application running on the smart device controls the display device 200 . The app can be configured to provide users with various controls in an intuitive user interface (UI) on the screen associated with the smart device.

In some embodiments, the terminal device 300 may install a software application with the display device 200 to implement connection communication through a network communication protocol, so as to achieve the purpose of one-to-one control operation and data communication. For example, the terminal device 300 and the display device 200 can be used to establish a control command protocol, the remote control keyboard can be synchronized to the terminal device 300, and the function of controlling the display device 200 can be realized by controlling the user interface on the terminal device 300. The audio and video content displayed on the terminal device 300 may also be transmitted to the display device 200 to implement a synchronous display function.

As also shown in FIG. 1 , the display device 200 also performs data communication with the server 400 through various communication methods. The display device 200 may be allowed to communicate via local area network (LAN), wireless local area network (WLAN), and other networks. The server 400 may provide various contents and interactions to the display device 200 . For example, the display device 200 interacts by sending and receiving information, and electronic program guide (EPG), receiving software program updates, or accessing a remotely stored digital media library. The server 400 provides other network service content such as video-on-demand and advertising services.

The display device 200 may be a liquid crystal display, an OLED display, or a projection display device. The specific display device type, size and resolution are not limited. Those skilled in the art can understand that the display device 200 can make some changes in performance and configuration as required.

The display device 200 may additionally provide a computer-supported Internet TV function, including but not limited to, Internet TV, Internet Protocol TV (IPTV), and the like, in addition to providing the broadcast receiving TV function.

FIG. 2 exemplarily shows a block diagram of the hardware configuration of the display device 200 according to the exemplary embodiment.

In some embodiments, the display device 200 includes a controller 250, a tuner 210, a communicator 220, a detector 230, an input/output interface 255, a display 275, an audio output interface 285, a memory 260, a power supply 290, At least one of the user interface 265 and the external device interface 240 .

In some embodiments, the display 275, for receiving the image signal from the output of the first processor, performs components for displaying video content and images and a menu manipulation interface.

In some embodiments, the display 275 includes a display component for presenting a picture, and a driving component for driving image display.

In some embodiments, the video content displayed may be from broadcast television content or various broadcast signals that may be received via wired or wireless communication protocols. Alternatively, various image contents sent from the network server side can be displayed and received from the network communication protocol.

In some embodiments, display 275 is used to present a user-manipulated UI interface generated in display device 200 and used to control display device 200 .

In some embodiments, depending on the type of display 275, a driving component for driving the display is also included.

In some embodiments, display 275 is a projection display, and may also include a projection device and projection screen.

In some embodiments, communicator 220 is a component for communicating with external devices or external servers according to various communication protocol types. For example, the communicator may include at least one of the WIFI module 221, the Bluetooth module 222, the wired Ethernet module 223 and other network communication protocol modules or near field communication protocol modules, and an infrared receiver, so that the communicator 220 can 250, receives the control signal from the control device 100, and implements the control signal as a signal type such as a WIFI signal, a Bluetooth signal, and a radio frequency signal.

In some embodiments, the display apparatus 200 may establish control signal and data signal transmission and reception between the communicator 220 and the external control apparatus 100 or the content providing apparatus.

In some embodiments, the user interface 265 may be used to receive infrared control signals from the control device 100 (eg, an infrared remote control, etc.).

In some embodiments, the detector 230 is a signal used by the display device 200 to collect the external environment or interact with the outside.

In some embodiments, the detector 230 includes a light receiver, a sensor for collecting ambient light intensity, and can adaptively display parameter changes and the like by collecting ambient light.

In some embodiments, the detector 230 may also include an image collector, such as a camera, a camera, etc., which can be used to collect external environment scenes, and used to collect user attributes or interactive gestures with the user, and can adaptively change display parameters, User gestures can also be recognized to implement functions that interact with users.

In some embodiments, detector 230 may also include a temperature sensor or the like, such as by sensing ambient temperature.

In some embodiments, the display device 200 can adaptively adjust the display color temperature of the image. For example, when the temperature is relatively high, the display device 200 can be adjusted to display a relatively cool color temperature of the image, or when the temperature is relatively low, the display device 200 can be adjusted to display a warmer color of the image.

In some embodiments, the detector 230 may also be a sound collector or the like, such as a microphone, which may be used to receive the user's voice. Exemplarily, it includes a voice signal of a control instruction of the user to control the display device 200, or collects ambient sounds, and is used to identify the type of the ambient scene, so that the display device 200 can adapt to the ambient noise.

In some embodiments, as shown in FIG. 2 , the input/output interface 255 is configured to enable data transfer between the controller 250 and other external devices or other controllers 250 . Such as receiving video signal data and audio signal data of external equipment, or command instruction data, etc.

In some embodiments, the external device interface 240 may include, but is not limited to, the following: any one or more of a high-definition multimedia interface HDMI interface, an analog or data high-definition component input interface, a composite video input interface, a USB input interface, an RGB port, etc. interface. It is also possible to form a composite input/output interface by a plurality of the above-mentioned interfaces.

In some embodiments, as shown in FIG. 2 , the tuner and demodulator 210 is configured to receive broadcast television signals through wired or wireless reception, and can perform modulation and demodulation processing such as amplification, frequency mixing, and resonance, and can perform modulation and demodulation processing from multiple wireless receivers. Or demodulate the audio and video signal from the cable broadcast TV signal, the audio and video signal may include the TV audio and video signal carried in the frequency of the TV channel selected by the user, and the EPG data signal.

In some embodiments, the frequency demodulated by the tuner-demodulator 210 is controlled by the controller 250, and the controller 250 can send a control signal according to the user's selection, so that the modem responds to the user-selected TV signal frequency and modulates and demodulates the frequency. The frequency carried by the television signal.

In some embodiments, broadcast television signals may be classified into terrestrial broadcast signals, cable broadcast signals, satellite broadcast signals, or Internet broadcast signals, etc. according to different broadcast formats of the television signals. Or according to different modulation types, it can be divided into digital modulation signal, analog modulation signal, etc. Or, it can be divided into digital signals, analog signals, etc. according to different types of signals.

In some embodiments, the controller 250 and the tuner 210 may be located in different separate devices, that is, the tuner 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box Wait. In this way, the set-top box outputs the modulated and demodulated television audio and video signals of the received broadcast television signals to the main device, and the main device receives the audio and video signals through the first input/output interface.

In some embodiments, the controller 250, through various software control programs stored on the memory, controls the operation of the display device and responds to user operations. The controller 250 may control the overall operation of the display apparatus 200 . For example, in response to receiving a user command for selecting a UI object to be displayed on the display 275, the controller 250 may perform an operation related to the object selected by the user command.

In some embodiments, the object may be any of the selectable objects, such as a hyperlink or an icon. Operations related to the selected object, such as displaying operations linked to hyperlinked pages, documents, images, etc., or executing operations corresponding to the icon. The user command for selecting the UI object may be an input command through various input devices (eg, a mouse, a keyboard, a touchpad, etc.) connected to the display device 200 or a voice command corresponding to a voice spoken by the user.

As shown in FIG. 2 , the controller 250 includes a random access memory 251 (Random Access Memory, RAM), a read-only memory 252 (Read-Only Memory, ROM), a video processor 270, an audio processor 280, and other processors 253 (For example: at least one of graphics processing unit (Graphics Processing Unit, GPU), central processing unit 254 (Central Processing Unit, CPU), communication interface (Communication Interface), and communication bus 256 (Bus). Wherein, the communication bus Connect the parts.

In some embodiments, RAM 251 is used to store temporary data for the operating system or other running programs.

In some embodiments, ROM 252 is used to store various system startup instructions.

In some embodiments, ROM 252 is used to store a basic input output system, called a Basic Input Output System (BIOS). It is used to complete the power-on self-test of the system, the initialization of each functional module in the system, the driver program of the basic input/output of the system and the booting of the operating system.

In some embodiments, when the power-on signal is received, the power supply of the display device 200 is started, and the CPU executes the system start-up instruction in the ROM 252, and copies the temporary data of the operating system stored in the memory to the RAM 251, so as to facilitate the start-up or running operation system. After the startup of the operating system is completed, the CPU copies the temporary data of various application programs in the memory to the RAM 251, so as to facilitate starting or running various application programs.

In some embodiments, processor 254 executes operating system and application program instructions stored in memory. And various application programs, data and content are executed according to various interactive instructions received from the external input, so as to finally display and play various audio and video content.

In some exemplary embodiments, processor 254 may include multiple processors. The plurality of processors may include a main processor and one or more sub-processors. The main processor is used to perform some operations of the display device 200 in the pre-power-on mode, and/or an operation of displaying a picture in the normal mode. One or more sub-processors for an operation in a state such as standby mode.

In some embodiments, the graphics processor 253 is used to generate various graphic objects, such as: icons, operation menus, and user input instructions to display graphics and the like. It includes an operator, which performs operations by receiving various interactive instructions input by the user, and displays various objects according to the display properties. and includes a renderer, which renders various objects obtained based on the operator, and the rendered objects are used for displaying on a display.

In some embodiments, the video processor 270 is configured to receive the external video signal and perform decompression, decoding, scaling, noise reduction, frame rate conversion, resolution conversion, image synthesis, etc. according to the standard codec protocol of the input signal. After video processing, a signal that can be directly displayed or played on the display device 200 can be obtained.

In some embodiments, the video processor 270 includes a demultiplexing module, a video decoding module, an image synthesis module, a frame rate conversion module, a display formatting module, and the like.

In some embodiments, the graphics processor 253 and the video processor can be integrated and set up, and can also be set up separately. When the integrated setting is performed, the processing of the graphics signal output to the display can be performed. When the separate setting is performed, different functions can be performed respectively. For example, GPU+FRC (Frame Rate Conversion)) architecture.

In some embodiments, the audio processor 280 is configured to receive an external audio signal, perform decompression and decoding, and noise reduction, digital-to-analog conversion, and amplification processing according to a standard codec protocol of the input signal to obtain a The sound signal played in the speaker.

In some embodiments, the video processor 270 may comprise one or more chips. The audio processor may also include one or more chips.

In some embodiments, the video processor 270 and the audio processor 280 may be separate chips, or may be integrated into one or more chips together with the controller.

In some embodiments, the audio output, under the control of the controller 250, receives the sound signal output by the audio processor 280, such as the speaker 286, and in addition to the speaker carried by the display device 200 itself, can be output to an external device. The external audio output terminal of the device, such as an external audio interface or an earphone interface, etc., may also include a short-range communication module in the communication interface, such as a Bluetooth module for outputting sound from a Bluetooth speaker.

The power supply 290, under the control of the controller 250, provides power supply support for the display device 200 with the power input from the external power supply. The power supply 290 may include a built-in power supply circuit installed inside the display device 200 , or may be an external power supply installed in the display device 200 to provide an external power supply interface in the display device 200 .

The user interface 265 is used for receiving user input signals, and then sending the received user input signals to the controller 250. The user input signal may be a remote control signal received through an infrared receiver, and various user control signals may be received through a network communication module.

In some embodiments, the user inputs user commands through the control apparatus 100 or the terminal device 300 , the user input interface is based on the user's input, and the display device 200 responds to the user's input through the controller 250 .

In some embodiments, the user may input user commands on a graphical user interface (GUI) displayed on the display 275, and the user input interface receives the user input commands through the graphical user interface (GUI). Alternatively, the user may input a user command by inputting a specific sound or gesture, and the user input interface recognizes the sound or gesture through a sensor to receive the user input command.

In some embodiments, a "user interface" is a media interface for interaction and information exchange between an application or operating system and a user. The commonly used form of user interface is Graphical User Interface (GUI), which refers to a user interface related to computer operations displayed in a graphical manner.

The memory 260 includes storing various software modules for driving the display device 200 . For example, various software modules stored in the first memory include at least one of a basic module, a detection module, a communication module, a display control module, a browser module, and various service modules.

The basic module is used for signal communication between various hardwares in the display device 200, and is a low-level software module that sends processing and control signals to the upper-layer module. The detection module is a management module used to collect various information from various sensors or user input interfaces, perform digital-to-analog conversion, and analyze and manage.

FIG. 3 exemplarily shows a configuration block diagram of the control apparatus 100 according to an exemplary embodiment. As shown in FIG. 3 , the control device 100 includes a controller 110 , a communication interface 130 , a user input/output interface, a memory, and a power supply.

The control apparatus 100 is configured to control the display device 200 , and can receive the user's input operation instructions, and convert the operation instructions into instructions that the display device 200 can recognize and respond to, so as to play an intermediary role between the user and the display device 200 . For example, the user operates the channel addition and subtraction keys on the control device 100, and the display device 200 responds to the channel addition and subtraction operation.

In some embodiments, the control apparatus 100 may be a smart device. For example, the control apparatus 100 may install various applications for controlling the display device 200 according to user requirements.

In some embodiments, as shown in FIG. 1 , the terminal device 300 or other intelligent electronic device can perform a similar function of the control apparatus 100 after installing the application for operating the display device 200 . For example, the user can install the application, various function keys or virtual buttons of the graphical user interface provided on the terminal device 300 or other intelligent electronic devices, so as to realize the function of the physical key of the control apparatus 100 .

The controller 110 includes a processor 112 and RAM 113 and ROM 114, a communication interface 130, and a communication bus. The controller is used to control the operation and operation of the control device 100, as well as the communication and cooperation between the internal components and the external and internal data processing functions.

The communication interface 130 realizes the communication of control signals and data signals with the display device 200 under the control of the controller 110 . For example, the received user input signal is sent to the display device 200 . The communication interface 130 may include at least one of other near field communication modules such as a WiFi chip 131 , a Bluetooth module 132 , and an NFC module 133 .

The user input/output interface 140, wherein the input interface includes at least one of other input interfaces such as a microphone 141, a touch panel 142, a sensor 143, and a key 144. For example, the user can implement the user command input function through actions such as voice, touch, gesture, pressing, etc. The input interface converts the received analog signal into a digital signal, and converts the digital signal into a corresponding command signal, and sends it to the display device 200.

The output interface includes an interface for transmitting received user instructions to the display device 200 . In some embodiments, it can be an infrared interface or a radio frequency interface. For example, when the infrared signal interface is used, the user input instruction needs to be converted into an infrared control signal according to the infrared control protocol, and sent to the display device 200 through the infrared sending module. For another example, when a radio frequency signal interface is used, the user input command needs to be converted into a digital signal, and then modulated according to the radio frequency control signal modulation protocol, and then sent to the display device 200 by the radio frequency transmission terminal.

In some embodiments, the control device 100 includes at least one of a communication interface 130 and an input-output interface 140 . The control device 100 is configured with a communication interface 130, such as modules such as WiFi, Bluetooth, NFC, etc., which can send user input instructions to the display device 200 through WiFi protocol, Bluetooth protocol, or NFC protocol encoding.

The memory 190 is used to store various operating programs, data and applications for driving and controlling the control device 100 under the control of the controller. The memory 190 can store various control signal instructions input by the user.

The power supply 180 is used to provide operating power support for each element of the control device 100 under the control of the controller. Can battery and related control circuit.

In some embodiments, a system may include a kernel (Kernel), a command parser (shell), a file system, and applications. Together, the kernel, shell, and file system make up the basic operating system structures that allow users to manage files, run programs, and access the system. After power-on, the kernel starts, activates the kernel space, abstracts hardware, initializes hardware parameters, etc., runs and maintains virtual memory, scheduler, signals and inter-process communication (IPC). After the kernel starts, the shell and user applications are loaded. An application is compiled into machine code after startup, forming a process.

Referring to FIG. 4 , in some embodiments, the system is divided into four layers, from top to bottom, they are an application layer (referred to as “application layer”), an application framework layer (referred to as “framework layer”) ”), the Android runtime and the system library layer (referred to as the “system runtime layer”), and the kernel layer.

In some embodiments, at least one application program runs in the application program layer, and these application programs may be a Window program, a system setting program, a clock program, a camera application, etc. built into the operating system; they may also be developed by a third party The application programs developed by the author, such as the Hijian program, the K song program, the magic mirror program, etc. During specific implementation, the application package in the application layer is not limited to the above examples, and may actually include other application packages, which is not limited in this embodiment of the present application.

The framework layer provides an application programming interface (Aplication Pogramming Iterface, API) and a programming framework for the applications of the application layer. The application framework layer includes some predefined functions. The application framework layer is equivalent to a processing center, which decides to let the applications in the application layer take action. The application program can access the resources in the system and obtain the services of the system during execution through the API interface.

As shown in FIG. 4 , the application framework layer in this embodiment of the present application includes managers (Managers), content providers (Content Providers), and view systems (View Systems), etc., wherein the manager includes at least one of the following modules: activity The Activity Manager is used to interact with all activities running in the system; the Location Manager is used to provide system services or applications with access to system location services; the Package Manager is used to It is used to retrieve various information related to the application package currently installed on the device; the Notification Manager is used to control the display and removal of notification messages; the Window Manager is used to manage the icons on the user interface. , windows, toolbars, wallpapers, and desktop widgets.

In some embodiments, software programs and/or modules corresponding to the software architecture in FIG. 4 are stored in the first memory or the second memory shown in FIG. 2 or FIG. 3 .

In some embodiments, for a display device with a touch function, taking a split-screen operation as an example, the display device receives an input operation (such as a split-screen operation) performed by the user on the display, and the kernel layer can generate corresponding input according to the input operation. event and report the event to the application framework layer. The window mode (such as multi-window mode) and window position and size corresponding to the input operation are set by the activity manager of the application framework layer. The window management of the application framework layer draws the window according to the settings of the activity manager, and then sends the drawn window data to the display driver of the kernel layer, and the display driver displays the corresponding application interface in different display areas of the display.

In some embodiments, as shown in FIG. 5 , the application layer includes at least one application that can display corresponding icon controls on the display, such as: Live TV application icon controls, Video On Demand (VOD) applications Program icon control, media center application icon control, application center icon control, game application icon control, etc.

In some embodiments, the live TV application may provide live TV from different sources. For example, a live TV application can access input from cable, over-the-air, satellite services, or other types of live TV services to provide TV signals. And, the live TV application may display the video of the live TV signal on the display device 200 .

In some embodiments, a video-on-demand application may provide video from various storage sources. Unlike live TV applications, video-on-demand provides a display of video from certain storage sources. For example, video-on-demand can come from the server side of cloud storage, from local hard disk storage containing existing video programs.

In some embodiments, the media center application may provide various multimedia content playback applications. For example, a media center may provide services other than live TV or video-on-demand, where users can access various images or audio through a media center application.

In some embodiments, the application center may provide storage of various applications. An application can be a game, an application, or some other application that is related to a computer system or other device but can be run on a Smart TV. The application center can obtain these applications from various sources, store them in local storage, and then run them on the display device 200 .

The above embodiments describe the hardware/software architecture and function implementation of a conventional display device. The difference between laser TV and conventional TV mainly lies in the imaging method of the screen image. Laser TV separates the video signal into RGB three primary color image signals, respectively controls the RGB three primary color semiconductor laser generators to emit lasers of corresponding intensity, and modulates the output and signal synchronization. After control, the laser beam is projected onto the screen based on the ultra-short-focus projection technology, so that the user can watch the projected picture. Other hardware/software configurations of the laser TV can be adapted with reference to conventional TVs or existing laser TV products, and the same or similar parts will not be repeated here.

Since the laser TV uses laser projection imaging, the projected image may exceed or be too smaller than the frame of the screen frame, resulting in a large area deviation between the projected image and the screen, thereby reducing the projection display effect. Therefore, it is necessary to perform geometric correction of the projected image. Make the corrected projected image better match the screen border.

FIG. 6 to FIG. 9 show the existing UI operations of automatic geometric correction. Laser TVs are generally equipped with geometric correction application 1. After the user opens geometric correction application 1 in the Settings menu, the operation guide page shown in Figure 6 will be displayed, and the operation guide page will display operation instructions and QR codes. , such as "use the terminal to scan the QR code below and operate according to the prompts", the premise of automatic geometric correction is that the laser TV and the terminal device must be in the same local area network (WiFi) to realize the communication connection between the two, so as to use The terminal is used to cooperate with the laser TV to calibrate the projected image.

The terminal device can be a mobile terminal such as a mobile phone and a tablet computer, so that the user can hold the terminal and adjust the image collector to aim at the screen of the laser TV. The image collector of the terminal generally adopts a rear camera. When both the laser TV and the terminal are connected to the Internet under the same WiFi, the user can scan the QR code in Figure 6 through the browser in the terminal device or other applications with QR code scanning function, such as WeChat, QQ, etc. , after scanning the code successfully, the communication connection between the terminal and the laser TV is automatically established. At this time, the laser TV will switch from the operation guide page to the display interface of the calibration chart shown in Figure 7. The dotted line in Fig. 7 is the boundary of the current projection image. Within the projection range, the image card generally includes a plurality of cross feature points arranged in an array. Obviously, the projection image has exceeded the frame range of the screen frame, and there are crosses that exceed the limit outside the screen frame. Feature points that meet the conditions for geometric correction. The corresponding correction parameters can be given by the distribution of the cross feature points and the screen frame.

When the terminal device scans the QR code successfully, it will recognize the link address corresponding to the QR code, and then jump to the page shown in Figure 8. The page in Figure 8 has operation information prompting the user to cooperate with the correction, such as "please Take a picture of the picture card horizontally about 3 meters in front of the TV screen and upload it”, and remind the user to pay attention to “please ensure that the TV screen and the picture card are completely captured”, that is, ensure that the screen border and the picture card edge in Figure 7 are captured, so that To ensure the accuracy of subsequent calibration. There is an image acquisition window in the middle part of Figure 8. After the user clicks the "+" in the image acquisition window, the image preview and shooting interface will pop up automatically. The preview ensures that the calibration chart is completely captured, and you can click the camera button to take an image of the calibration chart. As shown in Figure 9, the captured image will be directly displayed in the image capture window, and the user can check the shooting effect of the image. If the user confirms, click the "Finish" control in Figure 9, and the terminal will automatically upload the captured image to the laser TV. terminal; if the user clicks the "Cancel" control, the previously captured image will be removed from the image capture window, the terminal will restore the display interface shown in Figure 8, and the user can re-shoot the calibration chart until a satisfactory shooting effect is achieved.

After receiving the picture card image uploaded by the terminal, the laser TV uses the relevant image processing technology to process the picture card image, such as image binarization, edge detection, geometric transformation, etc. The coordinates of the feature points are processed and transformed, and the correction parameters are finally calculated. The laser TV calls the underlying interface and transmits the correction parameters to the relevant hardware to complete the correction of the laser signal. After the calibration is completed, the projection image displayed on the TV screen will be updated, and the user can watch the projection image to determine whether he is satisfied with the calibration effect. If the user is satisfied, he can end the calibration and exit the geometric calibration application 1; If satisfied, repeat the aforementioned correction operations on the page in Figure 8 until a satisfactory effect is achieved.

It can be known from the above existing automatic geometric correction scheme that the mobile terminal shoots the projection correction chart displayed on the laser TV screen, and then sends the projection correction chart to the laser TV, and the laser TV processes the chart to give the correction parameters. , on the one hand, due to the large image data of the image card captured by the terminal, which may reach tens of megabytes, and considering factors such as the speed and stability of WiFi, there must be a problem that the image card transmission takes a long time, which affects the correction efficiency; On the other hand, it will also increase the computational pressure of the laser TV, and the aforementioned calibration solution requires the user to cooperate with many manual operations, which cannot bring a better and more automated calibration operation experience to the user.

Considering the above technical problems, in some embodiments of the present application, as shown in FIG. 10 , the software architecture of the calibration process may involve three links of the display device 200 , the terminal device 300 and the server 400 .

In some embodiments, the display device 200 (ie, the laser TV end) includes at least a bottom-level hardware interface, a connection layer, a control layer, and a UI layer; the UI layer is used to display pages, such as the operation guide page and the correction diagram in the geometric correction application 1 The connection layer realizes the communication connection between the display device 200 and the terminal device 300, such as a TCP (Transmission Control Protocol, Transmission Control Protocol) connection, which is used to receive the correction parameters calculated and sent by the terminal device 300; the control layer is used to realize UI control, connection control and calibration control, including receiving the user's calibration request, controlling the display and update of the calibration chart, parsing the calibration parameters, and transmitting the calibration parameters to the underlying hardware interface, thereby controlling the hardware to calibrate the laser signal.

In some embodiments, the server 400 is used for training and storage of the calibration parameter calculation model, providing access to the calibration page of the terminal and a related JS (JavaScript) script library, and storing related graphics cards and the like. The deep learning model takes the calibration chart as input, and calculates and outputs the calibration parameters through the model; the calibration page is the page displayed on the calibration chart when the terminal device 300 cooperates with calibration. After the terminal device 300 establishes a TCP connection with the display device 200, it needs to load resources such as calibration pages, deep learning models, and JS script libraries from the server 400 to the local memory, and then formally start the calibration process. In the present application, resources such as training models and JS scripts for calibration parameters are preset in the server 400, and the terminal device 300 only needs to load the resources into the memory when using it. This method can relieve the storage burden of the terminal device 300 and does not affect the terminal device 300. Operational performance of device 300 . The correction parameter calculation model may use a deep learning model.

In some embodiments, the terminal device 300 communicates with the display device 200 and the server 400 respectively, and the terminal device 300 and the display device 200 need to be in the same local area network. For example, the display device 200 may periodically send a broadcast packet in the form of UDP, so that other devices connected to the same local area network can receive the broadcast packet, thereby requesting to establish a communication connection with these other devices. That is to say, the terminal device 300 will also receive the broadcast packet, and establish a TCP connection with the display device 200 according to the broadcast packet. After the TCP connection is successful, the terminal device 300 loads the correction page, the deep learning model, and the JS script library from the server 400. Resource to the local memory, and then use the rear camera of the local end to automatically capture the calibration chart; preprocess the calibration chart, identify the screen frame in the calibration chart, and perform affine transformation on the screen frame, and then use the depth The learning model calculates correction parameters and transmits the correction parameters to the display device 200 . A geometric correction application 2 may be installed in the terminal device 300 , and the terminal device 300 may execute a related correction process by controlling the geometric correction application 2 . The geometric correction application 1 in the display device 200 and the geometric correction application 2 in the terminal device 300 cooperate with each other to realize the linkage correction between the display device 200 and the terminal device 300, so as to realize automatic and efficient geometric correction of the projected image and improve the quality of the projected image. display effect.

In some embodiments, as shown in the interactive logic diagram in FIG. 11 , the following is a detailed description of the correction logic for the display device 200 , the terminal device 300 and the server 400 to realize the linkage and cooperation:

(A) Display device 200 side: the user starts the geometric correction application 1 in the Settings menu (this application names this behavior operation as the first correction operation), which is equivalent to initiating a correction request; after the geometric correction application 1 is started, The controller 250 controls the display 275 to display the operation guide page, and sends a UDP (User Datagram Protocol, User Datagram Protocol) broadcast packet to the terminal device 300 regularly, and the broadcast packet contains the identification information of the display device 200, and the identification information includes, for example, MAC address, IP address and/or device name, etc., so that the terminal device 300 can identify the device that initiated the connection request, and after establishing a TCP connection with the terminal device 300, it controls the display 275 to display the calibration chart; as shown in FIG. 12 , in the present application Only prompt information is displayed on the operation guide page, such as "Please open the geometric correction application 2 of the terminal, and start the coordination correction of the terminal." After viewing the prompt information, the user can start the geometric correction application 2 on the terminal device 300 (this application will This behavior operation is named as the second correction operation), the operation guide page in FIG. 12 does not contain a two-dimensional code, that is, the application does not connect the display device 200 and the terminal device 300 by scanning the two-dimensional code, but the display device 200 and the terminal The device 300 must be online under the same WiFi.

(B) Terminal device 300: After the geometric correction application 2 is started, it receives the broadcast packet sent by the display device 200, and parses the identification information of the display device 200 contained in the broadcast packet; adds the identification information of the display device 200 to the device list , the device list stores the identification information of several devices in the same network as the terminal device 300, and the device list is in a dynamic update state. When a device in the same WiFi scene initiates a connection request to the terminal device 300, the device will The identification information of the device is recorded in the device list, which is convenient for the user to select the connection object from the device list; the terminal device 300 pops up the device list shown in Figure 13 on the application interface, and the device list shows the identification of all devices that have currently initiated connections. information, the user selects the identification information of the display device 200 in the device list, and the terminal device 300 establishes a TCP connection with the selected display device 200 in response to the user's selection operation. For example, the device list shown in Figure 13 includes three devices, and the device names are Laser TV1, Laser TV2 and Laser TV3. If the user wants to correct the projection screen of Laser TV1, click Laser TV1 to make Laser TV1 get the selected focus, and then Click the "Confirm Calibration" control, then the laser TV1 becomes the calibration object.

(C) Terminal device 300: Load resources such as the deep learning model and JS script library from the server 400 to the local memory, and load the resources of the calibration page from the server 400, and control the display of the local to display as shown in Figure 14 The calibration page, the calibration page in Figure 14 includes the first control, the second control and the window; when all the resources are loaded, the JS script automatically turns on the rear camera of the terminal device 300, enters the shooting preview, and the window of the calibration page will The preview screen of the calibration chart is always displayed, and the user can preview in the window to adjust the shooting position/angle of the rear camera. After that, the rear camera automatically captures a frame of the calibration chart every preset period, and will capture the image of the calibration chart. The image frames are sent to the controller in the terminal device 300 for image processing and calculation of correction parameters. Since the rear camera is automatically started using JS script after the calibration page is loaded, and the rear camera will automatically capture the picture card image at regular intervals, the user only needs to adjust the rear camera to face the TV screen and ensure the integrity of the screen and the picture card. , no other additional operations are required, which improves the user experience.

(D) 300 side of the terminal device: use OpenCV.JS to preprocess the correction chart, and the preprocessing may include image processing methods such as denoising, binarization, and edge detection. user privacy information, thereby protecting user privacy and improving the security when displaying images. The preprocessing process can be implemented with reference to the prior art, which will not be repeated in this embodiment; Border; if the screen border is not recognized in the current frame, the correction parameters cannot be calculated, and the current frame is an invalid image card, the invalid image card can be filtered out, and the calibration image card can continue to be collected and executed (D); if the current frame The screen frame that satisfies the conditions can be identified in the frame. If the conditions described here are constraints from the screen size, it means that the current frame belongs to a valid image card, and the screen frame identified in the current preprocessed corrected image card is simulated. The correction chart obtained after the affine transformation is input into the deep learning model for calculation, and the deep learning model outputs correction parameters, and then sends the correction parameters to the display device 200 through the TCP connection channel. Among them, the correction parameters are, for example, the vertex position of the projection surface and the midpoint position of each boundary on the projection surface. By adjusting the positions of these key points, the range and boundary of the projection surface can be changed, so that the projection surface and the screen frame can be better matching, so as to improve the display effect of the projection screen.

Since the user is holding the terminal device 300 to adjust the rear camera, manual operation cannot guarantee that the captured calibration chart is positive. For example, the rear camera may be tilted in a certain direction/angle, resulting in the existence of a screen frame in the calibration chart. Obviously skewed or not presented as a strict rectangle, it may appear as a parallelogram, which will lead to inaccurate recognition of the actual range of the screen frame, and it is impossible to accurately calculate the correction parameters. In this regard, the present application performs affine transformation on the screen frame identified in the image card image, thereby transforming the screen frame in the corrected image card into a positive rectangle, thereby improving the accuracy and reliability of the calculation result of the deep learning model. Here, the affine transformation can be implemented with reference to the prior art, which is not repeated in this application.

(E) Display device 200 end: receive and parse the correction parameters sent by the terminal device 300, call the underlying hardware interface, and transmit the correction parameters to the corresponding hardware, thereby controlling the hardware to correct the laser signal according to the correction parameters; then based on the corrected parameters The laser signal generates the target chart; the display device 275 is controlled to update and display the target chart, that is, the target chart replaces the before-corrected chart, thereby completing the update of the corrected chart on the display device side.

(F) Terminal device 300: control the rear camera to capture the target image card, and control the window in the calibration page to display the target image card, the user can see the corrected and updated target image card, and can visually judge this calibration Whether to make the projection screen achieve its satisfactory effect; if the user is satisfied with the correction effect, trigger the first control in the correction page, that is, "Satisfied" in FIG. 14 , thereby ending the current correction process, and the terminal device 300 can exit the geometric correction application 2, Similarly, the display device 200 can exit the geometric correction application 1; if the user is not satisfied with the correction effect, the second control in the correction page is triggered, that is, “re-correction” in FIG. ) ~ (F) to re-calibrate until the user is satisfied with the results.

In some embodiments, the terminal device 300 should at least include a display for displaying a calibration page, a communicator for communicating with the display device 200 and the server 400 , and a controller, and the controller is configured to execute the aforementioned terminal device 300 The control flow in the related projection picture correction method. Other software and hardware configurations of the terminal device 300 are not limited.

Embodiments of a display device, an embodiment of a method for calibrating a projection image executed in a display device, an embodiment of a terminal device, and an embodiment of an embodiment of a method for calibrating a projection image executed in a terminal device can be verified by mutual reference between these embodiments. For details, please refer to the foregoing related descriptions , this article will not repeat them. In addition, the image processing process of the calibration chart is not limited in this application, and the method of calculating the calibration parameters is not limited to the deep learning model. The server 400 can flexibly formulate the calculation model of the calibration parameters according to the actual application. The UI provided in this application is only exemplary, and the actual product configuration and application shall prevail.

The embodiment of the present application is to correct the projection screen of the laser TV through the linkage and cooperation between the multi-terminal devices. After the display device terminal receives the first correction operation, the first correction operation is, for example, starting the geometric correction application 1 in the display device, and the display device will display On the operation guide page, after the user sees the instructions on the operation guide page, he will enter the second calibration operation in the terminal device. The second calibration operation is, for example, starting the geometric calibration application 2 in the terminal, and then realizes the display device and the terminal through the device list. Establish a communication connection between each other without scanning a QR code. At this time, the display device side projects and displays the calibration chart, and the image collector in the terminal (ie, the rear camera) captures the image of the calibration chart, and then performs a series of image processing on the calibration chart to improve the calculation accuracy of the calibration parameters. Then, the correction parameters are calculated in the terminal according to the correction chart, and the correction parameters are sent to the display device. The display device can directly use the correction parameters to correct the laser signal, thereby improving the effect of screen projection. This application does not require manual correction by the user. After the terminal collects and calibrates the graphic card, it does not send it to the display device, but directly calculates the correction parameters based on the graphic card at the local end, which not only eliminates the time-consuming transmission of the graphic card, but also does not require the display device to calculate the correction. The parameters reduce the computing pressure of the display device, and the display device can achieve rapid correction according to the parameters fed back by the terminal, which improves the correction efficiency, accuracy and correction effect of the laser TV projection screen, and the user operation is simpler and faster, which improves the user experience.

Those skilled in the art can clearly understand that the technology in the embodiments of the present application can be implemented by means of software plus a necessary general hardware platform. In a specific implementation, the present application also provides a computer-readable non-volatile storage medium, where a program can be stored in the computer storage medium. When the computer storage medium is located in the display device 200, the program may include program steps involved in the projection image correction method that the controller 250 is configured to execute; when the computer storage medium is located in the terminal device 300, the program may be executed when the program is executed. It includes the program steps involved in the projection picture correction method that the controller in the terminal device 300 is configured to execute. The computer storage medium may be a magnetic disk, an optical disc, a read-only memory (English: Read-Only Memory, ROM for short) or a random access memory (English: Random Access Memory, RAM for short).

Claims (10)

1. A terminal device including:

   a display for displaying the calibration page;

   Image grabber, used to capture the calibration chart on the display device;

   A communicator for communicating with the display device and the server;

   A controller, configured to execute:

   In response to the second correction operation input by the user according to the operation guide page, a communication connection is established with the display device; wherein the operation guide page is displayed by the display device after receiving the first correction operation and is used to guide the user to initiate cooperation on the terminal device Corrected pages;

   Control the image collector to collect the calibration chart displayed by the display device, and calculate the calibration parameters according to the calibration chart;

   The correction parameters are sent to the display device, so that the display device uses the correction parameters to correct the laser signal.
2. The terminal device of claim 1, the controller further configured to perform:

   After establishing a communication connection with the display device, load a deep learning model and a correction page from the server; wherein, the deep learning model is used to calculate correction parameters; the correction page includes a first control, a second control and a display for correction A window for image card shooting preview, the first control is used to end the calibration process when triggered, and the second control is used to re-collect and calibrate the image card when triggered, and perform calibration again.
3. The terminal device of claim 1, the controller further configured to perform:

   Before calculating the correction parameters according to the correction chart, use OpenCV.JS to preprocess the correction chart, and the preprocessing includes binarization and edge detection;

   If the screen frame is not recognized in the preprocessed calibration chart, continue to collect the calibration chart;

   Correction parameters are calculated if screen borders are identified in the preprocessed correction chart.
4. The terminal device of claim 3, the controller is configured to calculate the correction parameter as follows:

   Perform affine transformation on the screen border in the preprocessed correction chart;

   The correction chart obtained after the affine transformation is input to the deep learning model for calculation, and finally the deep learning model outputs the correction parameters.
5. The terminal device of claim 2, the controller further configured to perform:

   Controlling the image collector to collect a target image card, the target image card is the image card updated and displayed by the display device after correcting the laser signal;

   controlling the window to display the target chart;

   In response to the user triggering the operation of the first control after viewing the target image card, the current calibration process is ended;

   In response to the user triggering the operation of the second control after viewing the target image card, recalibration is performed according to the collected target image card.
6. The terminal device according to claim 1, wherein the controller is configured to establish a communication connection with the display device in the following manner:

   receiving a broadcast packet sent by the display device;

   Parse the identification information of the display device contained in the broadcast packet;

   adding the identification information of the display device to the device list; the device list stores the identification information of several devices in the same network as the terminal device;

   In response to a selection operation on the identification information of the display device in the device list, a TCP connection between the terminal device and the display device is established.
7. A display device comprising:

   A display for projecting and displaying operation guide pages and calibration charts;

   A communicator for communicating with terminal equipment;

   A controller, configured to execute:

   In response to receiving the first correction operation, the display is controlled to display an operation guide page, and a broadcast packet is sent to the terminal device; the operation guide page is used to guide the user to start a coordinated correction on the terminal device; the broadcast packet is used to request and the terminal device. establishing a communication connection, the broadcast packet contains identification information of the display device;

   After establishing a communication connection with the terminal device, control the display to display the calibration chart;

   The receiving terminal device corrects the laser signal according to the correction parameters calculated by the correction chart, and uses the correction parameters.
8. The display device of claim 7, the controller further configured to perform:

   Based on the corrected laser signal, generate a target chart;

   The display is controlled to update and display the target image card, and after the target image card is collected by the image collector of the terminal device, it is determined to end the current calibration process or to re-calibrate.
9. A projection image correction method in a terminal device, comprising:

   In response to the second correction operation input by the user according to the operation guide page, a communication connection is established with the display device; wherein the operation guide page is displayed by the display device after receiving the first correction operation and is used to guide the user to initiate cooperation on the terminal device Corrected pages;

   Collect the calibration chart displayed by the display device, and calculate the calibration parameters according to the calibration chart;

   The correction parameters are sent to the display device, so that the display device uses the correction parameters to correct the laser signal.
10. A projection image correction method in a display device, comprising:

    In response to receiving the first correction operation, an operation guide page is displayed, and a broadcast packet is sent to the terminal device; the operation guide page is used to guide the user to initiate coordination correction on the terminal device; the broadcast packet is used to request the establishment of communication with the terminal device connection, the broadcast packet contains the identification information of the display device;

    After establishing a communication connection with the terminal device, display a calibration chart;

    The receiving terminal device corrects the laser signal according to the correction parameters calculated by the correction chart, and uses the correction parameters.

Images