CN113473196A

CN113473196A - Screen projection data processing method and display device

Info

Publication number: CN113473196A
Application number: CN202110823509.6A
Authority: CN
Inventors: 张伟丽
Original assignee: Qingdao Hisense Media Network Technology Co Ltd
Current assignee: Vidaa Netherlands International Holdings BV
Priority date: 2021-07-21
Filing date: 2021-07-21
Publication date: 2021-10-01

Abstract

The application provides a screen projection data processing method and display equipment. After screen projection data sent by the mobile terminal are received, effective data in the screen projection data can be extracted by the display equipment, and whether a picture needing screen projection is a horizontal screen picture or a vertical screen picture is determined by the effective data. And then, under the condition that the screen projection picture is a vertical screen picture, the display equipment rotates the display equipment which is in the horizontal screen state to be in the vertical screen state, meanwhile, invalid data in the screen projection data are cut, and only pictures corresponding to the valid data are displayed. Or, under the condition that the screen projection picture is the vertical screen picture, if the display equipment is currently in the vertical screen state, the display equipment can directly cut invalid data in the screen projection data, and only displays the picture corresponding to the valid data. And then can avoid appearing staying black problem when showing the screen shot picture on display device, guarantee user's use and experience.

Description

Screen projection data processing method and display device

Technical Field

The application relates to the technical field of display, in particular to a screen projection data processing method and display equipment.

Background

The screen projection function can realize that the picture on the mobile terminal is shared on the display equipment for displaying, so that the user can watch the content currently displayed on the mobile terminal on the display equipment in real time.

At present, most screen projection methods are that after a mobile terminal sends data of a display picture to a display device, the display device displays the data. Generally, data sent by a mobile terminal to a display device are all cross-screen, and if the mobile terminal displays the cross-screen, the data represent a cross-screen picture at the moment; if the mobile terminal displays the vertical screen, the mobile terminal can add some filling data in the data of the original vertical screen picture so as to add filling pictures on the left side and the right side of the vertical screen picture, and the filled whole picture is a horizontal screen picture. The filler picture has no actual content, and is usually black in order to not affect the content of the original portrait screen. Such as shown in fig. 6. The filling manner may also be referred to as a blackening process.

However, leaving a large amount of black on the display device can cause a problem of poor image picture display effect, thereby affecting the viewing experience of the user.

Disclosure of Invention

The application provides a display method and display equipment for a screen projection image, which are used for solving the problem that when a mobile terminal projects a screen on the display equipment at present, a screen projection image is not adaptive to the size of a screen of the display equipment.

In a first aspect, the present application provides a display device comprising:

a display;

a rotating component configured to rotate the display to change a display state of the display, wherein the display state comprises a landscape screen state and a portrait screen state;

a controller configured to:

receiving screen projection data sent by a mobile terminal;

determining the type of the picture to be projected by the mobile terminal by utilizing effective data which is used for representing the display picture of the mobile terminal in the screen projection data; the screen projection data further comprise invalid data used for filling a picture under the condition that the picture type is a vertical screen picture;

and under the condition that the picture type is a vertical screen picture, determining whether the display needs to be controlled to rotate or not according to the display state of the display so as to enable the display state of the display to be matched with the picture type, and simultaneously cutting invalid data in the screen projection data.

In some embodiments, the controller is further configured to:

detecting the current display state of the display under the condition that the picture type is a vertical screen picture;

under the condition that the display is in a horizontal screen state at present, controlling the display to rotate to a vertical screen state, and cutting invalid data in the screen projection data;

and under the condition that the display is in a vertical screen state at present, cutting invalid data in the screen projection data.

In some embodiments, the controller is further configured to:

and under the condition that the picture type is a vertical screen picture and the display is in a horizontal screen state at present, displaying the screen projection picture on the display rotating to be in the vertical screen state by utilizing the residual effective data after cutting in the screen projection data.

In some embodiments, the controller is further configured to:

and under the condition that the picture type is a vertical screen picture and the display is in a vertical screen state at present, displaying the screen projection picture on the display in the vertical screen state at present by using the residual effective data after cutting in the screen projection data.

In some embodiments, the controller is further configured to:

detecting the current display state of the display under the condition that the picture type is a horizontal screen picture;

under the condition that the display is in a horizontal screen state at present, displaying a screen projection picture on the display by using the screen projection data;

under the condition that the display is in a vertical screen state at present, controlling the display to rotate to a horizontal screen state;

and displaying a screen projection picture on a display in a horizontal screen state by using the screen projection data.

In some embodiments, the controller is further configured to:

controlling a display to display a prompt message to a user before controlling the display to rotate; the prompt message is used for inquiring whether the user agrees to rotate the display;

and controlling the display to rotate when a confirmation instruction input to the display device by a user is received.

In some embodiments, the controller is further configured to:

determining the color of an invalid picture for filling the display picture of the mobile terminal as a target color;

performing a summation operation on a screen projection picture corresponding to the screen projection data and the target color image, and identifying a target area corresponding to the target color in the screen projection picture;

and determining that the data corresponding to the target area is invalid data, and determining that the data except the invalid data in the screen projection data is valid data.

In some embodiments, the controller is further configured to:

identifying a target area corresponding to a target color from a screen projection picture corresponding to the screen projection data by using an image identification algorithm;

In some embodiments, the controller is further configured to:

determining whether the resource content of the screen projection data sent by the mobile terminal is changed;

and under the condition that the resource content is changed, re-determining effective data in the screen projection data.

In a second aspect, the present application further provides a method for processing screen projection data, where the method includes:

receiving screen projection data sent by a mobile terminal;

As can be seen from the above, the technical solution of the present application provides a method for processing screen projection data and a display device. Wherein the display device may be a rotating display device. After screen projection data sent by the mobile terminal are received, effective data in the screen projection data can be extracted by the display equipment, and whether a picture needing screen projection is a horizontal screen picture or a vertical screen picture is determined by the effective data. And then, under the condition that the screen projection picture is a vertical screen picture, the display equipment rotates the display equipment which is in the horizontal screen state to be in the vertical screen state, and meanwhile, invalid data in the screen projection data are cut, so that only pictures corresponding to the valid data are displayed. Or, under the condition that the screen projection picture is the vertical screen picture, if the display equipment is currently in the vertical screen state, the display equipment can directly cut invalid data in the screen projection data, and only displays the picture corresponding to the valid data. According to the technical scheme, when the mobile terminal delivers the vertical screen picture to the display device, invalid data can be cut off, only valid data are reserved, the display state of the display device is matched with the picture type of the screen picture, the problem that black is reserved when the screen picture is projected through display on the display device can be avoided, and the use experience of a user is guaranteed.

Drawings

In order to more clearly explain the technical solution of the present application, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious to those skilled in the art that other drawings can be obtained according to the drawings without creative efforts.

FIG. 1 illustrates a schematic diagram of a usage scenario of a display device 200 according to some embodiments;

FIG. 2 illustrates a rear view of a display device 200 according to some embodiments;

fig. 3 shows a block diagram of a hardware configuration of the control apparatus 100 according to some embodiments;

fig. 4 illustrates a hardware configuration block diagram of the display apparatus 200 according to some embodiments;

FIG. 5 illustrates a software configuration diagram in the display device 200 according to some embodiments;

FIG. 6 is a schematic diagram illustrating a landscape screen shot by a portrait screen mobile terminal according to some embodiments;

FIG. 7 illustrates a process diagram for a portrait screen mobile terminal sending screen projection data to a landscape display device 200, according to some embodiments;

FIG. 8 illustrates a process diagram for a landscape mobile terminal to send screen projection data to a portrait display device 200, according to some embodiments;

FIG. 9 illustrates a schematic diagram of processing screen projection data in a display device 200 according to some embodiments;

FIG. 10 illustrates a schematic diagram of a display device 200 extracting valid data according to some embodiments;

FIG. 11 illustrates another schematic diagram of the display device 200 extracting valid data according to some embodiments;

FIG. 12 illustrates a schematic diagram of an active screen in a projected screen of a portrait mobile terminal according to some embodiments;

FIG. 13 illustrates another schematic diagram of an active screen in a projected screen of a portrait screen mobile terminal, in accordance with some embodiments;

FIG. 14 illustrates a schematic diagram of prompting information on a display device 200 according to some embodiments.

Detailed Description

To make the purpose and embodiments of the present application clearer, the following will clearly and completely describe the exemplary embodiments of the present application with reference to the attached drawings in the exemplary embodiments of the present application, and it is obvious that the described exemplary embodiments are only a part of the embodiments of the present application, and not all of the embodiments.

It should be noted that the brief descriptions of the terms in the present application are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of the present application. These terms should be understood in their ordinary and customary meaning unless otherwise indicated.

The terms "first," "second," "third," and the like in the description and claims of this application and in the above-described drawings are used for distinguishing between similar or analogous objects or entities and not necessarily for describing a particular sequential or chronological order, unless otherwise indicated. It is to be understood that the terms so used are interchangeable under appropriate circumstances.

The terms "comprises" and "comprising," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a product or apparatus that comprises a list of elements is not necessarily limited to all elements expressly listed, but may include other elements not expressly listed or inherent to such product or apparatus.

The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and/or software code that is capable of performing the functionality associated with that element.

FIG. 1 illustrates a schematic diagram of a usage scenario of a display device 200 according to some embodiments. As shown in fig. 1, the display apparatus 200 is also in data communication with a server 400, and a user can operate the display apparatus 200 through the smart device 300 or the control device 100.

In some embodiments, the control apparatus 100 may be a remote controller, and the communication between the remote controller and the display device includes at least one of an infrared protocol communication or a bluetooth protocol communication, and other short-distance communication methods, and controls the display device 200 in a wireless or wired manner. The user may control the display apparatus 200 by inputting a user instruction through at least one of a key on a remote controller, a voice input, a control panel input, and the like.

In some embodiments, the smart device 300 may include any of a mobile terminal, a tablet, a computer, a laptop, an AR/VR device, and the like.

In some embodiments, the smart device 300 may also be used to control the display device 200. For example, the display device 200 is controlled using an application program running on the smart device.

In some embodiments, the smart device 300 and the display device may also be used for communication of data.

In some embodiments, the display device 200 may also be controlled in a manner other than the control apparatus 100 and the smart device 300, for example, the voice instruction control of the user may be directly received by a module configured inside the display device 200 to obtain a voice instruction, or may be received by a voice control apparatus provided outside the display device 200.

In some embodiments, the display device 200 is also in data communication with a server 400. The display device 200 may be allowed to be communicatively connected through a Local Area Network (LAN), a Wireless Local Area Network (WLAN), and other networks. The server 400 may provide various contents and interactions to the display apparatus 200. The server 400 may be a cluster or a plurality of clusters, and may include one or more types of servers.

In some embodiments, software steps executed by one step execution agent may be migrated on demand to another step execution agent in data communication therewith for execution. Illustratively, software steps performed by the server may be migrated to be performed on a display device in data communication therewith, and vice versa, as desired.

In some embodiments, as shown in FIG. 2, display device 200 includes a base, display 260, terminal interfaces 278 extending from the gaps in the backplane, and a rotating assembly 276 coupled to the backplane. The display 260 may be a liquid crystal display, an organic light emitting display, a projection device, or the like, among others. The rotation assembly 276 may rotate the display 260 without limitation to the particular display device type, size, resolution, etc. From the perspective of the front view of the display device, the rotating component 276 can rotate the display screen to a vertical screen state, that is, a state where the vertical side length of the screen is greater than the horizontal side length, or to a horizontal screen state, that is, a state where the horizontal side length of the screen is greater than the vertical side length.

In some exemplary embodiments, the controller 250 is a control chip, and is contained inside the base.

Fig. 3 illustrates a block diagram of a hardware configuration of the control apparatus 100 according to some embodiments. As shown in fig. 3, the control device 100 includes a controller 110, a communication interface 130, a user input/output interface 140, a memory, and a power supply. The control apparatus 100 may receive an input operation instruction from a user and convert the operation instruction into an instruction recognizable and responsive by the display device 200, serving as an interaction intermediary between the user and the display device 200.

In some embodiments, the communication interface 130 is used for external communication, and includes at least one of a WIFI chip, a bluetooth module, NFC, or an alternative module.

In some embodiments, the user input/output interface 140 includes at least one of a microphone, a touchpad, a sensor, a key, or an alternative module.

Fig. 4 illustrates a hardware configuration block diagram of the display apparatus 200 according to some embodiments. As shown in fig. 4, the display apparatus 200 includes at least one of a tuner demodulator 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output interface 270, a rotating component 276, a memory, a power supply, and a user interface.

The rotating assembly 276 may include a driving motor, a rotating shaft, and the like. Wherein, the driving motor can be connected to the controller 250 and output the rotation angle under the control of the controller 250; one end of the rotating shaft is connected to a power output shaft of the driving motor, and the other end is connected to the display 260, so that the display 260 can be fixedly mounted on a wall or a bracket through the rotating assembly 276.

The rotating assembly 276 may also include other components, such as a transmission component, a detection component, and the like. Wherein, the transmission component can adjust the rotating speed and the torque output by the rotating component 276 through a specific transmission ratio, and can be in a gear transmission mode; the detection means may be composed of a sensor, such as an angle sensor, an attitude sensor, or the like, provided on the rotation shaft. These sensors may detect parameters such as the angle at which the rotating assembly 276 is rotated and transmit the detected parameters to the controller 250, so that the controller 250 can determine or adjust the state of the display apparatus 200 according to the detected parameters. In practice, rotating assembly 276 may include, but is not limited to, one or more of the components described above.

In some embodiments, the controller includes a central processor, a video processor, an audio processor, a graphic processor, a RAM, a ROM, a first interface to an nth interface for input/output.

In some embodiments, the display 260 includes a display screen component for displaying pictures, and a driving component for driving image display, a component for receiving image signals from the controller output, displaying video content, image content, and menu manipulation interface, and a user manipulation UI interface, etc.

In some embodiments, the display 260 may be at least one of a liquid crystal display, an OLED display, and a projection display, and may also be a projection device and a projection screen.

In some embodiments, the tuner demodulator 210 receives broadcast television signals via wired or wireless reception, and demodulates audio/video signals, such as EPG data signals, from a plurality of wireless or wired broadcast television signals.

In some embodiments, communicator 220 is a component for communicating with external devices or servers according to various communication protocol types. For example: the communicator may include at least one of a Wifi module, a bluetooth module, a wired ethernet module, and other network communication protocol chips or near field communication protocol chips, and an infrared receiver. The display apparatus 200 may establish transmission and reception of control signals and data signals with the control device 100 or the server 400 through the communicator 220.

In some embodiments, the detector 230 is used to collect signals of the external environment or interaction with the outside. For example, detector 230 includes a light receiver, a sensor for collecting ambient light intensity; alternatively, the detector 230 includes an image collector, such as a camera, which may be used to collect external environment scenes, attributes of the user, or user interaction gestures, or the detector 230 includes a sound collector, such as a microphone, which is used to receive external sounds.

In some embodiments, the external device interface 240 may include, but is not limited to, the following: high Definition Multimedia Interface (HDMI), analog or data high definition component input interface (component), composite video input interface (CVBS), USB input interface (USB), RGB port, and the like. The interface may be a composite input/output interface formed by the plurality of interfaces.

In some embodiments, the controller 250 and the modem 210 may be located in different separate devices, that is, the modem 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

In some embodiments, the controller 250 controls the operation of the display device and responds to user operations through various software control programs stored in memory. The controller 250 controls the overall operation of the display apparatus 200. For example: in response to receiving a user command for selecting a UI object to be displayed on the display 260, the controller 250 may perform an operation related to the object selected by the user command.

In some embodiments, the object may be any one of selectable objects, such as a hyperlink, an icon, or other actionable control. The operations related to the selected object are: displaying an operation connected to a hyperlink page, document, image, or the like, or performing an operation of a program corresponding to the icon.

In some embodiments, the controller includes at least one of a Central Processing Unit (CPU), a video processor, an audio processor, a Graphic Processing Unit (GPU), a RAM Random Access Memory (RAM), a ROM (Read-Only Memory), a first interface to an nth interface for input/output, a communication Bus (Bus), and the like.

And the CPU is used for executing the operating system and the application program instructions stored in the memory and executing various application programs, data and contents according to various interaction instructions for receiving external input so as to finally display and play various audio and video contents. The CPU processor may include a plurality of processors. E.g. comprising a main processor and one or more sub-processors.

In some embodiments, a graphics processor for generating various graphics objects, such as: at least one of an icon, an operation menu, and a user input instruction display figure. The graphic processor comprises an arithmetic unit, which performs operation by receiving various interactive instructions input by a user and displays various objects according to display attributes; the system also comprises a renderer for rendering various objects obtained based on the arithmetic unit, wherein the rendered objects are used for being displayed on a display.

In some embodiments, the video processor is configured to receive an external video signal, and perform at least one of video processing such as decompression, decoding, scaling, noise reduction, frame rate conversion, resolution conversion, and image synthesis according to a standard codec protocol of the input signal, so as to obtain a signal displayed or played on the direct display device 200.

In some embodiments, the video processor includes at least one of a demultiplexing module, a video decoding module, an image composition module, a frame rate conversion module, a display formatting module, and the like. The demultiplexing module is used for demultiplexing the input audio and video data stream. And the video decoding module is used for processing the video signal after demultiplexing, including decoding, scaling and the like. And the image synthesis module is used for carrying out superposition mixing processing on the GUI signal input by the user or generated by the user and the video image after the zooming processing by the graphic generator so as to generate an image signal for display. And the frame rate conversion module is used for converting the frame rate of the input video. And the display formatting module is used for converting the received video output signal after the frame rate conversion, and changing the signal to be in accordance with the signal of the display format, such as an output RGB data signal.

In some embodiments, the audio processor is configured to receive an external audio signal, decompress and decode the received audio signal according to a standard codec protocol of the input signal, and perform at least one of noise reduction, digital-to-analog conversion, and amplification processing to obtain a sound signal that can be played in the speaker.

In some embodiments, a user may enter user commands on a Graphical User Interface (GUI) displayed on display 260, and the user input interface receives the user input commands through the Graphical User Interface (GUI). Alternatively, the user may input the user command by inputting a specific sound or gesture, and the user input interface receives the user input command by recognizing the sound or gesture through the sensor.

In some embodiments, a "user interface" is a media interface for interaction and information exchange between an application or operating system and a user that enables conversion between an internal form of information and a form that is acceptable to the user. A commonly used presentation form of the User Interface is a Graphical User Interface (GUI), which refers to a User Interface related to computer operations and displayed in a graphical manner. It may be an interface element such as an icon, a window, a control, etc. displayed in the display screen of the electronic device, where the control may include at least one of an icon, a button, a menu, a tab, a text box, a dialog box, a status bar, a navigation bar, a Widget, etc. visual interface elements.

In some embodiments, user interface 280 is an interface that may be used to receive control inputs (e.g., physical buttons on the body of the display device, or the like).

In some embodiments, a system of a display device may include a Kernel (Kernel), a command parser (shell), a file system, and an application program. The kernel, shell, and file system together make up the basic operating system structure that allows users to manage files, run programs, and use the system. After power-on, the kernel is started, kernel space is activated, hardware is abstracted, hardware parameters are initialized, and virtual memory, a scheduler, signals and interprocess communication (IPC) are operated and maintained. And after the kernel is started, loading the Shell and the user application program. The application program is compiled into machine code after being started, and a process is formed.

As shown in fig. 5, the system of the display device may include a Kernel (Kernel), a command parser (shell), a file system, and an application program. The kernel, shell, and file system together make up the basic operating system structure that allows users to manage files, run programs, and use the system. After power-on, the kernel is started, kernel space is activated, hardware is abstracted, hardware parameters are initialized, and virtual memory, a scheduler, signals and interprocess communication (IPC) are operated and maintained. And after the kernel is started, loading the Shell and the user application program. The application program is compiled into machine code after being started, and a process is formed.

As shown in fig. 5, the system of the display device is divided into three layers, i.e., an application layer, a middleware layer and a hardware layer from top to bottom.

The Application layer mainly includes common applications on the television and an Application Framework (Application Framework), wherein the common applications are mainly applications developed based on the Browser, such as: HTML5 APPs; and Native APPs (Native APPs);

an Application Framework (Application Framework) is a complete program model, and has all basic functions required by standard Application software, such as: file access, data exchange …, and interfaces (toolbars, status bars, menus, dialog boxes) for use of these functions.

Native APPs (Native APPs) may support online or offline, message push, or local resource access.

The middleware layer comprises various television protocols, multimedia protocols, system components and other middleware. The middleware can use basic service (function) provided by system software to connect each part of an application system or different applications on a network, and can achieve the purposes of resource sharing and function sharing.

The hardware layer mainly comprises an HAL interface, hardware and a driver, wherein the HAL interface is a unified interface for butting all the television chips, and specific logic is realized by each chip. The driving mainly comprises: audio drive, display driver, bluetooth drive, camera drive, WIFI drive, USB drive, HDMI drive, sensor drive (like fingerprint sensor, temperature sensor, pressure sensor etc.) and power drive etc..

The screen projection function enables a screen on the mobile terminal to be shared on the display device 200 for display, so that a user can view the content currently displayed on the mobile terminal on the display device 200 in real time.

At present, most of screen projection methods are that after the mobile terminal transmits data of a display screen to the display device 200, the display device 200 displays the data. Generally, the data sent by the mobile terminal to the display device 200 are all cross-screen, and if the mobile terminal displays the cross-screen, the data themselves represent the cross-screen picture at the moment; if the mobile terminal displays the vertical screen, the mobile terminal can add some filling data in the data of the original vertical screen picture so as to add filling pictures on the left side and the right side of the vertical screen picture, and the filled whole picture is a horizontal screen picture. The filler picture has no actual content, and is usually black in order to not affect the content of the original portrait screen. Such as shown in fig. 6. The filling manner may also be referred to as a blackening process.

However, leaving a large amount of black on the display device 200 may cause a problem of poor image picture display effect, thereby affecting the viewing experience of the user.

The present embodiment provides a display device 200, which specifically includes a display 260, a rotating assembly 276, and a controller 250. The display 260 may display a screen projection picture sent by the mobile terminal, and the rotating component 276 may be controlled by the controller 250 to further drive the display 260 to rotate to change a display state of the display 260, where the display state of the display 260 includes a landscape screen state or a portrait screen state.

The display device in this embodiment may be a rotating television, a rotatable computer, a tablet computer, or the like. And the mobile terminal can be a smart phone, a tablet computer and other devices with a display function.

The screen projection data sent by the mobile terminal with vertical screen display to the display device 200 usually includes valid data for representing a display screen and invalid data for filling the screen; and the projection data transmitted from the mobile terminal for landscape display to the display device 200 may include only valid data representing the display screen.

The display device 200 cannot directly determine whether the mobile terminal is in a horizontal screen display or a vertical screen display, or whether a screen projection picture sent by the mobile terminal is a horizontal screen picture or a vertical screen picture. Furthermore, effective data is extracted from the screen projection data, and the effective data has the specific size of the screen projection picture, so that the display device 200 can judge whether the screen projection picture is a horizontal screen picture or a vertical screen picture according to the specific size of the screen projection picture in the effective data.

When the length-height ratio in the specific size is greater than 1, namely the length/height is greater than 1, the screen projection picture is a transverse screen picture; and when the length-height ratio in the specific size is less than 1, namely the length/height is less than 1, the screen projection picture is a vertical screen picture.

In order to enable the display device 200 to better adapt to the screen-projected picture, after determining the type of the screen-projected picture, the display device 200 further needs to adjust the display state of the display 260 so that the display state of the display 260 matches with the picture type of the screen-projected picture.

Fig. 7 is a schematic view illustrating a process of sending screen projection data to the horizontal screen display device 200 by the vertical screen mobile terminal according to some embodiments, and as shown in fig. 7, after the display device 200 determines that a screen projection picture is a vertical screen picture according to valid data, the current horizontal screen state is rotated to a vertical screen state, so as to match the vertical screen picture.

Fig. 8 is a schematic view illustrating a process of sending screen projection data to the vertical screen display device 200 by the horizontal screen mobile terminal according to some embodiments, and as shown in fig. 8, after determining that a screen projection picture is a horizontal screen picture according to the valid data, the display device 200 may rotate from a current vertical screen state to a horizontal screen state, so as to match the horizontal screen picture.

Aiming at a vertical screen picture to be projected by a vertical screen mobile terminal, in order to change the vertical screen picture into a horizontal screen picture, the mobile terminal needs to fill pure-color pictures at the left side and the right side of the vertical screen picture so as to change the filled pictures into the horizontal screen picture, wherein the filled pure-color picture data is invalid data, and the data with picture content is valid data.

If the display device 200 directly displays the projection data with invalid data, a pure color picture is displayed on the left and right sides of the picture corresponding to the valid data, and this display manner obviously makes the picture size of the valid data too small compared with the size of the display 260, thereby affecting the user to view the picture content. Meanwhile, the filled pure color picture also affects the viewing experience of the user.

In the embodiment of the present application, after the display state of the display 260 is matched with the picture type, if the screen projection data has invalid data, the display device 200 further needs to cut the invalid data in the screen projection data, and only the valid data is retained for displaying. And further, the influence and interference of invalid data on valid data can be avoided.

In the above-described process of screen projection data processing, the controller 250 of the display device 200 may be configured to: firstly, screen projection data sent by a mobile terminal are received. Then, effective data used for representing the display screen of the mobile terminal in the screen projection data is utilized to determine the type of the screen required to be projected by the mobile terminal. Finally, in the case that the screen type is a vertical screen, it is determined whether the display 260 needs to be controlled to rotate according to the display state of the display 260 so that the display state of the display 260 matches the screen type, and invalid data in the screen projection data is cut.

Fig. 9 is a schematic diagram illustrating the display device 200 according to some embodiments, and as shown in fig. 9, after the mobile terminal such as a mobile phone sends the screen projection data to the display device 200 such as a rotating television, the controller 250 extracts effective data from the screen projection data, and then determines the type of the picture to be projected and determines the current display state of the display 260 by using the effective data. Judging the current display state of the display 260 under the condition that the picture type is a horizontal screen picture, and if the current display state of the display 260 is a vertical screen state, the controller 250 needs to rotate the display 260 to the horizontal screen state; if the current display state of the display 260 is the landscape state, the controller 250 does not need to rotate the display 260. Judging the current display state of the display 260 under the condition that the picture type is a vertical screen picture, and if the current display state of the display 260 is a horizontal screen state, the controller 250 needs to rotate the display 260 to a vertical screen state; if the current display state of the display 260 is the portrait state, the controller 250 does not need to rotate the display 260.

Since the screen projection data includes invalid data when the picture type is a vertical screen picture, the controller 250 needs to cut the invalid data in the screen projection data and retain the valid data for displaying no matter whether the display 260 needs to be rotated or not when the picture type is a vertical screen picture.

However, when the frame type is a horizontal screen, the projection data only includes valid data, and therefore, when the frame type is a vertical screen, the controller 250 does not need to cut invalid data in the projection data regardless of whether the display 260 needs to be rotated.

After trimming the invalid data, the display apparatus 200 displays a screen-projected picture using only the valid data; or when the screen projection data does not include invalid data, the display device 200 directly displays the screen projection screen using the screen projection data.

In some embodiments, whether the horizontal-screen mobile terminal or the vertical-screen mobile terminal is used, the actual size of the screen-projected picture corresponding to the valid data in the screen-projected data may not be the same as the size of the screen of the display 260, that is, the screen-projected picture corresponding to the valid data may not be displayed on the display 260 in a full screen. Therefore, the display device 200 may also appropriately zoom the screen projection according to the size of the display 260, so that the screen projection is displayed in a full screen, thereby enhancing the viewing experience of the user.

In the embodiment of the present application, the frame playing a filling role is an invalid frame, which corresponds to invalid data. The ineffective picture is a pure color picture, such as black, gray, etc. In general, a specific color of the invalid screen is set by the mobile terminal, and the display device 200 is notified of the specific color.

In the embodiment of the present application, there are various ways for the display device 200 to extract valid data, for example, a color recognition way is used to recognize invalid data first, and then the rest of the projected data is valid data.

In some embodiments, the display device 200 may determine the color of an invalid screen, i.e., a target color, in the screens projected by the mobile terminal. And performing an and operation by using the target color image and the screen projection picture, wherein the size of the target color image can be the same as or larger than that of the screen projection picture. And the operation is to identify a target area with the same color as the target in the screen projection picture, and determine the target area as an invalid picture, wherein the data corresponding to the invalid picture is invalid data. Finally, the display device 200 may determine data other than the invalid data in the screen projection data as valid data, thereby completing the extraction process of the valid data.

FIG. 10 illustrates a schematic diagram of the display device 200 extracting valid data according to some embodiments. In the above-described valid data extraction process, as shown in fig. 10, the controller 250 of the display apparatus 200 may be further configured to: first, a color of an invalid screen for filling a display screen of the mobile terminal is determined as a target color. And then, performing sum operation on the screen projection picture corresponding to the screen projection data and the target color image, and identifying a target area corresponding to the target color in the screen projection picture. And finally, determining that the data corresponding to the target area is invalid data, and determining that the data except the invalid data in the screen projection data is valid data.

In some embodiments, the display device 200 may also determine a color of an invalid picture, i.e., a target color, in the pictures projected by the mobile terminal. And then, directly identifying a target area corresponding to the target color from the screen projection picture by using an image identification algorithm, and determining the target area as an invalid picture, wherein data corresponding to the invalid picture is invalid data. Finally, the display device 200 may determine data other than the invalid data in the screen projection data as valid data, thereby completing the extraction process of the valid data.

FIG. 11 illustrates another schematic diagram of the display device 200 extracting valid data according to some embodiments. In the above-described valid data extraction process, as shown in fig. 11, the controller 250 of the display apparatus 200 may be further configured to: first, a color of an invalid screen for filling a display screen of the mobile terminal is determined as a target color. And then, identifying a target area corresponding to the target color from a screen projection picture corresponding to the screen projection data by using an image identification algorithm. And finally, determining that the data corresponding to the target area is invalid data, and determining that the data except the invalid data in the screen projection data is valid data.

In some embodiments, the foregoing manner of extracting valid data may also be regarded as a region division process for an invalid picture and a valid picture, where the valid picture is a picture corresponding to the valid data. If the contents displayed on the mobile terminal all belong to the contents of the same resource, when the mobile terminal is always in the vertical screen state or the horizontal screen state, the resolution of each frame of image in the resource is generally the same, i.e. the size is the same. The area range or size of the active picture should also be the same.

For example, when the mobile terminal displays the video XX on the vertical screen, the resolution of each frame of image in the video displayed on the mobile terminal is 1080 × 2430. In this case, when the mobile terminal projects the video image onto the display device 200, the display device 200 only needs to determine the range and size of the effective image displayed on the display device 200 for the first frame image, and since the resolution of each frame image is the same, the ranges of the effective images corresponding to the images of the other frames belonging to the same video "XX" are the same as those of the first frame image, and the display device 200 can directly determine the effective image of each subsequent frame image and the effective data corresponding to the effective image by using the determined effective image range.

However, when the contents displayed on the mobile terminal before and after do not belong to the same resource content, the resolution of the screen projected by the mobile terminal twice before and after to the display device 200 may be considered to be different, that is, the size is different no matter the mobile terminal is in the vertical screen state or the horizontal screen state. The area range or size of the effective picture is also different.

For example, the mobile terminal first projects a picture of a video "YYY" displayed in a vertical screen onto the display device 200, the resolution of each frame of image displayed on the mobile terminal is 1080 × 2430, and the display device 200 refers to fig. 12 for the range of valid pictures determined by the projected picture of the video "YYY". Then, the mobile terminal projects the picture 1 displayed in the vertical screen to the display device 200, the resolution of the picture 1 displayed on the mobile terminal is 1280 × 720, and the display device 200 refers to fig. 13 for the range of the effective picture determined by the projection picture of the picture 1. Since the video image and the picture 1 are respectively projected by the mobile terminal twice, the display device 200 needs to determine the area of the effective picture and the effective data again when displaying the projected picture of the picture 1.

In the above-described process of extracting effective data using the effective screen division, the controller 250 of the display apparatus 200 may be further configured to: firstly, whether resource content to which screen projection data sent by a mobile terminal belongs is changed or not is determined. And under the condition that the resource content is changed, re-determining effective data in the screen projection data. In the case that the resource content is not changed, effective data in the screen projection data does not need to be determined again.

In some embodiments, the screen of the display device 200 is clickable or touch operable. The user can select pictures belonging to different resources to display by sliding the content on the screen or clicking the content according to the requirements of the user. The display device 200 may also determine whether the resolution, the size, or the like of the current screen projection screen changes by determining whether the user operates the screen, and thus determine whether the area of the effective screen on the screen projection screen needs to be determined.

For example, after the mobile terminal projects a video picture to the display device 200, the user performs a sliding operation on the screen of the display device 200 to control the currently displayed picture to be changed into a picture associated with the projected video, and since the picture does not belong to the same resource content as the projected video, the display device 200 needs to determine an effective picture of the projected picture and effective data again after the effective picture of the video projected picture is determined.

In some embodiments, display device 200, while a rotatable display device, may also be present where a user does not wish to rotate display 260. Further, before rotating the display 260, the display device 200 may also issue a prompt to the user, which is displayed on the display 260 in a prompt message manner for reminding the user whether the display 260 needs to be rotated. If the user agrees, a confirmation instruction may be input to the display device 200, and the display device 200 may continue to rotate the display 260 to the landscape state or the portrait state; if the user does not input any instruction or input a cancel instruction, the display device 200 may continue to display the screen shot of the mobile terminal in the current display state of the display 260.

Referring to fig. 14, fig. 14 shows a schematic diagram of a prompt on a display device 200 according to some embodiments, which may be, for example, "a display is rotated to a landscape state to present a better display effect, is it rotated? "etc., and controls such as" confirm "and" cancel "may also be provided within the prompt box containing the prompt information.

In the embodiment of the present application, a user may input a confirmation instruction or a cancel instruction to the display device 200 through the remote controller or the like by controlling the display device 100, and the specific method is as follows: the user positions the direction key control focus on the prompt message to the 'confirm' control by pressing the direction key on the control device 100, and then presses the confirm key on the control device 100 to input a confirm instruction to the display device 200; alternatively, the user may input a cancel instruction to the display device 200 by pressing a direction key control focus on the control apparatus 100 to position the "cancel" control on the prompt message, and then pressing a confirmation key on the control apparatus 100.

In the above-described process of rotating the display 260 according to the prompt information, the control of the display device 200 may be further configured to: first, before the control display 260 is rotated, the control display 260 displays a prompt message to the user. Then, in a case where a confirmation instruction input to the display apparatus 200 by the user is received, the display 260 is controlled to rotate.

As can be seen from the above, the display device 200 in the embodiment of the present application may be a rotary display device. After receiving the screen projection data sent by the mobile terminal, the display device 200 extracts effective data in the screen projection data, and determines whether a picture to be projected is a horizontal screen picture or a vertical screen picture by using the effective data. Then, when the screen projection picture is the vertical screen picture, the display device 200 rotates the display device currently in the horizontal screen state to the vertical screen state, and cuts the invalid data in the screen projection data, and only displays the picture corresponding to the valid data. Or, in the case that the screen projection picture is a vertical screen picture, if the display device 200 is currently in a vertical screen state, the display device 200 may directly crop invalid data in the screen projection data, and only display a picture corresponding to the valid data. Further, the problem of black display when the screen projection picture is displayed on the display device 200 can be avoided, and the use experience of the user is guaranteed.

In order to solve the problem that the screen projection picture cannot adapt to the screen size of the display device, an embodiment of the present application further provides a method for processing screen projection data, where the method may be applied to the display device 200 in the foregoing embodiment, and specifically may include the following steps: receiving screen projection data sent by a mobile terminal; determining the type of a picture to be projected by the mobile terminal by utilizing effective data which is used for representing a display picture of the mobile terminal in the screen projection data; in the case that the screen type is a vertical screen, it is determined whether it is necessary to control the display 260 to rotate according to the display state of the display 260 so that the display state of the display 260 matches the screen type while trimming invalid data in the screen projection data.

In some embodiments, the method may further comprise: detecting the current display state of the display 260 under the condition that the picture type is a vertical screen picture; under the condition that the display 260 is in the horizontal screen state at present, controlling the display 260 to rotate to the vertical screen state, and meanwhile cutting invalid data in the screen projection data; in the case where the display 260 is currently in the portrait state, invalid data in the screen projection data is clipped.

In some embodiments, the method may further comprise: and under the condition that the picture type is a vertical screen picture and the display is in a horizontal screen state currently, displaying the screen projection picture on the display 260 which is rotated to be in the vertical screen state by using the residual effective data after being cut in the screen projection data.

In some embodiments, the method may further comprise: and under the condition that the picture type is a vertical screen picture and the display 260 is in a vertical screen state at present, displaying the screen projection picture on the display 260 in the vertical screen state at present by using the residual effective data after being cut in the screen projection data.

In some embodiments, the method may further comprise: detecting the current display state of the display 260 under the condition that the picture type is a landscape screen picture; displaying a screen projection picture on the display 260 by using the screen projection data under the condition that the display 260 is currently in the landscape state; controlling the display 260 to rotate to a landscape state in a case where the display 260 is currently in a portrait state; the screen projection data is used to display a screen projection screen on the display 260 in the landscape state.

In some embodiments, the method may further comprise: before controlling the display 260 to rotate, controlling the display 260 to display a prompt to the user; in the case where a confirmation instruction input to the display apparatus 200 by the user is received, the display 260 is controlled to rotate.

In some embodiments, the method may further comprise: determining the color of an invalid picture for filling the display picture of the mobile terminal as a target color; performing sum operation on a screen projection picture corresponding to the screen projection data and a target color image, and identifying a target area corresponding to the target color in the screen projection picture; and determining that the data corresponding to the target area is invalid data, and determining that the data except the invalid data in the screen projection data is valid data.

In some embodiments, the method may further comprise: determining the color of an invalid picture for filling the display picture of the mobile terminal as a target color; identifying a target area corresponding to a target color from a screen projection picture corresponding to the screen projection data by using an image identification algorithm; and determining that the data corresponding to the target area is invalid data, and determining that the data except the invalid data in the screen projection data is valid data.

In some embodiments, the method may further comprise: determining whether resource content of screen projection data sent by a mobile terminal is changed; and in the case of the change of the resource content, re-determining the effective data in the screen projection data.

As the method for processing screen projection data in the embodiment of the present application can be applied to the display device 200 in the foregoing embodiment, other contents related to the method for processing screen projection data in the embodiment of the present application can refer to the contents of the foregoing embodiment, and are not described herein again.

Finally, it should be noted that: the above embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present application.

The foregoing description, for purposes of explanation, has been presented in conjunction with specific embodiments. However, the illustrative discussions above are not intended to be exhaustive or to limit the embodiments to the precise forms disclosed above. Many modifications and variations are possible in light of the above teaching. The embodiments were chosen and described in order to best explain the principles and the practical application, to thereby enable others skilled in the art to best utilize the embodiments and various embodiments with various modifications as are suited to the particular use contemplated.

Claims

1. A display device, comprising:

a display;

a controller configured to:

receiving screen projection data sent by a mobile terminal;

2. The display device of claim 1, wherein the controller is further configured to:

3. The display device of claim 2, wherein the controller is further configured to:

4. The display device of claim 2, wherein the controller is further configured to:

5. The display device of claim 1, wherein the controller is further configured to:

6. The display device of claim 1, wherein the controller is further configured to:

7. The display device of claim 1, wherein the controller is further configured to:

8. The display device of claim 1, wherein the controller is further configured to:

9. The display device of claim 1, wherein the controller is further configured to:

10. A method for processing screen projection data, which is characterized by comprising the following steps:

receiving screen projection data sent by a mobile terminal;