CN115705129A

CN115705129A - Display device and window background display method

Info

Publication number: CN115705129A
Application number: CN202110898231.9A
Authority: CN
Inventors: 王光强; 李荣华
Original assignee: Juhaokan Technology Co Ltd
Current assignee: Juhaokan Technology Co Ltd
Priority date: 2021-08-05
Filing date: 2021-08-05
Publication date: 2023-02-17

Abstract

The application provides a display device and a window background display method, wherein the method can detect color values of a plurality of target points in a current user interface after a user inputs a control instruction for displaying a menu window, and screen out color picking points for calculating background colors from the detected color values, so that the background of the menu window is set according to the colors corresponding to the color picking points. According to the method, when the menu window is displayed, the background color can be adjusted according to the content displayed in the bottom user interface, so that the background form of the menu window is richer, the coordination between the menu window and the user interface is improved, and the user experience is improved.

Description

Display device and window background display method

Technical Field

The present application relates to the field of display device technologies, and in particular, to a display device and a window background display method.

Background

The display device is a terminal device capable of outputting a specific display picture, and may be a terminal device such as a smart television, a communication terminal, a smart advertisement screen, and a projector. Taking an intelligent television as an example, the intelligent television is based on an Internet application technology, is provided with an open operating system and a chip, is provided with an open application platform, can realize a bidirectional man-machine interaction function, integrates multiple functions of video, entertainment, data and the like, and is a television product for meeting diversified and personalized requirements of users.

The display device may display various User Interfaces (UIs) based on the operating system to facilitate User interaction. The user interface may present a menu window when the user enters a particular interactive action. For example, when the user controls the display device to enable the search function, a results window may be displayed in a lower area of the user interface. In the results window, a number of options may be included that are compatible with the graph search function, including image recognition results, search term bars, and the like.

The menu window, as a kind of UI item, may follow a theme style set in the operating system, with a specific display background. For example, when the theme style of the operating system is blue, the background of the menu window may also be blue. However, the background following the theme style of the operating system has a single form, and the color of the background of the menu window is inconsistent with that of the user interface under part of the user interface, so that the user experience is reduced.

Disclosure of Invention

The application provides display equipment and a window background display method, and aims to solve the problems that the traditional window background is single in form, and the window background and the user interface are not coordinated in color.

In one aspect, the present application provides a display device, comprising: a display, a communicator, and a controller. Wherein the display is configured to display a user interface; the communicator is configured to establish a communication connection with a server; the controller is configured to perform the following program steps:

acquiring a control instruction which is input by a user and used for displaying a menu window;

detecting color values of a plurality of target points in a current user interface in response to the control instruction;

acquiring a background color, wherein the background color is generated by the controller or the server according to color values of color points acquired from a plurality of target points, and the color points are the target points with color difference values smaller than a first color difference threshold value, which are determined by comparing the color values of the plurality of target points;

and setting the background of the menu window according to the background color.

Optionally, the display device further comprises a communicator configured to establish a communication connection with a server, and the controller is further configured to execute the following program steps:

after a background picture is obtained, sending the background picture to the server to trigger the server to execute image recognition on the background picture;

receiving an image recognition result fed back by the server aiming at the background picture, wherein the image recognition result comprises a characteristic target;

dividing a color taking area in the background picture according to the image identification result, wherein the color taking area is an area where the characteristic target is located in the background picture or an area outside the area where the characteristic target is located;

and adding a plurality of target points in the color sampling area, wherein the target points are uniformly distributed in the color sampling area.

On the other hand, the application also provides a window background display method, which is applied to the display device, and the window background display method comprises the following steps:

obtaining a background color, wherein the background color is generated according to color values of color points obtained from a plurality of target points, and the color point obtained is the target point of which the color difference value determined by comparing the color values of the plurality of target points is smaller than a first color difference threshold value;

According to the technical scheme, the display device and the window background display method can detect the color values of a plurality of target points in the current user interface after a user inputs a control instruction for displaying the menu window, and screen out the color picking points for calculating the background color from the detected color values, so that the background of the menu window is set according to the color corresponding to the color picking points. According to the method, when the menu window is displayed, the background color can be adjusted according to the content displayed in the bottom user interface, so that the background form of the menu window is richer, the coordination between the menu window and the user interface is improved, and the user experience is improved.

Drawings

In order to more clearly explain the technical solution of the present application, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious to those skilled in the art that other drawings can be obtained according to the drawings without creative efforts.

FIG. 1 is a schematic diagram of an operation scenario between a display device and a control apparatus in an embodiment of the present application;

fig. 2 is a block diagram of a hardware configuration of a display device in an embodiment of the present application;

FIG. 3 is a schematic diagram of a software configuration of a display device in an embodiment of the present application;

FIG. 4 is a schematic diagram of an icon control interface display of an application program of a display device in an embodiment of the present application;

FIG. 5 is a schematic diagram of a home page in an embodiment of the present application;

FIG. 6 is a schematic view of a graphical interface in an embodiment of the present application;

FIG. 7 is a schematic diagram of a menu window according to an embodiment of the present application;

FIG. 8 is a schematic diagram of another menu window in the embodiment of the present application;

FIG. 9 is a flowchart illustrating a window background display method according to an embodiment of the present application;

FIG. 10 is a schematic view of target points in an embodiment of the present application;

FIG. 11 is a schematic diagram of a color point screening process in the embodiment of the present application;

FIG. 12 is a schematic flow chart of dividing color areas in the embodiment of the present application;

fig. 13 is a schematic flow chart illustrating color filling for the fixed color region in the embodiment of the present application.

Detailed Description

To make the purpose and embodiments of the present application clearer, the following will clearly and completely describe the exemplary embodiments of the present application with reference to the attached drawings in the exemplary embodiments of the present application, and it is obvious that the described exemplary embodiments are only a part of the embodiments of the present application, and not all of the embodiments.

It should be noted that the brief descriptions of the terms in the present application are only for convenience of understanding of the embodiments described below, and are not intended to limit the embodiments of the present application. These terms should be understood in their ordinary and customary meaning unless otherwise indicated.

The terms "first," "second," "third," and the like in the description and claims of this application and in the above-described drawings are used for distinguishing between similar or analogous objects or entities and not necessarily for describing a particular sequential or chronological order, unless otherwise indicated. It is to be understood that the terms so used are interchangeable under appropriate circumstances.

The terms "comprises" and "comprising," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a product or apparatus that comprises a list of elements is not necessarily limited to all elements expressly listed, but may include other elements not expressly listed or inherent to such product or apparatus.

The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware or/and software code that is capable of performing the functionality associated with that element.

Fig. 1 is a schematic diagram of an operation scenario between a display device and a control apparatus according to an embodiment. As shown in fig. 1, a user may operate the display apparatus 200 through the smart device 300 or the control device 100.

In some embodiments, the control apparatus 100 may be a remote controller, and the communication between the remote controller and the display device includes an infrared protocol communication or a bluetooth protocol communication, and other short-distance communication methods, and controls the display device 200 in a wireless or wired manner. The user may input a user instruction through a key on a remote controller, voice input, control panel input, etc., to control the display apparatus 200.

In some embodiments, the smart device 300 (e.g., a communication terminal, a tablet, a computer, a laptop, etc.) may also be used to control the display device 200. For example, the display device 200 is controlled using an application program running on the smart device.

In some embodiments, the display device 200 may also be controlled in a manner other than the control apparatus 100 and the smart device 300, for example, the voice command control of the user may be directly received by a module configured inside the display device 200 to obtain a voice command, or may be received by a voice control device provided outside the display device 200.

In some embodiments, the display device 200 is also in data communication with a server 400. The display device 200 may be allowed to be communicatively connected through a Local Area Network (LAN), a Wireless Local Area Network (WLAN), and other networks. The server 400 may provide various contents and interactions to the display apparatus 200. The server 400 may be a cluster or a plurality of clusters, and may include one or more types of servers.

Fig. 2 exemplarily shows a block diagram of a configuration of the control apparatus 100 according to an exemplary embodiment. As shown in fig. 2, the control device 100 includes a controller 110, a communication interface 130, a user input/output interface 140, a memory, and a power supply. The control apparatus 100 may receive an input operation instruction from a user and convert the operation instruction into an instruction recognizable and responsive by the display device 200, serving as an interaction intermediary between the user and the display device 200.

Fig. 3 illustrates a hardware configuration block diagram of the display apparatus 200 according to an exemplary embodiment.

In some embodiments, the display apparatus 200 includes at least one of a tuner 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output interface 270, a memory, a power supply, and a user interface.

In some embodiments the controller comprises a processor, a video processor, an audio processor, a graphics processor, a RAM, a ROM, a first interface to an nth interface for input/output.

In some embodiments, the display 260 includes a display screen component for presenting a picture, and a driving component for driving an image display, a component for receiving an image signal from the controller output, performing display of video content, image content, and a menu manipulation interface, and a user manipulation UI interface.

In some embodiments, the display 260 may be a liquid crystal display, an OLED display, and a projection display, and may also be a projection device and a projection screen.

In some embodiments, communicator 220 is a component for communicating with external devices or servers according to various communication protocol types. For example: the communicator may include at least one of a Wifi module, a bluetooth module, a wired ethernet module, and other network communication protocol chips or near field communication protocol chips, and an infrared receiver. The display apparatus 200 may establish transmission and reception of control signals and data signals with the external control apparatus 100 or the server 400 through the communicator 220.

In some embodiments, the user interface may be configured to receive control signals for controlling the apparatus 100 (e.g., an infrared remote control, etc.).

In some embodiments, the detector 230 is used to collect signals of the external environment or interaction with the outside. For example, detector 230 includes a light receiver, a sensor for collecting the intensity of ambient light; alternatively, the detector 230 includes an image collector, such as a camera, which may be used to collect external environment scenes, attributes of the user, or user interaction gestures, or the detector 230 includes a sound collector, such as a microphone, which is used to receive external sounds.

In some embodiments, the external device interface 240 may include, but is not limited to, the following: high Definition Multimedia Interface (HDMI), analog or data high definition component input interface (component), composite video input interface (CVBS), USB input interface (USB), RGB port, and the like. The interface may be a composite input/output interface formed by the plurality of interfaces.

In some embodiments, the tuner demodulator 210 receives broadcast television signals via wired or wireless reception, and demodulates audio/video signals, such as EPG data signals, from a plurality of wireless or wired broadcast television signals.

In some embodiments, the controller 250 and the modem 210 may be located in different separate devices, that is, the modem 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

In some embodiments, the controller 250 controls the operation of the display device and responds to user operations through various software control programs stored in memory. The controller 250 controls the overall operation of the display apparatus 200. For example: in response to receiving a user command for selecting a UI object to be displayed on the display 260, the controller 250 may perform an operation related to the object selected by the user command.

In some embodiments, the object may be any one of selectable objects, such as a hyperlink, an icon, or other actionable control. The operations related to the selected object are: displaying an operation connected to a hyperlink page, document, image, or the like, or performing an operation of a program corresponding to the icon.

In some embodiments the controller comprises at least one of a Central Processing Unit (CPU), a video processor, an audio processor, a Graphics Processing Unit (GPU), a RAM Random Access Memory (RAM), a ROM (Read-Only Memory), a first to nth interface for input/output, a communication Bus (Bus), and the like.

A CPU processor. For executing operating system and application program instructions stored in the memory, and executing various application programs, data and contents according to various interactive instructions receiving external input, so as to finally display and play various audio-video contents. The CPU processor may include a plurality of processors. E.g. comprising a main processor and one or more sub-processors.

In some embodiments, a graphics processor for generating various graphics objects, such as: icons, operation menus, user input instruction display graphics, and the like. The graphic processor comprises an arithmetic unit, which performs operation by receiving various interactive instructions input by a user and displays various objects according to display attributes; the system also comprises a renderer for rendering various objects obtained based on the arithmetic unit, wherein the rendered objects are used for being displayed on a display.

In some embodiments, the video processor is configured to receive an external video signal, and perform video processing such as decompression, decoding, scaling, noise reduction, frame rate conversion, resolution conversion, and image synthesis according to a standard codec protocol of the input signal, so as to obtain a signal that can be displayed or played on the directly displayable device 200.

In some embodiments, the video processor includes a demultiplexing module, a video decoding module, an image synthesis module, a frame rate conversion module, a display formatting module, and the like. The demultiplexing module is used for demultiplexing the input audio and video data stream. And the video decoding module is used for processing the video signal after demultiplexing, including decoding, scaling and the like. And the image synthesis module is used for carrying out superposition mixing processing on the GUI signal input by the user or generated by the user and the video image after the zooming processing by the graphic generator so as to generate an image signal for display. And the frame rate conversion module is used for converting the frame rate of the input video. And the display formatting module is used for converting the received video output signal after the frame rate conversion, and changing the signal to be in accordance with the signal of the display format, such as outputting an RGB data signal.

In some embodiments, the audio processor is configured to receive an external audio signal, decompress and decode the received audio signal according to a standard codec protocol of the input signal, and perform noise reduction, digital-to-analog conversion, and amplification processing to obtain an audio signal that can be played in the speaker.

In some embodiments, a user may enter user commands on a Graphical User Interface (GUI) displayed on display 260, and the user input interface receives the user input commands through the Graphical User Interface (GUI). Alternatively, the user may input a user command by inputting a specific sound or gesture, and the user input interface receives the user input command by recognizing the sound or gesture through the sensor.

In some embodiments, a "user interface" is a media interface for interaction and information exchange between an application or operating system and a user that enables conversion between an internal form of information and a form that is acceptable to the user. A commonly used presentation form of the User Interface is a Graphical User Interface (GUI), which refers to a User Interface related to computer operations and displayed in a graphical manner. It may be an interface element such as an icon, a window, a control, etc. displayed in a display screen of the electronic device, where the control may include a visual interface element such as an icon, a button, a menu, a tab, a text box, a dialog box, a status bar, a navigation bar, a Widget, etc.

In some embodiments, the system of the display device may include a Kernel (Kernel), a command parser (shell), a file system, and an application. The kernel, shell, and file system together make up the basic operating system structure that allows users to manage files, run programs, and use the system. After power-on, the kernel starts, activates kernel space, abstracts hardware, initializes hardware parameters, etc., runs and maintains virtual memory, scheduler, signals and inter-process communication (IPC). And after the kernel is started, loading the Shell and the user application program. The application program is compiled into machine code after being started, and a process is formed.

Referring to fig. 4, in some embodiments, the system is divided into four layers, which are an Application (Applications) layer (abbreviated as "Application layer"), an Application Framework (Application Framework) layer (abbreviated as "Framework layer"), an Android runtime (Android runtime) and system library layer (abbreviated as "system runtime library layer"), and a kernel layer from top to bottom.

In some embodiments, at least one application program runs in the application program layer, and the application programs may be windows (Window) programs carried by an operating system, system setting programs, clock programs or the like; or an application developed by a third party developer. In particular implementations, the application packages in the application layer are not limited to the above examples.

The framework layer provides an Application Programming Interface (API) and a programming framework for the application. The application framework layer includes a number of predefined functions. The application framework layer acts as a processing center that decides to let the applications in the application layer act. The application program can access the resources in the system and obtain the services of the system in execution through the API interface.

As shown in fig. 4, in the embodiment of the present application, the application framework layer includes a manager (Managers), a Content Provider (Content Provider), and the like, where the manager includes at least one of the following modules: an Activity Manager (Activity Manager) is used for interacting with all activities running in the system; the Location Manager (Location Manager) is used for providing the system service or application with the access of the system Location service; a Package Manager (Package Manager) for retrieving various information related to an application Package currently installed on the device; a Notification Manager (Notification Manager) for controlling display and clearing of Notification messages; a Window Manager (Window Manager) is used to manage the icons, windows, toolbars, wallpapers, and desktop components on a user interface.

In some embodiments, the activity manager is used to manage the lifecycle of the various applications as well as general navigational fallback functions, such as controlling exit, opening, fallback, etc. of the applications. The window manager is used for managing all window programs, such as obtaining the size of a display screen, judging whether a status bar exists, locking the screen, intercepting the screen, controlling the change of the display window (for example, reducing the display window, displaying a shake, displaying a distortion deformation, and the like), and the like.

In some embodiments, the system runtime layer provides support for the upper layer, i.e., the framework layer, and when the framework layer is used, the android operating system runs the C/C + + library included in the system runtime layer to implement the functions to be implemented by the framework layer.

In some embodiments, the kernel layer is a layer between hardware and software. As shown in fig. 4, the core layer includes at least one of the following drivers: audio drive, display driver, bluetooth drive, camera drive, WIFI drive, USB drive, HDMI drive, sensor drive (like fingerprint sensor, temperature sensor, pressure sensor etc.) and power drive etc..

Based on the above-described display apparatus 200, the user can control the display apparatus 200 to display various user interfaces during the use of the display apparatus 200. For example, as shown in fig. 5, the display apparatus 200 may display a top page of a different channel. Different media assets can be displayed on the configuration front page column position through operation configuration so as to recommend the media assets. The user can select the column position to expand or view the next level of page. The user can also enter the search page through the search page arranged on the display page to search the media assets. The user can also directly obtain feedback results through voice search.

The display apparatus 200 may display the menu window on the basis of the user interface while displaying the user interface along with the interactive operation performed by the user based on the user interface. The menu window is an extended interactive window in which a plurality of user interactive items can be built.

For example, the display apparatus 200 may perform an image recognition operation on a currently displayed image or a specific image to be recognized according to a recognition operation input by a user to determine whether an object satisfying a specific feature, such as a person, a commodity, a building, or the like, is included in the image. After performing the recognition operation on the image, the display apparatus 200 may display a window for presenting the image recognition result in the user interface. That is, as shown in fig. 6, a person link, a product link, and the like identified from the image may be included in the menu window for selection by the user. When the user clicks on the person link option, the display device 200 may jump to a person details interface or a media asset recommendation interface associated with the identified person.

It should be noted that, in the embodiment of the present application, the menu window has a broad sense, and may be broadly referred to as a UI window displayed based on a user interface. The menu window includes but is not limited to: a recommendation column, a result column, a setting menu, a status column, a shortcut setting menu, a prompt column, etc. In the embodiment of the present application, the status bar may be any one of the UI windows described above, unless otherwise specified.

The menu window may present different specific display forms according to different uses. For example, as shown in fig. 7, the menu window for presenting the image recognition result may be in the form of a bar frame displayed in the bottom area of the user interface. And the menu window for shortcut setting may be in the form of a rectangular box displayed in the middle area of the user interface, as shown in fig. 8.

The items in the menu window may be presented in a particular layout. For example, when the display apparatus 200 displays a menu window for presenting an image recognition result, a commodity recognition result may be displayed in a left area of the menu window, a recognized character image and a name may be displayed in a right area, and a thumbnail of the recognized picture or a search term may be displayed in a middle area of the menu window for a user to perform different interactive operations in different areas.

The display apparatus 200 may also set the background of the menu window in a specific color or pattern in order to highlight items in the menu window. In some embodiments, the background of the menu window may be set in accordance with the operating system theme style of the display device 200. For example, when the UI style of the operating system of the display device 200 is blue, the display device 200 may set the menu window to be also a blue background when the menu window is displayed.

However, when the display device 200 sets the background of the menu window based on the theme style of the operating system, when the menu window is displayed under a portion of the user interface due to a single display form of the theme style, the color difference between the main colors of the menu window and the user interface is large, and thus an inconsistent display effect may be formed. Moreover, since the specific content presented in the user interface may change with the change of the user operation, even for the same type of user interface, an inconsistent display effect may be caused when displaying the partial items. For example, when display device 200 displays a asset list interface, the items in the asset list interface are typically posters, cover sheets, screenshots, etc. of the asset items. The pattern form and color style of different media asset items also differ significantly, thus further exacerbating this sense of incongruity.

In order to enrich the background form of the menu window and make the display effect of the menu window more coordinated, some embodiments of the present application further provide a window background display method, which can be applied to the display device 200 described above. Obviously, in order for the display apparatus 200 to be able to apply the above-described window background display method, in the present embodiment, the display apparatus 200 should include at least the display 260 and the controller 250. Wherein the display 260 is configured to display a user interface and to display a menu window or other type of window when a user enters a particular interactive operation. The controller 250 may be configured to execute the window background display method to adjust the background content of the menu window according to the content of the user interface, as shown in fig. 9, specifically including the following steps:

and acquiring a control instruction which is input by a user and used for displaying the menu window. The display device 200 may respond according to various control instructions input by the user and trigger the display of specific content according to the control instructions. Wherein part of the control instructions may be used to trigger the display device 200 to display a menu window. The user can input a control command for displaying the menu window through different interactive manners.

In some embodiments, the control command for displaying the menu window may be input through a key on the control apparatus 100 associated with the display device 200. For example, the control device 100 may be provided with a screen capture key (or a map recognition key). When the user presses the screen capture key, the display device 200 may be triggered to enable the image recognition function, and enter the screen capture scene. In the screenshot scenario, the recognition application may first perceive the screenshot event and then broadcast the screenshot command within the display device 200. After receiving the screenshot command, the service operating system of the display device 200 executes screenshot and image recognition operations, and triggers the display device 200 to display a menu window for displaying the image recognition result.

It will be apparent that for other types of menu windows, the user may complete the input by controlling other keys on the device 100. For example, the control apparatus 100 may be directly provided with a menu key, and when the user presses the menu key, the display device 200 may be triggered to display a menu window. For example, when the user presses the "up" direction key of the control device 100 to trigger the display device 200 to display the status bar when the focus cursor controlling the interface of the home page is located at the top of the interface, it is equivalent to input a control instruction for displaying the menu window through the keys on the control device 100.

In addition to inputting a control instruction for displaying a menu window through a key on the control apparatus 100, in some embodiments, the display device 200 may further display various options that can be clicked (selected) in the user interface, and when the user clicks some of the options, the display device 200 may be triggered to display the menu window corresponding to the function of the option, that is, a control instruction for displaying the menu window is input.

In some embodiments, the display device 200 may also perform the input of the control command for displaying the menu window by a special form of interaction. For example, when the display device 200 is equipped with an intelligent voice system, the user may input voice "display shortcut menu", "recognize a picture", etc., and trigger the display device 200 to execute a corresponding function through the intelligent voice system, so as to display a corresponding menu window, i.e., input a control instruction for displaying the menu window.

After the user inputs a control instruction for displaying the menu window, the display device 200 may detect color values of a plurality of target points in the current user interface in response to the input control instruction. The target point refers to a plurality of sampling points selected on the current user interface according to a specific arrangement rule.

In order to obtain the color values, the display device 200 may generate a background picture for extracting color information according to the currently displayed user interface. The background picture can be obtained by screenshot operation or analysis operation on the display screen. For example, for a user interface displayed through a single layer, the display device 200 may directly extract a display screen of the layer where the user interface is located to obtain a background picture.

In some embodiments, for a user interface formed by combining multiple image layers, the display device 200 may perform screenshot processing on the final multi-image layer image fusion effect to generate a background image in order to obtain complete image content. That is, the display device 200 may perform a screen capture operation on the user interface, acquire a background picture, and locate a plurality of target points in the background picture to extract a color value of each target point. For example, after the user inputs a control instruction of a graphic recognizing operation, the display apparatus 200 may screen-capture a current display screen in response to the control instruction. And the picture obtained by screenshot can be used for executing image recognition on one hand, and can be used as a background picture to acquire a target point color value on the other hand.

While obtaining the background picture, the display device 200 may also locate the target point in the background picture. Since the content displayed in the user interface may have different colors at different positions, for example, in the asset list interface, the background area portion is generally a blue gradient pattern, the main body color extracted in the background area is blue, and in the asset item icon portion, the main body color is displayed in different colors according to the specific pattern of the asset cover. Therefore, in order to make the background of the menu window more harmonious with the display effect of the user interface, as shown in fig. 10, the display apparatus 200 needs to locate a plurality of target points in the background picture. Through a plurality of target points, the color in the background picture can be detected at different positions.

The number of target points for detecting color values may be set to a fixed value according to the screen size specification and resolution of the display device 200. For example, for a smart television of 2k display level (resolution of 1920 × 1080), the number of target points may be set to 20; for a smart tv with a display level of 4K (resolution 3840 × 2160), the number of destination points may be set to 30.

The number of target points can also be adjusted in real time according to data such as the type of the display device, the type of the user interface, and the picture content in the user interface. For example, when the display device 200 is a reading device or displays a reading user interface, the number of target points may be reduced to reduce the data processing amount because the display content of the reading interface is simple and the user interface includes fewer colors.

In the embodiments of the present application, the color value refers to a numerical value or a combination of numerical values that can be used to represent a specific color. For example, the color value may be an RGB value, that is, the color value may be a combination of color values corresponding to three channels of red, green, and blue. For the convenience of subsequent determination, the color value may also be a value obtained by converting the RGB value through a specific weighted summation algorithm, that is:

C＝W ₁ ×R+W ₂ ×G+W ₃ ×B。

wherein C is a color value; w ₁ 、W ₂ 、W ₃ The calculated weights of the red, green and blue color channels are respectively, wherein R is a red channel numerical value, G is a green channel numerical value, and B is a blue channel numerical value.

After obtaining the color values of the target points, the display device 200 may further filter out the color-picking points used for calculating the background color by comparing the color values of the plurality of target points. In order to be able to screen out color points from a plurality of target points, the display device 200 may preset a screening rule for the target points. For example, the display device 200 may delete one or more points with the highest color value and one or more points with the lowest color value from the plurality of target points by running the filtering rule program, so as to use the remaining target points as color-picking points.

As shown in fig. 11, in some embodiments, the display device 200 may further compare the target points with each other to determine color difference values between the target points, and determine which target points are color sampling points according to the color difference values, that is, the color sampling points are target points with color difference values smaller than the first color difference threshold. The display apparatus 200 may calculate an average value of a plurality of target point color values; calculating the variance of the color value of each target point according to the average value to obtain a color difference value; comparing the color difference value with a first color difference threshold value; if the color difference value is greater than or equal to the first color difference threshold value, namely the color corresponding to the current target point has larger color difference with other target points, the target point can be deleted; and if the color difference value is smaller than the first color difference threshold value, namely the color corresponding to the current target point has smaller color difference with other target points, marking the target point corresponding to the color difference value as a color picking point.

For example, after adding the 20 target points P1 and P2 … … P20 to the background picture, the average value of the color values of the 20 target points, that is, the average value AV = (C1 + C2 +), can be calculated first… … + C20)/20. Then, according to the calculated average value AV, calculating the variance corresponding to each target point as a color difference judgment value, namely the color difference judgment value sigma 1= (C1-AV) corresponding to the target point P1 ² (ii) a Color difference determination value σ 2= (C2-AV) corresponding to target point P2 ² … … finally, the color difference values σ 1, σ 2 … … σ 20 are compared to a first color difference threshold D0. When the sigma 1 is larger than or equal to the D0, the chromatic aberration between the P1 point and the rest target points is determined to be larger, so that the target point P1 can be deleted. Similarly, the target point P2 continues to be compared in the above comparison manner, and if σ 2 < D0, it is determined that the color of the P2 point is less in color difference from the rest of the points, so that the P2 point can be marked as the color sampling point. By analogy, other target points are judged, and therefore color taking points with small color difference can be screened out from the mark points.

After the color fetching point is screened out, the display device 200 may set the background of the menu window according to the color value of the color fetching point. When setting the background of the menu window, the specific color matching mode can be determined according to the background form of the menu window. For example, when the menu window adopts a solid background, the display device 200 may calculate an average value according to the color value of the color point, and use a color corresponding to the calculated average value as the color of the solid background. When the menu window adopts a gradient background, the display device 200 may set a plurality of region colors according to the position of the color selection point, and render the background pattern of the whole menu window through gradient processing.

It should be noted that the process of filtering the color picking point and setting the background of the menu window according to the color value of the color picking point can also be performed by the server 400 connected to the display device 200. That is, in some embodiments, the display device 200 further includes a communicator 220, the communicator 220 is configured to establish a communication connection with the server 400, and after the display device 200 detects color values of a plurality of target points in the current user interface in response to the control instruction, the detected color values may be sent to the server 400, and the server 400 performs subsequent processing. That is, the server 400 may compare and determine the detected color values, screen out a target point with a color difference value smaller than the first color difference threshold as a color sampling point, generate a background color according to a color value of the color sampling point, and send the generated background color to the display device 200, so that the display device 200 may set the background of the menu window according to the background color after acquiring the background color.

According to the technical scheme, the extracted color is the color from the color sampling point position with small color difference, so that the background set for the menu window based on the extracted color value is consistent with the main color phase in the user interface, the background of the menu window can be changed in real time according to the display content of the user interface, the display form of the menu window is enriched, and meanwhile, the display effect of the menu window can be better coordinated with the user interface.

In some embodiments, the display device 200 may further perform image recognition on the user interface in the process of extracting the color value of the target point from the user interface, and set the setting position of the target point according to the image recognition result. That is, the display device 200 may further have an image recognition model built therein, and after a screenshot operation is performed on the user interface to obtain a background picture, the background picture may be input into the image recognition model, so that specific content included in the background picture is recognized by the image recognition model. For example, whether the user interface contains characteristic objects such as people, commodities, buildings and the like can be identified through the image identification model.

In some embodiments, the image recognition process may also be accomplished by the server 400. That is, as shown in fig. 11, the server 400 may store an image recognition model therein, and after acquiring the background picture, the display device 200 may send the background picture to the server 400 to trigger the server 400 to perform image recognition on the background picture.

The specific background picture sending mode may be completed by a picture recognition request, that is, the display device 200 may generate a picture recognition request according to the background picture and send the picture recognition request to the server 400. After receiving the image recognition request, the server 400 parses the image recognition request to obtain a background image that needs to be subjected to image recognition. And inputting the background picture into the image recognition model so as to determine whether the background picture contains characteristic targets such as people, commodities, buildings and the like and the specific positions of the characteristic targets. After the image recognition processing, the server 400 feeds back the image recognition result to the display apparatus 200.

The display device 200 divides the color sampling area in the background picture according to the image recognition result after receiving the image recognition result fed back by the server for the background picture. The color taking area is an area where the characteristic target is located in the background picture or an area outside the area where the characteristic target is located. The display device 200 may determine which area is to be the color extraction area according to a user interface type, a custom policy, and the like. For example, when the display device 200 displays the media asset list interface, since the pattern of the media asset item cover is not favorable for maintaining the coordination effect, after the position of the media asset item cover is determined by image recognition, the area other than the media asset item cover is used as the color sampling area, so that the color difference in the color sampling area is small. When the display device 200 displays a media asset playing interface with a frame mainly including a building, the position of the building can be determined through image recognition, and the position of the building is used as a color sampling area, so that content interference outside the building is eliminated.

After the color-taking area is divided, the display device 200 adds a plurality of target points to the color-taking area, and the target points can be uniformly distributed in the color-taking area to obtain the dominant hue in the color-taking area, so that the display device 200 can render the background of the menu window according to the dominant hue in the color-taking area. Therefore, through image recognition of the background picture, the display device 200 can intelligently set the position of the target point according to the content contained in the user interface, so that the extracted color value better conforms to the display characteristics of the user interface, and the finally presented background effect of the menu window is more coordinated.

In some embodiments, while the target point position is set by the image recognition result, the number of target points may also be determined according to the number of feature targets recognized in the image recognition result. That is, in the step of adding the plurality of target points in the color sampling region, the display device 200 may traverse the number of the feature targets identified in the image identification result, and then calculate the number of the target points according to the number of the feature targets, thereby adding the target points in the color sampling region according to the number of the target points.

Wherein the number of target points is linearly related to the number of feature targets. Obviously, when the color sampling area is the area where the characteristic target is located, the number of the target points is linearly and positively correlated with the number of the characteristic targets. That is, the larger the number of feature targets, the larger the proportion of the user interface range occupied by the feature targets, and therefore, the larger the number of target points set correspondingly. Similarly, when the color sampling area is an area outside the area where the characteristic target is located, the number of the target points and the number of the characteristic targets are linearly and negatively correlated. That is, the larger the number of the feature targets, the larger the proportion of the user interface range occupied by the feature targets, and the smaller the area outside the area where the feature targets are located, the smaller the number of the target points correspondingly set.

For example, in the asset list interface, the display device 200 may use a region other than the region where the asset cover is located as the color-taking point, and when it is recognized that the current user interface includes n asset items, the number of the obtained feature targets is n. The display device 200 then queries the number m of target points needed by the picture corresponding to the current user interface according to the obtained number of the characteristic targets, and uniformly arranges m target points in the color sampling area.

In some embodiments, the number of the target points added in the color sampling region may also be determined according to the resolution of the background picture, that is, the display device 200 may obtain the resolution of the background picture in the step of adding a plurality of target points to the color sampling region, query the specification interval to which the background picture belongs according to the resolution, and extract the number of the target points in the specification interval to which the background picture belongs, so as to add the target points in the color sampling region according to the number of the target points.

Obviously, when the display resolution of the background picture is higher, the specific expressible picture content is more, so a larger number of target points need to be set to reflect the main display content in the user interface. For this reason, in the display apparatus 200 or the server 400, a database may be provided for storing the number of target points that need to be added at different resolutions. For example, when the resolution of the acquired background picture is 1920 × 1080, a query may be made in the database according to the resolution to determine that the number of target points is 20.

The two setting modes of the number of the target points can be applied independently or in combination. That is, in some embodiments, after acquiring the background picture, the display device 200 may determine the required target point base number according to the resolution of the background picture, and then adjust the base number according to the image recognition result, that is, determine the target point number that needs to be set finally based on the base number.

In some embodiments, in the step of setting the background of the menu window according to the color value of the color picking point, the display device 200 may first calculate an average value of the color values of the plurality of color picking points; rendering a pure color picture according to the average value so as to add the pure color picture to a display area of the menu window; meanwhile, the display apparatus 200 may also calculate a foreground color from the average value to render items in the menu window in the foreground color.

In order to clearly display the items contained in the menu window, the foreground color should be easily distinguished from the background color of the menu window, that is, the difference between the color value of the foreground color and the average value is greater than the first color difference threshold. For example, when the average value of the cyan color calculated from the color values of the plurality of color points is (3,245,237), the color may be filled as the solid background of the menu window. Meanwhile, the display device 200 may determine a foreground color that is convenient for the user to distinguish according to the color corresponding to the average value. For example, white against a dark background (0,0,0) or black against a light background (255, 255, 255), and displays text and UI item graphics in the menu window in the determined foreground color.

To determine the foreground color, the display device 200 may have at least two color schemes built in, such as a dark color scheme and a light color scheme. Under each color scheme, the display device 200 may set a corresponding default foreground color scheme, so that after the average value is obtained through calculation, the display device 200 may determine the color scheme to which the current background color belongs according to the average value, and search for the foreground color corresponding to the color scheme.

In some embodiments, the display device 200 may further add a plurality of colors to the menu window background, that is, as shown in fig. 12, in the step of setting the background of the menu window according to the color value of the color-picking point, the display device 200 may traverse the position of the color-fixing area in the menu window. The color fixing area is used for indicating a specific color presented in a certain part of the menu window. The display apparatus 200 may set a fixed color region for each type of menu window, i.e., may include at least one fixed color region in the menu window, according to the effect of the final presentation. Moreover, the colors displayed in different color fixing areas can be the same or different.

For example, the background color of the menu window popped up by the display device 200 may be set to 10 color values by default, and the 10 color values have a corresponding relationship with all the fixed color regions of the RGB colors, so that the display device 200 may extract the RGB color values of each point by capturing 20 points of the background color, and eliminate the value having a larger difference from most colors. And inquiring the corresponding relation according to the average value, thereby determining that the corresponding color value is the color value of the background color of the menu window.

The specific number and position of the fixed color regions can be determined according to the specific display function in the menu window. For example, for a recognition result presentation window, the window may include a product area on the left, a character area on the right, and a search area (or screenshot presentation area) in the middle. At this time, the display apparatus 200 may set three color fixing regions for the menu window, respectively for setting colors that need to be set in the left, right, and middle regions.

In some embodiments, the position of the fixing region may be further set according to the image recognition result, that is, the display device 200 may obtain the image recognition result of the user interface; traversing the feature targets in the image recognition result to calculate the number of the fixed color areas according to the number of the feature targets and/or the type number of the feature targets; thereby setting the position of each fixed color area within the menu window according to the number of the fixed color areas.

For example, when person recognition is performed on the background image by the image recognition method, the display device 200 may determine the number of the fixed color regions according to the number of the recognized portrait targets after recognizing the portrait target, that is, may set the number of the fixed color regions to be equal to the number of the portrait targets or to satisfy a specific correlation.

After determining the position of the fixed color region, the display device 200 searches for a related target point among the plurality of target points according to the position of the fixed color region. Each fixing color area can be correspondingly provided with a plurality of associated target points. The specific association relationship may be determined according to the distance between the target point and the fixing color region. For example, the user interface may be divided into regions corresponding to the left, right, and middle regions on the menu window to obtain a left-side related region, a right-side related region, and a middle related region. And traversing the target point in each associated area to determine the associated target point of each fixed color area.

After querying the associated target point, the display device 20 may further calculate a filling color value according to the associated target point color value of each color fixing area, so as to fill a color in each color fixing area according to the filling color value. When the number of the associated target points of the color fixing areas is multiple, the filling color value may be set to be an average value of color values of the associated target points of each color fixing area.

In some embodiments, after determining the background color of the menu window, the display device 200 may further determine that the determined background color is favorable for the user to perform an interactive operation, that is, in the step of obtaining the background color, the display device 200 may calculate the color to be rendered according to a color value of a color sampling point after comparing color values of a plurality of target points to obtain the color sampling point of the plurality of target points whose color difference value is smaller than a first color difference threshold value. For different background rendering forms, the color to be rendered may be a single color or a color group composed of a plurality of colors. For example, for a solid background, the color to be rendered is a single color corresponding to the average of the color values of the plurality of color points; and for the background comprising a plurality of fixed color areas, the color to be rendered represents a color corresponding to each fixed color area.

After the color to be rendered is obtained through calculation, the display device 200 may further compare the color to be rendered with a preset extreme color. The preset extreme colors may be several colors that are not favorable for display and interactive operation, such as pure white, pure red, pure black, and the like, according to the UI interface style of the operating system of the display device 200. The preset extreme colors may also include other colors that are the same as or too similar to the colors of the contents in the specific menu, such as text, graphics, etc.

By comparing the color value corresponding to the color to be rendered and the preset extreme color, the display device 200 may determine whether the color to be rendered is the same as or similar to the preset extreme color, thereby determining whether the color to be rendered is suitable for display. If the color difference between the color to be rendered and the preset extreme color is smaller than a second color difference threshold value, namely the rendered color is the same as or similar to the preset extreme color and is not beneficial to display, the color to be rendered can be replaced by the preset alternative color to be used as the background color. Similarly, if the color difference between the color value to be rendered and the preset extreme color value is greater than or equal to the second color difference threshold value, that is, the rendering color is different from or not similar to the preset extreme color, the display and the user interaction operation are facilitated, and therefore the color to be rendered can be directly used as the background color to be rendered.

In some embodiments, the extreme color may be labeled by using a color coordinate range, that is, after the color to be rendered is obtained, if a difference between the color coordinate of the color to be rendered and the color coordinate of the extreme color satisfies a preset condition, a color difference representing the two satisfies a second color difference threshold.

In some embodiments, the color coordinate range may be defined by the color coordinate of the extreme color and a second color difference threshold, and after the color to be rendered is obtained, if the color coordinate of the color to be rendered is not in the defined color coordinate range, the color difference characterizing the two satisfies the second color difference threshold.

In some embodiments, the RGB channels may be utilized to calculate color differences.

In some embodiments, when the control instruction for displaying the menu window triggers the display device 200 to set the background of the menu window by the method in the above embodiments, the display device 200 may also render the foreground content of the menu window in response to the control instruction input by the user. For example, after the user presses a map recognition key on the control apparatus 100, the display device 200 may receive an image recognition result of the server 400 capturing a current user interface screen in response to a control instruction when the map recognition key is pressed, so as to obtain a foreground screen of the menu window.

The foreground picture comprises an image recognition result of the current display picture, and comprises characteristic target information recognized from a user interface, such as a person name, a building name, a commodity name and the like. Meanwhile, the foreground picture can also comprise associated recommended items determined based on the image recognition result, including media asset items, webpage links, detailed texts and the like associated with the characteristic target information.

After acquiring the foreground screen, the display device 200 may further extract the background color determined in the above embodiments to render the menu window to be presented. In the process of rendering the menu window, the display device 200 may create a display layer on an upper layer of the layer where the user interface is located, call a display template of the menu window, add a foreground picture and a background color to the display template, generate a menu picture, and display the menu picture in the display layer.

For example, the user interface presented by the display apparatus 200 may be displayed on a video layer, and the graphic menu window may be displayed through an OSD layer located on an upper layer of the video layer. After the user inputs a map recognition instruction, the display apparatus 200 may invoke the OSD layer for interface rendering in response to the map recognition instruction. In the rendering process, the display device 200 may call a display template of the graph recognition menu window, and obtain a graph recognition result output by the graph recognition model and associated recommended item data fed back by the server 400. And finally, rendering a specific menu window picture according to the display template and the image recognition result, and displaying the menu window picture on an OSD layer.

Based on the above window background display method, in some embodiments of the present application, there is further provided a display device 200, where the display device 200 includes a display 260 and a controller 250, where the display 260 is used for displaying a user interface, and the controller 250 is configured to perform the following program steps:

comparing the color values of the target points to screen out color picking points used for calculating the background color, wherein the color picking points are the target points of which the color difference values are smaller than a first color difference threshold value;

and setting the background of the menu window according to the color value of the color picking point.

As can be seen from the foregoing technical solutions, the display device and the window background display method provided in the foregoing embodiments may detect color values of a plurality of target points in a current user interface after a user inputs a control instruction for displaying a menu window, and screen out a color fetching point used for calculating a background color from the detected color values, so as to set a background of the menu window according to a color corresponding to the color fetching point. According to the method, when the menu window is displayed, the background color can be adjusted according to the content displayed in the bottom user interface, so that the background form of the menu window is richer, the coordination between the menu window and the user interface is improved, and the user experience is improved.

The detailed description provided above is only a few examples under the general concept of the present application, and does not constitute a limitation to the scope of the present application. Any other embodiments extended according to the scheme of the present application without inventive efforts will be within the scope of protection of the present application for a person skilled in the art.

Claims

1. A display device, comprising:

a display configured to display a user interface;

a communicator configured to establish a communication connection with a server;

a controller configured to:

responding to the control instruction, and detecting color values of a plurality of target points in the current user interface;

acquiring a background color, wherein the background color is generated by the controller or the server according to color values of color sampling points in the target points, and the color sampling points are the target points of which the color difference values determined by comparing the color values of the target points are smaller than a first color difference threshold value;

2. The display device of claim 1, wherein the controller is further configured to:

in the step of detecting color values of a plurality of target points in a current user interface, performing screen capture operation on the user interface to acquire a background picture;

and positioning a plurality of target points in the background picture to extract the color value of each target point.

3. The display device of claim 2, wherein the controller is further configured to:

in the step of positioning a plurality of target points in the background picture, sending the background picture to the server to trigger the server to execute image recognition on the background picture;

and adding a plurality of target points in the color taking area, wherein the target points are uniformly distributed in the color taking area.

4. The display device of claim 3, wherein the controller is further configured to:

traversing the number of the characteristic targets identified in the image identification result in the step of adding a plurality of target points in the color picking area;

calculating the number of the target points according to the number of the characteristic targets, wherein the number of the target points is linearly related to the number of the characteristic targets;

and adding the target points in the color taking area according to the number of the target points.

5. The display device of claim 3, wherein the controller is further configured to:

in the step of adding a plurality of target points in the color taking area, acquiring the resolution of the background picture;

inquiring the specification interval to which the background picture belongs according to the resolution;

extracting the number of target points in the specification interval to which the background picture belongs;

6. The display device according to claim 1, wherein the controller is further configured to:

in the step of setting the background of the menu window according to the background color, calculating the average value of the color values of the plurality of color points;

rendering a pure color picture according to the average value;

adding the pure color picture to a display area of the menu window;

calculating a foreground color according to the average value, wherein the difference value of the color value of the foreground color and the average value is larger than a first color difference threshold value;

and rendering the items in the menu window according to the foreground color.

7. The display device of claim 1, wherein the controller is further configured to:

in the step of setting the background of the menu window according to the background color, traversing the position of a fixed color area in the menu window;

inquiring a related target point in the target points according to the position of the color fixing area;

calculating a filling color value according to the associated target point color value of each color fixing area, wherein the filling color value is an average value of the associated target point color values of each color fixing area;

and filling colors in each color fixing area according to the filling color values.

8. The display device of claim 1, wherein the controller is further configured to:

in the step of obtaining the background color, comparing color values of a plurality of target points to obtain color sampling points of the plurality of target points, wherein the color difference value of the color sampling points is smaller than a first color difference threshold value;

calculating the color to be rendered according to the color value of the color taking point;

when the color difference between the color to be rendered and the preset extreme color is smaller than a second color difference threshold value, marking a preset alternative color as the background color;

and when the color difference between the color value to be rendered and the preset extreme color value is greater than or equal to the second color difference threshold value, marking the color to be rendered as the background color.

9. The display device of claim 1, wherein the controller is further configured to:

responding to the control instruction, acquiring a foreground picture of the menu window, wherein the foreground picture comprises an image recognition result of a current display picture and a recommendation item related to the image recognition result;

calling a display template of the menu window, and adding the foreground picture and the background color to the display template to generate a menu picture;

creating a display layer on the upper layer of the layer where the user interface is located;

and displaying the menu picture in the display layer.

10. A method for displaying a window background, comprising: