CN115776585A - Display device and content presentation method - Google Patents

Display device and content presentation method Download PDF

Info

Publication number
CN115776585A
CN115776585A CN202211519109.7A CN202211519109A CN115776585A CN 115776585 A CN115776585 A CN 115776585A CN 202211519109 A CN202211519109 A CN 202211519109A CN 115776585 A CN115776585 A CN 115776585A
Authority
CN
China
Prior art keywords
display
content
information
server
video
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202211519109.7A
Other languages
Chinese (zh)
Inventor
付延松
穆聪聪
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hisense Visual Technology Co Ltd
Original Assignee
Hisense Visual Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hisense Visual Technology Co Ltd filed Critical Hisense Visual Technology Co Ltd
Priority to CN202211519109.7A priority Critical patent/CN115776585A/en
Publication of CN115776585A publication Critical patent/CN115776585A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/422—Input-only peripherals, i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS]
    • H04N21/42204—User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G06F3/04845—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range for image manipulation, e.g. dragging, rotation, expansion or change of colour
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/22—Image preprocessing by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition
    • G06V10/235—Image preprocessing by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition based on user input or interaction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/94—Hardware or software architectures specially adapted for image or video understanding
    • G06V10/95—Hardware or software architectures specially adapted for image or video understanding structured as a network, e.g. client-server architectures
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/422—Input-only peripherals, i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS]
    • H04N21/42202—Input-only peripherals, i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS] environmental sensors, e.g. for detecting temperature, luminosity, pressure, earthquakes
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/422—Input-only peripherals, i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS]
    • H04N21/42203—Input-only peripherals, i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS] sound input device, e.g. microphone
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/422—Input-only peripherals, i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS]
    • H04N21/42204—User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor
    • H04N21/42206—User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor characterized by hardware details
    • H04N21/42221—Transmission circuitry, e.g. infrared [IR] or radio frequency [RF]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/422—Input-only peripherals, i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS]
    • H04N21/4223—Cameras
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H04N21/440218—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by transcoding between formats or standards, e.g. from MPEG-2 to MPEG-4
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H04N21/440245—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display the reformatting operation being performed only on part of the stream, e.g. a region of the image or a time segment

Abstract

The invention discloses a display device and a content display method, wherein the display device comprises: the display device includes: a display; a controller communicatively coupled with the display, the controller configured to: receiving a screenshot instruction; responding to the screenshot instruction, and performing screenshot operation on a current display picture displayed by the display to obtain a screenshot image; when the screenshot image comprises a picture generated by playing a video, sending an information acquisition request to a server, wherein the information acquisition request comprises scene information corresponding to the screenshot image; receiving response information sent by the server in response to the information acquisition request, wherein the response information comprises recommended content corresponding to the scene information; and controlling the display to display recommended content contained in the response information. By adopting the display equipment and the content display method provided by the application, the recommended content displayed by the display equipment can be richer.

Description

Display device and content presentation method
The application is a divisional application of domestic application (patent application No. 202011461720.X, application date: 2020-12-11, patent name: display device and content display method).
Technical Field
The embodiment of the application relates to a display technology. And more particularly, to a display apparatus and a content presentation method.
Background
The television is a common household appliance in daily life, and can integrate the functions of video, entertainment, games and the like. For example, a television can play a video, and can also acquire a frame of still picture in the played video for content identification.
In the related art, when a television identifies content of a frame of still picture in a played video, only one or more objects included in the still picture are identified, and then recommended content corresponding to an identification result is displayed. For example, it is possible to recognize the content of an object such as a person, an animal, or a plant included in a still picture, and display the recognized person name, animal name, or plant name, and recommended content determined based on the person name, animal name, or plant name.
However, the accuracy and success rate of the still picture recognition are limited by many factors. If an object contained in the still picture is not clear enough or is small in size, it may result in that the object cannot be recognized or only a part of the object can be recognized, resulting in less recommended content being presented.
Disclosure of Invention
An exemplary embodiment of the present application provides a display device and a corresponding content display method, so as to solve the problems in the related art that when content identification is performed on a frame of still picture in a played video by using a television, the identified content is relatively single, and the number of displayable content is small.
In order to solve the technical problem, the embodiment of the application discloses the following technical scheme:
in some embodiments of the present application, a display apparatus is disclosed, the display apparatus comprising: a display and a controller, wherein the controller is communicatively coupled to the display, the controller configured to: receiving a screenshot instruction; responding to the screenshot instruction, and performing screenshot operation on a current display picture displayed by the display to obtain a screenshot image; when the screenshot image comprises a picture generated by playing a video, sending an information acquisition request to a server, wherein the information acquisition request comprises scene information corresponding to the screenshot image; receiving response information sent by the server in response to the information acquisition request, wherein the response information comprises recommended content corresponding to the scene information; and controlling the display to display recommended content contained in the response information.
In some embodiments, the information acquisition request further includes the screenshot image; or, the response information further includes a recognition result of the target object recognized and obtained from the screenshot image.
In some embodiments, the information acquisition request further includes auxiliary information for assisting the server in content identification of the screenshot image.
In some embodiments, in the sending of the information acquisition request to the server, the controller is further configured to: sending the screenshot image and auxiliary information for assisting the server in identifying the content of the screenshot image to a content identification server; and sending the scene information to a content recommendation server.
In some embodiments, in the step of receiving response information sent by the server in response to the information acquisition request, the controller is further configured to: receiving an identification result sent by a content identification server, wherein the identification result is obtained by identifying the screenshot image based on the auxiliary information; and receiving the recommended content sent by the content recommendation server based on the scene information.
In some embodiments, in the step of controlling the display to display the recommended content included in the response information, the controller is further configured to: and controlling the display to display the identification result in a first display area, and controlling the display to display the recommended content in a second display area.
In some embodiments, in the controlling of the display to display the recommended content included in the response information, the controller is further configured to: controlling the display to display the recognition result; and after receiving a switching instruction, controlling the display to switch to display the recommended content.
Corresponding to the foregoing embodiments of the display device, in some embodiments of the present application, a content presentation method is disclosed, where the method includes: receiving a screenshot instruction; responding to the screenshot instruction, and performing screenshot operation on a current display picture displayed by the display to obtain a screenshot image; when the screenshot image comprises a picture generated by playing a video, sending an information acquisition request to a server, wherein the information acquisition request comprises scene information corresponding to the screenshot image; receiving response information sent by the server in response to the information acquisition request, wherein the response information comprises recommended content corresponding to the scene information; and displaying the recommended content contained in the response information.
In some embodiments, the information acquisition request further includes the screenshot image; the response information also comprises an identification result of the target object identified from the screenshot image.
In some embodiments, the information acquisition request further includes auxiliary information for assisting the server in content identification of the screenshot image.
In some embodiments, the step of sending an information acquisition request to the server includes: sending the screenshot image and auxiliary information for assisting the server in identifying the content of the screenshot image to a content identification server; and sending the scene information to a content recommendation server.
In some embodiments, the step of receiving response information sent by the server in response to the information acquisition request includes: receiving an identification result sent by a content identification server, wherein the identification result is obtained by identifying the screenshot image based on the auxiliary information; and receiving the recommended content sent by the content recommendation server based on the scene information.
In some embodiments, the step of controlling the display to display the recommended content included in the response message includes: and controlling the display to display the identification result in a first display area, and controlling the display to display the recommended content in a second display area.
In some embodiments, the step of controlling the display to display the recommended content included in the response message includes: controlling the display to display the recognition result; and after receiving a switching instruction, controlling the display to switch to display the recommended content.
The application provides a display device and a content display method, which can control a display to display recommended content determined based on scene information, and not only display the recommended content determined by an identification result, so that the recommended content displayed by the display device is richer.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.
Drawings
In order to more clearly illustrate the embodiments of the present application or the implementation manner in the related art, a brief description will be given below of the drawings required for the description of the embodiments or the related art, and it is obvious that the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art according to the drawings.
FIG. 1 illustrates a usage scenario of a display device according to some embodiments;
fig. 2 illustrates a hardware configuration block diagram of the control apparatus 100 according to some embodiments;
fig. 3 illustrates a hardware configuration block diagram of the display apparatus 200 according to some embodiments;
FIG. 4 illustrates a software configuration diagram in the display device 200 according to some embodiments;
FIG. 5 illustrates an icon control interface display of an application in the display device 200, in accordance with some embodiments;
FIG. 6 illustrates a network architecture diagram of some embodiments;
FIG. 7 illustrates a screenshot image display effect diagram of some embodiments;
FIGS. 8A to 8F are diagrams illustrating a recommended content display effect according to some embodiments;
FIG. 9 is a diagram illustrating recommended content display effects in further embodiments;
FIG. 10 is a flow diagram that illustrates a method for content presentation in some embodiments.
Detailed Description
To make the purpose and embodiments of the present application clearer, the following will clearly and completely describe the exemplary embodiments of the present application with reference to the attached drawings in the exemplary embodiments of the present application, and it is obvious that the described exemplary embodiments are only a part of the embodiments of the present application, and not all the embodiments.
It should be noted that the brief descriptions of the terms in the present application are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of the present application. These terms should be understood in their ordinary and customary meaning unless otherwise indicated.
The terms "first," "second," "third," and the like in the description and claims of this application and in the above-described drawings are used for distinguishing between similar or analogous objects or entities and not necessarily for describing a particular sequential or chronological order, unless otherwise indicated. It is to be understood that the terms so used are interchangeable under appropriate circumstances.
The terms "comprises" and "comprising," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a product or apparatus that comprises a list of elements is not necessarily limited to all elements expressly listed, but may include other elements not expressly listed or inherent to such product or apparatus.
The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and/or software code that is capable of performing the functionality associated with that element.
Fig. 1 is a schematic diagram of a usage scenario of a display device according to an embodiment. As shown in fig. 1, the display apparatus 200 is also in data communication with a server 400, and a user can operate the display apparatus 200 through the smart device 300 or the control device 100.
In some embodiments, the control apparatus 100 may be a remote controller, and the communication between the remote controller and the display device includes at least one of an infrared protocol communication or a bluetooth protocol communication, and other short-distance communication methods, and controls the display device 200 in a wireless or wired manner. The user may control the display apparatus 200 by inputting a user instruction through at least one of a key on a remote controller, a voice input, a control panel input, and the like.
In some embodiments, the smart device 300 may include any of a mobile terminal, a tablet, a computer, a laptop, an AR/VR device, and the like.
In some embodiments, the smart device 300 may also be used to control the display device 200. For example, the display device 200 is controlled using an application program running on the smart device.
In some embodiments, the smart device 300 and the display device may also be used for communication of data.
In some embodiments, the display device 200 may also be controlled in a manner other than the control apparatus 100 and the smart device 300, for example, the voice instruction control of the user may be directly received by a module configured inside the display device 200 to obtain a voice instruction, or may be received by a voice control apparatus provided outside the display device 200.
In some embodiments, the display device 200 is also in data communication with a server 400. The display device 200 may be allowed to be communicatively connected through a Local Area Network (LAN), a Wireless Local Area Network (WLAN), and other networks. The server 400 may provide various contents and interactions to the display apparatus 200. The server 400 may be a cluster or a plurality of clusters, and may include one or more types of servers.
In some embodiments, software steps executed by one step execution agent may be migrated on demand to another step execution agent in data communication therewith for execution. Illustratively, software steps performed by the server may be migrated on demand to be performed on the display device in data communication therewith, and vice versa.
Fig. 2 exemplarily shows a block diagram of a configuration of the control apparatus 100 according to an exemplary embodiment. As shown in fig. 2, the control device 100 includes a controller 110, a communication interface 130, a user input/output interface 140, a memory, and a power supply. The control apparatus 100 may receive an input operation instruction of a user and convert the operation instruction into an instruction recognizable and responsive to the display device 200, serving as an interaction intermediary between the user and the display device 200.
In some embodiments, the communication interface 130 is used for external communication, and includes at least one of a WIFI chip, a bluetooth module, NFC, or an alternative module.
In some embodiments, the user input/output interface 140 includes at least one of a microphone, a touchpad, a sensor, a key, or an alternative module.
Fig. 3 shows a hardware configuration block diagram of the display apparatus 200 according to an exemplary embodiment.
In some embodiments, the display apparatus 200 includes at least one of a tuner demodulator 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output interface 270, a memory, a power supply, a user interface.
In some embodiments the controller comprises a central processor, a video processor, an audio processor, a graphics processor, a RAM, a ROM, a first interface to an nth interface for input/output.
In some embodiments, the display 260 includes a display screen component for displaying pictures, and a driving component for driving image display, and is used for receiving image signals from the controller output, displaying video content, image content, and components of a menu manipulation interface, and a user manipulation UI interface, etc.
In some embodiments, the display 260 may be at least one of a liquid crystal display, an OLED display, and a projection display, and may also be a projection device and a projection screen.
In some embodiments, the tuner demodulator 210 receives broadcast television signals via wired or wireless reception, and demodulates audio/video signals, such as EPG data signals, from a plurality of wireless or wired broadcast television signals.
In some embodiments, communicator 220 is a component for communicating with external devices or servers according to various communication protocol types. For example: the communicator may include at least one of a Wifi module, a bluetooth module, a wired ethernet module, and other network communication protocol chips or near field communication protocol chips, and an infrared receiver. The display apparatus 200 may establish transmission and reception of control signals and data signals with the control device 100 or the server 400 through the communicator 220.
In some embodiments, the detector 230 is used to collect signals of the external environment or interaction with the outside. For example, detector 230 includes a light receiver, a sensor for collecting the intensity of ambient light; alternatively, the detector 230 includes an image collector, such as a camera, which may be used to collect external environment scenes, attributes of the user, or user interaction gestures, or the detector 230 includes a sound collector, such as a microphone, which is used to receive external sounds.
In some embodiments, the external device interface 240 may include, but is not limited to, the following: high Definition Multimedia Interface (HDMI), analog or data high definition component input interface (component), composite video input interface (CVBS), USB input interface (USB), RGB port, and the like. The interface may be a composite input/output interface formed by the plurality of interfaces.
In some embodiments, the controller 250 and the modem 210 may be located in different separate devices, that is, the modem 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.
In some embodiments, the controller 250 controls the operation of the display device and responds to user operations through various software control programs stored in memory. The controller 250 controls the overall operation of the display apparatus 200. For example: in response to receiving a user command for selecting a UI object displayed on the display 260, the controller 250 may perform an operation related to the object selected by the user command.
In some embodiments, the object may be any one of selectable objects, such as a hyperlink, an icon, or other actionable control. The operations related to the selected object are: displaying an operation connected to a hyperlink page, document, image, or the like, or performing an operation of a program corresponding to the icon.
In some embodiments the controller comprises at least one of a Central Processing Unit (CPU), a video processor, an audio processor, a Graphics Processing Unit (GPU), a RAM Random Access Memory (RAM), a ROM (Read-Only Memory), a first to nth interface for input/output, a communication Bus (Bus), and the like.
A CPU processor. For executing operating system and application program instructions stored in the memory, and executing various application programs, data and contents according to various interactive instructions receiving external input, so as to finally display and play various audio-video contents. The CPU processor may include a plurality of processors. E.g. comprising a main processor and one or more sub-processors.
In some embodiments, a graphics processor for generating various graphical objects, such as: at least one of an icon, an operation menu, and a user input instruction display figure. The graphic processor comprises an arithmetic unit, which performs operation by receiving various interactive instructions input by a user and displays various objects according to display attributes; the system also comprises a renderer for rendering various objects obtained based on the arithmetic unit, wherein the rendered objects are used for being displayed on a display.
In some embodiments, the video processor is configured to receive an external video signal, and perform at least one of decompression, decoding, scaling, noise reduction, frame rate conversion, resolution conversion, image synthesis, and other video processing according to a standard codec protocol of the input signal, so as to obtain a signal that can be displayed or played on the direct display device 200.
In some embodiments, the video processor includes at least one of a demultiplexing module, a video decoding module, an image composition module, a frame rate conversion module, a display formatting module, and the like. The demultiplexing module is used for demultiplexing the input audio and video data stream. And the video decoding module is used for processing the video signal after demultiplexing, including decoding, scaling and the like. And the image synthesis module is used for carrying out superposition mixing processing on the GUI signal input by the user or generated by the user and the video image after the zooming processing by the graphic generator so as to generate an image signal for display. And the frame rate conversion module is used for converting the frame rate of the input video. And the display formatting module is used for converting the received video output signal after the frame rate conversion, and changing the signal to be in accordance with the signal of the display format, such as outputting an RGB data signal.
In some embodiments, the audio processor is configured to receive an external audio signal, decompress and decode the received audio signal according to a standard codec protocol of the input signal, and perform at least one of noise reduction, digital-to-analog conversion, and amplification processing to obtain a sound signal that can be played in the speaker.
In some embodiments, a user may enter user commands on a Graphical User Interface (GUI) displayed on display 260, and the user input interface receives the user input commands through the Graphical User Interface (GUI). Alternatively, the user may input the user command by inputting a specific sound or gesture, and the user input interface receives the user input command by recognizing the sound or gesture through the sensor.
In some embodiments, a "user interface" is a media interface for interaction and information exchange between an application or operating system and a user that enables conversion between an internal form of information and a form that is acceptable to the user. A commonly used presentation form of the User Interface is a Graphical User Interface (GUI), which refers to a User Interface related to computer operations and displayed in a graphical manner. It may be an interface element such as an icon, a window, a control, etc. displayed in the display screen of the electronic device, where the control may include at least one of an icon, a button, a menu, a tab, a text box, a dialog box, a status bar, a navigation bar, a Widget, etc. visual interface elements.
In some embodiments, user interface 280 is an interface that may be used to receive control inputs (e.g., physical buttons on the body of the display device, or the like).
In some embodiments, the system of the display device may include a Kernel (Kernel), a command parser (shell), a file system, and an application. The kernel, shell, and file system together make up the basic operating system structure that allows users to manage files, run programs, and use the system. After power-on, the kernel starts, activates kernel space, abstracts hardware, initializes hardware parameters, etc., runs and maintains virtual memory, scheduler, signals and inter-process communication (IPC). And after the kernel is started, loading the Shell and the user application program. The application program is compiled into machine code after being started, and a process is formed.
Referring to fig. 4, in some embodiments, the system is divided into four layers, which are an Application (Applications) layer (abbreviated as "Application layer"), an Application Framework (Application Framework) layer (abbreviated as "Framework layer"), an Android runtime (Android runtime) and system library layer (abbreviated as "system runtime library layer"), and a kernel layer from top to bottom.
In some embodiments, at least one application program runs in the application program layer, and the application programs may be windows (Window) programs carried by an operating system, system setting programs, clock programs or the like; or an application developed by a third party developer. In particular implementations, the application packages in the application layer are not limited to the above examples.
The framework layer provides an Application Programming Interface (API) and a programming framework for the application program of the application layer. The application framework layer includes a number of predefined functions. The application framework layer acts as a processing center that decides to let the applications in the application layer act. The application program can access the resources in the system and obtain the services of the system in execution through the API interface.
As shown in fig. 4, in the embodiment of the present application, the application framework layer includes a manager (Managers), a Content Provider (Content Provider), and the like, where the manager includes at least one of the following modules: an Activity Manager (Activity Manager) is used for interacting with all activities running in the system; the Location Manager (Location Manager) is used for providing the system service or application with the access of the system Location service; a Package Manager (Package Manager) for retrieving various information about an application Package currently installed on the device; a Notification Manager (Notification Manager) for controlling display and clearing of Notification messages; a Window Manager (Window Manager) is used to manage the icons, windows, toolbars, wallpapers, and desktop components on a user interface.
In some embodiments, the activity manager is used to manage the lifecycle of the various applications and the usual navigation fallback functions, such as controlling exit, opening, fallback, etc. of the applications. The window manager is used for managing all window programs, such as obtaining the size of a display screen, judging whether a status bar exists, locking the screen, intercepting the screen, controlling the change of the display window (for example, reducing the display window, displaying a shake, displaying a distortion deformation, and the like), and the like.
In some embodiments, the system runtime layer provides support for an upper layer, i.e., the framework layer, and when the framework layer is used, the android operating system runs the C/C + + library included in the system runtime layer to implement the functions to be implemented by the framework layer.
In some embodiments, the kernel layer is a layer between hardware and software. As shown in fig. 4, the core layer includes at least one of the following drivers: audio drive, display driver, bluetooth drive, camera drive, WIFI drive, USB drive, HDMI drive, sensor drive (like fingerprint sensor, temperature sensor, pressure sensor etc.) and power drive etc..
In some embodiments, the display device may directly enter an interface of a preset video-on-demand program after being started, and the interface of the video-on-demand program may include at least a navigation bar 510 and a content display area located below the navigation bar 510, as shown in fig. 5, where content displayed in the content display area may change according to a change of a selected control in the navigation bar. The programs in the application program layer can be integrated in the video-on-demand program and displayed through one control of the navigation bar, and can also be further displayed after the application control in the navigation bar is selected.
In some embodiments, the display device may directly enter a display interface of a signal source selected last time after being started, or a signal source selection interface, where the signal source may be a preset video-on-demand program, or may be at least one of an HDMI interface and a live tv interface, and after a user selects a different signal source, the display may display content obtained from the different signal source.
For clarity of explanation of the embodiments of the present application, a network architecture provided by the embodiments of the present application is described below with reference to fig. 6.
Referring to fig. 6, fig. 6 is a schematic diagram of a network architecture according to an embodiment of the present application. In fig. 6, the smart device is configured to receive input information and output a processing result of the information; the voice recognition service equipment is electronic equipment with voice recognition service deployed, the semantic service equipment is electronic equipment with semantic service deployed, and the business service equipment is electronic equipment with business service deployed. The electronic device may include a server, a computer, and the like, and the speech recognition service, the semantic service (also referred to as a semantic engine), and the business service are web services that can be deployed on the electronic device, wherein the speech recognition service is used for recognizing audio as text, the semantic service is used for performing semantic parsing on the text, and the business service is used for providing specific services such as a weather query service for ink weather, a music query service for QQ music, and the like. In one embodiment, there may be multiple business service devices deployed with different business services in the architecture shown in FIG. 6. If no special description is provided, the service device is a server of each type in the embodiment of the present application.
The following describes, by way of example, a process for processing information input to an intelligent device based on the architecture shown in fig. 6, where the information input to the intelligent device is an example of a query statement input by voice, the process may include the following three stages:
1. stage of speech recognition
The intelligent device can upload the audio of the query sentence to the voice recognition service device after receiving the query sentence input by voice, so that the voice recognition service device can recognize the audio as a text through the voice recognition service and then return the text to the intelligent device.
In one embodiment, before the audio of the query statement is uploaded to the speech recognition service device, the smart device may perform denoising processing on the audio of the query statement, where the denoising processing may include removing echo and environmental noise.
2. Semantic understanding phase
The intelligent device uploads the text of the query sentence identified by the voice identification service to the semantic service device, and the semantic service device performs semantic analysis on the text through semantic service to obtain the service field, intention and the like of the text.
3. Response phase
And the semantic service equipment issues a query instruction to corresponding business service equipment according to the semantic analysis result of the text of the query statement so as to obtain a query result given by the business service. The intelligent device can obtain the query result from the semantic service device and output the query result, for example, the query result is output to the display device in a wireless or infrared mode. As an embodiment, the semantic service device may further send a semantic parsing result of the query statement to the intelligent device, so that the intelligent device outputs a feedback statement in the semantic parsing result. The semantic service equipment can also send the semantic analysis result of the query statement to the display equipment, so that the intelligent equipment outputs the feedback statement in the semantic analysis result.
It should be noted that the architecture shown in fig. 6 is only an example, and does not limit the scope of the present application. In the embodiment of the present application, other architectures may also be used to implement similar functions, which are not described herein.
The controller 250 of the display device 200 is communicatively connected to the display 275. If not specifically stated, the steps performed by the display device in the following embodiments are all understood to be performed by the controller 250 or the controller 250 cooperating with other components of the display device 200.
In some embodiments of the present application, the display device 200 may perform a screenshot operation on a current display screen displayed by the display 275 in response to the screenshot instruction, so as to obtain a screenshot image; then, an information acquisition request is sent to the server 400, and response information sent by the server 400 in response to the information acquisition request is received, and then the display 275 is controlled to display recommended content included in the response information. The information acquisition request comprises scene information corresponding to the screenshot image, and the response information comprises recommended content corresponding to the scene information.
The technical solutions provided by the embodiments of the present application are described below with reference to the accompanying drawings.
The controller of the display device 200 in the present application may receive various forms of screenshot instructions. After receiving the screenshot command, performing screenshot operation on the current display interface of the display 275 in response to the received screenshot command, and obtaining a screenshot image. After the screenshot image is obtained, the display device may display the screenshot image or a thumbnail of the screenshot image, for example, as shown in fig. 7, the thumbnail of the screenshot image may be displayed in a stacked display manner in an upper left corner of the display screen 275, or may not be displayed, which is not limited in this application.
The screenshot instruction may be sent to the display device 200 directly by the user, for example, a speech-type screenshot instruction such as "who the person is," where the piece of clothing is bought from, "what is in the screen" is sent to the display device 200 directly by a speech mode, or a screenshot instruction may be sent to the display device 200 by the user by operating a designated key or a function button of a device such as a mobile phone or a remote controller. The form of the screenshot command or the manner in which the display device 200 acquires the screenshot command is not limited in the present application.
According to different application scenes, the screenshot image may include all the content displayed on the current display interface or only include part of the content displayed on the current display interface. In order to reduce the data transmission amount and reduce the data processing amount in the image recognition process, the screenshot image may only include the content displayed in a partial area in the current display interface, for example, only include the content displayed in the video playing window in the current display interface, and not include the content outside the video playing window.
Due to the influence of the operation delay of the user or the data processing delay of the display device 200, the screenshot image may not include a target object, wherein the target object refers to an object that may be of interest to the user. For example, there may be a long time delay from when the user views the screen displayed on the display 275, when the user issues a screenshot command, and when the display device 200 actually performs the screenshot operation. The existence of the time delay may also cause that the finally obtained screenshot image is inconsistent with the display picture that the user wants to intercept, and may also cause that the image of the target object in the screenshot image is unclear or even does not contain the target object. The server 400 may not be able to identify the content of such a screenshot image, and may not be able to target objects from it, thereby providing the user with information that may be of interest to the user.
In order to avoid such a situation, when the display device 200 acquires the screenshot image, scene information corresponding to the screenshot image may also be acquired and then sent to the server 400. The server 400 may generate recommended content or complete image recognition based on the scene information, and further generate corresponding response information, and provide information that may be of interest to the user through the response information. Thus, regardless of the content or quality of the screenshot image, and regardless of whether the server 400 can identify the target object from the screenshot image, the server 400 may feed back recommended content or identification results that may be of interest to the display device 200 for display by the display device 200.
The technical solution of the present application will be further described with reference to some embodiments.
In some embodiments of the present application, the context information is a basis for the server 400 to provide recommended content, and after the server 400 acquires the context information, the recommended content or the identification result corresponding to the context information may be provided. That is, after acquiring the scene information, the server 400 may provide different recommended content or identification result to the display apparatus 200 according to the content of the scene information. The scene information may refer to any information other than the screenshot image, and in general, the scene information may include information associated with the video, information associated with the screenshot image, or operation state information of the display device 200, etc.
For example, the scene information may include one or more pieces of information associated with a video, such as a video ID that the display device 200 is playing the video, the video name, the video playing progress, or whether the video is a local video; one or more pieces of information associated with the screenshot image, such as the time when the display device 200 receives the screenshot instruction, the resolution of the screenshot image, the APP name for implementing the video playing, and the like, may also be included; or may also include information on one or more operation states related to information on an APP in which the display apparatus 200 is operating, a time for which the display apparatus 200 has been continuously operated, and the like.
In addition to sending the scene information to the server 400 through the information obtaining request, the display device 200 may also send other information such as a screenshot image to the server 400 through the information obtaining request, so that the server 400 performs content identification on the screenshot image, and further feeds back an identification result or feeds back recommended content determined based on the identification result to the display device 200. In order to improve the recognition effect of the screenshot image, in addition to sending the screenshot image to the server 400, auxiliary information for assisting the server 400 to perform content recognition on the screenshot image may be sent to the server 400. The auxiliary information may also be of various types, and for example, may include an image related to the screenshot image (e.g., a key frame in the video closest to the screenshot image, an image frame next to the screenshot image, a video clip including the screenshot image, etc.), or may also include information related to the video such as a video ID, a name, a source, etc. of the video.
There may be multiple ways for the display device 200 to send the information acquisition request, and in a general case, the display device 200 may send an information acquisition request including the scene information to the server 400 after acquiring the screenshot image, so as to send the scene information to the server 400 through the information acquisition request. Besides the scene information, the information acquisition request may also include other information such as the screenshot image or the auxiliary information. The information acquisition request may only include the screenshot image and the auxiliary information, but not the scene information, and the application is not limited thereto.
In some embodiments, the display apparatus 200 may transmit the information acquisition request to the server 400 only when a predetermined condition is met. For example, the display device 200 may send an information acquisition request to the server 400 only when the screenshot image includes a picture generated by playing a video, and may send the screenshot image to a content recognition server for content recognition in a normal manner if the screenshot image does not include the picture generated by playing the video.
In other embodiments, the display device 200 may also send the information obtaining request to the server 400 only after receiving a confirmation instruction sent by the user; if a confirmation instruction of the user is not received, the screenshot image can be sent to a content recognition server for content recognition in a form other than the information acquisition request in a normal mode after being acquired, and the information acquisition request is not sent; alternatively, neither the information acquisition request nor the screen shot image may be transmitted to the server 400. This application is not limited thereto.
In various embodiments of the present application, the video may be a video that is already stored in the display device 200 in advance, or may be a video that is generated (for example, a game picture) or captured immediately by the display device 200 (for example, an image captured by a camera), or may be a video corresponding to a streaming media, a live broadcast signal, or a television signal, and the type of the video is not limited in this application. The video locally stored in the display device 200 may also be various videos such as a streaming video played by the display device 200, a live television picture displayed by the display device 200, and a video image captured by a local camera of the display device 200.
The determination of whether the screen generated by playing the video is included in the screenshot image may be performed in various manners, and the display apparatus 200 may determine whether the screen generated by playing the video is included in the screenshot image according to an operating state of the display apparatus 200, a program being executed, or an instruction that has been received, and the like. For example, when the display device 200 is in a video playing state (i.e., a certain video is being played), it may be determined that the screenshot image includes a picture generated by playing the video; or when the current display picture contains a video playing window, determining that the screenshot image contains a picture generated by playing a video; alternatively, the display device 200 may determine whether the screenshot image includes a picture generated by playing a video through image recognition. Various specific implementation processes of the determination method are not described herein again.
The present application does not limit the types and the number of the servers 400, and the number and the types of the servers 400 may be different in different application scenarios. The server 400 may be independent of the display device 200, or may be a part of the display device 200. The number of the servers 400 may be one or more, a plurality of the servers 400 may be respectively used to implement different functions or provide different information, and the same server 400 may also be used to implement a plurality of functions or provide a plurality of different information. The display apparatus 200 may transmit the information acquisition request to all the servers 400, or may transmit the information acquisition request only to a part of the servers 400.
According to different contents contained in the information acquisition request or different specific types of the server 400, the server 400 processes the information acquisition request in different ways. The present application does not limit the functions that can be implemented by the server 400 and the implementation process of the functions. Accordingly, the content included in the information acquisition request and the content included in the response information may also be different.
The technical solution of the present application is further described below with reference to some specific examples.
In some embodiments, the information acquisition request includes a screenshot image, and the response information may include an identification result of a target object identified from the screenshot image; the corresponding server 400 may then comprise a content recognition server.
In this embodiment, the content recognition server is configured to perform content recognition on the screenshot image, and generate response information. The number of the content recognition servers may be multiple, and each content recognition server may be used for recognizing only one specific type of target object, for example, only one type of target object such as characters, persons, articles, and the like. The display apparatus 200 may select one or more content recognition servers as the selected server according to information such as the content of the screenshot command, the content of the confirmation command, and the like, and then transmit the information acquisition request to the selected server. For example, when the instruction of the screen capture is a voice instruction of "who this actor is", the character recognition server 400 for character recognition may be selected from the plurality of servers 400, and the information acquisition request may be sent to the character recognition server 400. Further, the response information may further include a content recommendation server, and then the content recommendation server determines recommended content according to the identification result, and then the content identification server or the content recommendation server sends the response information to the display device 200, where the response information may include the identification result and/or the recommended content.
By adopting the technical scheme in the embodiment, the server 400 which accords with the user information acquisition intention can be selected to identify the screenshot image, so that the identified result is more accordant with the expectation of the user.
In other embodiments, the information obtaining request includes scene information such as a video ID and a video playing progress, the response information may include an identification result of the target object, and the server 400 may include a content identification server.
When a user plays an online video, the user may select a version with lower definition (i.e., lower resolution or lower code rate) to play, but not select a version with highest definition (i.e., highest resolution or highest code rate) to play, depending on the restrictive conditions such as traffic or bandwidth. In this case, the sharpness of the captured image is also relatively poor, which results in an increase in difficulty of image recognition or a decrease in accuracy of image recognition. In this case, after receiving the information acquisition request, the content identification server may find the highest-definition version of the video according to the video ID of the video, then acquire the highest-definition version of the screenshot image from the highest-definition version of the video according to the playing progress of the video, and further perform content identification on the highest-definition version of the screenshot image to obtain a corresponding identification result. Further, the response information may also include a content recommendation server, and the content recommendation server determines recommended content according to the identification result.
By adopting the technical scheme in the embodiment, the server 400 does not need to directly obtain the screenshot image from the display device 200, and the display device 200 can only send the video ID of the video and the playing progress of the video to realize the content identification of the screenshot image, so that the data transmission amount can be reduced, and the traffic consumption of the display device 200 in a wireless network scene can be saved.
In other embodiments, the information obtaining request includes scene information such as a video ID and video description information of a video, the response information may include an identification result of a target object identified from the screenshot image, and the server 400 may include a content identification server.
Since the same target object may have different meanings in different scenes, the recognition result may be very simple or limited if only the content recognition is performed on the screenshot image. For example, the same actor may play different roles in different episodes, and if content recognition is performed only on the screenshot image, it is usually only possible to recognize who the actor is, but it is not possible to determine from which episode the screenshot image came from and who the actor plays in the episode. In this case, the display apparatus 200 may use the video ID, name, source, and the like description information of the video as the auxiliary information. When the information acquisition request includes the description information, the server 400 may first identify the screenshot image to generate a preliminary result, and then expand or process the preliminary result based on the auxiliary information to obtain an identification result. For example, the server 400 may first recognize the screenshot image to obtain an initial recognition result that an actor in the screenshot image is "zhangsan", then determine an episode corresponding to the screenshot image according to the description information, and further determine that the actor plays a role "lie four" in the episode according to the episode, so that the finally obtained recognition result may be "zhangsan" in the screenshot image and "lie four" in the actor in the episode. Further, the response information may further include a content recommendation server, and then determine recommended content according to the identification result, for example, an episode having the same or similar role as "liqu" is used as the recommended content, and then the content identification server or the content recommendation server sends the response information to the display device 200, where the response information may include the identification result and/or the recommended content, so as to enrich the content included in the identification result.
In other embodiments, the information obtaining request includes auxiliary information such as a screenshot image and at least one key frame, the response information may include an identification result of the target object, and the server 400 may include a content identification server.
According to different encoding modes, the video may include a key frame and a transition frame, and if the screenshot image is an image corresponding to the transition frame, the target object in the screenshot image may be unclear, so that the target object is low in recognition success rate. In this case, after receiving the information acquisition request, the content recognition server may directly perform content recognition on the key frames one by one without recognizing the screenshot image; or when the target object is not identified from the screenshot image, the key frame can be identified again. If a target object is identified from the key frame, response information containing the identification result of the target object in the key frame may be generated.
Further, the server 400 may further include a content identification server, and the response information may further include recommended content determined based on the identification result. The content recommendation server may determine recommended content according to the recognition result, and then transmit the response information to the display device 200 by the content recognition server or the content recommendation server. It should be noted that, in this embodiment, the key frame may also be replaced by an adjacent frame of the frame corresponding to the screenshot image, and the specific process is not described herein again. In this embodiment, when the screenshot image is an image corresponding to a transition frame, the nearest key frame or adjacent frame of the transition frame may be used as auxiliary information, so that the server 400 may perform content identification on the screenshot image in addition to content identification on the key frame, thereby improving the success rate of identifying the target object and avoiding that an identification result cannot be obtained due to poor screenshot operation opportunity of a user.
In other embodiments, the information obtaining request includes scene information such as a playing progress of the video, the response information may include recommended content determined based on the playing progress of the video, and the server 400 may include a content recommendation server.
In this embodiment, the content recommendation server may pre-store preset content associated with a different playing progress interval from the video. The playing progress interval can be a time period or a time point, different playing progress intervals can be discontinuous or overlapped with each other, the types of the preset contents associated with different playing progress intervals can be the same or different, and the preset contents associated with each playing progress interval can change along with user operation and time change.
For example, a first progress interval (e.g., 0 th to 15 th minutes of the video) of the video may be associated with some recommended videos as recommended content, a second progress interval (e.g., 5 th to 20 th minutes of the video) may be associated with some commodity recommendation information or purchase link as recommended content, and a third progress interval (e.g., 25 th to 30 th minutes of the video) may be associated with some keywords, which may be used to determine the recommended content. The playing progress of the video can be represented in a form of a video ID plus a playing time length. After the information acquisition request is acquired, the content recommendation server firstly determines which playing progress interval the playing progress of the video falls into according to the video ID and the playing duration, and then takes the preset content associated with the falling playing progress interval as the recommended content. For example, if the play progress falls within a first progress interval, the recommended video may be taken as recommended content; if the playing progress falls into a second progress interval, the commodity recommendation information or the purchase link can be used as recommended content; if the playing progress falls into a third progress interval, content searching can be performed by using the keyword, and then a search result is used as the recommended content.
For another example, a set of preset content may be associated with a specific video segment in the video, where the preset content may include actors appearing in the video segment and corresponding role information, and may further include media recommendation information, a commodity purchase link, and the like determined based on the interface content in the video segment. If the video segment corresponding to the playing progress is the specific video segment, the preset content can be used as recommended content.
By adopting the technical scheme of the embodiment, the screenshot image recognition function and the content recommendation function can be separated, so that the effect same as or similar to that of the screenshot image recognition can be realized even if the screenshot image is not recognized or is recognized in other modes except the embodiment of the application.
It should be noted that the above embodiments are only some embodiments of the present application, and do not represent all technical solutions of the present application, and the solutions or steps in different embodiments may be combined with each other, so as to form a new technical solution, which is not limited to this application, and is not described in detail again.
After receiving the response information, the display device 200 may display content such as recommended content included in the response information through the display 275. Besides displaying the recommended content and other content included in the response information, the display device 200 may also display the screenshot image or a thumbnail of the screenshot image, or display other information that has been generated or acquired by the display device 200.
The types of the recommended content may be various according to different application scenarios or different content recommendation servers, and the response information may include a plurality of recommended contents of different types. For example, the recommended content may include media asset recommendation information, a commodity purchase link, travel recommendation information, and the like. The display mode of the recommended content can be different according to the type of the recommended content.
In some embodiments, the display device 200 may be used to display the recommended content through a content recommendation interface. The content recommendation interface may have at least one display area for displaying the recommended content, and when there are more recommended content or a plurality of different types of recommended content, different display areas may be used to display different types of recommended content, for example, as shown in fig. 8A, or the same display area may be used to display different recommended content in a loop, for example, as shown in fig. 8B to 8C; besides the display area for displaying the recommended content, the content recommendation interface may further have at least one display area for displaying other information such as a thumbnail of the screenshot image, for example, as shown in fig. 8D. The content recommendation interface further includes a display area for displaying a thumbnail of the screenshot image, and a display area for displaying an operation button and prompt information, for example, as shown in fig. 8E.
The content recommendation interface may be displayed in various ways, and the following description is only given by way of example.
In some embodiments, the content recommendation interface may be displayed in a layer manner on the other interfaces in an overlapping manner. The layer may be a translucent, opaque, or partially area transparent layer, such as shown in fig. 8F. When the content recommendation interface is displayed on other interfaces in an overlapping manner, the content or the content display mode displayed by other interfaces may be kept unchanged, or may be paused until the content recommendation interface is no longer displayed. For example, if the content recommendation interface is superimposed on the video playing interface, the video playing interface may still maintain the playing state of the video (i.e., does not pause or exit the video playing), or may not pause or exit the video playing; if the content recommendation interface is superposed on the menu interface, the menu interface can still keep the periodical switching of the contents of the window or the control, or the contents of the menu interface can be frozen, so that the menu interface is not changed any more.
In other embodiments, the content recommendation interface may be displayed in the form of a pop-up window, i.e., or the content recommendation interface may occupy only a partial area of the display screen, for example, as shown in fig. 9. When the content recommendation interface is displayed in the form of a pop-up window, the pop-up window can be displayed on other interfaces in an overlapped mode. Similarly, when the pop-up window is displayed, the content or the content display mode displayed by other interfaces may remain unchanged.
In other embodiments, the content recommendation interface may be a specific display interface, the display device 200 may jump from the currently displayed interface to the content recommendation interface, and during the interface jump, the display device 200 may further display a corresponding transition effect or transition animation. And will not be described in detail herein.
Corresponding to the display device in the foregoing embodiment, the present application also provides a content display method. As shown in fig. 10, the content presentation method includes the steps of:
step 1001, receiving a screenshot command.
The manner in which the display device receives the screenshot instruction may be referred to above, and is not described herein again.
And step 1002, responding to the screenshot instruction, and performing screenshot operation on a current display picture displayed by the display.
After receiving the screenshot instruction, the display device may perform screenshot operation on a current display screen displayed by the display to obtain a screenshot image. The specific implementation manner of the screenshot operation and the acquisition manner of the screenshot image are not limited in the present application, and are not described herein.
Step 1003, sending an information acquisition request to the server.
The display device may transmit an information acquisition request to the server when a predetermined condition is satisfied. The predetermined condition may include that the screenshot image includes a graphic generated by playing a video, that a confirmation operation of the user is received, and the like. The information acquisition request may include scene information corresponding to the screenshot image, and may further include information such as the screenshot image and auxiliary information.
Step 1004, receiving response information sent by the server in response to the information acquisition request.
The number of the response messages may be only one or multiple, and when the number of the corresponding messages is multiple, different response messages may be sent by different servers. The number of the servers can be multiple, and the types can be multiple. The response information may include recommended content corresponding to the scene information, may include a recognition result of the screenshot image, or may include other information. For the related points, reference is made to the foregoing embodiments, and details are not repeated here.
Step 1005, displaying the content contained in the response information.
After receiving the response message, the display device may display all or part of the content included in the response message. For example, recommended content or content such as a recognition result included in the response information may be displayed.
Displaying the content included in the response information may be displaying the content included in the response information, or displaying a processing result obtained by further processing the content included in the response information, or displaying a content obtained by further searching based on the content included in the response information, which is not limited in this application.
By adopting the content display method provided by the application, the problem that the displayable content is too single due to the fact that the content is displayed based on the identification result of the screenshot image can be avoided, and the identified content is richer.
Finally, it should be noted that: the above embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and these modifications or substitutions do not depart from the scope of the technical solutions of the embodiments of the present application.
The foregoing description, for purposes of explanation, has been presented in conjunction with specific embodiments. However, the illustrative discussions above are not intended to be exhaustive or to limit the embodiments to the precise forms disclosed above. Many modifications and variations are possible in light of the above teaching. The embodiments were chosen and described in order to best explain the principles and the practical application, to thereby enable others skilled in the art to best utilize the embodiments and various embodiments with various modifications as are suited to the particular use contemplated.

Claims (10)

1. A display device, characterized in that the display device comprises:
the display is used for presenting a playing picture of video content, wherein the video comprises key frames and transition frames;
a communicator configured to communicate with a server;
a controller in communicative connection with the display, the controller configured to:
acquiring an instruction sent by a user for acquiring recommended content according to the playing picture;
responding to the instruction for acquiring the recommended content, performing screenshot operation on a current display picture displayed by the display, and generating a screenshot image;
sending an information acquisition request related to playing the video, which comprises the screenshot image and the current display picture, to a server, so that the server does not identify a target object from the screenshot image, and when detecting that the acquired screenshot image is a transition frame, performing content identification on a key frame close to the transition frame;
receiving response information sent by the server in response to the information acquisition request, wherein the response information comprises recommended content corresponding to target images or scene information in key frames associated with the screenshot images;
and controlling the display to display recommended content contained in the response information.
2. The display device of claim 1,
the information acquisition request also comprises auxiliary information for assisting the server to identify the content of the screenshot image, wherein when the screenshot image is an image corresponding to a transition frame, the key frame or adjacent frame closest to the transition frame is used as the auxiliary information.
3. The display device according to claim 2, wherein sharpness information of the play-back picture is further included in the auxiliary information.
4. The display device according to claim 1, wherein in the step of controlling the display to display the recommended content included in the response information, the controller is further configured to:
and controlling the display to display the identification result in a first display area, and controlling the display to display the recommended content in a second display area.
5. The display device according to claim 4, wherein in the step of controlling the display to display the recommended content included in the response information, the controller is further configured to:
controlling the display to display the recognition result;
and after receiving a switching instruction, controlling the display to switch to display the recommended content.
6. The content presentation method is applied to a display device, wherein the display device comprises a display, a communicator and a controller, and the communicator is configured to communicate with a server; the controller is connected with the display in a communication mode and comprises:
acquiring an instruction sent by a user for acquiring recommended content according to the playing picture;
responding to the instruction for acquiring the recommended content, and performing screenshot operation on a current display picture displayed by the display to generate a screenshot image;
sending an information acquisition request related to playing the video, which comprises the screenshot image and the current display picture, to a server, so that the server does not identify a target object from the screenshot image, and when detecting that the acquired screenshot image is a transition frame, performing content identification on a key frame close to the transition frame;
receiving response information sent by the server in response to the information acquisition request, wherein the response information comprises recommended content corresponding to target images or scene information in key frames associated with the screenshot images;
and controlling the display to display recommended content contained in the response information.
7. The content presentation method according to claim 6, wherein the information acquisition request further includes auxiliary information for assisting a server in content identification of the screenshot image, and wherein when the screenshot image is an image corresponding to a transition frame, a key frame or an adjacent frame closest to the transition frame is used as the auxiliary information.
8. The content presentation method of claim 7, wherein the auxiliary information further includes sharpness information of the playing screen.
9. An image recognition method is applied to a server, and is characterized in that:
after receiving a screenshot image and an information acquisition request related to playing the video in a current display picture, identifying the content of the screenshot image;
if the target object is identified, the recommended content corresponding to the target is issued to the display equipment;
and if the target object is not identified, detecting whether the screenshot image is a transition frame, if so, searching a key frame associated with the transition frame, and identifying the content.
10. The server according to claim 9,
and if the key frame is still identified as the target object, detecting whether the definition of the video is smaller than a definition threshold value of image analysis, searching the key frame at the corresponding position of the screenshot image in the playing video with the highest definition under the condition that the definition is smaller than the definition threshold value, and carrying out image analysis according to the key frame, wherein the playing video with the highest definition is the playing video with the highest definition searched by the server according to the ID.
CN202211519109.7A 2020-12-11 2020-12-11 Display device and content presentation method Pending CN115776585A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211519109.7A CN115776585A (en) 2020-12-11 2020-12-11 Display device and content presentation method

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202011461720.XA CN112601117B (en) 2020-12-11 2020-12-11 Display device and content presentation method
CN202211519109.7A CN115776585A (en) 2020-12-11 2020-12-11 Display device and content presentation method

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
CN202011461720.XA Division CN112601117B (en) 2020-07-14 2020-12-11 Display device and content presentation method

Publications (1)

Publication Number Publication Date
CN115776585A true CN115776585A (en) 2023-03-10

Family

ID=75192972

Family Applications (2)

Application Number Title Priority Date Filing Date
CN202011461720.XA Active CN112601117B (en) 2020-07-14 2020-12-11 Display device and content presentation method
CN202211519109.7A Pending CN115776585A (en) 2020-12-11 2020-12-11 Display device and content presentation method

Family Applications Before (1)

Application Number Title Priority Date Filing Date
CN202011461720.XA Active CN112601117B (en) 2020-07-14 2020-12-11 Display device and content presentation method

Country Status (1)

Country Link
CN (2) CN112601117B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2022012299A1 (en) * 2020-07-14 2022-01-20 海信视像科技股份有限公司 Display device and person recognition and presentation method
CN117271872A (en) * 2022-06-15 2023-12-22 北京有竹居网络技术有限公司 Recommendation method, recommendation device, recommendation apparatus, recommendation storage medium and recommendation computer program product

Family Cites Families (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8667054B2 (en) * 2010-07-12 2014-03-04 Opus Medicus, Inc. Systems and methods for networked, in-context, composed, high resolution image viewing
KR102120771B1 (en) * 2013-02-13 2020-06-09 삼성전자주식회사 Display apparatus, server and control method thereof
US10452706B2 (en) * 2013-06-04 2019-10-22 Oath Inc. Method and system for handling images on a multi-touch device
JP6562289B2 (en) * 2013-12-26 2019-08-21 パナソニックＩｐマネジメント株式会社 Video editing device
US9591349B2 (en) * 2014-12-23 2017-03-07 Intel Corporation Interactive binocular video display
KR102088443B1 (en) * 2015-04-01 2020-03-12 삼성전자주식회사 Display apparatus for performing a search and Method for controlling display apparatus thereof
US10582201B2 (en) * 2016-05-19 2020-03-03 Qualcomm Incorporated Most-interested region in an image
CN108322806B (en) * 2017-12-20 2020-04-07 海信视像科技股份有限公司 Smart television and display method of graphical user interface of television picture screenshot
CN108040125A (en) * 2017-12-28 2018-05-15 北京陌上花科技有限公司 Content recognition and method for pushing and TV syndrome AI assistant devices
WO2020113020A1 (en) * 2018-11-29 2020-06-04 Google Llc Providing content related to objects detected in images
CN111010610B (en) * 2019-12-18 2022-01-28 维沃移动通信有限公司 Video screenshot method and electronic equipment

Also Published As

Publication number Publication date
CN112601117A (en) 2021-04-02
CN112601117B (en) 2022-10-28

Similar Documents

Publication Publication Date Title
CN114302190B (en) Display equipment and image quality adjusting method
CN111836109A (en) Display device, server and method for automatically updating column frame
CN112667184A (en) Display device
CN113014939A (en) Display device and playing method
CN113535019A (en) Display device and display method of application icons
CN112601117B (en) Display device and content presentation method
CN112653906A (en) Video hotspot playing method on display device and display device
CN113301420A (en) Content display method and display equipment
CN113490042A (en) Display device and channel searching method
CN112272331B (en) Method for rapidly displaying program channel list and display equipment
CN113111214A (en) Display method and display equipment for playing records
CN112580625A (en) Display device and image content identification method
CN113490032A (en) Display device and medium resource display method
CN112584213A (en) Display device and display method of image recognition result
CN113453069B (en) Display device and thumbnail generation method
CN112911381B (en) Display device, mode adjustment method, device and medium
CN114390332A (en) Display device and method for rapidly switching split-screen application
CN112601116A (en) Display device and content display method
CN113542901A (en) Display device and fast switching display method of network channels
CN113596559A (en) Method for displaying information in information bar and display equipment
CN112668546A (en) Video thumbnail display method and display equipment
CN112199560A (en) Setting item searching method and display device
CN111914565A (en) Electronic equipment and user statement processing method
CN113766164B (en) Display equipment and signal source interface display method
CN114302131A (en) Display device and black screen detection method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination