CN111163367B - Information searching method, device, equipment and medium based on playing video - Google Patents
Information searching method, device, equipment and medium based on playing video Download PDFInfo
- Publication number
- CN111163367B CN111163367B CN202010018478.2A CN202010018478A CN111163367B CN 111163367 B CN111163367 B CN 111163367B CN 202010018478 A CN202010018478 A CN 202010018478A CN 111163367 B CN111163367 B CN 111163367B
- Authority
- CN
- China
- Prior art keywords
- video
- information
- user
- query
- determining
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 65
- 230000008569 process Effects 0.000 claims abstract description 21
- 230000001960 triggered effect Effects 0.000 claims abstract description 9
- 230000015654 memory Effects 0.000 claims description 19
- 238000012545 processing Methods 0.000 claims description 9
- 230000000694 effects Effects 0.000 abstract description 7
- 238000010586 diagram Methods 0.000 description 7
- 230000003993 interaction Effects 0.000 description 7
- 230000008901 benefit Effects 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 5
- 238000004891 communication Methods 0.000 description 4
- 238000004590 computer program Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000015572 biosynthetic process Effects 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 238000003786 synthesis reaction Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000977 initiatory effect Effects 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000001151 other effect Effects 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/85—Assembly of content; Generation of multimedia applications
- H04N21/854—Content authoring
- H04N21/8547—Content authoring involving timestamps for synchronizing content
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/21—Server components or server architectures
- H04N21/218—Source of audio or video content, e.g. local disk arrays
- H04N21/2187—Live feed
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/232—Content retrieval operation locally within server, e.g. reading video streams from disk arrays
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
- H04N21/23418—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/431—Generation of visual interfaces for content selection or interaction; Content or additional data rendering
- H04N21/4312—Generation of visual interfaces for content selection or interaction; Content or additional data rendering involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/47—End-user applications
- H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Databases & Information Systems (AREA)
- Human Computer Interaction (AREA)
- Computer Security & Cryptography (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the application discloses an information searching method, device, equipment and medium based on playing video, relating to the technical field of information searching, wherein the method comprises the following steps: identifying audio information sent by the intelligent sound box, and determining a target video; acquiring a query instruction sent by an intelligent sound box, wherein the query instruction is triggered based on a currently played video picture in the process of watching a target video by a user, and the query instruction carries a time stamp corresponding to the video picture; determining a video picture corresponding to the time stamp in the target video; and searching information based on the identification result of the query instruction and the video picture. The embodiment of the application can realize the effect of conveniently and accurately searching the information related to the video picture on the premise of not influencing the user to watch the video, and meets the searching requirement of the user on the unknown information in the currently played video picture.
Description
Technical Field
The embodiment of the application relates to a computer technology, in particular to an information searching technology, and especially relates to an information searching method, device, equipment and medium based on playing video.
Background
Currently, a video playing device does not support interaction with a user during the process of watching a video. For example, when a user views a television, the user wants to know the name of a certain star in the current playing picture or the information of a background point in the current picture, and the conventional television and the set-top box cannot interact with the user and cannot timely feed back the information required by the user.
Disclosure of Invention
The embodiment of the application discloses an information searching method, device, equipment and medium based on a playing video, which are used for realizing the effect of conveniently and accurately searching information related to a video picture on the premise of not influencing the watching of the video by a user and meeting the searching requirement of the user on unknown information in the currently played video picture.
In a first aspect, an embodiment of the present application discloses an information searching method based on playing video, including:
identifying audio information sent by the intelligent sound box, and determining a target video;
acquiring a query instruction sent by the intelligent sound box, wherein the query instruction is triggered based on a currently played video picture in the process of watching the target video by a user, and the query instruction carries a timestamp corresponding to the video picture;
determining a video picture corresponding to the time stamp in the target video;
and searching information based on the identification result of the query instruction and the video picture.
One embodiment of the above application has the following advantages or benefits: the method and the device have the advantages that the effect of conveniently and accurately searching information related to the video picture on the premise that the user watches the video is not affected, the searching requirement of the user on unknown information in the currently played video picture is met, and the existing information searching mode is enriched.
Optionally, based on the identification result of the query instruction and the video frame, performing information search includes:
identifying the query instruction and determining the query requirement of a user, wherein the query requirement comprises a query object;
determining an image sub-region in the video frame based on the query object;
and searching information meeting the query requirement based on the image subareas.
Optionally, the audio information includes a video playing voice command sent by the user, where the video playing voice command is used to instruct the intelligent sound box to control the playing device to play the target video;
correspondingly, the identifying the audio information sent by the intelligent sound box and determining the target video comprise the following steps:
acquiring the video playing voice instruction sent by the intelligent sound box;
and identifying the program name or channel information included in the video playing voice instruction, and determining the target video.
Optionally, the audio information includes the audio of the program currently played by the playing device;
correspondingly, the identifying the audio information sent by the intelligent sound box and determining the target video comprise the following steps:
acquiring the program audio frequency currently played by the playing equipment and sent by the intelligent sound box;
and identifying the content of the program audio and determining the target video.
Optionally, the method further comprises:
and sending the search result to video superposition equipment in real time, wherein the video superposition equipment is used for carrying out superposition processing on video information received in real time and the search result according to the timestamp in the query instruction, so that the search result is displayed on a video picture corresponding to the timestamp in a superposition mode.
Optionally, the method further comprises:
and sending the search result to the intelligent sound box or the user terminal.
One embodiment of the above application has the following advantages or benefits: the diversity and flexibility of the feedback mode of the search result in the scheme of the embodiment are reflected, so that a user can conveniently acquire unknown information in a video picture on the premise of not influencing the current video watching.
Optionally, the target video comprises live program video.
In a second aspect, the embodiment of the present application further discloses an information searching apparatus based on playing video, including:
the target video determining module is used for identifying the audio information sent by the intelligent sound box and determining a target video;
the query instruction acquisition module is used for acquiring a query instruction sent by the intelligent sound box, wherein the query instruction is triggered based on a currently played video picture in the process of watching the target video by a user, and the query instruction carries a time stamp corresponding to the video picture;
a video picture determining module, configured to determine, in the target video, a video picture corresponding to the timestamp;
and the information searching module is used for searching information based on the identification result of the query instruction and the video picture.
In a third aspect, an embodiment of the present application further discloses an electronic device, including:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method for searching information based on playing video according to any one of the embodiments of the present application.
In a fourth aspect, embodiments of the present application further disclose a non-transitory computer readable storage medium storing computer instructions for causing a computer to perform the method for searching information based on playing video according to any of the embodiments of the present application.
According to the technical scheme provided by the embodiment of the application, the query instruction about the video picture sent by the user in the video watching process is obtained, the video picture is determined according to the timestamp in the query instruction, and the information required by the user is determined based on the image search, so that the problems that the traditional television and the set top box cannot interact with the user and cannot timely feed back the information required by the user are solved, the effect of conveniently and accurately searching the information related to the video picture on the premise that the user watches the video is not influenced, and the search requirement of the user on the unknown information in the currently played video picture is met; meanwhile, the intelligent sound box is used as a bridge between video playing equipment and search service, so that the existing information search mode is enriched. Other effects of the above alternative will be described below in connection with specific embodiments.
Drawings
The drawings are included to provide a better understanding of the present application and are not to be construed as limiting the application. Wherein:
fig. 1 is a flowchart of a method for searching information based on a play video according to an embodiment of the present application;
FIG. 2 is a flow chart of another method for searching information based on playing video according to an embodiment of the present application;
FIG. 3 is a schematic diagram of a process for superimposing video information received in real time and search results according to an embodiment of the present application;
FIG. 4 is a schematic diagram of an interaction between a smart speaker, a server and a user terminal according to an embodiment of the present application;
fig. 5 is a schematic structural view of an information searching apparatus based on a play video according to an embodiment of the present application;
fig. 6 is a block diagram of an electronic device disclosed in accordance with an embodiment of the application.
Detailed Description
Exemplary embodiments of the present application will now be described with reference to the accompanying drawings, in which various details of the embodiments of the present application are included to facilitate understanding, and are to be considered merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the application. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
Fig. 1 is a flowchart of an information searching method based on playing video according to an embodiment of the present application, where the embodiment may be applied to a case where, in a process of watching video by a user, unknown information in a currently playing video frame is searched by an intelligent sound box, the video watched by the user includes a television program and an online video. The method of the embodiment can be executed by an information searching device based on playing video, the device can be realized in a software and/or hardware mode, and can be integrated on any electronic equipment with computing capability, such as a background server corresponding to an intelligent sound box, and the like.
As shown in fig. 1, the information searching method based on playing video disclosed in this embodiment includes:
s101, identifying audio information sent by the intelligent sound box, and determining a target video.
In this embodiment, the user may control the video playing device to play the video content that the user wants to see through the smart speaker. The audio information sent by the intelligent sound box to the server is used for determining the target video by the server, and the server can simultaneously determine the playing progress of the target video in the process of determining the target video. The specific content of the audio information is related to the current state of the playing device, for example, if the video waiting to be played, the audio information may include a video playing voice command sent by a user to the intelligent sound box, the video playing progress corresponding to the command sending time is 0, and the video playing progress increases along with time recursion; if the video playing is medium, the audio information may include the program audio currently played by the playing device, and the corresponding target video and the video playing progress may be determined based on the program audio. Further, the target video includes, but is not limited to, live program video.
As for the identification of the audio information, any available audio identification and processing technique in the prior art may be used, and the present embodiment is not particularly limited.
The first example, the audio information sent by the intelligent sound box includes a video playing voice command sent by the user, where the video playing voice command is used to instruct the intelligent sound box to control the playing device to play the target video, that is, after the intelligent sound box receives the video playing voice command of the user, the intelligent sound box directly controls the playing device to play the target video, and meanwhile, the server triggers the monitoring of the current video playing condition of the playing device according to the video playing voice command. Correspondingly, identifying the audio information sent by the intelligent sound box, and determining the target video comprises the following steps: acquiring a video playing voice instruction sent by an intelligent sound box; and identifying the program name or channel information included in the video playing voice instruction, and determining the target video.
Exemplary two, the audio information includes the program audio currently played by the playback device; correspondingly, identifying the audio information sent by the intelligent sound box, and determining the target video comprises the following steps: acquiring the program audio frequency currently played by the playing equipment sent by the intelligent sound box; and identifying the content of the program audio and determining the target video. The server can identify the received program audio, determine the keywords of the video program based on the identification result, then search in the video database based on the keywords, determine the target video currently played by the playing device, and determine the video playing progress based on the matching of the received program audio and the complete audio in the target video. The time length of the program audio received by the server can be flexibly set on the basis of ensuring that the target video can be determined, for example, a reasonable video duration value can be determined based on duration statistics, and the embodiment is not particularly limited.
S102, acquiring a query instruction sent by the intelligent sound box, wherein the query instruction is triggered based on a currently played video picture in the process of watching a target video by a user, and the query instruction carries a time stamp corresponding to the video picture.
The user can send a query instruction to the intelligent sound box based on any unknown information in the currently watched video picture, and the current video watching of the user is not influenced. For example, in the process of watching a video, a user wants to know the name of a certain female star in a current video picture, and can send a query instruction to the intelligent sound box: what name is the female star of this red dress? After receiving the inquiry command of the user, the intelligent sound box can automatically mark a time stamp on the inquiry command to indicate the receiving time of the inquiry command, and then the time stamp is carried in the inquiry command and sent to the server. In addition, the user may further include current time information in the query instruction as a time stamp corresponding to the currently viewed video picture when the query instruction is issued. For example, the user may send a query instruction to the smart speaker: what is the name of a 7-point 35-out red dress-penetrating female star?
S103, in the target video, determining a video picture corresponding to the time stamp.
The corresponding relation exists between the time stamp in the query instruction and the video playing progress, so that after the server analyzes the time stamp in the user query instruction, the corresponding video picture can be determined based on the time stamp.
In the playing process of the target video, the intelligent sound box can also monitor the audio sent by the playing equipment in real time and forward the audio to the server in real time, so that the server can determine the playing progress of the target video based on the program audio received in real time through audio identification and matching. Of course, the intelligent sound box can also select to send only the audio frequency when the video starts to be played, namely the initially detected program audio frequency, to the server, so that the monitoring operation of the server on the video playing progress is triggered, and then the server monitors the video playing progress based on the time lapse. For live program video, the video playing start time is considered to be determined, so that after the server determines the target video, the video playing progress can also be determined directly based on the current time.
And S104, searching information based on the identification result of the query instruction and the video picture.
The server can identify the image elements in the video picture by any available image identification technology in the prior art, then determines the image subarea queried by the user by combining the identification result of the user query instruction, and then invokes the search engine to search information so as to meet the query requirement of the user. Here, it should be noted that, in the prior art, the search is generally performed based on information known to the user, such as keywords, and for the video screen information unknown to the user, the user cannot determine the corresponding keywords, and thus cannot initiate an effective information search. Even if the user initiates the search by using the fuzzy description information, the search results fed back by the search engine have larger deviation from the information required by the user ideal, or the number of the fed back search results is larger, and the user needs to automatically screen the information, so that the search efficiency of the user is lower and the search results are not ideal. According to the scheme, the corresponding video picture is determined based on the time stamp in the query instruction, so that the information required by the user is accurately searched based on the initiation of image search, the search efficiency is improved, and the search accuracy is improved.
Illustratively, performing information search based on the identification result of the query instruction and the video picture includes:
identifying a query instruction, and determining a query requirement of a user, wherein the query requirement comprises a query object;
determining an image subarea in the video picture based on the query object;
based on the image sub-regions, information satisfying the user query needs is searched.
Continuing with the above example, the server identifies the query instruction of the user, and can determine that the query object is "female star wearing a red dress", then determine the image sub-area containing the query object on the video frame based on the identification result of each image element on the video frame, and then determine the information of the female star by using the image search technology.
According to the technical scheme provided by the embodiment of the application, the query instruction about the video picture sent by the user in the video watching process is obtained, the video picture is determined according to the timestamp in the query instruction, and the information required by the user is determined based on the image search, so that the problems that the traditional television and the set top box cannot interact with the user and cannot timely feed back the information required by the user are solved, the effect of conveniently and accurately searching the information related to the video picture on the premise that the user watches the video is not influenced, and the search requirement of the user on the unknown information in the currently played video picture is met; meanwhile, the intelligent sound box is used as a bridge between video playing equipment and search service, so that the existing information search mode is enriched.
Fig. 2 is a flowchart of another information searching method based on playing video according to an embodiment of the present application, which is further optimized and expanded based on the above technical solution, and may be combined with the above various alternative embodiments. As shown in fig. 2, the method includes:
s201, identifying audio information sent by the intelligent sound box, and determining a target video.
S202, acquiring a query instruction sent by the intelligent sound box, wherein the query instruction is triggered based on a currently played video picture in the process of watching a target video by a user, and the query instruction carries a time stamp corresponding to the video picture.
S203, in the target video, determining a video picture corresponding to the time stamp.
S204, searching information based on the identification result of the query instruction and the video picture.
And S205, sending the search result to video superposition equipment in real time, wherein the video superposition equipment is used for carrying out superposition processing on video information received in real time and the search result according to the time stamp in the query instruction, so that the search result is superposed and displayed on a video picture corresponding to the time stamp.
S206, sending the search result to the intelligent sound box or the user terminal.
The operations S205 and S206 may be alternatively executed according to actual needs, or may be executed simultaneously, which is not limited in this embodiment.
Fig. 3 is a schematic diagram of a process for superimposing video information received in real time and search results according to an embodiment of the present application. As shown in fig. 3, the video overlaying device, that is, the new hardware, may perform analog-to-digital conversion processing or decoding processing on video information (including a television program signal or a network video signal) received in real time, so that the video signal may be converted into a data format that can be identified and operated by the video overlaying device, then the search result is displayed on a new image layer, and video synthesis is implemented in a mode of overlaying the image layer according to a timestamp in a user query instruction, so that the search result is displayed on a corresponding video picture, and finally output to the video playing device, thereby indirectly implementing interaction between the video playing device and a user. It should be noted that, regarding the superposition or synthesis of the original video frame and the search result, any available video processing technology in the prior art may be used, and the embodiment is not limited specifically.
Fig. 4 is a schematic diagram of interaction among a smart speaker, a server and a user terminal according to an embodiment of the present application, which should not be construed as a specific limitation of the embodiment of the present application. The user can preset the receiver of the search result through an application program which is installed on the terminal and is relevant to the intelligent sound box function implementation. The server can send the search result to the intelligent sound box or the user terminal, so that the diversity and flexibility of feedback on the search requirement of the user in the scheme of the embodiment are reflected. Further, after the intelligent sound box or the user terminal receives the search result sent by the server, an information prompt can be timely sent to the user to prompt the user to check the search result, and the search result can also be directly subjected to voice broadcasting.
According to the technical scheme provided by the embodiment of the application, the query instruction about the video picture sent by the user in the video watching process is obtained, the video picture is determined according to the timestamp in the query instruction, and the information required by the user is determined based on the image search, so that the problems that the traditional television and the set top box cannot interact with the user and cannot timely feed back the information required by the user are solved, the effect of conveniently and accurately searching the information related to the video picture on the premise that the user watches the video is not influenced, and the search requirement of the user on the unknown information in the currently played video picture is met; meanwhile, the intelligent sound box is used as a bridge between video playing equipment and search service, so that the existing information search mode is enriched; and the search results can be displayed on the corresponding video pictures in a superimposed manner, and can also be sent to the intelligent sound box or the user terminal, so that the diversity and flexibility of the feedback mode of the search results in the scheme of the embodiment are reflected, and the user can conveniently acquire the unknown information in the video pictures.
Fig. 5 is a schematic structural diagram of an information searching apparatus 300 based on playing video according to an embodiment of the present application, which can be applied to a case of searching unknown information in a currently playing video frame through an intelligent speaker in a process of watching video by a user. The device can be realized in a software and/or hardware mode, and can be integrated on any electronic equipment with computing capability, such as a background server corresponding to the intelligent sound box.
As shown in fig. 5, the information searching apparatus 300 based on playing video disclosed in the present embodiment may include a target video determining module 301, a query instruction acquiring module 302, a video screen determining module 303, and an information searching module 304, wherein:
the target video determining module 301 is configured to identify audio information sent by the intelligent speaker, and determine a target video;
the query instruction obtaining module 302 is configured to obtain a query instruction sent by the intelligent sound box, where the query instruction is triggered based on a currently played video picture in a process of watching a target video by a user, and the query instruction carries a timestamp corresponding to the video picture;
a video picture determining module 303, configured to determine, in the target video, a video picture corresponding to the timestamp;
the information searching module 304 is configured to perform information searching based on the identification result of the query instruction and the video frame.
Optionally, the information search module 304 includes:
the query requirement determining unit is used for identifying a query instruction and determining the query requirement of a user, wherein the query requirement comprises a query object;
an image sub-region determining unit configured to determine an image sub-region in a video picture based on a query object;
and the information searching unit is used for searching information meeting the query requirement based on the image subareas.
Optionally, the audio information includes a video playing voice command sent by the user, where the video playing voice command is used to instruct the intelligent sound box to control the playing device to play the target video;
accordingly, the target video determining module 301 is specifically configured to:
acquiring a video playing voice instruction sent by an intelligent sound box;
and identifying the program name or channel information included in the video playing voice instruction, and determining the target video.
Optionally, the audio information includes audio of a program currently played by the playing device;
accordingly, the target video determining module 301 is specifically configured to:
acquiring the program audio frequency currently played by the playing equipment sent by the intelligent sound box;
and identifying the content of the program audio and determining the target video.
Optionally, the apparatus disclosed in this embodiment further includes:
the first search result sending module is used for sending the search result to the video superposition equipment in real time, wherein the video superposition equipment is used for superposing the video information received in real time and the search result according to the time stamp in the query instruction, so that the search result is superposed and displayed on a video picture corresponding to the time stamp.
Optionally, the apparatus disclosed in this embodiment further includes:
and the second search result sending module is used for sending the search result to the intelligent sound box or the user terminal.
Optionally, in this embodiment, the target video includes live program video.
The information searching device 300 based on playing video disclosed in the embodiment of the application can execute the information searching method based on playing video disclosed in the embodiment of the application, and has the corresponding functional modules and beneficial effects of the executing method. Reference is made to the description of any method embodiment of the application for details not described in this embodiment.
According to an embodiment of the present application, the present application also provides an electronic device and a readable storage medium.
As shown in fig. 6, fig. 6 is a block diagram of an electronic device for implementing an information search method based on playing video in an embodiment of the present application. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the embodiments of the application described and/or claimed herein. Preferably, the electronic device disclosed in the embodiment includes a background server capable of performing information interaction with the intelligent sound box.
As shown in fig. 6, the electronic device includes: one or more processors 401, memory 402, and interfaces for connecting the components, including a high-speed interface and a low-speed interface. The various components are interconnected using different buses and may be mounted on a common motherboard or in other manners as desired. The processor may process instructions executing within the electronic device, including instructions stored in or on memory to display graphical information of a graphical user interface (Graphical User Interface, GUI) on an external input/output device, such as a display device coupled to the interface. In other embodiments, multiple processors and/or multiple buses may be used, if desired, along with multiple memories and multiple memories. Also, multiple electronic devices may be connected, each providing a portion of the necessary operations, e.g., as a server array, a set of blade servers, or a multiprocessor system. One processor 401 is illustrated in fig. 6.
Memory 402 is a non-transitory computer readable storage medium provided by embodiments of the present application. The memory stores instructions executable by at least one processor to cause the at least one processor to perform the information searching method based on playing video provided by the embodiment of the application. The non-transitory computer-readable storage medium of the embodiment of the present application stores computer instructions for causing a computer to execute the information search method based on playing video provided by the embodiment of the present application.
The memory 402 is used as a non-transitory computer readable storage medium, and may be used to store non-transitory software programs, non-transitory computer executable programs, and modules, such as program instructions/modules corresponding to the information searching method based on playing video in the embodiment of the present application, for example, the target video determining module 301, the query instruction acquiring module 302, the video frame determining module 303, and the information searching module 304 shown in fig. 5. The processor 401 executes various functional applications of the server and data processing by running non-transitory software programs, instructions, and modules stored in the memory 402, that is, implements the information search method based on playing video in the above-described method embodiment.
Memory 402 may include a storage program area that may store an operating system, at least one application program required for functionality, and a storage data area; the storage data area may store data created according to the use of the electronic device based on the information search method of playing video, and the like. In addition, memory 402 may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory 402 may optionally include a memory remotely located with respect to the processor 401, which may be connected via a network to an electronic device for implementing the information searching method based on playing video in the present embodiment. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The electronic device for implementing the information searching method based on playing video in this embodiment may further include: an input device 403 and an output device 404. The processor 401, memory 402, input device 403, and output device 404 may be connected by a bus or otherwise, for example in fig. 6.
The input device 403 may receive input numeric or character information and generate key signal inputs related to user settings and function controls of an electronic device for implementing the information search method based on playing video in this embodiment, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, a pointer stick, one or more mouse buttons, a track ball, a joystick, etc. The output means 404 may include a display device, auxiliary lighting means, such as light emitting diodes (Light Emitting Diode, LEDs), tactile feedback means, and the like; haptic feedback devices such as vibration motors and the like. The display device may include, but is not limited to, a liquid crystal display (Liquid Crystal Display, LCD), an LED display, and a plasma display. In some implementations, the display device may be a touch screen.
Various implementations of the systems and techniques described here can be implemented in digital electronic circuitry, integrated circuitry, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.
These computing programs, also referred to as programs, software applications, or code, include machine instructions for a programmable processor, and may be implemented in a high-level procedural and/or object-oriented programming language, and/or in assembly/machine language. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and/or device for providing machine instructions and/or data to a programmable processor, e.g., magnetic discs, optical disks, memory, programmable logic devices (Programmable Logic Device, PLD), including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and/or data to a programmable processor.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device for displaying information to a user, for example, a Cathode Ray Tube (CRT) or an LCD monitor; and a keyboard and pointing device, such as a mouse or trackball, by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback, e.g., visual feedback, auditory feedback, or tactile feedback; and input from the user may be received in any form, including acoustic input, speech input, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a background component, e.g., as a data server, or that includes a middleware component, e.g., an application server, or that includes a front-end component, e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here, or any combination of such background, middleware, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication, e.g., a communication network. Examples of communication networks include: local area network (Local Area Network, LAN), wide area network (Wide Area Network, WAN) and the internet.
The computer system may include a client and a server. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.
According to the technical scheme provided by the embodiment of the application, the query instruction about the video picture sent by the user in the video watching process is obtained, the video picture is determined according to the timestamp in the query instruction, and the information required by the user is determined based on the image search, so that the problems that the traditional television and the set top box cannot interact with the user and cannot timely feed back the information required by the user are solved, the effect of conveniently and accurately searching the information related to the video picture on the premise that the user watches the video is not influenced, and the search requirement of the user on the unknown information in the currently played video picture is met; meanwhile, the intelligent sound box is used as a bridge between video playing equipment and search service, so that the existing information search mode is enriched; and the search results can be displayed on the corresponding video pictures in a superimposed manner, and can also be sent to the intelligent sound box or the user terminal, so that the diversity and flexibility of the feedback mode of the search results in the scheme of the embodiment are reflected, and the user can conveniently acquire the unknown information in the video pictures.
It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps described in the present application may be performed in parallel, sequentially, or in a different order, so long as the desired results of the technical solution disclosed in the present application can be achieved, and are not limited herein.
The above embodiments do not limit the scope of the present application. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application should be included in the scope of the present application.
Claims (6)
1. An information searching method based on playing video, comprising:
acquiring the program audio frequency currently played by the playing equipment sent by the intelligent sound box;
identifying the content of the program audio, determining a target video and determining the video playing progress of the target video based on the matching of the program audio and the complete audio in the target video; wherein the target video comprises live program video;
acquiring a query instruction sent by the intelligent sound box, wherein the query instruction is triggered based on a currently played video picture in the process of watching the target video by a user, and the query instruction carries a timestamp corresponding to the video picture; wherein, there is a correspondence between the time stamp in the inquiry instruction and the video playing progress;
determining a video picture corresponding to the time stamp in the target video;
identifying the query instruction, and determining a query requirement containing unknown information of a user, wherein the query requirement comprises a query object;
determining an image sub-region in the video frame based on the query object;
and searching information meeting the query requirement based on the image subareas.
2. The method according to claim 1, wherein the method further comprises:
and sending the search result to video superposition equipment in real time, wherein the video superposition equipment is used for carrying out superposition processing on video information received in real time and the search result according to the timestamp in the query instruction, so that the search result is displayed on a video picture corresponding to the timestamp in a superposition mode.
3. The method according to claim 1, wherein the method further comprises:
and sending the search result to the intelligent sound box or the user terminal.
4. An information search apparatus based on playing video, comprising:
the target video determining module is used for acquiring the program audio currently played by the playing equipment sent by the intelligent sound box; identifying the content of the program audio, determining a target video and determining the video playing progress of the target video based on the matching of the program audio and the complete audio in the target video; wherein the target video comprises live program video;
the query instruction acquisition module is used for acquiring a query instruction sent by the intelligent sound box, wherein the query instruction is triggered based on a currently played video picture in the process of watching the target video by a user, and the query instruction carries a time stamp corresponding to the video picture; wherein, there is a correspondence between the time stamp in the inquiry instruction and the video playing progress;
a video picture determining module, configured to determine, in the target video, a video picture corresponding to the timestamp;
the query requirement determining unit is used for identifying a query instruction and determining a query requirement containing unknown information of a user, wherein the query requirement comprises a query object;
an image sub-region determining unit configured to determine an image sub-region in a video picture based on a query object;
and the information searching unit is used for searching information meeting the query requirement based on the image subareas.
5. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of searching for information based on playing video of any one of claims 1-3.
6. A non-transitory computer-readable storage medium storing computer instructions for causing the computer to perform the broadcast video-based information search method of any one of claims 1-3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010018478.2A CN111163367B (en) | 2020-01-08 | 2020-01-08 | Information searching method, device, equipment and medium based on playing video |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010018478.2A CN111163367B (en) | 2020-01-08 | 2020-01-08 | Information searching method, device, equipment and medium based on playing video |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111163367A CN111163367A (en) | 2020-05-15 |
| CN111163367B true CN111163367B (en) | 2023-08-29 |
Family
ID=70561877
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010018478.2A Active CN111163367B (en) | 2020-01-08 | 2020-01-08 | Information searching method, device, equipment and medium based on playing video |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111163367B (en) |
Families Citing this family (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111696549A (en) * | 2020-06-02 | 2020-09-22 | 深圳创维－Rgb电子有限公司 | Picture searching method and device, electronic equipment and storage medium |
| CN111694983B (en) * | 2020-06-12 | 2023-12-19 | 百度在线网络技术（北京）有限公司 | Information display method, information display device, electronic equipment and storage medium |
| CN111866568B (en) * | 2020-07-23 | 2023-03-31 | 聚好看科技股份有限公司 | Display device, server and video collection acquisition method based on voice |
| CN111859159A (en) * | 2020-08-05 | 2020-10-30 | 上海连尚网络科技有限公司 | Information pushing method, video processing method and equipment |
| CN111818358A (en) * | 2020-08-07 | 2020-10-23 | 广州酷狗计算机科技有限公司 | Audio file playing method and device, terminal and storage medium |
| CN112423137B (en) * | 2020-10-26 | 2024-06-04 | 深圳Tcl新技术有限公司 | Video playing method, device and system, intelligent television and server |
| CN112866762A (en) * | 2020-12-31 | 2021-05-28 | 北京达佳互联信息技术有限公司 | Processing method and device for acquiring video associated information, electronic equipment and server |
| CN114125819B (en) * | 2022-01-25 | 2022-09-06 | 深圳市云动创想科技有限公司 | Playing method and device based on mobile communication technology and electronic equipment |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103108229A (en) * | 2013-02-06 | 2013-05-15 | 上海云联广告有限公司 | Method for identifying video contents in cross-screen mode through audio frequency |
| CN106488332A (en) * | 2016-10-25 | 2017-03-08 | 华为软件技术有限公司 | TV shopping method and apparatus |
| CN108012173A (en) * | 2017-11-16 | 2018-05-08 | 百度在线网络技术（北京）有限公司 | A kind of content identification method, device, equipment and computer-readable storage medium |
| CN109658932A (en) * | 2018-12-24 | 2019-04-19 | 深圳创维－Rgb电子有限公司 | A kind of apparatus control method, device, equipment and medium |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108255922A (en) * | 2017-11-06 | 2018-07-06 | 优视科技有限公司 | Video frequency identifying method, equipment, client terminal device, electronic equipment and server |
-
2020
- 2020-01-08 CN CN202010018478.2A patent/CN111163367B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103108229A (en) * | 2013-02-06 | 2013-05-15 | 上海云联广告有限公司 | Method for identifying video contents in cross-screen mode through audio frequency |
| CN106488332A (en) * | 2016-10-25 | 2017-03-08 | 华为软件技术有限公司 | TV shopping method and apparatus |
| CN108012173A (en) * | 2017-11-16 | 2018-05-08 | 百度在线网络技术（北京）有限公司 | A kind of content identification method, device, equipment and computer-readable storage medium |
| CN109658932A (en) * | 2018-12-24 | 2019-04-19 | 深圳创维－Rgb电子有限公司 | A kind of apparatus control method, device, equipment and medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111163367A (en) | 2020-05-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111163367B (en) | Information searching method, device, equipment and medium based on playing video | |
| WO2021068558A1 (en) | Simultaneous subtitle translation method, smart television, and storage medium | |
| CN111901615A (en) | Live video playing method and device | |
| US20170019450A1 (en) | Media production system with social media feature | |
| US20230061691A1 (en) | Display Apparatus with Intelligent User Interface | |
| CN110996136B (en) | Video resource display method and device | |
| US11706489B2 (en) | Display apparatus with intelligent user interface | |
| CN110501918B (en) | Intelligent household appliance control method and device, electronic equipment and storage medium | |
| CN111935551A (en) | Video processing method and device, electronic equipment and storage medium | |
| US20210264951A1 (en) | Computing System With DVE Template Selection And Video Content Item Generation Feature | |
| CN104202676A (en) | Channel switching method and system | |
| CN110913259A (en) | Video playing method and device, electronic equipment and medium | |
| CN103631526A (en) | Device and method for displaying search information | |
| CN110933227A (en) | Assistance method, device, equipment and medium for intelligent terminal | |
| CN111083538A (en) | Background image display method and device | |
| CN110750719A (en) | IPTV-based information accurate pushing system and method | |
| CN112383825B (en) | Video recommendation method and device, electronic equipment and medium | |
| CN111770388B (en) | Content processing method, device, equipment and storage medium | |
| US20210392394A1 (en) | Method and apparatus for processing video, electronic device and storage medium | |
| CN113536037A (en) | Video-based information query method, device, equipment and storage medium | |
| CN111901482B (en) | Function control method and device, electronic equipment and readable storage medium | |
| CN111352685B (en) | Display method, device, equipment and storage medium of input method keyboard | |
| CN112929717B (en) | Focus management method and display device | |
| CN110741652A (en) | Display device with intelligent user interface | |
| CN114598936B (en) | Subtitle batch generation and management method, system, device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20210512 Address after: 100085 Baidu Building, 10 Shangdi Tenth Street, Haidian District, Beijing Applicant after: BAIDU ONLINE NETWORK TECHNOLOGY (BEIJING) Co.,Ltd. Applicant after: Shanghai Xiaodu Technology Co.,Ltd. Address before: 100085 Baidu Building, 10 Shangdi Tenth Street, Haidian District, Beijing Applicant before: BAIDU ONLINE NETWORK TECHNOLOGY (BEIJING) Co.,Ltd. |
|
| TA01 | Transfer of patent application right | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |