CN113190695B - Multimedia data searching method and device, computer equipment and medium - Google Patents
Multimedia data searching method and device, computer equipment and medium Download PDFInfo
- Publication number
- CN113190695B CN113190695B CN202110492379.2A CN202110492379A CN113190695B CN 113190695 B CN113190695 B CN 113190695B CN 202110492379 A CN202110492379 A CN 202110492379A CN 113190695 B CN113190695 B CN 113190695B
- Authority
- CN
- China
- Prior art keywords
- multimedia data
- searched
- key frame
- frame sequence
- audio
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 56
- 230000008859 change Effects 0.000 claims abstract description 29
- 230000001360 synchronised effect Effects 0.000 claims abstract description 8
- 238000004590 computer program Methods 0.000 claims description 30
- 230000004044 response Effects 0.000 claims description 13
- 238000000605 extraction Methods 0.000 claims description 11
- 238000001514 detection method Methods 0.000 claims description 4
- 238000013473 artificial intelligence Methods 0.000 abstract description 8
- 238000004891 communication Methods 0.000 description 13
- 238000005516 engineering process Methods 0.000 description 12
- 238000012545 processing Methods 0.000 description 9
- 238000010586 diagram Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 6
- 230000008569 process Effects 0.000 description 5
- 238000013459 approach Methods 0.000 description 4
- 238000004422 calculation algorithm Methods 0.000 description 3
- 238000010801 machine learning Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000001413 cellular effect Effects 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000010267 cellular communication Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000008878 coupling Effects 0.000 description 1
- 238000010168 coupling process Methods 0.000 description 1
- 238000005859 coupling reaction Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 239000012634 fragment Substances 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 230000003924 mental process Effects 0.000 description 1
- 230000003278 mimic effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003058 natural language processing Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/40—Information retrieval; Database structures therefor; File system structures therefor of multimedia data, e.g. slideshows comprising image and additional audio data
- G06F16/43—Querying
- G06F16/432—Query formulation
- G06F16/433—Query formulation using audio data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/40—Information retrieval; Database structures therefor; File system structures therefor of multimedia data, e.g. slideshows comprising image and additional audio data
- G06F16/48—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/483—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Library & Information Science (AREA)
- Mathematical Physics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The disclosure provides a multimedia data searching method and device, computing equipment and medium, relates to the technical field of artificial intelligence, and particularly relates to the field of intelligent searching. The implementation scheme is as follows: acquiring multimedia data to be searched, wherein the multimedia data to be searched comprises a video to be searched and an audio to be searched which are synchronous in time; extracting a first key frame sequence from a video to be searched of the multimedia data to be searched according to a preset rule based on the change of the sound characteristics of the audio to be searched in a time domain; acquiring a second key frame sequence corresponding to each candidate multimedia data in the plurality of candidate multimedia data, wherein the second key frame sequence of each candidate multimedia data is extracted according to the preset rule; and determining multimedia data matched with the multimedia data to be searched in the plurality of candidate multimedia data based on the first key frame sequence and a second key frame sequence corresponding to each candidate multimedia data in the plurality of candidate multimedia data.
Description
Technical Field
The present disclosure relates to the field of artificial intelligence, and in particular, to the field of intelligent searching. And more particularly to a method, apparatus, electronic device, computer readable storage medium and computer program product for multimedia data searching.
Background
Artificial intelligence is the discipline of studying the process of making a computer mimic certain mental processes and intelligent behaviors (e.g., learning, reasoning, thinking, planning, etc.) of a person, both hardware-level and software-level techniques. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing, etc.: the artificial intelligence software technology mainly comprises a computer vision technology, a voice recognition technology, a natural language processing technology, a machine learning/deep learning technology, a big data processing technology, a knowledge graph technology and the like.
Multimedia can provide more rich information content than information transmission media such as text, images, etc. In some scenarios, a user wishes to obtain multimedia data he or she needs by searching. Existing multimedia data search methods are typically text-based searches, i.e., search results are obtained by matching search terms entered by a user with text labels of individual multimedia data in a multimedia database. This search method is independent of the content of the multimedia data itself, and only depends on the accuracy of the search word input by the user and the label marking of the multimedia data, so that the search result is generally difficult to satisfy the user.
The approaches described in this section are not necessarily approaches that have been previously conceived or pursued. Unless otherwise indicated, it should not be assumed that any of the approaches described in this section qualify as prior art merely by virtue of their inclusion in this section. Similarly, the problems mentioned in this section should not be considered as having been recognized in any prior art unless otherwise indicated.
Disclosure of Invention
The present disclosure provides a method, apparatus, computer device, computer readable storage medium and computer program product for multimedia data searching.
According to an aspect of the present disclosure, there is provided a multimedia data searching method, including: acquiring multimedia data to be searched, wherein the multimedia data to be searched comprises a video to be searched and an audio to be searched which are synchronous in time; extracting a first key frame sequence from a video to be searched of the multimedia data to be searched according to a preset rule based on the change of the sound characteristics of the audio to be searched in a time domain; acquiring a second key frame sequence corresponding to each candidate multimedia data in the plurality of candidate multimedia data, wherein the second key frame sequence of each candidate multimedia data is extracted according to the preset rule; and determining multimedia data matched with the multimedia data to be searched in the plurality of candidate multimedia data based on the first key frame sequence and a second key frame sequence corresponding to each candidate multimedia data in the plurality of candidate multimedia data.
According to another method of the present disclosure, there is provided a multimedia data searching apparatus including: a first acquisition unit configured to acquire multimedia data to be searched, wherein the multimedia data to be searched includes a video to be searched and an audio to be searched which are synchronized in time; the first extraction unit is configured to extract a first key frame sequence from the video to be searched of the multimedia data to be searched according to a preset rule based on the change of the sound characteristics of the audio to be searched in the time domain; the second acquisition unit is configured to acquire a second key frame sequence corresponding to each candidate multimedia data in the plurality of candidate multimedia data, wherein the second key frame sequence of each candidate multimedia data is extracted according to a preset rule; and a determining unit configured to determine multimedia data matching the multimedia data to be searched among the plurality of candidate multimedia data based on the first key frame sequence and the second key frame sequence corresponding to each of the plurality of candidate multimedia data.
According to another aspect of the present disclosure, there is provided a computer apparatus comprising: a memory, a processor and a computer program stored on the memory, wherein the processor is configured to execute the computer program to implement the steps of the above method.
According to another aspect of the present disclosure, there is provided a non-transitory computer readable storage medium having stored thereon a computer program, wherein the computer program when executed by a processor implements the steps of the above-described method.
According to another aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein the computer program, when executed by a processor, implements the steps of the above-described method.
According to one or more embodiments of the present disclosure, a multimedia data search scheme for "searching multimedia in multimedia" based on sound characteristics is provided. Matching between multimedia data is performed based on key frame sequences respectively extracted from the multimedia data to be searched and each candidate multimedia data, so that the data amount required to be processed in the process of multimedia data searching is effectively reduced, computing resources are saved, and the multimedia data searching efficiency is improved.
It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the disclosure, nor is it intended to be used to limit the scope of the disclosure. Other features of the present disclosure will become apparent from the following specification.
Drawings
The accompanying drawings illustrate exemplary embodiments and, together with the description, serve to explain exemplary implementations of the embodiments. The illustrated embodiments are for exemplary purposes only and do not limit the scope of the claims. Throughout the drawings, identical reference numerals designate similar, but not necessarily identical, elements.
FIG. 1 illustrates a schematic diagram of an exemplary system in which various methods described herein may be implemented, in accordance with an embodiment of the present disclosure;
fig. 2 shows a flowchart of a multimedia data search method according to an embodiment of the present disclosure;
fig. 3 illustrates a schematic diagram of matching types of multimedia data to be searched and candidate multimedia data according to an embodiment of the present disclosure;
fig. 4 shows a block diagram of a structure of a multimedia data search apparatus according to an embodiment of the present disclosure;
fig. 5 illustrates a block diagram of an exemplary electronic device that can be used to implement embodiments of the present disclosure.
Detailed Description
Exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present disclosure to facilitate understanding, and should be considered as merely exemplary. Accordingly, one of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
In the present disclosure, the use of the terms "first," "second," and the like to describe various elements is not intended to limit the positional relationship, timing relationship, or importance relationship of the elements, unless otherwise indicated, and such terms are merely used to distinguish one element from another. In some examples, a first element and a second element may refer to the same instance of the element, and in some cases, they may also refer to different instances based on the description of the context.
The terminology used in the description of the various examples in this disclosure is for the purpose of describing particular examples only and is not intended to be limiting. Unless the context clearly indicates otherwise, the elements may be one or more if the number of the elements is not specifically limited. Furthermore, the term "and/or" as used in this disclosure encompasses any and all possible combinations of the listed items.
Embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.
Fig. 1 illustrates a schematic diagram of an exemplary system 100 in which various methods and apparatus described herein may be implemented, in accordance with an embodiment of the present disclosure. Referring to fig. 1, the system 100 includes one or more client devices 101, 102, 103, 104, 105, and 106, a server 120, and one or more communication networks 110 coupling the one or more client devices to the server 120. Client devices 101, 102, 103, 104, 105, and 106 may be configured to execute one or more applications.
In an embodiment of the present disclosure, the server 120 may run one or more services or software applications that enable execution of the multimedia data search method.
In some embodiments, server 120 may also provide other services or software applications that may include non-virtual environments and virtual environments. In some embodiments, these services may be provided as web-based services or cloud services, for example, provided to users of client devices 101, 102, 103, 104, 105, and/or 106 under a software as a service (SaaS) model.
In the configuration shown in fig. 1, server 120 may include one or more components that implement the functions performed by server 120. These components may include software components, hardware components, or a combination thereof that are executable by one or more processors. A user operating client devices 101, 102, 103, 104, 105, and/or 106 may in turn utilize one or more client applications to interact with server 120 to utilize the services provided by these components. It should be appreciated that a variety of different system configurations are possible, which may differ from system 100. Accordingly, FIG. 1 is one example of a system for implementing the various methods described herein and is not intended to be limiting.
The user may use the client devices 101, 102, 103, 104, 105 and/or 106 to obtain multimedia data to be searched or to present page information about the search results. The client device may provide an interface that enables a user of the client device to interact with the client device. The client device may also output information to the user via the interface. Although fig. 1 depicts only six client devices, those skilled in the art will appreciate that the present disclosure may support any number of client devices.
Client devices 101, 102, 103, 104, 105, and/or 106 may include various types of computer devices, such as portable handheld devices, general purpose computers (such as personal computers and laptop computers), workstation computers, wearable devices, gaming systems, thin clients, various messaging devices, sensors or other sensing devices, and the like. These computer devices may run various types and versions of software applications and operating systems, such as Microsoft Windows, apple iOS, UNIX-like operating systems, linux, or Linux-like operating systems (e.g., google Chrome OS); or include various mobile operating systems such as Microsoft Windows Mobile OS, iOS, windows Phone, android. Portable handheld devices may include cellular telephones, smart phones, tablet computers, personal Digital Assistants (PDAs), and the like. Wearable devices may include head mounted displays and other devices. The gaming system may include various handheld gaming devices, internet-enabled gaming devices, and the like. The client device is capable of executing a variety of different applications, such as various Internet-related applications, communication applications (e.g., email applications), short Message Service (SMS) applications, and may use a variety of communication protocols.
Network 110 may be any type of network known to those skilled in the art that may support data communications using any of a number of available protocols, including but not limited to TCP/IP, SNA, IPX, etc. For example only, the one or more networks 110 may be a Local Area Network (LAN), an ethernet-based network, a token ring, a Wide Area Network (WAN), the internet, a virtual network, a Virtual Private Network (VPN), an intranet, an extranet, a Public Switched Telephone Network (PSTN), an infrared network, a wireless network (e.g., bluetooth, WIFI), and/or any combination of these and/or other networks.
The server 120 may include one or more general purpose computers, special purpose server computers (e.g., PC (personal computer) servers, UNIX servers, mid-end servers), blade servers, mainframe computers, server clusters, or any other suitable arrangement and/or combination. The server 120 may include one or more virtual machines running a virtual operating system, or other computing architecture that involves virtualization (e.g., one or more flexible pools of logical storage devices that may be virtualized to maintain virtual storage devices of the server). In various embodiments, server 120 may run one or more services or software applications that provide the functionality described below.
The computing units in server 120 may run one or more operating systems including any of the operating systems described above as well as any commercially available server operating systems. Server 120 may also run any of a variety of additional server applications and/or middle tier applications, including HTTP servers, FTP servers, CGI servers, JAVA servers, database servers, etc.
In some implementations, server 120 may include one or more applications to analyze and consolidate data feeds and/or event updates received from users of client devices 101, 102, 103, 104, 105, and 106. Server 120 may also include one or more applications to display data feeds and/or real-time events via one or more display devices of client devices 101, 102, 103, 104, 105, and 106.
In some implementations, the server 120 may be a server of a distributed system or a server that incorporates a blockchain. The server 120 may also be a cloud server, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology. The cloud server is a host product in a cloud computing service system, so as to solve the defects of large management difficulty and weak service expansibility in the traditional physical host and Virtual special server (VPS PRIVATE SERVER) service.
The system 100 may also include one or more databases 130. In some embodiments, these databases may be used to store data and other information. For example, one or more of databases 130 may be used to store information such as audio files and video files. The data store 130 may reside in a variety of locations. For example, the data store used by the server 120 may be local to the server 120, or may be remote from the server 120 and may communicate with the server 120 via a network-based or dedicated connection. The data store 130 may be of different types. In some embodiments, the data store used by server 120 may be a database, such as a relational database. One or more of these databases may store, update, and search the databases and data from the databases in response to the commands.
In some embodiments, one or more of databases 130 may also be used by applications to store application data. The databases used by the application may be different types of databases, such as key value stores, object stores, or conventional stores supported by the file system.
The system 100 of fig. 1 may be configured and operated in various ways to enable application of the various methods and apparatus described in accordance with the present disclosure.
For purposes of embodiments of the present disclosure, in the example of fig. 1, client applications for conducting multimedia data searches may be included in client devices 101, 102, 103, 104, 105, and 106. The client application may be, for example, an application program that needs to be downloaded and installed before running, a video website that is accessible through a browser, a lightweight applet that runs in a host application, and so on. The client application may provide various functions based on the multimedia data, such as searching, viewing, uploading, downloading, clipping, etc., of the multimedia data. In response, the server 120 may be a server for use with the client application. The server 120 may provide multimedia services to client applications running in the client devices 101, 102, 103, 104, 105 and 106 based on stored multimedia data resources, multimedia editing tools, etc. Specifically, the server 120 may perform the multimedia data searching method 200 according to the embodiment of the present disclosure based on the stored multimedia resources, and provide the multimedia data searching service to the user, so as to implement fast and accurate multimedia data searching.
Fig. 2 is a flowchart illustrating a multimedia data search method 200 according to an exemplary embodiment. The method 200 may be performed at a server (e.g., the server 120 shown in fig. 1), i.e., the subject of execution of the steps of the method 200 may be the server 120 shown in fig. 1.
As shown in fig. 2, the method 200 includes:
step 201, obtaining multimedia data to be searched, wherein the multimedia data to be searched comprises a video to be searched and an audio to be searched which are synchronous in time;
Step 202, extracting a first key frame sequence from a video to be searched of multimedia data to be searched according to a preset rule based on the change of the sound characteristics of the audio to be searched in a time domain;
Step 203, obtaining a second key frame sequence corresponding to each candidate multimedia data in the plurality of candidate multimedia data, wherein the second key frame sequence of each candidate multimedia data is extracted according to a preset rule; and
Step 204, determining multimedia data matched with the multimedia data to be searched in the plurality of candidate multimedia data based on the first key frame sequence and the second key frame sequence corresponding to each candidate multimedia data in the plurality of candidate multimedia data.
According to an embodiment of the present disclosure, a multimedia data search scheme for searching multimedia in multimedia based on sound features in multimedia data is provided. The matching can be performed based on the key frame sequences respectively extracted from the multimedia data to be searched and each candidate multimedia data, so that the data quantity required to be processed in the process of multimedia data searching is effectively reduced, the computing resources are saved, and the multimedia data searching efficiency is improved.
The multimedia data searching method 200 of the embodiment of the disclosure relates to the technical field of multimedia data processing, in particular to artificial intelligence and computer vision technology. The method 200 may be applied in a multimedia data understanding scenario, for example, for searching multimedia data matching multimedia data to be searched specified by a user, or making multimedia data recommendations to the user based on the content of the multimedia data, etc.
For example, in some scenes, a user may watch a highlight movie clip through a channel and wish to search for a movie or episode containing the movie clip. In this case, the multimedia data to be searched is the movie fragment; candidate multimedia data may be all multimedia data stored in a server or associated database, or all movies/television shows, movies/television shows of a certain type (e.g., comedy, suspense etc.) or starring by certain actors, etc.; the multimedia data matched with the multimedia data to be searched is the multimedia data matched with the highlight video segment obtained by searching from the candidate multimedia data.
As another example, in other scenarios, a user may view a highlight in a sporting event, such as a goal segment in a football event, a basketball event, etc., through some channel, and may wish to search for a more complete event video containing the highlight, such as full-field event multimedia data, half-field event multimedia data, etc., containing the highlight. In this case, the multimedia data to be searched is a highlight of the sports event; the candidate multimedia data may be all multimedia data stored in a server or an associated database, or all sports event multimedia data, a certain type of sports event multimedia data, etc.; the multimedia data matched with the multimedia data to be searched is event multimedia data matched with the highlight, which is searched from the candidate multimedia data.
The various steps of method 200 are described in detail below.
Referring to fig. 2, multimedia data to be searched in step 210 may be acquired in various ways.
According to some embodiments, a user may upload multimedia data to be searched through a client device and initiate a multimedia data search request requesting to search for multimedia data matching the multimedia data to be searched. Accordingly, in step 210, the server may directly acquire the multimedia data to be searched uploaded by the user.
According to further embodiments, a user may specify an address of multimedia data to be searched through a client device and initiate a multimedia data search request. Accordingly, in step 210, the server may acquire multimedia data to be searched from the corresponding address.
According to still other embodiments, the server may take any multimedia data that the user has viewed as multimedia data to be searched for without specification by the user. In this case, the server may determine multimedia data matching the multimedia data to be searched through the method 200 and push the multimedia data to the client device to provide the multimedia intelligent recommendation service to the user.
According to some embodiments, the video to be searched and the audio to be searched included in the multimedia data to be searched are kept synchronized by respective time stamps.
In step 202, according to some embodiments, based on the change of the sound feature of the audio to be searched in the time domain, extracting the first key frame sequence from the video to be searched of the multimedia data to be searched according to the preset rule may include: determining at least one mute period in the multimedia data to be searched based on the change of the sound characteristic of the audio to be searched in the time domain; and determining at least one video frame, close to the mute period, in the video to be searched as a first key frame for each of the at least one mute period, so as to form a first key frame sequence. Therefore, the key frames can be extracted based on the mute period in the multimedia data, and the data quantity required to be processed for matching between the subsequent multimedia data can be reduced.
Because the non-mute period in the multimedia data often has more abundant information, the extraction of the video frames is carried out based on the mute period, and the video frames in the mute period can be prevented from being extracted in the process of extracting the video frames in a targeted manner so as to extract more information in the multimedia data.
According to some embodiments, a video frame of the video to be searched that is adjacent to one of the two ends of the silence period may be determined as a first key frame.
According to some embodiments, a video frame having a predetermined distance from one of both ends of the silence period in the video to be searched may be determined as the first key frame.
According to some embodiments, determining at least one silence period in the multimedia data to be searched based on a change in a time domain of a sound characteristic of the audio to be searched may include: at least one silence period in the multimedia data to be searched is identified using voice boundary detection (VAD) based on the change in the time domain of the sound characteristics of the audio to be searched. Thereby, the silence period in the multimedia data to be searched can be conveniently identified.
According to some embodiments, the sound feature comprises an energy value, and determining at least one silence period in the multimedia data to be searched based on a change in the energy value of the audio to be searched in the time domain may comprise: and for any time period within the duration range of the multimedia data to be searched, determining the time period as a mute period in the multimedia data to be searched in response to the energy value of the audio to be searched being not greater than a first preset energy threshold value in the time period and the energy value of the audio to be searched being greater than a second preset energy threshold value at a time point close to the time period, wherein the first preset energy threshold value is not greater than the second preset energy threshold value. The silence period in the multimedia data can thus be effectively identified based on the change in the real-time energy value of the audio to be searched.
According to some embodiments, the sound features may further comprise a loudness, and determining at least one period of silence in the multimedia data to be searched based on the change in the loudness of the audio to be searched in the time domain may comprise: and for any time period within the duration range of the multimedia data to be searched, determining the time period as a mute period in the multimedia data to be searched in response to the loudness of the audio to be searched being not more than a third preset energy threshold in the time period and the loudness of the audio to be searched being more than a fourth preset energy threshold at a time point close to the time period, wherein the third preset energy threshold is not more than the fourth preset energy threshold.
According to some embodiments, based on the change of the sound feature of the audio to be searched in the time domain, extracting the first key frame sequence from the video to be searched of the multimedia data to be searched according to the preset rule may include: performing voice recognition on the audio to be searched based on the change of the sound characteristics of the audio to be searched in the time domain; responding to the voice recognition result of the audio to be searched, wherein the voice recognition result comprises preset characters, and determining the time point corresponding to the recognized preset characters in the audio to be searched as a key time point; and extracting a first key frame sequence from the video to be searched based on the determined key time point. Therefore, the key frames can be extracted based on the characters recognized by the voice recognition technology, and the data quantity required to be processed for matching between the subsequent multimedia data can be reduced.
According to some embodiments, the preset characters may be specific characters specified in advance, for example, "you", "me", "home", etc. characters that are frequently used in daily life.
According to some embodiments, the preset character may also be a high frequency character in the audio to be searched. Specifically, according to a voice recognition result of audio to be searched, at least one character with higher occurrence frequency in the recognition result is determined as a preset character.
In step 203, a corresponding second key frame sequence may be extracted from each of the plurality of candidate multimedia data based on the same preset rule. The specific manner of extracting the second key frame sequence is the same as that of extracting the first key frame sequence, and will not be described herein.
According to some embodiments, for each candidate multimedia data in the plurality of candidate multimedia data, a second key frame sequence corresponding to the candidate multimedia data may be extracted from the candidate multimedia data in advance based on a preset rule. Therefore, by extracting the second key frame sequence corresponding to each candidate multimedia data in advance, the extraction of key frames to the candidate multimedia data can be avoided during each search, and the search efficiency is improved.
According to some embodiments, the second key frame sequence corresponding to each candidate multimedia data is stored in the designated database in advance, and the stored second key frame sequence is associated with the candidate multimedia data corresponding thereto.
In step 204, according to some embodiments, determining, based on the first key frame sequence and the second key frame sequence corresponding to each of the plurality of candidate multimedia data, multimedia data of the plurality of candidate multimedia data that matches the multimedia data to be searched may include: for each candidate multimedia data in the plurality of candidate multimedia data, determining the candidate multimedia data as multimedia data matched with the multimedia data to be searched in response to the fact that the second key frame sequence corresponding to the candidate multimedia data and the first key frame sequence meet a preset matching condition. Therefore, the matching relation between the multimedia data to be searched and the candidate multimedia data can be judged based on the matching result between the first key frame sequence and the second key frame sequence, and the calculation amount of multimedia data searching is effectively reduced.
Wherein the matching of the candidate multimedia data with the multimedia data to be searched may include having portions overlapping each other between the candidate multimedia data and the multimedia data to be searched. Specifically, the matching between the candidate multimedia data and the multimedia data to be searched may include various types, for example, as shown in fig. 3, the candidate multimedia data a is included in the multimedia data to be searched, the candidate multimedia data B includes the entire multimedia data to be searched, and the candidate multimedia data C partially overlaps with the multimedia data to be searched. The candidate multimedia data A, the candidate multimedia data B and the candidate multimedia data C are matched with the multimedia data to be searched.
According to some embodiments, the meeting of the preset matching condition between the second key frame sequence and the first key frame sequence corresponding to the candidate multimedia data may include: aiming at a second sub-sequence with a first preset length in a second key frame sequence, the first sub-sequence with the first preset length corresponding to the second sub-sequence exists in the first key frame sequence, wherein the similarity between every two frames sequentially corresponding to the second sub-sequence and the first sub-sequence is larger than a preset threshold value.
It is understood that the first sub-sequence in the first key frame sequence and the second sub-sequence in the second key frame sequence may be located at different relative positions in the first key frame sequence and the second key frame sequence, respectively. By comparing the similarity between every two frames sequentially corresponding to the second sub-sequence and the first sub-sequence, whether the candidate multimedia data and the multimedia data to be searched have an overlapping part with a first preset length or not can be judged, and whether the candidate multimedia data is matched with the multimedia data to be searched or not can be further determined.
It is understood that the actual overlapping portion between the candidate multimedia data and the multimedia data to be searched may be greater than the first preset length. In an exemplary embodiment of the present disclosure, based on the similarity between each two frames sequentially corresponding to a first sub-sequence and a second sub-sequence within a first preset length being greater than a preset threshold, it may be determined that an "overlap" portion exists between candidate multimedia data and multimedia data to be searched, without performing additional comparison of video frames, so that multimedia data searching efficiency may be effectively improved.
According to some embodiments, for a second sub-sequence of a second preset length located at one end of a second key frame sequence, a first sub-sequence of the second preset length corresponding to the second sub-sequence exists at the other end of the first key frame sequence, wherein a similarity between every two frames sequentially corresponding to the second sub-sequence and the first sub-sequence is greater than a preset threshold, and the second preset length is smaller than the first preset length.
When the matching relationship between the candidate multimedia data and the multimedia data to be searched is partially overlapped, for example, as in the relationship between the candidate multimedia data C and the multimedia data to be searched in fig. 3, the overlapping portion between the candidate multimedia data and the multimedia data to be searched is limited, and the first preset length may not be reached. In this case, therefore, it is possible to avoid missing candidate multimedia data partially overlapping the multimedia data to be searched during the multimedia data search by reducing the requirement of the length of the overlapping portion, i.e., making the second preset length smaller than the first preset length.
According to some embodiments, the similarity of images may be determined by a structural similarity algorithm (SSIM), a peak signal to noise ratio algorithm (PSNR), or various machine learning approaches, which are not limited herein.
According to some embodiments, in a case that the first key frame sequence includes a timestamp of each first key frame in the multimedia data to be searched, and the second key frame sequence includes a timestamp corresponding to each second key frame in the corresponding candidate multimedia data, the satisfaction of the preset condition between the second key frame sequence corresponding to the candidate multimedia data and the first key frame sequence may include: for a fourth sub-sequence of a third preset length in the second key frame sequence, a third sub-sequence of the third preset length corresponding to the fourth sub-sequence exists in the first key frame sequence, wherein for any two frames in the fourth sub-sequence, the time difference between the two frames in the fourth sub-sequence is the same as the time difference between two frames in the third sub-sequence, which correspond to the two frames in sequence.
According to some embodiments, when determining whether the second key frame sequence and the first key frame sequence corresponding to the candidate multimedia data meet the preset condition, the determination may be performed based on the methods described in the above embodiments at the same time.
According to some embodiments, after determining multimedia data matching the multimedia data to be searched among the plurality of candidate multimedia data, the feedback is based on the determined page information of the multimedia data matching the multimedia data to be searched. Thus, the user can intuitively acquire the search result information.
According to some embodiments, the page information may include matching type information between candidate multimedia data and multimedia data to be searched. For example, the overlapping portion between the multimedia data to be searched and the candidate multimedia data may be intuitively presented to the user by way of illustration (as shown in fig. 3).
According to another aspect of the present disclosure, there is also provided a multimedia data searching apparatus 400, as shown in fig. 4, the apparatus 400 including:
A first obtaining unit 401 configured to obtain multimedia data to be searched, wherein the multimedia data to be searched includes a video to be searched and an audio to be searched which are synchronized in time;
A first extraction unit 402, configured to extract a first key frame sequence from a video to be searched for of the multimedia data to be searched for according to a preset rule based on a change of a sound feature of the audio to be searched for in a time domain;
A second obtaining unit 403, configured to obtain a second key frame sequence corresponding to each candidate multimedia data in the plurality of candidate multimedia data, where the second key frame sequence of each candidate multimedia data is extracted according to a preset rule; and
The determining unit 404 is configured to determine, based on the first key frame sequence and the second key frame sequence corresponding to each of the plurality of candidate multimedia data, multimedia data matching the multimedia data to be searched among the plurality of candidate multimedia data.
According to some embodiments, the extraction unit comprises: a first determination subunit configured to determine at least one silence period in the multimedia data to be searched based on a change in a time domain of a sound feature of the audio to be searched; and a second determining subunit configured to determine, for each of the at least one silence period, at least one video frame of the video to be searched that is close to the silence period as a first key frame to constitute a first key frame sequence.
According to some embodiments, the first determining subunit is further configured to: at least one silence period in the multimedia data to be searched is identified using voice boundary detection (VAD) based on the change in the time domain of the sound characteristics of the audio to be searched.
According to some embodiments, the sound characteristic comprises an energy value, the first determining subunit is further configured to: and for any time period within the duration range of the multimedia data to be searched, determining the time period as a mute period in the multimedia data to be searched in response to the energy value of the audio to be searched being not greater than a first preset energy threshold value within the time period and the energy value of the audio to be searched being greater than a second preset energy threshold value at a time point close to the time period, wherein the first preset energy threshold value is not greater than the second preset energy threshold value.
According to some embodiments, the extraction unit comprises: the recognition subunit is configured to perform voice recognition on the audio to be searched based on the change of the sound characteristics of the audio to be searched in the time domain; a third determining subunit configured to determine, as a key time point, a time point corresponding to the identified preset character in the audio to be searched in response to the preset character being included in the speech recognition result of the audio to be searched; and a fourth determination subunit configured to extract a first key frame sequence from the video to be searched based on the determined key time point.
According to some embodiments, the determining unit is further configured to: for each candidate multimedia data in the plurality of candidate multimedia data, determining the candidate multimedia data as multimedia data matched with the multimedia data to be searched in response to the fact that the second key frame sequence corresponding to the candidate multimedia data and the first key frame sequence meet a preset matching condition.
According to some embodiments, the meeting the preset matching condition between the second key frame sequence and the first key frame sequence corresponding to the candidate multimedia data includes: aiming at a second sub-sequence with a preset length in a second key frame sequence, a first sub-sequence with a preset length corresponding to the second sub-sequence exists in a first key frame sequence, wherein the similarity between every two frames sequentially corresponding to the second sub-sequence and the first sub-sequence is larger than a preset threshold value.
According to some embodiments, the multimedia data search apparatus further comprises: and the second extraction unit is configured to extract a second key frame sequence corresponding to each candidate multimedia data from the candidate multimedia data in advance based on a preset rule for each candidate multimedia data in the plurality of candidate multimedia data.
According to another aspect of the present disclosure, there is also provided a computer apparatus including: a memory, a processor and a computer program stored on the memory, wherein the processor is configured to execute the computer program to implement the steps of the above method.
According to another aspect of the present disclosure, there is also provided a non-transitory computer readable storage medium having stored thereon a computer program, wherein the computer program when executed by a processor implements the steps of the above method.
According to another aspect of the present disclosure, there is also provided a computer program product comprising a computer program, wherein the computer program when executed by a processor realizes the steps of the above method.
With reference to fig. 5, a block diagram of a computer device 500 that may be a server or a client of the present disclosure will now be described, which is an example of a hardware device that may be applied to aspects of the present disclosure. Electronic devices are intended to represent various forms of digital electronic computer devices, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the disclosure described and/or claimed herein.
As shown in fig. 5, the apparatus 500 includes a computing unit 501 that can perform various suitable actions and processes according to a computer program stored in a Read Only Memory (ROM) 502 or a computer program loaded from a storage unit 508 into a Random Access Memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the device 500 can also be stored. The computing unit 501, ROM 502, and RAM 503 are connected to each other by a bus 504. An input/output (I/O) interface 505 is also connected to bus 504.
Various components in the device 500 are connected to the I/O interface 505, including: an input unit 506, an output unit 507, a storage unit 508, and a communication unit 509. The input unit 506 may be any type of device capable of inputting information to the device 500, the input unit 506 may receive input numeric or character information and generate key signal inputs related to user settings and/or function control of the electronic device, and may include, but is not limited to, a mouse, a keyboard, a touch screen, a track pad, a track ball, a joystick, a microphone, and/or a remote control. The output unit 507 may be any type of device capable of presenting information and may include, but is not limited to, a display, speakers, video/audio output terminals, vibrators, and/or printers. Storage unit 508 may include, but is not limited to, magnetic disks, optical disks. The communication unit 509 allows the device 500 to exchange information/data with other devices via a computer network such as the internet and/or various telecommunications networks, and may include, but is not limited to, modems, network cards, infrared communication devices, wireless communication transceivers and/or chipsets, such as bluetooth (TM) devices, 1302.11 devices, wiFi devices, wiMax devices, cellular communication devices, and/or the like.
The computing unit 501 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of computing unit 501 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 501 performs the various methods and processes described above, such as multimedia data searching. For example, in some embodiments, the multimedia data search may be implemented as a computer software program tangibly embodied on a machine-readable medium, such as storage unit 508. In some embodiments, part or all of the computer program may be loaded and/or installed onto the device 500 via the ROM 502 and/or the communication unit 509. When the computer program is loaded into the RAM 503 and executed by the computing unit 501, one or more steps of the multimedia data search method described above may be performed. Alternatively, in other embodiments, the computing unit 501 may be configured to perform the multimedia data search by any other suitable means (e.g., by means of firmware).
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuit systems, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.
Program code for carrying out methods of the present disclosure may be written in any combination of one or more programming languages. These program code may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus such that the program code, when executed by the processor or controller, causes the functions/operations specified in the flowchart and/or block diagram to be implemented. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), and the internet.
The computer system may include a client and a server. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.
It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps recited in the present disclosure may be performed in parallel, sequentially or in a different order, provided that the desired results of the disclosed aspects are achieved, and are not limited herein.
Although embodiments or examples of the present disclosure have been described with reference to the accompanying drawings, it is to be understood that the foregoing methods, systems, and apparatus are merely exemplary embodiments or examples, and that the scope of the present invention is not limited by these embodiments or examples but only by the claims following the grant and their equivalents. Various elements of the embodiments or examples may be omitted or replaced with equivalent elements thereof. Furthermore, the steps may be performed in a different order than described in the present disclosure. Further, various elements of the embodiments or examples may be combined in various ways. It is important that as technology evolves, many of the elements described herein may be replaced by equivalent elements that appear after the disclosure.
Claims (18)
1. A multimedia data search method, comprising:
acquiring multimedia data to be searched, wherein the multimedia data to be searched comprises a video to be searched and an audio to be searched which are synchronous in time;
Extracting a first key frame sequence from the video to be searched of the multimedia data to be searched according to a preset rule based on the change of the sound characteristics of the audio to be searched in the time domain;
Acquiring a second key frame sequence corresponding to each candidate multimedia data in the plurality of candidate multimedia data, wherein the second key frame sequence of each candidate multimedia data is extracted according to the preset rule; and
Based on the first key frame sequence and a second key frame sequence corresponding to each candidate multimedia data in the plurality of candidate multimedia data, determining multimedia data matched with the multimedia data to be searched in the plurality of candidate multimedia data comprises:
And for each candidate multimedia data in the plurality of candidate multimedia data, determining the candidate multimedia data as the multimedia data matched with the multimedia data to be searched in response to the fact that the second key frame sequence corresponding to the candidate multimedia data and the first key frame sequence meet a preset matching condition.
2. The method of claim 1, wherein the extracting the first key frame sequence from the video to be searched for the multimedia data according to the preset rule based on the change of the sound feature of the audio to be searched for in the time domain comprises:
determining at least one mute period in the multimedia data to be searched based on the change of the sound characteristic of the audio to be searched in the time domain; and
For each mute period of the at least one mute period, determining at least one video frame, close to the mute period, in the video to be searched as a first key frame, so as to form the first key frame sequence.
3. The method of claim 2, wherein the determining at least one mute period in the multimedia data to be searched based on the change in the sound characteristic of the audio to be searched in the time domain comprises:
At least one silence period in the multimedia data to be searched is identified using voice boundary detection (VAD) based on the change in the time domain of the sound characteristics of the audio to be searched.
4. The method of claim 2, wherein the sound feature comprises an energy value, and determining at least one silence period in the multimedia data to be searched based on a change in the energy value of the audio to be searched in the time domain comprises:
And for any time period within the duration range of the multimedia data to be searched, determining the time period as a mute period in the multimedia data to be searched in response to the energy value of the audio to be searched being not greater than a first preset energy threshold value within the time period and the energy value of the audio to be searched being greater than a second preset energy threshold value at a time point close to the time period, wherein the first preset energy threshold value is not greater than the second preset energy threshold value.
5. The method of claim 1, wherein the extracting the first key frame sequence from the video to be searched for the multimedia data according to the preset rule based on the change of the sound feature of the audio to be searched for in the time domain comprises:
Performing voice recognition on the audio to be searched based on the change of the voice characteristics of the audio to be searched in the time domain;
responding to the voice recognition result of the audio to be searched, wherein the voice recognition result comprises preset characters, and determining the time point corresponding to the recognized preset characters in the audio to be searched as a key time point; and
And extracting a first key frame sequence from the video to be searched based on the determined key time point.
6. The method of claim 1, wherein the satisfaction of the preset matching condition between the second key frame sequence corresponding to the candidate multimedia data and the first key frame sequence comprises:
For a second sub-sequence with a preset length in the second key frame sequence, a first sub-sequence with the preset length corresponding to the second sub-sequence exists in the first key frame sequence, wherein the similarity between every two frames sequentially corresponding to the second sub-sequence and the first sub-sequence is larger than a preset threshold value.
7. The method of claim 1, further comprising:
And extracting a second key frame sequence corresponding to each candidate multimedia data from the candidate multimedia data in advance based on the preset rule aiming at each candidate multimedia data in the plurality of candidate multimedia data.
8. The method of claim 1, further comprising:
after determining multimedia data matching the multimedia data to be searched among the plurality of candidate multimedia data, feeding back page information based on the determined multimedia data matching the multimedia data to be searched.
9. A multimedia data search apparatus comprising:
a first acquisition unit configured to acquire multimedia data to be searched, wherein the multimedia data to be searched includes a video to be searched and an audio to be searched which are synchronized in time;
The first extraction unit is configured to extract a first key frame sequence from the video to be searched of the multimedia data to be searched according to a preset rule based on the change of the sound characteristics of the audio to be searched in the time domain;
the second acquisition unit is configured to acquire a second key frame sequence corresponding to each candidate multimedia data in the plurality of candidate multimedia data, wherein the second key frame sequence of each candidate multimedia data is extracted according to the preset rule; and
A determining unit configured to determine, based on the first key frame sequence and a second key frame sequence corresponding to each of the plurality of candidate multimedia data, multimedia data that matches the multimedia data to be searched among the plurality of candidate multimedia data, wherein the determining unit is further configured to:
And for each candidate multimedia data in the plurality of candidate multimedia data, determining the candidate multimedia data as the multimedia data matched with the multimedia data to be searched in response to the fact that the second key frame sequence corresponding to the candidate multimedia data and the first key frame sequence meet a preset matching condition.
10. The apparatus of claim 9, wherein the first extraction unit comprises:
A first determining subunit configured to determine at least one silence period in the multimedia data to be searched based on a change in a time domain of a sound feature of the audio to be searched; and
And the second determining subunit is configured to determine, for each mute period in the at least one mute period, at least one video frame, close to the mute period, in the video to be searched as a first key frame, so as to form the first key frame sequence.
11. The apparatus of claim 10, wherein the first determination subunit is further configured to:
means for identifying at least one silence period in the multimedia data to be searched using voice boundary detection (VAD) based on a change in a time domain of a sound characteristic of the audio to be searched.
12. The apparatus of claim 10, wherein the sound characteristic comprises an energy value, the first determination subunit further configured to:
And means for determining, for any time period within a range of durations of the multimedia data to be searched, the time period as a mute period in the multimedia data to be searched in response to an energy value of the audio to be searched being not greater than a first preset energy threshold within the time period and an energy value of the audio to be searched being greater than a second preset energy threshold at a point in time near the time period, wherein the first preset energy threshold is not greater than the second preset energy threshold.
13. The apparatus of claim 9, wherein the first extraction unit comprises:
An identification subunit configured to perform speech recognition on the audio to be searched based on a change in a time domain of sound features of the audio to be searched;
A third determining subunit configured to determine, as a key time point, a time point corresponding to the identified preset character in the audio to be searched in response to the preset character being included in the speech recognition result of the audio to be searched; and
And a fourth determination subunit configured to extract a first key frame sequence from the video to be searched based on the determined key time point.
14. The apparatus of claim 9, wherein the satisfaction of the preset matching condition between the second key frame sequence corresponding to the candidate multimedia data and the first key frame sequence comprises:
For a second sub-sequence with a preset length in the second key frame sequence, a first sub-sequence with the preset length corresponding to the second sub-sequence exists in the first key frame sequence, wherein the similarity between every two frames sequentially corresponding to the second sub-sequence and the first sub-sequence is larger than a preset threshold value.
15. The apparatus of claim 9, further comprising:
and a second extraction unit configured to extract, for each candidate multimedia data of the plurality of candidate multimedia data, a second key frame sequence corresponding to the candidate multimedia data from the candidate multimedia data in advance based on the preset rule.
16. A computer device, comprising:
a memory, a processor and a computer program stored on the memory,
Wherein the processor is configured to execute the computer program to implement the steps of the method of any of claims 1-8.
17. A non-transitory computer readable storage medium having stored thereon a computer program, wherein the computer program when executed by a processor implements the steps of the method of any of claims 1-8.
18. A computer program product comprising a computer program, wherein the computer program when executed by a processor implements the steps of the method of any of claims 1-8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110492379.2A CN113190695B (en) | 2021-05-06 | 2021-05-06 | Multimedia data searching method and device, computer equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110492379.2A CN113190695B (en) | 2021-05-06 | 2021-05-06 | Multimedia data searching method and device, computer equipment and medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113190695A CN113190695A (en) | 2021-07-30 |
| CN113190695B true CN113190695B (en) | 2024-05-14 |
Family
ID=76983815
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110492379.2A Active CN113190695B (en) | 2021-05-06 | 2021-05-06 | Multimedia data searching method and device, computer equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113190695B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113627363B (en) * | 2021-08-13 | 2023-08-15 | 百度在线网络技术（北京）有限公司 | Video file processing method, device, equipment and storage medium |
| CN114003739A (en) * | 2021-10-29 | 2022-02-01 | 北京达佳互联信息技术有限公司 | Multimedia resource processing method and device |
| CN114186093B (en) * | 2021-12-13 | 2023-04-28 | 北京百度网讯科技有限公司 | Method, device, equipment and medium for processing multimedia data |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2015058332A1 (en) * | 2013-10-21 | 2015-04-30 | Microsoft Technology Licensing, Llc | Mobile video search |
| CN112214695A (en) * | 2019-07-12 | 2021-01-12 | 北京字节跳动网络技术有限公司 | Information processing method and device and electronic equipment |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20200311123A1 (en) * | 2019-03-28 | 2020-10-01 | Wipro Limited | Method and a system for multimodal search key based multimedia content extraction |
-
2021
- 2021-05-06 CN CN202110492379.2A patent/CN113190695B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2015058332A1 (en) * | 2013-10-21 | 2015-04-30 | Microsoft Technology Licensing, Llc | Mobile video search |
| CN112214695A (en) * | 2019-07-12 | 2021-01-12 | 北京字节跳动网络技术有限公司 | Information processing method and device and electronic equipment |
Non-Patent Citations (1)
| Title |
|---|
| 多媒体网络教学资源的内容特征提取与搜索研究;孟祥增;;电化教育研究(第12期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113190695A (en) | 2021-07-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113190695B (en) | Multimedia data searching method and device, computer equipment and medium | |
| CN112559800B (en) | Method, apparatus, electronic device, medium and product for processing video | |
| CN112836072B (en) | Information display method and device, electronic equipment and medium | |
| JP2021034003A (en) | Human object recognition method, apparatus, electronic device, storage medium, and program | |
| CN116501960B (en) | Content retrieval method, device, equipment and medium | |
| CN113810765B (en) | Video processing method, device, equipment and medium | |
| CN108197105B (en) | Natural language processing method, device, storage medium and electronic equipment | |
| CN111639234B (en) | Method and device for mining core entity attention points | |
| CN115269989B (en) | Object recommendation method, device, electronic equipment and storage medium | |
| CN113139093A (en) | Video search method and apparatus, computer device, and medium | |
| CN114245171B (en) | Video editing method and device, electronic equipment and medium | |
| CN113139092B (en) | Video searching method and device, electronic equipment and medium | |
| CN113868453B (en) | Object recommendation method and device | |
| CN114219046B (en) | Model training method, matching method, device, system, electronic equipment and medium | |
| CN113312511B (en) | Method, apparatus, device and computer readable storage medium for recommending content | |
| CN113190713B (en) | Video searching method and device, electronic equipment and medium | |
| CN114998963A (en) | Image detection method and method for training image detection model | |
| CN113139094B (en) | Video searching method and device, electronic equipment and medium | |
| CN114238745A (en) | Method and device for providing search result, electronic equipment and medium | |
| CN113378001B (en) | Video playing progress adjusting method and device, electronic equipment and medium | |
| CN113641933B (en) | Abnormal webpage identification method, abnormal site identification method and device | |
| CN113139095A (en) | Video retrieval method and device, computer equipment and medium | |
| CN113190713A (en) | Video searching method and device, electronic equipment and medium | |
| CN113722534B (en) | Video recommendation method and device | |
| CN113326417B (en) | Method and device for updating webpage library |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |