CN112818166B - Video information query method and device, electronic equipment and storage medium - Google Patents
Video information query method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN112818166B CN112818166B CN202110144158.6A CN202110144158A CN112818166B CN 112818166 B CN112818166 B CN 112818166B CN 202110144158 A CN202110144158 A CN 202110144158A CN 112818166 B CN112818166 B CN 112818166B
- Authority
- CN
- China
- Prior art keywords
- video
- data
- query
- external service
- service system
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/70—Information retrieval; Database structures therefor; File system structures therefor of video data
- G06F16/73—Querying
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/70—Information retrieval; Database structures therefor; File system structures therefor of video data
- G06F16/78—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/7867—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using information manually generated, e.g. tags, keywords, comments, title and artist information, manually generated time, location and usage information, user ratings
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/546—Message passing systems or structures, e.g. queues
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/54—Indexing scheme relating to G06F9/54
- G06F2209/547—Messaging middleware
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/54—Indexing scheme relating to G06F9/54
- G06F2209/548—Queue
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Multimedia (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Library & Information Science (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The embodiment of the invention provides a video information query method, a video information query device, a storage system and a storage medium, and relates to the technical field of advertisement delivery. The method comprises the following steps: when a target video query request is received, querying target videos in prestored hot spot video data; the hot spot video data are obtained in advance according to historical data of video information inquiry in the advertisement putting process; when the target video cannot be queried in the hot spot video data, querying the target video in the pre-stored video update data; the video update data are acquired from an external service system when the external service system accessed by the advertisement delivery system has video update data; and querying the target video from an external service system when the target video cannot be queried in the video update data. The scheme of the invention avoids inquiry delay, reduces the request amount of inquiry from an external service system, saves server resources and improves reliability and stability.
Description
Technical Field
The present invention relates to the field of advertisement delivery technologies, and in particular, to a method and apparatus for querying video information, an electronic device, and a storage medium.
Background
The acquisition of video information plays a vital role in the delivery flow of an advertisement system, such as whether a user enjoys the content attribute of video information required by the free rights and interests, the video time required by the delivery of patch advertisements, labels, album ids, the characteristic labels of the required video required by political sensitive information shielding, and the like, so how to improve the reliability and stability of the video information query service is a necessary work of the video advertisement delivery system.
Because of the very large video data capacity, video queries for advertisement delivery are typically conducted in real-time depending on external query interfaces. However, the timeout rate of the external Query interface is about one thousandth, the timeout of the peak time is serious, and the external call is limited by strict QPS (Query Per Second). Therefore, due to serious overtime and QPS limitation in peak time, the video query of advertisement delivery in real time by simply relying on an external query interface can cause query delay, and the problem needs to be solved.
Disclosure of Invention
The invention provides a video information query method, a device, electronic equipment and a storage medium, which solve the problem that in the prior art, video query which simply relies on an external query interface to conduct advertisement delivery in real time can cause query delay.
In a first aspect of the present invention, a video information query method is provided, applied to an advertisement delivery system, and the method includes:
when a target video query request is received, querying the target video in prestored hot spot video data; the hot spot video data are obtained in advance according to historical data of video information inquiry in the advertisement putting process, and are stored in the advertisement putting system;
querying the target video in pre-stored video update data when the target video cannot be queried in the hot spot video data; the video update data are acquired from an external service system accessed by the advertisement delivery system when the video update is carried out on the external service system, and the video update data are stored in a database;
and querying the target video from the external service system when the target video cannot be queried in the video update data.
Preferably, before receiving the target video query request, the method further includes:
according to historical data of video information inquiry in the advertisement putting process, acquiring hot spot video data with inquiry frequency meeting preset conditions, and storing the hot spot video data in the advertisement putting system locally;
The preset condition is that the query frequency of the video is greater than or equal to a preset threshold value, or the preset condition is that the video is arranged according to the sequence from high query frequency to low query frequency, the query frequency of the video is arranged in the first N bits, and N is an integer greater than or equal to 1.
Preferably, after storing the hotspot video data locally to the advertisement delivery system, the method further comprises:
updating historical data of video information inquiry in the advertisement putting process according to a preset period, and re-acquiring hot spot video data of which the inquiry frequency meets preset conditions according to the updated historical data;
and updating the hot spot video data locally stored in the advertisement delivery system according to the re-acquired hot spot video data.
Preferably, the method further comprises:
when the external service system accessed by the advertisement delivery system has video update data, the video update data is acquired from the external service system, and the video update data is stored in the database.
Preferably, the method further comprises:
setting validity period information of each video stored in a database while storing the video update data in the database;
And deleting any video from the database when the time stored in the database reaches the corresponding validity period.
Preferably, when the target video is not queried in the video update data, after querying the target video from the external service system, the method further comprises:
and storing the inquired target video in the database.
Preferably, after storing the hotspot video data locally to the advertisement delivery system, the method further comprises:
when the external service system accessed by the advertisement delivery system has video update data and the video in the video update data comprises the video in the hot spot video data, the hot spot video data locally stored by the advertisement delivery system is updated according to the video update data.
Preferably, when the target video is not queried in the video update data, the step of querying the target video from the external service system includes:
writing a task for inquiring the target video into a message queue; the task of inquiring the target video, which is written into a message queue, comprises version information, ID information and/or validity period information of the target video, which are stored in the database;
And acquiring video data corresponding to the task in the message queue from the external service system.
Preferably, before writing the task of querying the target video into the message queue, the method further comprises:
and filtering the received request for repeatedly inquiring the same target video within the preset time.
Preferably, when the external service system accessed by the advertisement delivery system has video update data, the step of acquiring the video update data from the external service system includes:
subscribing the task of video update data written in the message queue by an external service system;
and acquiring video data corresponding to the task in the message queue from the external service system.
Preferably, the step of obtaining video data corresponding to the task in the message queue from the external service system includes:
sending task requests in the message queue to the external service system in batches, and carrying out query rate QPS (quality of service) current limitation on the number of task requests sent to the external service system;
and receiving video data corresponding to the task in the message queue returned by the external service system.
In a third aspect of the present invention, there is also provided a video information query apparatus applied to an advertisement delivery system, the apparatus comprising:
The first query module is used for querying the target video in the prestored hot spot video data when receiving a target video query request; the hot spot video data are obtained in advance according to historical data of video information inquiry in the advertisement putting process, and are stored in the advertisement putting system;
the second query module is used for querying the target video in the pre-stored video update data when the target video cannot be queried in the hot spot video data; the video update data are acquired from an external service system accessed by the advertisement delivery system when the video update is carried out on the external service system, and the video update data are stored in a database;
and the third query module is used for querying the target video from the external service system when the target video cannot be queried in the video update data.
In a third aspect of the present invention, there is also provided an electronic device, including: a processor, a communication interface, a memory, and a communication bus; the processor, the communication interface and the memory complete communication with each other through a communication bus;
A memory for storing a computer program;
a processor for implementing the steps in the video information query method as described in any one of the above when executing a program stored on a memory.
In a fourth aspect of the present invention, there is also provided a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the video information query method as described in any of the above.
Aiming at the prior art, the invention has the following advantages:
in the embodiment of the invention, when a target video query request is received, a target video is queried in prestored hot spot video data; the hot spot video data are obtained in advance according to historical data of video information inquiry in the advertisement putting process, and are stored in the advertisement putting system; when the target video cannot be queried in the hot spot video data, querying the target video in the pre-stored video update data; when the external service system accessed by the advertisement delivery system has video update, the video update data is acquired from the external service system and is stored in a database; and when the target video cannot be queried in the video update data, querying the target video from an external service system. Therefore, the video query request can be responded in real time through the cached hot spot video data and the video update data, the query delay is reduced, the query requirement can be basically met through the cached hot spot video data and the video update data, the request amount queried from an external service system is reduced, the server resource is saved, the reliability and the stability are improved, and certain video query errors are allowed.
The foregoing description is only an overview of the present invention, and is intended to be implemented in accordance with the teachings of the present invention in order that the same may be more clearly understood and to make the same and other objects, features and advantages of the present invention more readily apparent.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments will be briefly described below.
Fig. 1 is a flow chart of a video information query method according to an embodiment of the present invention;
FIG. 2 is a flowchart illustrating a sub-step of a video information query method according to an embodiment of the present invention;
FIG. 3 is a flowchart illustrating another sub-step of the video information query method according to an embodiment of the present invention;
fig. 4 is a flowchart illustrating another sub-step of the video information query method according to an embodiment of the present invention;
FIG. 5 is a schematic block diagram of an advertisement delivery system provided by an embodiment of the present invention;
fig. 6 is a schematic block diagram of a video information query apparatus according to an embodiment of the present invention;
fig. 7 is a schematic block diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.
Fig. 1 is a flowchart of a video information query method according to an embodiment of the present invention, where the video information query method is applied to an advertisement delivery system, and referring to fig. 1, the method includes:
step 101: when a target video query request is received, querying the target video in prestored hot spot video data; the hot spot video data are obtained in advance according to historical data of video information inquiry in the advertisement putting process, and are stored in the advertisement putting system.
Here, a batch of hot spot video data is obtained and stored locally in the advertisement delivery system in advance by analyzing historical data of video information query in the advertisement delivery process, and when a target video query request is received, query is firstly performed in the locally cached hot spot video data. The hot spot video data cached locally can respond to the video query request in real time, so that the query speed is high, the query delay is reduced, the hot spot video belongs to the video with higher query heat, most query requests can be met, and the request amount queried from an external service system is reduced compared with the mode of sending the query request to the external service system and waiting for response.
In the process of advertisement delivery, how to deliver advertisements in videos is closely related to video information, such as whether users enjoy advertising rights or not needs to combine with content attribute judgment of videos, how long advertisements are delivered in videos, what types of advertisements are delivered need to combine with information comprehensive decision of video duration, labels, content and the like, so that the video information needs to be queried in the process of advertisement delivery. The target video refers to the video which requests inquiry in the advertisement putting process, and can be any video.
The fact that the target video cannot be queried in the hot spot video data includes two cases, one is that the target video does not exist in the hot spot video data, and the other is that the version of the target video stored in the hot spot video data is not the latest version, and the two cases can lead to the fact that the target video cannot be queried and further query is needed in the video update data.
The version of the video refers to a storage version set during each updating and storing, and since the originally stored video information may be updated and changed during each updating and storing, for example, the original 3 pieces of attribute information are increased to 5 pieces of attribute information, in order to distinguish the video stored each time, the storage version information of the video, for example, version 1.1, version 1.2 and the like, are set simultaneously.
The data volume of the stored hot spot video can be set according to the requirement. For example, when the advertisement delivery system locally stores the hot spot video data, the data volume of the hot spot video can be adjusted according to the scale of the advertisement delivery system, so that the influence of excessive memory occupation on other functions of the advertisement delivery system is avoided.
Step 102: querying the target video in pre-stored video update data when the target video cannot be queried in the hot spot video data; wherein the video update data is acquired from an external service system accessed by the advertisement delivery system when the external service system has video update, and the video update data is stored in a database.
Here, since some new videos or cold videos are often targeted during advertisement delivery, the videos are characterized by being new and low in access frequency, and therefore, in the embodiment of the invention, the new videos also have a buffer function, namely, when an external service system accessed by the advertisement delivery system has video update data, the video update data is acquired from the external service system and stored. When the target video cannot be queried in the hot spot video data, the target video is queried in the video update data, so that the hot spot video data is supplemented through the cached video update data, the query hit rate is improved, the request amount queried by an external service system is further reduced, the delay rate of the video query request can be reduced through the cached video update data, and the query delay is avoided.
The fact that the target video cannot be queried in the video update data also comprises two cases, namely that the target video does not exist in the video update data, and that the version of the target video stored in the video update data is not the latest version, both cases can cause that the target video cannot be queried, and further query from an external service system is needed.
The external service system accessed by the advertisement delivery system refers to an external service system for providing a video source for the advertisement delivery system, the structure and the number of the external service systems are not limited, and the advertisement delivery system can be accessed into any structure and any number of the external service systems according to requirements.
Step 103: and querying the target video from the external service system when the target video cannot be queried in the video update data.
Here, the embodiment of the invention allows a certain video query error, and if the target video is not queried from the hot spot video data and the video update data, the target video is queried from an external service system, thereby ensuring the effective acquisition of the target video.
According to the video information query method, the video query request can be responded in real time through the cached hot spot video data and the video update data, query delay is avoided, the query requirement can be basically met through the cached hot spot video data and the video update data, the request amount queried from an external service system is reduced, server resources are saved, and reliability and stability are improved.
Preferably, as shown in fig. 2, before the step 101, the method further includes:
step 1001: according to historical data of video information inquiry in the advertisement putting process, acquiring hot spot video data with inquiry frequency meeting preset conditions, and storing the hot spot video data in the advertisement putting system locally;
the preset condition is that the query frequency of the video is greater than or equal to a preset threshold value, or the preset condition is that the video is arranged according to the sequence from high query frequency to low query frequency, the query frequency of the video is arranged in the first N bits, and N is an integer greater than or equal to 1.
At the moment, through analyzing the historical data of the video information query in the advertisement putting process, the hot spot video data are obtained and stored locally, when a video query request is received, the query is firstly carried out in the local hot spot video data, the real-time response can be achieved, the query delay is avoided, and the query request quantity of calling an external interface is reduced. The hot spot video data is selected video with the query frequency being greater than or equal to a preset threshold value or the query frequency being ranked in the front N bits, the hot spot video selected in the mode can meet the video query requirement of most advertisements, the hit rate of local query is ensured, the real-time response of the query is ensured, and the system performance is improved.
The preset threshold value can be set according to requirements and can be set to any value. For example, assuming that the preset threshold is set to 2 tens of millions, when the preset condition is that the query frequency of the video is greater than or equal to the preset threshold, according to the historical data of the video information query, obtaining the video with the query frequency of more than 2 tens of millions as the hot spot video to store locally.
The value of N may be set as desired, or may be any value. For example, assuming that N is set to 100, when the preset condition is that videos are arranged in order of from high query frequency to low query frequency, the query frequency of the videos is arranged in the first N bits, and according to the historical data of the video information query, the videos with the query frequency arranged in the first 100 bits are acquired and stored locally as hot spot videos.
Of course, other manners may be adopted to select the hotspot video data in the embodiment of the present invention, for example, whether the video belongs to the hotspot video is determined according to the tag of the video, which is not described herein.
In the embodiment of the invention, the historical data of the video information inquiry in the advertisement putting process can only store the basic information such as the ID of each video so as to save storage resources, at this time, step 1001 can firstly acquire the set of the basic information such as the ID of the hot spot video according to the historical data, and then acquire the service data of the hot spot video from the external service system in batches according to the set of the basic information such as the ID of the hot spot video for local storage.
Specifically, the external service system may store the service data of the video in a service database, for example, a Mysql (My Structured Query Language, structured query language) database may be used in the service database, and the advertisement delivery system may derive, from the service database, a data file (for example, a source code file mm format file) of the service data business data of the hotspot video through the video derivation service, and synchronize the data file to a local machine of the video information query service of the advertisement delivery system.
Preferably, after the hotspot video data is stored locally in the advertisement delivery system in step 1001, the method further includes:
updating historical data of video information inquiry in the advertisement putting process according to a preset period, and re-acquiring hot spot video data of which the inquiry frequency meets preset conditions according to the updated historical data;
and updating the hot spot video data locally stored in the advertisement delivery system according to the re-acquired hot spot video data.
At this time, the locally stored hot spot video data is updated regularly according to the preset period, so that the locally stored hot spot video data can be ensured to be the latest hot spot video all the time, the hit rate of local query of the video is improved, and the system performance is improved.
The preset period can be set according to requirements and can be any value. For example, if the preset period is one day, the hot spot video data is obtained again according to the historical data of the video information inquiry in the daily updated advertisement putting process and is synchronized to the local memory, so that the accuracy of the local hot spot video data is ensured.
Preferably, the method further comprises:
step 1002: when the external service system accessed by the advertisement delivery system has video update data, the video update data is acquired from the external service system, and the video update data is stored in the database.
At this time, the advertisement delivery system captures the external video update data in real time, that is, acquires the video update data from the external service system in real time and records the video update data in the database, thereby supplementing the hot spot video data, and when the target video cannot be queried in the hot spot video data, querying in the video update data, improving the query hit rate, further reducing the request amount queried from the external service system, and reducing the delay rate of the video query request through the cached video update data, thereby avoiding the query delay. And video update data is stored in the database, so that local storage resources are saved.
When the external service system has video update, a notification message is issued to the advertisement delivery system, and after the advertisement delivery system receives the notification message, video update data can be acquired from the external service system.
The database for storing the video update data according to the embodiment of the present invention may use a kv (Key-Value) storage system, for example, dis (Remote Dictionary Server, remote dictionary service), but is not limited thereto.
In particular, in order to minimize the risk of dependency on an external system, the embodiment of the present invention has an emergency downgrade function for a kv storage system, and the configuration center can directly switch the reading of the kv storage system to a local cache, that is, when step 1002 stores video update data in a database, or when step 102 queries a target video in the database, or any step involving database reading, the method can switch to local cache execution, so as to ensure the functional integrity of the whole service.
When the kv storage system is in downtime and other risks, the kv storage system is closed through the configuration center, and related reading operations are switched to the local cache, namely, the local cache is used for storing video update data, inquiring target videos and other functions. The emergency standby function is realized through the local cache memory, and the functional integrity and the safety of the whole service are ensured.
Preferably, after the hotspot video data is stored locally in the advertisement delivery system in step 1001, the method further includes:
when the external service system accessed by the advertisement delivery system has video update data and the video in the video update data comprises the video in the hot spot video data, the hot spot video data locally stored by the advertisement delivery system is updated according to the video update data.
At this time, when the external service system updates the locally stored hot spot video data, the updated data is synchronized to the locally stored hot spot video data, so as to ensure that the locally stored hot spot video data is the latest and most accurate video data, further improve the hit rate of local query, thereby ensuring the real-time response of the query and improving the system performance.
When the external service system has video update, a notification message is issued to the advertisement delivery system, and after the advertisement delivery system receives the notification message, the advertisement delivery system can acquire video update data from the external service system and detect whether the acquired video update data comprises hot spot video data.
Preferably, the method further comprises:
step 1003: setting validity period information of each video stored in a database while storing the video update data in the database;
step 1004: and deleting any video from the database when the time stored in the database reaches the corresponding validity period.
Here, since some new videos or cold videos which are updated newly are stored in the database, the access frequency of the videos is low, in order to save database resources, the validity period information of the videos is simultaneously stored in the database, and when a certain video reaches the corresponding validity period, the video is deleted from the database to release the database storage resources.
By the method provided by the embodiment of the invention, the latest updated video can be stored in the database continuously, the outdated video is deleted, resources are released, the hot video is supplemented, the hit rate of video query is ensured, and the optimal performance of the database is ensured.
For example, for national celebration propaganda videos, a longer effective period, such as one month, can be set due to low update frequency, and for small videos uploaded by users, a shorter effective period, such as one week, can be set due to higher update frequency. The method can be flexibly set according to requirements, and the setting mode of the effective period is not limited in the embodiment of the invention.
Preferably, after the step 103, the method further includes:
and storing the inquired target video in the database.
At this time, if the target video is not queried from the hot video data and the video update data, namely when video query errors occur, the data update in the database is triggered through the video query error event, namely, some cold videos with lower access frequency are continuously supplemented into the database, and when the same video is queried again, the videos are acquired from the database, so that instant response can be realized without locally storing all the videos, the cost of video storage resources is reduced, and the contradiction between the low delay requirement of real-time query and the high delay of the video information update in the existing scheme is decoupled.
When the target video is not inquired in the hot spot video data and the video update data, the data in the database can be updated in an asynchronous processing mode, and the target video is supplemented into the database, so that the response speed is improved, and the processing efficiency is improved.
In the embodiment of the invention, when the video is required to be inquired from an external service system, the video can be realized by a message queue mode, and the specific description is as follows.
Preferably, as shown in fig. 3, the step 103 includes:
step 1031: writing a task for inquiring the target video into a message queue; the task of inquiring the target video, which is written into a message queue, comprises version information, ID information and/or validity period information of the target video, which are stored in the database;
step 1032: and acquiring video data corresponding to the task in the message queue from the external service system.
At this time, when the target video is not queried from the hot spot video data and the video update data, the query task is written into the message queue, the query request is submitted to an external service system through the message queue, the orderly processing of the query request can be ensured by utilizing the caching function of the message queue, and the query pressure on a service party is reduced.
The message queue used in step 1031 may be, for example, a distributed publish-subscribe message system kafka message queue, but is not limited to this.
The message queues may include a first message queue for updating video basic information and a second message queue for updating video feature library information.
The first message queue typically issues update messages for basic information such as ID, version, URL, etc. of the video. The second message queue typically issues updated messages of feature information such as labels of the video, which may include custom type labels, NLP (Natural Language Processing ) labels obtained through machine learning, and so on.
Since the basic information such as the ID of the video is different from the information source of the characteristic information such as the tag of the video, two message queues are used for issuing the update message.
Wherein step 1031 writing the task of querying the target video to the message queue may comprise: and writing the task for inquiring the target video into a first message queue for updating the video basic information.
In this case, when the target video task query task is written, a query message of the basic information such as the ID and version of the video may be written in the first message queue, and the video may be queried.
Preferably, before step 1031 writes the task of querying the target video to the message queue, the method further comprises:
and filtering the received request for repeatedly inquiring the same target video within the preset time.
At this time, for the sudden hot spot new video, pulse flow can be generated in a short time, and in order to prevent the excessive repeated inquiry tasks from being submitted to the message queue, the embodiment of the invention supports the filtering processing of the repeated tasks in a short time, namely, the repeated tasks are not submitted after the same task is submitted once, and the subsequent repeated tasks are filtered out.
The preset time length can be set according to requirements and can be any time length.
In the embodiment of the invention, when the external service system needs to update the video to the database, the video can also be realized by a message queue mode, which is specifically described as follows.
Preferably, as shown in fig. 4, when the external service system accessed by the advertisement delivery system has video update data in step 1002, the step of acquiring the video update data from the external service system includes:
step 10021: subscribing the task of video update data written in the message queue by an external service system;
step 10022: and acquiring video data corresponding to the task in the message queue from the external service system.
At this time, when the external service system has video update data, the external service system writes the task of the video update data into the message queue, the video information update service of the advertisement delivery system subscribes to the task in the message queue, the video update data of the external service system is queried through the message queue, the orderly processing of query requests can be ensured by utilizing the caching function of the message queue, and the query pressure on the service party is reduced.
The message queue used in step 10021 may be, for example, a distributed message middleware locktmq message queue, but is not limited thereto.
The message queues may include a first message queue for updating video basic information and a second message queue for updating video feature library information. Writing tasks of video update data to the message queue by the external business system may include: the external service system writes the task of the video update data into a first message queue for updating the video basic information and/or a second message queue for updating the video feature library information. Accordingly, the subscribing task of step 10021 may include: a task of subscribing to video update data written in a first message queue of video basic information updates and/or a second message queue of video feature library information updates. At this time, the first message queue can obtain the messages updated by the basic information such as the duration of the video and the URL, and the second message queue can obtain the messages updated by the feature information such as the tag of the video, and the tag information can include a custom type tag, an NLP tag obtained by machine learning, and the like.
For step 1032 and step 10022, the step of obtaining the video data corresponding to the task in the message queue from the external service system may include:
sending task requests in the message queue to the external service system in batches, and carrying out query rate QPS (quality of service) current limitation on the number of task requests sent to the external service system;
And receiving video data corresponding to the task in the message queue returned by the external service system.
At this time, tasks in the kafka or the locktmq message queues can be asynchronously executed in batches after being combined, and the consumed QPS is limited in the execution process, so that the processing efficiency is improved, the timeliness of the system is improved, and the query pressure to a business party is reduced.
The above functions of batch processing tasks in the message queue and performing QPS current limiting may be performed by the task execution manager Task Excution Manager, that is, the task of the consumed kafka or locktq message queue may be submitted to Task Excution Manager, and Task Excution Manager is to perform a layer of encapsulation on the execution queue, so as to provide functions of serial synchronous execution, parallel asynchronous execution, batch serial synchronous execution, batch parallel asynchronous execution, and QPS current limiting in the execution process. For the tasks of the kafka or locktmq message queues, task Excution Manager would limit the QPS consumed and merge the tasks to construct a batch request to the business system for execution.
The implementation flow of the video information query method of the present invention is illustrated below in conjunction with an application scenario.
As shown in fig. 5, it is assumed that the advertisement delivery system of the embodiment of the present invention includes a video information query service video summary, a hot spot video export service exporter, a video information update service video info updater, a video update data cache database hikv, message middleware kafka and dockmq, and the external service system accessed includes a video service data source.
The advertisement delivery system can set a plurality of video information inquiry services so as to bear more video inquiry service demands based on the plurality of inquiry services, and achieve faster response speed. The advertisement delivery system can also set a plurality of video information updating services, so that the processing efficiency of video updating is improved.
The video information query method of the embodiment of the invention has the following specific execution flow:
s21: firstly, a video information inquiry service derives a data file of hot spot video service data from a service database (Mysql database can be adopted) through a hot spot video derivation service, and synchronizes to a local machine of the video information inquiry service.
S22: when there is a query request of the target video sent by the adapter, the video information query service queries the local hotspot video service data first, if the target video is queried, the target video is directly returned, otherwise, the process goes to S23.
S23: the video information inquiry service inquires the video update data cache database, if the target video is inquired, the video information inquiry service returns directly, otherwise, the process goes to S24.
Here, S22 and S23 query for the target video includes two cases where there is no target video or where the version is not up to date.
S24: the video information inquiry service writes a task of inquiring the target video into a message queue of the video basic information update of kafka, wherein the kafka comprises the message queue entity of the video basic information update and the message queue feature of the video feature library information update, the writing task comprises version information and id information of the video and validity period information stored in a video update data cache database.
S25: the video information update service subscribes to the task of kafka, and when a task message is acquired, jumps to S27.
S26: the video information update service subscribes to the task of locktmq and when a task message is acquired, it jumps to S27.
When the video service data source has video update data, the video service data source update notification service writes an update task in the locktmq.
S27: the video information updating service queries the read/feature interface of the video service data source and then updates the queried video data into the video update data cache database.
When the step S27 is executed, the video information update service may submit the task of the kafka or the locktmq to the task execution manager, and the task execution manager may limit the QPS consumed, and merge the tasks and construct a batch request to the video service data source for execution.
Wherein, kafka and locktmq support multi-process parallel consumption under the same group id, and the joining and exiting of the video information updating service automatically triggers the balance of consumption partitions.
When a query request of a target video is received, the video information query method firstly queries the hot spot video data cached locally, when the query is not completed, then queries the video update data cached in the database, and when the query is not completed in the local database and the database, the data can be queried from the outside and the database update data is triggered, so that the video query request can be responded in real time, the query delay is avoided, the request amount queried from an external service system is reduced, the server resource is saved, the reliability and the stability are improved, a certain video query error is allowed, the video update is triggered through the video query error event, the cost of video storage resources is effectively reduced, and the contradiction between the low delay requirement of the real-time query and the high delay of the video information update of the existing scheme is decoupled.
Referring to fig. 6, the embodiment of the present invention further provides a video information query apparatus 600, applied to an advertisement delivery system, where the apparatus includes:
the first query module 601 is configured to query, when receiving a target video query request, a target video in pre-stored hotspot video data; the hot spot video data are obtained in advance according to historical data of video information inquiry in the advertisement putting process, and are stored in the advertisement putting system;
a second query module 602, configured to query the target video in pre-stored video update data when the target video is not queried in the hotspot video data; the video update data are acquired from an external service system accessed by the advertisement delivery system when the video update is carried out on the external service system, and the video update data are stored in a database;
and a third query module 603, configured to query the target video from the external service system when the target video is not queried in the video update data.
The video information query device of the embodiment of the invention can respond to the video query request in real time through the cached hot spot video data and the video update data, thereby avoiding query delay, basically meeting query requirements through the cached hot spot video data and the video update data, reducing the request amount queried from an external service system, saving server resources, and improving reliability and stability.
Preferably, the apparatus further comprises:
the first storage module is used for acquiring hot spot video data with the query frequency meeting the preset condition according to the historical data of the video information query in the advertisement putting process, and storing the hot spot video data locally in the advertisement putting system;
the preset condition is that the query frequency of the video is greater than or equal to a preset threshold value, or the preset condition is that the video is arranged according to the sequence from high query frequency to low query frequency, the query frequency of the video is arranged in the first N bits, and N is an integer greater than or equal to 1.
Preferably, the apparatus further comprises:
the first acquisition module is used for updating historical data of video information inquiry in the advertisement putting process according to a preset period, and re-acquiring hot spot video data with inquiry frequency meeting preset conditions according to the updated historical data;
and the first updating module is used for updating the hot spot video data locally stored in the advertisement delivery system according to the acquired hot spot video data.
Preferably, the apparatus further comprises:
and the second storage module is used for acquiring the video update data from the external service system when the external service system accessed by the advertisement delivery system has the video update data, and storing the video update data in the database.
Preferably, the apparatus further comprises:
the first setting module is used for setting validity period information of each video stored in the database while storing the video update data in the database;
and the deleting module is used for deleting any video from the database when the time stored in the database reaches the corresponding validity period.
Preferably, the apparatus further comprises:
and the third storage module is used for storing the inquired target video in the database.
Preferably, the apparatus further comprises:
and the second updating module is used for updating the hot spot video data locally stored in the advertisement delivery system according to the video updating data when the video updating data is available in the external service system accessed by the advertisement delivery system and the video in the video updating data comprises the video in the hot spot video data.
Preferably, the third query module 603 is specifically configured to:
writing a task for inquiring the target video into a message queue; the task of inquiring the target video, which is written into a message queue, comprises version information, ID information and/or validity period information of the target video, which are stored in the database;
And acquiring video data corresponding to the task in the message queue from the external service system.
Preferably, the apparatus further comprises:
and the filtering module is used for filtering the received request for repeatedly inquiring the same target video within the preset time.
Preferably, the second storage module is specifically configured to:
subscribing the task of video update data written in the message queue by an external service system;
and acquiring video data corresponding to the task in the message queue from the external service system.
Preferably, the step of obtaining, by the third query module 603 or the second storage module, the video data corresponding to the task in the message queue from the external service system includes:
sending task requests in the message queue to the external service system in batches, and carrying out query rate QPS (quality of service) current limitation on the number of task requests sent to the external service system;
and receiving video data corresponding to the task in the message queue returned by the external service system.
When a query request of a target video is received, the video information query device firstly queries the hot spot video data cached locally, when the query is not completed, then queries the video update data cached in the database, and when the query is not completed in the local database and the database, the data can be queried from the outside and the database update data is triggered, so that the video query request can be responded in real time, the query delay is avoided, the request amount queried from an external service system is reduced, the server resource is saved, the reliability and the stability are improved, a certain video query error is allowed, the video update is triggered through the video query error event, the cost of video storage resources is effectively reduced, and the contradiction between the low delay requirement of the real-time query and the high delay of the video information update of the existing scheme is decoupled.
For the device embodiments described above, reference is made to the description of the method embodiments for the relevant points, since they are substantially similar to the method embodiments.
The embodiment of the invention also provides electronic equipment which can be a server. As shown in fig. 7, the device includes a processor 701, a communication interface 702, a memory 703 and a communication bus 704, wherein the processor 701, the communication interface 702 and the memory 703 communicate with each other through the communication bus 704.
A memory 703 for storing a computer program.
The processor 701 is configured to execute the program stored in the memory 703, and implement the following steps:
when a target video query request is received, querying the target video in prestored hot spot video data; the hot spot video data are obtained in advance according to historical data of video information inquiry in the advertisement putting process, and are stored in the advertisement putting system;
querying the target video in pre-stored video update data when the target video cannot be queried in the hot spot video data; the video update data are acquired from an external service system accessed by the advertisement delivery system when the video update is carried out on the external service system, and the video update data are stored in a database;
And querying the target video from the external service system when the target video cannot be queried in the video update data.
The communication bus mentioned by the above electronic device may be a peripheral component interconnect standard (Peripheral Component Interconnect, abbreviated as PCI) bus or an extended industry standard architecture (Extended Industry Standard Architecture, abbreviated as EISA) bus, or the like. The communication bus may be classified as an address bus, a data bus, a control bus, or the like. For ease of illustration, the figures are shown with only one bold line, but not with only one bus or one type of bus.
The communication interface is used for communication between the electronic device and other devices.
The memory may include random access memory (Random Access Memory, RAM) or non-volatile memory (non-volatile memory), such as at least one disk memory. Optionally, the memory may also be at least one memory device located remotely from the aforementioned processor.
The processor may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU for short), a network processor (Network Processor, NP for short), etc.; but also digital signal processors (Digital Signal Processing, DSP for short), application specific integrated circuits (Application Specific Integrated Circuit, ASIC for short), field-programmable gate arrays (Field-Programmable Gate Array, FPGA for short) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
In yet another embodiment of the present invention, a computer readable storage medium is provided, in which instructions are stored, which when executed on a computer, cause the computer to perform the video information query method described in the above embodiment.
In yet another embodiment of the present invention, a computer program product containing instructions that, when run on a computer, cause the computer to perform the video information query method described in the above embodiment is also provided.
In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, produces a flow or function in accordance with embodiments of the present invention, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another, for example, by wired (e.g., coaxial cable, optical fiber, digital Subscriber Line (DSL)), or wireless (e.g., infrared, wireless, microwave, etc.). The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains an integration of one or more available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid State Disk (SSD)), etc.
It is noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
In this specification, each embodiment is described in a related manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for system embodiments, since they are substantially similar to method embodiments, the description is relatively simple, as relevant to see a section of the description of method embodiments.
The foregoing description is only of the preferred embodiments of the present invention and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.
Claims (14)
1. A video information query method applied to an advertisement delivery system, the method comprising:
when a target video query request is received, querying the target video in prestored hot spot video data; the hot spot video data are obtained in advance according to historical data of video information inquiry in the advertisement putting process, and are stored in the advertisement putting system;
querying the target video in pre-stored video update data when the target video cannot be queried in the hot spot video data; the video update data are acquired from an external service system accessed by the advertisement delivery system when the video update is carried out on the external service system, and the video update data are stored in a database;
and querying the target video from the external service system when the target video cannot be queried in the video update data.
2. The video information query method of claim 1, wherein prior to receiving the target video query request, the method further comprises:
according to historical data of video information inquiry in the advertisement putting process, acquiring hot spot video data with inquiry frequency meeting preset conditions, and storing the hot spot video data in the advertisement putting system locally;
the preset condition is that the query frequency of the video is greater than or equal to a preset threshold value, or the preset condition is that the video is arranged according to the sequence from high query frequency to low query frequency, the query frequency of the video is arranged in the first N bits, and N is an integer greater than or equal to 1.
3. The method of video information query of claim 2, wherein after storing the hotspot video data locally at the advertisement delivery system, the method further comprises:
updating historical data of video information inquiry in the advertisement putting process according to a preset period, and re-acquiring hot spot video data of which the inquiry frequency meets preset conditions according to the updated historical data;
and updating the hot spot video data locally stored in the advertisement delivery system according to the re-acquired hot spot video data.
4. The video information query method of claim 1, wherein the method further comprises:
when the external service system accessed by the advertisement delivery system has video update data, the video update data is acquired from the external service system, and the video update data is stored in the database.
5. The video information query method of claim 4, wherein the method further comprises:
setting validity period information of each video stored in a database while storing the video update data in the database;
and deleting any video from the database when the time stored in the database reaches the corresponding validity period.
6. The video query method of claim 1, wherein when the target video is not queried in the video update data, after querying the target video from the external service system, the method further comprises:
and storing the inquired target video in the database.
7. The method of video information query of claim 2, wherein after storing the hotspot video data locally at the advertisement delivery system, the method further comprises:
When the external service system accessed by the advertisement delivery system has video update data and the video in the video update data comprises the video in the hot spot video data, the hot spot video data locally stored by the advertisement delivery system is updated according to the video update data.
8. The video query method according to claim 1, wherein the step of querying the target video from the external service system when the target video is not queried in the video update data comprises:
writing a task for inquiring the target video into a message queue; the task of inquiring the target video, which is written into a message queue, comprises version information, ID information and/or validity period information of the target video, which are stored in the database;
and acquiring video data corresponding to the task in the message queue from the external service system.
9. The video query method of claim 8, wherein prior to writing the task of querying the target video to a message queue, the method further comprises:
and filtering the received request for repeatedly inquiring the same target video within the preset time.
10. The video query method of claim 4, wherein when the external service system accessed by the advertisement delivery system has video update data, the step of acquiring the video update data from the external service system comprises:
subscribing the task of video update data written in the message queue by an external service system;
and acquiring video data corresponding to the task in the message queue from the external service system.
11. The video information query method according to claim 8 or 10, wherein the step of acquiring video data corresponding to the task in the message queue from the external service system comprises:
sending task requests in the message queue to the external service system in batches, and carrying out query rate QPS (quality of service) current limitation on the number of task requests sent to the external service system;
and receiving video data corresponding to the task in the message queue returned by the external service system.
12. A video information query apparatus for use in an advertising system, the apparatus comprising:
the first query module is used for querying the target video in the prestored hot spot video data when receiving a target video query request; the hot spot video data are obtained in advance according to historical data of video information inquiry in the advertisement putting process, and are stored in the advertisement putting system;
The second query module is used for querying the target video in the pre-stored video update data when the target video cannot be queried in the hot spot video data; the video update data are acquired from an external service system accessed by the advertisement delivery system when the video update is carried out on the external service system, and the video update data are stored in a database;
and the third query module is used for querying the target video from the external service system when the target video cannot be queried in the video update data.
13. An electronic device, comprising: a processor, a communication interface, a memory, and a communication bus; the processor, the communication interface and the memory complete communication with each other through a communication bus;
a memory for storing a computer program;
a processor for implementing the steps in the video information query method as claimed in any one of claims 1 to 11 when executing a program stored on a memory.
14. A computer-readable storage medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, implements a video information query method as claimed in any one of claims 1 to 11.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110144158.6A CN112818166B (en) | 2021-02-02 | 2021-02-02 | Video information query method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110144158.6A CN112818166B (en) | 2021-02-02 | 2021-02-02 | Video information query method and device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112818166A CN112818166A (en) | 2021-05-18 |
| CN112818166B true CN112818166B (en) | 2023-07-25 |
Family
ID=75860647
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110144158.6A Active CN112818166B (en) | 2021-02-02 | 2021-02-02 | Video information query method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112818166B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115174982B (en) * | 2022-06-30 | 2024-04-09 | 咪咕文化科技有限公司 | Real-time video association display method, device, computing equipment and storage medium |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104216957A (en) * | 2014-08-20 | 2014-12-17 | 北京奇艺世纪科技有限公司 | Query system and query method for video metadata |
| WO2016123894A1 (en) * | 2015-02-06 | 2016-08-11 | 百度在线网络技术（北京）有限公司 | Advertisement delivery method and system |
| CN105933538A (en) * | 2016-06-15 | 2016-09-07 | 维沃移动通信有限公司 | Video finding method for mobile terminal and mobile terminal |
| CN106326462A (en) * | 2016-08-30 | 2017-01-11 | 北京奇艺世纪科技有限公司 | Video index grading method and device |
| WO2017096836A1 (en) * | 2015-12-07 | 2017-06-15 | 乐视控股（北京）有限公司 | Method and system for caching video file |
| CN108132958A (en) * | 2016-12-01 | 2018-06-08 | 阿里巴巴集团控股有限公司 | A kind of multi-level buffer data storage, inquiry, scheduling and processing method and processing device |
| CN109992695A (en) * | 2019-04-10 | 2019-07-09 | 北京奇艺世纪科技有限公司 | A kind of video information querying method and device |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090113475A1 (en) * | 2007-08-21 | 2009-04-30 | Yi Li | Systems and methods for integrating search capability in interactive video |
| US10225582B1 (en) * | 2017-08-30 | 2019-03-05 | Microsoft Technology Licensing, Llc | Processing live video streams over hierarchical clusters |
| US20190297131A1 (en) * | 2018-03-23 | 2019-09-26 | Satori Worldwide, Llc | System and Method for Querying and Updating a Live Video Stream Using a Structured Query Language |
-
2021
- 2021-02-02 CN CN202110144158.6A patent/CN112818166B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104216957A (en) * | 2014-08-20 | 2014-12-17 | 北京奇艺世纪科技有限公司 | Query system and query method for video metadata |
| WO2016123894A1 (en) * | 2015-02-06 | 2016-08-11 | 百度在线网络技术（北京）有限公司 | Advertisement delivery method and system |
| WO2017096836A1 (en) * | 2015-12-07 | 2017-06-15 | 乐视控股（北京）有限公司 | Method and system for caching video file |
| CN105933538A (en) * | 2016-06-15 | 2016-09-07 | 维沃移动通信有限公司 | Video finding method for mobile terminal and mobile terminal |
| CN106326462A (en) * | 2016-08-30 | 2017-01-11 | 北京奇艺世纪科技有限公司 | Video index grading method and device |
| CN108132958A (en) * | 2016-12-01 | 2018-06-08 | 阿里巴巴集团控股有限公司 | A kind of multi-level buffer data storage, inquiry, scheduling and processing method and processing device |
| CN109992695A (en) * | 2019-04-10 | 2019-07-09 | 北京奇艺世纪科技有限公司 | A kind of video information querying method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112818166A (en) | 2021-05-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109684358B (en) | Data query method and device | |
| CN101493826B (en) | Database system based on WEB application and data management method thereof | |
| CN109947668B (en) | Method and device for storing data | |
| CN101650741B (en) | Method and system for updating index of distributed full-text search in real time | |
| CN101364217B (en) | Data maintenance method, equipment and system thereof in data-base | |
| CN111563102A (en) | Cache updating method, server, system and storage medium | |
| CN103338243A (en) | Method and system for updating cache data of Web node | |
| CN106775498B (en) | Cache data synchronization method and system | |
| CN107092628B (en) | Time series data processing method and device | |
| WO2019057193A1 (en) | Data deletion method and distributed storage system | |
| CN112818166B (en) | Video information query method and device, electronic equipment and storage medium | |
| CN110309156A (en) | Database Systems, database update, expansion method and equipment | |
| CN113312370B (en) | Information acquisition method and device, electronic equipment and storage medium | |
| CN111367691B (en) | Data feedback method and device, electronic equipment and storable medium | |
| CN112395337B (en) | Data export method and device | |
| CN112865927B (en) | Message delivery verification method, device, computer equipment and storage medium | |
| AU2020203282A1 (en) | Method and system for matching multi-dimensional data units in electronic information system | |
| CN113037420B (en) | Reading time stamp obtaining method and device, electronic equipment and storage medium | |
| CN112949326B (en) | Information query method, device, equipment and computer readable medium | |
| CN110781238B (en) | Client view caching method and device based on combination of Redis and Hbase | |
| CN113742376A (en) | Data synchronization method, first server and data synchronization system | |
| CN114077587A (en) | Rule engine based business processing method, rule engine, medium and device | |
| CN112688982A (en) | User request processing method and device | |
| CN112578996B (en) | Metadata sending method of storage system and storage system | |
| CN115002137B (en) | Offline message processing method and device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |