WO2021114552A1 - 信息处理方法及装置、电子设备和存储介质 - Google Patents

信息处理方法及装置、电子设备和存储介质 Download PDF

Info

Publication number
WO2021114552A1
WO2021114552A1 PCT/CN2020/087486 CN2020087486W WO2021114552A1 WO 2021114552 A1 WO2021114552 A1 WO 2021114552A1 CN 2020087486 W CN2020087486 W CN 2020087486W WO 2021114552 A1 WO2021114552 A1 WO 2021114552A1
Authority
WO
WIPO (PCT)
Prior art keywords
information
target material
video stream
video
reference image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2020/087486
Other languages
English (en)
French (fr)
Inventor
张帆
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Sensetime Technology Development Co Ltd
Original Assignee
Beijing Sensetime Technology Development Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Sensetime Technology Development Co Ltd filed Critical Beijing Sensetime Technology Development Co Ltd
Priority to KR1020217018772A priority Critical patent/KR20210090262A/ko
Priority to JP2021534223A priority patent/JP2022516701A/ja
Publication of WO2021114552A1 publication Critical patent/WO2021114552A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/431—Generation of visual interfaces for content selection or interaction; Content or additional data rendering
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/23424—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/23418—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/44008—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/44016—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving splicing one content stream with another content stream, e.g. for substituting a video clip
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/45—Management operations performed by the client for facilitating the reception of or the interaction with the content or administrating data related to the end-user or to the client device itself, e.g. learning user preferences for recommending movies, resolving scheduling conflicts
    • H04N21/458—Scheduling content for creating a personalised stream, e.g. by combining a locally stored advertisement with an incoming stream; Updating operations, e.g. for OS modules ; time-related management operations
    • H04N21/4586—Content update operation triggered locally, e.g. by comparing the version of software modules in a DVB carousel to the version stored locally
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/81—Monomedia components thereof
    • H04N21/812—Monomedia components thereof involving advertisement data
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/81—Monomedia components thereof
    • H04N21/8126—Monomedia components thereof involving additional data, e.g. news, sports, stocks, weather forecasts
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
    • H04N21/845—Structuring of content, e.g. decomposing content into time segments
    • H04N21/8456—Structuring of content, e.g. decomposing content into time segments by decomposing the content in the time domain, e.g. in time segments

Definitions

  • the present disclosure relates to the field of information processing technology, and in particular to an information processing method and device, electronic equipment, and storage medium.
  • streaming media information such as a video stream
  • other streaming media information other than the video stream such as advertising materials
  • OTT over-the-top
  • third-party service providers since TVs and set-top boxes cannot obtain relevant content of third-party advertising materials, automatic insertion of advertising materials cannot be realized.
  • the present disclosure proposes a technical solution for information processing.
  • an information processing method comprising: performing face detection on a first video image in a played video stream to obtain face information of the first video image;
  • the server sends the face information; receives the target material sent by the server, wherein the target material is obtained based on the face information; in response to detecting that the video stream is played to the start playing time corresponding to the target material To trigger the display of the target material.
  • the target material includes an advertising material.
  • triggering the display of the target material includes: sending the target material to a terminal device so that the terminal device displays the target material; or, after the triggering of the display of the target material , The method further includes: displaying the target material.
  • the display mode of the target material includes: the target material is displayed in the corner of the video stream playback interface in the form of a corner mark; or the target material is associated with the playback content of the playback interface
  • the ground is displayed on the video stream playback interface in the form of band-aids; or, the target material is displayed on the video stream playback interface in the form of banners and spans the playback interface.
  • the method further includes: receiving playback time information of the target material sent by the server, where the playback time information includes reference image information corresponding to a preset playback time period of the target material; The playing time information detects whether the video stream is played to the start playing time corresponding to the target material.
  • the reference image information includes: feature information of multiple frames of reference images corresponding to the preset playing time period of the target material.
  • the multi-frame reference image is obtained by sampling multiple times on at least one time period adjacent to the preset playback time period.
  • detecting whether the video stream is played to the start playback time corresponding to the target material includes: comparing the second video image to be played of the video stream with the playback time The reference image information included in the information is compared to obtain a comparison result; based on the comparison result, it is determined whether the video stream is played to the start playing time corresponding to the target material. After the first video image is played, the second video image is played.
  • determining whether the video stream is played to the start playing time corresponding to the target material includes: when the comparison result is the first in the multi-frame reference image When a reference image is successfully compared, it is determined that the video stream is played to the start time corresponding to the target material, wherein the first M reference images in the multi-frame reference image include the first reference image.
  • the display of the target material in response to detecting that the video stream is played to the start time corresponding to the target material, the display of the target material is triggered: in response to the second video image of the video stream and the multi-frame The first target reference image in the reference image is successfully compared, which triggers the display of the target material.
  • triggering the display of the target material includes: responding to detecting all If the second video image is successfully compared with the first target reference image, and the third video image of the video stream is successfully compared with the second target reference image in the multi-frame reference image, triggering the target material Displaying, wherein after the second video image is played, the third video image is played.
  • the method further includes: in a case where the comparison result is a successful comparison with a second reference image in the multi-frame reference image, determining to end the playback of the target material, wherein, The last N reference images in the multi-frame reference image include the second reference image. In some embodiments, the N reference images after the target reference image in the multi-frame reference image include the second reference image.
  • the end playback of the target material is triggered.
  • the method further includes: in response to detecting the advertisement material meeting the preset blocking condition, displaying the target material.
  • the preset interception condition includes at least one of the following:
  • a creative that has a different source or a different business from the target material has a different source or a different business from the target material.
  • an information processing method comprising: receiving face information of at least one first video image sent by a terminal device; and determining the face information based on the face information and a face template library; The video stream information to which at least one first video image belongs, wherein the face template library includes a plurality of preset face templates; obtain target material according to the video stream information; send the target to the terminal device material.
  • determining the video stream information to which the at least one first video image belongs based on the face information and the face template library includes: based on multiple preset faces included in the face template library Template to obtain the person identity information corresponding to the face information; search a video library based on the person identity information to obtain the video stream information to which the at least one first video image belongs.
  • obtaining the person identity information corresponding to the face information includes: determining that the face information corresponds to the multiple preset face templates. The first comparison result of the face template; based on the first comparison result, the person identity information corresponding to the face information is obtained.
  • the target material includes: advertising material.
  • the method further includes: sending play time information corresponding to the target material to the terminal device, wherein the play time information includes a reference image corresponding to a preset play time period of the target material information.
  • the reference image information includes: feature information of multiple frames of reference images corresponding to the preset playing time period of the target material.
  • the method further includes: sampling the video stream information in at least one adjacent time period of the preset playback time period to obtain a multi-frame reference image;
  • the feature extraction process obtains the feature information of the multi-frame reference image.
  • the method further includes: sending an advertisement blocking instruction to the terminal device, where the advertisement blocking instruction is used to indicate that when a specific advertisement material is detected in the video stream played by the terminal device, the The terminal device displays the target material.
  • an information processing device comprising: a first detection unit configured to perform face detection on a first video image in a played video stream to obtain the first video The face information of the image; the first sending unit is used to send the face information to the server; the first receiving unit is used to receive the target material sent by the server, wherein the target material is based on the face information Obtained; a first response unit for triggering the display of the target material in response to detecting that the video stream is played to the start time corresponding to the target material.
  • an information processing apparatus comprising: a third receiving unit, configured to receive face information of at least one first video image sent by a terminal device; and an information determining unit, configured to Determine the video stream information to which the at least one first video image belongs based on the face information and face template library, where the face template library includes a plurality of preset face templates; a material acquisition unit for Obtain the target material according to the video stream information; the second sending unit is configured to send the target material to the terminal device.
  • an electronic device including: a processor; a memory for storing executable instructions of the processor; wherein the processor is configured to execute the above-mentioned information processing method.
  • a computer-readable storage medium having computer program instructions stored thereon, and when the computer program instructions are executed by a processor, the foregoing information processing method is implemented.
  • face detection is performed on the first video image in the played video stream to obtain the face information of the first video image; the face information is sent to the server; the face information is received by the server The target material, wherein the target material is obtained based on the face information; in response to detecting that the video stream is played to the start time corresponding to the target material, the display of the target material is triggered, thereby facilitating the realization Automatic insertion of OTT creatives.
  • Fig. 1 shows a flowchart of an information processing method according to an embodiment of the present disclosure.
  • Fig. 2a is a schematic diagram showing a frame of video image in a video stream according to an embodiment of the present disclosure.
  • Fig. 2b shows a schematic diagram of a frame of video image displaying target material according to an embodiment of the present disclosure.
  • 3 to 5 show schematic diagrams of material display modes according to embodiments of the present disclosure.
  • Fig. 6 shows a flowchart of an information processing method according to an embodiment of the present disclosure.
  • Fig. 7 shows a schematic diagram of a sampling manner in a video search according to an embodiment of the present disclosure.
  • Fig. 8 shows a schematic diagram of a sampling manner in video stream information synchronization according to an embodiment of the present disclosure.
  • FIG. 9 shows a schematic diagram of the timing relationship of triggering the display of advertisement materials in the information processing method according to an embodiment of the present disclosure.
  • FIGS 10-12 show schematic diagrams of multiple user interfaces of an information processing method according to an embodiment of the present disclosure.
  • FIG. 13 shows a block diagram of an information processing apparatus according to an embodiment of the present disclosure.
  • FIG. 14 shows a block diagram of an electronic device according to an embodiment of the present disclosure.
  • FIG. 15 shows a block diagram of an electronic device according to an embodiment of the present disclosure.
  • Figure 1 shows a flow chart of an information processing method according to an embodiment of the present disclosure.
  • the method is applied to an information processing device.
  • the information processing device is deployed on a terminal device and can play video streams, perform video synchronization checks, and display materials, etc. .
  • terminal devices can include user equipment (UE, User Equipment), mobile devices, cellular phones, cordless phones, personal digital assistants (PDAs, Personal Digital Assistant), handheld devices, computing devices, in-vehicle devices, wearable devices, televisions, etc.
  • the information processing method may be implemented by a processor invoking computer-readable instructions stored in the memory.
  • the flow of the information processing method shown in FIG. 1 includes steps S101-S104.
  • Step S101 The terminal device performs face detection on the first video image in the played video stream to obtain face information of the first video image.
  • the video stream may include video features such as TV shows or movies, short videos, variety shows, etc., and the video stream may be composed of multiple frames of video images.
  • One or more frames of video images in a video stream can contain one or more face information.
  • feature extraction can be performed on the first video image through a feature extraction network to obtain multiple feature information, and then the multiple feature information Perform clustering, and obtain the face information according to the clustering results.
  • Step S102 The terminal device sends the face information to the server.
  • each individual face information can be extracted and sent to the server, and the server can The face information is compared with the preset face template in the face template library of the server, and the video stream information that matches the face information is obtained according to the comparison result. For example, you can search the cloud based on the comparison result.
  • the video stream information is used to obtain a cloud drama series matching the video stream played by the terminal device, and the target material (such as advertising material, etc.) is obtained accordingly.
  • Step S103 The terminal device receives the target material sent by the server, where the target material is obtained based on the face information.
  • the target material includes an advertising material
  • the advertising material can be obtained through facial information comparison performed by the server.
  • Step S104 In response to detecting that the video stream is played to the start time corresponding to the target material, the terminal device triggers the display of the target material.
  • the start playing time may be an advertising spot that displays the target material (such as an advertising material, etc.), and the starting time of playing the advertising material can be identified according to the advertising spot. Trigger the display of the creative at this ad point.
  • Fig. 2a is a schematic diagram showing a frame of video image in a video stream according to an embodiment of the present disclosure.
  • 2b is a schematic diagram showing a frame of video image displaying target material according to an embodiment of the present disclosure, wherein the played video stream is shown in the video feature 11, and the target material that triggers the display is shown in the advertisement material 12.
  • the server may send the target material to the terminal device that plays the video stream, so that the terminal device displays the target material.
  • the information processing method further includes: displaying the target material.
  • the above-mentioned target material may be an OTT advertising material.
  • OTT advertising materials are provided by third-party service providers via the Internet (such as operators).
  • an ordinary non-smart TV can not only obtain information through the Internet (such as the network provided by the operator)
  • the resources of big video content can also get more diversified effective information (such as target material) from a large number of third-party material content providers on the server.
  • face detection is performed on the first video image in the played video stream, and after obtaining the face information, the face information is sent to the server for comparison, so as to obtain the corresponding target material based on the face information , So as to realize the automatic and accurate insertion of OTT ads.
  • the display mode of the advertising material may include corner marks, band-aids, banners, etc., and Figures 3 to 5 show the display of the target material according to embodiments of the present disclosure.
  • the schematic diagram of the method is as follows:
  • the display mode may be: the advertising material 13 is displayed in the corner of the video stream playing interface in the form of a corner mark, as shown in FIG. 3.
  • the display mode may be: the advertisement material 14 is displayed on the video stream playback interface in the form of band-aids in association with the playback content of the playback interface, as shown in FIG. 4.
  • the key information "national liberation" included in the broadcast content
  • the key information will be queried for the key information or the creatives associated with the key information, and the queried creatives will be displayed to the broadcaster.
  • the interface to echo each other with the content to be played.
  • the ad word "Lifelong Loyalty" contained in the creative is related to the key message "National Liberation”.
  • the creative can be displayed on the playback interface to echo the content of the broadcast.
  • the display mode may be: the advertisement material 15 is displayed on the video stream playback interface in the form of a banner and spans the playback interface, as shown in FIG. 5.
  • the information processing method further includes: receiving playback time information of the target material sent by the server, where the playback time information includes a reference corresponding to a preset playback time period of the target material Image information; based on the playback time information, detect whether the video stream is played to the start playback time corresponding to the target material.
  • the reference image information includes: feature information of multiple frames of reference images corresponding to the preset playing time period of the target material. The multi-frame reference image is obtained by sampling multiple times on at least one time period adjacent to the preset playback time period.
  • detecting whether the video stream is played to the start play time corresponding to the target material includes: comparing the second video image to be played of the video stream with the The reference image information included in the playback time information is compared to obtain a comparison result; based on the comparison result, it is determined whether the video stream is played to the start playback time corresponding to the target material.
  • the playback of the second video image is after the first video image.
  • comparing the second video image to be played in the video stream with the reference image information to obtain the comparison result includes: performing feature extraction processing on the second video image to obtain the second video image.
  • the feature information of the video image based on the similarity between the feature information of the second video image and the feature information of at least one frame of reference image in the reference image information, a comparison result is obtained.
  • the terminal device can determine whether the video stream is played to the start playing time corresponding to the target material according to the comparison result, so as to trigger the display of the target material at the start playing time, and display the target material after the display of the target material is triggered.
  • the similarity between the feature information of the video image and the feature information of the reference image is greater than or equal to a preset threshold, it can be determined that the comparison between the video image and the reference image is successful.
  • the preset threshold may be determined according to the method of determining the similarity actually used.
  • the first M reference images in the multi-frame reference image include the first reference image.
  • the video image of the video stream may be compared with a multi-frame reference image to determine whether to trigger the display of the target material. For example, when the second video image is successfully compared with the first reference image, the display of the target material may be triggered, or when the second video image is successfully compared with the first reference image and the third video image is compared with the third When the reference image is successfully compared, the display of the target material is triggered, where the third video image is played after the second video image, and the first reference image and the third reference image may be included in multiple frames of reference images For any reference image within the first M reference images, or a preset reference image, the specific triggering mechanism can be determined according to actual requirements, which is not limited in the embodiment of the present disclosure.
  • the second video image of the video stream when the second video image of the video stream is successfully compared with the second reference image in the multi-frame reference image, it is determined to end the playback of the target material, and/or trigger the target End playback of the material, wherein the last N reference images in the multi-frame reference image include the second reference image.
  • the reference image information includes: feature information of a multi-frame reference image associated with the start playing time of the target material.
  • the multi-frame reference image is obtained by sampling multiple times on at least one time period adjacent to the preset playback time period.
  • the multi-frame reference image is obtained by sampling the first time period and the second time period adjacent to the start playing time for multiple times, wherein the end of the first time period is the Play start time, and the start of the second time period is the start play time.
  • the duration of the second time period may be the advertisement duration of the advertisement spot or slightly greater than the advertisement duration of the advertisement spot, or the start of the second time period is the next time point adjacent to the start playing time, At this time, optionally, in the above example, the first reference image is sampled in the first time period.
  • the second reference image is sampled in the second time period, but the embodiment of the present disclosure does not limit this.
  • the video synchronization check in response to detecting that the second video image is successfully compared with the first reference image, the video synchronization check is started.
  • the display of the target material is triggered.
  • the information processing method further includes: displaying the target material in response to detecting the advertising material that meets the preset blocking condition.
  • the preset blocking condition includes at least one of the following: preset merchants listed in the blacklist; preset advertisement types listed in the blacklist; or those with different sources or different merchants from the target material Creative.
  • the server sends an advertisement blocking instruction to the terminal device.
  • the advertisement blocking instruction is used to indicate that the terminal device displays the target material when an advertisement that meets the preset blocking condition is detected in the video stream played by the terminal device.
  • the advertising material played in the video stream is Nike sneakers, and the terminal device determines that the Nike sneakers are listed in the blacklist according to the advertisement blocking instructions, and the target material will be displayed in the form of corner tags, band-aids or banners, for example Ads for Adidas sneakers.
  • the advertising material played in the video stream is a Mercedes-Benz ad, which is a competing product with the target material "BMW Ad". Once found, the terminal device immediately displays the BMW ad in the form of a corner mark, band-aid or banner, etc. Improve the accuracy of playback of your own creatives.
  • the matched creative can be popped up, so as to perform precise blocking processing on the creative with the preset blocking condition.
  • FIG. 6 shows a flowchart of an information processing method according to an embodiment of the present disclosure, the method is applied to an information processing device, the information processing device is deployed on the server and can perform video stream information search and play the video stream and the displayed material Synchronize. As shown in Figure 6, the process includes steps S201-S204.
  • Step S201 The server receives the face information of at least one first video image sent by the terminal device.
  • the video stream played by the terminal device contains at least one first video image.
  • feature extraction can be performed on the first video image through a feature extraction network to obtain a corresponding video image.
  • Multiple feature information cluster the multiple feature information, and obtain face information according to the clustering result.
  • the face information of the at least one first video image is sent to the server, and the server receives the face information of the at least one first video image, and performs The face information is compared, and the target material corresponding to the video stream information (such as cloud drama) is obtained according to the comparison result.
  • the cloud drama series may be a video feature, a short video, a variety show, etc. of a TV series or movie.
  • Step S202 The server determines the video stream information to which the at least one first video image belongs based on the face information and face template library, where the face template library includes a plurality of preset face templates.
  • person identity information corresponding to the face information can be obtained.
  • the video library is searched based on the identity information of the person, and the video stream information to which at least one first video image belongs can be obtained.
  • the character identity information such as Huo Jianhua
  • a possible series matching the played TV series can be searched from the cloud series, for example, "Zhen Huan Biography”.
  • obtaining the person identity information corresponding to the face information may include: determining the first comparison result of the face information and the plurality of preset face templates ; Based on the first comparison result, the person identity information corresponding to the face information (such as Huo Jianhua) is obtained.
  • Step S203 The server obtains the target material according to the video stream information.
  • the target material can also be obtained according to the video stream information, where the target material includes advertising materials and the like.
  • the cloud drama series to be embedded with ads includes a total of 70 episodes, each with 8 advertising spots, and the placement of advertising materials can be completed by the operator.
  • the cast list, episode name, episode video, episode number and cast list of the episode are also uploaded to the server in advance by the operating staff. The synchronization between the playing video stream and the displayed material can be completed in advance.
  • Step S204 The server sends the target material to the terminal device.
  • the server sends the target material to the terminal device.
  • the terminal device After receiving the target material, the terminal device can respond to detecting that the video stream is played. To the start playing time corresponding to the target material, the display of the target material is triggered.
  • the server receives the face information of at least one first video image sent by the terminal device, and compares the face information with a plurality of preset face templates in the face template library, so as The comparison result obtains the corresponding video stream information, and the target material is obtained according to the video stream information.
  • the target material is an advertising material
  • the advertising material can be fed back to the terminal device to complete the display of the advertising material on the terminal device.
  • the target material such as advertising material, etc.
  • the target material can be accurately matched to the video stream, so that the user can get the information in real time More effective information related to the video stream (such as accurate delivery such as advertising materials, etc.).
  • the information processing method further includes: sending play time information corresponding to the target material to the terminal device, where the play time information includes a preset play time period of the target material Corresponding reference image information.
  • the reference image information includes: feature information of multiple frames of reference images corresponding to the preset playing time period of the target material.
  • the information processing method further includes: sampling the video stream information in at least one adjacent time period of the preset playing time period to obtain multiple frames of reference images, and The frame reference image is subjected to feature extraction processing to obtain the feature information of the above-mentioned multi-frame reference image.
  • the information processing method further includes: sending an advertisement blocking instruction to the terminal device, where the advertisement blocking instruction is used to indicate that when it is detected that a specific advertisement material appears in the video stream played by the terminal device
  • the terminal device displays the target material.
  • the advertising material played in the video stream is Nike sneakers.
  • the advertisement blocking instruction the terminal device inquires that the Nike sneakers are listed in the blacklist, and the target material will be displayed in the form of corner labels, band-aids or banners.
  • ads for Adidas sneakers For another example, the advertising material played in the video stream is a Mercedes-Benz ad, which is a competing product with the target material "BMW ad”.
  • the terminal device will immediately display it in the form of a corner mark, band-aid or banner.
  • BMW advertising to improve the accuracy of its own advertising material playback.
  • the advertisement blocking instruction once the advertisement material that meets the preset blocking condition is queried, the matching advertisement material can be popped up, so as to accurately intercept the advertisement material that meets the preset blocking condition.
  • the video stream played by the terminal device includes at least one first video image, and the first video image includes one or more facial information, and the facial information needs to be detected.
  • the target material can be an advertising material
  • the server searches for cloud video stream information (such as cloud drama) based on the face information detected by the terminal device. After determining the currently playing cloud drama series and the current playing time point, the server may send the advertising material and the reference image information to the terminal device. The terminal device compares the reference image feature information to determine the starting time according to the comparison result, so that the display of the advertising material is triggered at the starting time (or advertising point), and then the advertising material is displayed.
  • Video stream information search including episode search and a single episode search in the episode
  • the cast list, episode name, episode video, episode number, cast list, etc. of the cloud episode can be determined by The operations staff upload to the cloud server in advance.
  • the terminal device After the terminal device starts to play the video stream, it can use the SDK (Software Development Kit) or APP (Application) of the terminal device to detect the video played within 3 minutes, and detect the independent facial information in one or more frames of the video stream as The face information to be compared is uploaded to the server.
  • the server compares the face information to be compared with the actors in the cloud drama series to identify the identity information of the characters based on the comparison results, such as multiple actor names or the cast list of the drama.
  • the cast of each actor name or drama series matches the most likely cloud drama (such as the biography of Zhen Huan).
  • FIG. 7 shows a schematic diagram of a sampling method in a video search according to an embodiment of the present disclosure.
  • the server can sample all advertising spots in a cloud drama (for example, 70 episodes).
  • the server compares a frame of video image uploaded by the terminal device with all the sampled frames (33600 frames) of the cloud drama, and finds the most similar frame of sampled video image to this frame of video image among the sampled frames, thereby determining what is being played
  • the video stream is the single episode number corresponding to the sampled video image of the frame (such as the third episode of Zhen Huan's biography) and the playback time (accurate to the second).
  • FIG. 8 shows a schematic diagram of a sampling method in video stream information synchronization according to an embodiment of the present disclosure.
  • the server can perform a total of 8 seconds before and after each advertisement point, according to each second Two frames are sampled to obtain 16 reference images, and then feature information is extracted for each frame of these 16 reference images. For example, the key pixel information in each frame of reference image can be extracted to compress the amount of data to be processed. Improve the efficiency of synchronization processing.
  • the SDK or APP that the advertising material is delivered to the terminal device.
  • the currently played video stream information can be compared with the feature information of the reference image downloaded locally, and the similarity between the feature information can be obtained, and the comparison result can be obtained.
  • the terminal device confirms whether the video stream is played to
  • the video stream information played by the terminal device is an episode in the target drama series, and there is more than one advertisement spot in an episode, that is, there are multiple advertisement spots, multiple sampling is required.
  • the sampling in the video stream information synchronization is to find all the advertising spots in the set, and synchronize these advertising spots to the SDK or APP of the terminal device, and then continue to play the video stream on the SDK or APP of the terminal device Information, after comparing the video stream information with reference image feature information, the advertisement spot closest to the current playing point of the current video stream information can be obtained, so that the corresponding advertisement material can be displayed at the advertisement spot. This is different from the sampling in the above video search.
  • the sampling here is to compare the feature information of the reference image of a certain episode in the cloud drama with the video stream information played by the terminal device to search for the currently playing video
  • the flow information is the display of the advertising material of which advertising point should be concentrated.
  • FIG. 9 shows a schematic diagram of the relationship between time nodes that trigger advertisement display in an information processing method according to an embodiment of the present disclosure.
  • the video stream information played by the SDK or APP of the terminal device is played to the end of t1
  • the SDK keeps the video synchronization check until the end of t2.
  • the reference image information corresponding to the nearest advertising spot includes the feature information of 16 frames of reference images, these 16 frames of reference images include: the first frame of reference image, the second frame of reference image, ..., the sixth frame of reference image ,..., the sixteenth frame reference image.
  • the terminal device can determine that when it detects that the third video image in the current video stream is successfully compared with the sixth frame reference image, it will trigger the display of the advertising material, and The video synchronization check continues after the first comparison is successful.
  • the display of the advertising material is triggered.
  • the terminal device continues to check the video synchronization.
  • the display of the advertising material ends. If the video stream information played by the SDK or APP of the terminal device cannot be successfully compared with at least one frame of the reference image from the seventh frame to the 16th frame during the advertisement display process, that is, the terminal device is deemed to have switched to the content to be played, and the advertisement is stopped. The presentation of the material.
  • the video stream search and video stream information synchronization are initiated again.
  • the time period for the video synchronization check is from the playback point of the second video image to the end time of t2. If the video synchronization check within the time period of the video synchronization check is successful, the creative can be played to the end of t2.
  • the episode video has a total of 40 minutes, including 8 advertising spots, and the first advertising spot is the 5th minute and the 0th.
  • the second advertising point is the 0th second of the 10th minute to the 5th second of the 10th minute, and so on.
  • the terminal device After displaying the creative at the first advertising point, the terminal device will The video synchronization check of the second advertising spot starts at the 57th second of the 9th minute.
  • the frequency of comparing the video image with the reference image can be twice per second, which is consistent with the sampling frequency of the reference image.
  • FIG. 10-12 show schematic diagrams of examples of multiple user interfaces of the information processing method according to an embodiment of the present disclosure, where, as shown in FIG. 10, in the process of playing the video stream information on the terminal device, the playback of the video stream information is obtained Click "00:16" to request the server to query video resources, for example, which cloud episode is currently playing and which episode is currently playing, and you can also request the server to query the advertising point so that the corresponding advertising material can be triggered at the advertising point. display.
  • the server video search is completed and the terminal device receives the advertising material sent by the server
  • the video name and advertising point "00:30-00:35" are displayed on the terminal device, and the playback point is " 00:16".
  • the creative will be displayed. As shown in Figure 12, during the process of displaying the advertising material, that is, within the interval of the advertising point "00:30-00:35", the current playback point is displayed.
  • the writing order of the steps does not mean a strict execution order but constitutes any limitation on the implementation process.
  • the specific execution order of each step should be based on its function and possibility.
  • the inner logic is determined.
  • the present disclosure also provides information processing devices, electronic equipment, computer-readable storage media, and programs, all of which can be used to implement any information processing method provided in the present disclosure.
  • information processing devices electronic equipment, computer-readable storage media, and programs, all of which can be used to implement any information processing method provided in the present disclosure.
  • FIG. 13 shows a block diagram of an information processing device according to an embodiment of the present disclosure.
  • the processing device includes: a first detection unit 51, configured to perform human operations on the first video image in the played video stream. Face detection to obtain the face information of the first video image; the first sending unit 52 is used to send the face information to the server; the first receiving unit 53 is used to receive the target material sent by the server, where The target material is obtained based on the face information; the first response unit 54 is configured to trigger the display of the target material in response to detecting that the video stream is played to the start time corresponding to the target material.
  • the terminal device performs face detection on the first video image in the played video stream, the face information of the first video image can be obtained.
  • the advertising material can be obtained according to the face information comparison and sent to the terminal, and the terminal device responds to detecting that the video stream is played to The start playing time corresponding to the creative will trigger the display of the creative and then display the creative.
  • the display of the creatives can be accurate after the terminal device receives the creatives It is matched with the video stream being played, so that the user can get more effective information related to the video stream in real time (such as accurate delivery such as advertising materials, etc.).
  • the first response unit is configured to: send the target material to a terminal device so that the terminal device displays the target material; or, in the triggering of the target material After the display, the target material is displayed.
  • the display manner of the target material includes at least one of the following:
  • the target material is displayed in the corner of the video stream playback interface in the form of a corner mark
  • the target material is displayed on the video stream playback interface in the form of band-aids in association with the playback content of the playback interface;
  • the target material is displayed on the video stream playback interface in the form of a banner and spans the playback interface.
  • the device further includes: a second receiving unit, configured to receive playback time information of the target material sent by the server, where the playback time information includes preset playback of the target material Reference image information corresponding to the time period; a second detection unit, configured to detect whether the video stream is played to the start playing time corresponding to the target material based on the play time information.
  • the reference image information includes: feature information of multiple frames of reference images corresponding to the preset playing time period of the target material.
  • the multi-frame reference image is obtained by sampling multiple times on at least one time period adjacent to the preset playback time period.
  • the second detection unit is configured to: compare the second video image to be played in the video stream with the reference image information included in the playback time information to obtain a comparison result ; Based on the comparison result, it is determined whether the video stream is played to the start playing time corresponding to the target material.
  • the second detection unit is configured to: perform feature extraction processing on the second video image to obtain feature information of the second video image; and based on the feature of the second video image The similarity between the information and the feature information of at least one frame of reference image included in the reference image information, to obtain a comparison result.
  • the second detection unit is configured to determine the video stream when the comparison result is a successful comparison with the first reference image in the multi-frame reference image Play to the start time corresponding to the target material.
  • the comparison result is a successful comparison with the second reference image in the multi-frame reference image, it is determined to end the playback of the target material.
  • the first M reference images in the multi-frame reference image include the first reference image
  • the last N reference images in the multi-frame reference image include the second reference image
  • the device further includes a material display unit configured to display the target material in response to detecting the advertising material that meets the preset blocking condition.
  • the preset interception condition includes at least one of the following:
  • a creative that has a different source or a different business from the target material has a different source or a different business from the target material.
  • the present disclosure provides an information processing device.
  • the device includes: a third receiving unit for receiving face information of at least one first video image sent by a terminal device; an information determining unit for receiving face information based on the face information Determine the video stream information to which the at least one first video image belongs with the face template library, where the face template library includes a plurality of preset face templates; the material acquisition unit is configured to obtain information based on the video stream information , Obtain the target material; the second sending unit is configured to send the target material to the terminal device.
  • the target material may include an advertising material to determine the video stream information to which at least one first video image belongs based on the comparison of the face information on the server side, for example, based on the face information and the face template library, so that Obtain the creative according to the video stream information.
  • the server sends the advertisement material obtained by comparing the face information to the terminal device.
  • the display of the advertisement material is triggered to display the advertisement material.
  • the display of the creatives can be accurate after the terminal device receives the creatives It is matched with the video stream being played, so that the user can get more effective information related to the video stream in real time (such as accurate delivery such as advertising materials, etc.).
  • the information determining unit is configured to: obtain the person identity information corresponding to the face information based on a plurality of preset face templates included in the face template library;
  • the identity information searches the video library to obtain the video stream information to which the at least one first video image belongs.
  • the information determining unit is configured to: determine a first comparison result of the face information and the plurality of preset face templates; and obtain based on the first comparison result Personal identity information corresponding to the face information.
  • the apparatus further includes a third sending unit, configured to: send the play time information corresponding to the target material to the terminal device, where the play time information includes the time information of the target material The reference image information corresponding to the preset playing time period.
  • the reference image information includes: feature information of multiple frames of reference images corresponding to the preset playing time period of the target material.
  • the device further includes: a sampling unit, configured to sample the video stream information in at least one adjacent time period of the preset playback time period to obtain a multi-frame reference image;
  • the extraction unit is configured to perform feature extraction processing on the multi-frame reference image to obtain feature information of the multi-frame reference image.
  • the apparatus further includes a fourth sending unit, configured to send an advertisement blocking instruction to the terminal device, where the advertisement blocking instruction is used to indicate when a video stream played by the terminal device is detected When a specific advertising material appears in the display, the terminal device displays the target material.
  • the functions or modules contained in the device provided in the embodiments of the present disclosure can be used to execute the methods described in the above method embodiments.
  • the functions or modules contained in the device provided in the embodiments of the present disclosure can be used to execute the methods described in the above method embodiments.
  • the embodiments of the present disclosure also provide a computer-readable storage medium on which computer program instructions are stored, and the computer program instructions implement the above-mentioned method when executed by a processor.
  • the computer-readable storage medium may be a volatile computer-readable storage medium or a non-volatile computer-readable storage medium.
  • the embodiments of the present disclosure also provide a computer program product, which includes computer-readable code.
  • the processor in the device executes the information processing method for implementing the information processing method provided by any of the above embodiments. instruction.
  • the embodiments of the present disclosure also provide another computer program product for storing computer-readable instructions, which when executed, cause the computer to perform the operations of the information processing method provided by any of the foregoing embodiments.
  • the computer program product can be specifically implemented by hardware, software, or a combination thereof.
  • the computer program product is specifically embodied as a computer storage medium.
  • the computer program product is specifically embodied as a software product, such as a software development kit (SDK), etc. Wait.
  • SDK software development kit
  • An embodiment of the present disclosure also provides an electronic device, including: a processor; a memory for storing executable instructions of the processor; wherein the processor is configured as the above-mentioned method.
  • the electronic device can be provided as a terminal device, a server, or other types of devices.
  • Fig. 14 is a block diagram showing an electronic device 800 according to an exemplary embodiment.
  • the electronic device 800 may be a mobile phone, a computer, a digital broadcasting terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, and other terminal devices.
  • the electronic device 800 may include one or more of the following components: a processing component 802, a memory 804, a power supply component 806, a multimedia component 808, an audio component 810, an input/output (I/O) interface 812, and a sensor component 814 , And communication component 816.
  • the processing component 802 generally controls the overall operations of the electronic device 800, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations.
  • the processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the foregoing method.
  • the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and other components.
  • the processing component 802 may include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.
  • the memory 804 is configured to store various types of data to support operations in the electronic device 800. Examples of these data include instructions for any application or method to operate on the electronic device 800, contact data, phone book data, messages, pictures, videos, etc.
  • the memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable and Programmable Read Only Memory (EPROM), Programmable Read Only Memory (PROM), Read Only Memory (ROM), Magnetic Memory, Flash Memory, Magnetic Disk or Optical Disk.
  • SRAM static random access memory
  • EEPROM electrically erasable programmable read-only memory
  • EPROM erasable and Programmable Read Only Memory
  • PROM Programmable Read Only Memory
  • ROM Read Only Memory
  • Magnetic Memory Flash Memory
  • Magnetic Disk Magnetic Disk or Optical Disk.
  • the power supply component 806 provides power for various components of the electronic device 800.
  • the power supply component 806 may include a power management system, one or more power supplies, and other components associated with the generation, management, and distribution of power for the electronic device 800.
  • the multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and the user.
  • the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user.
  • the touch panel includes one or more touch sensors to sense touch, sliding, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure related to the touch or slide operation.
  • the multimedia component 808 includes a front camera and/or a rear camera. When the electronic device 800 is in an operation mode, such as a shooting mode or a video mode, the front camera and/or the rear camera can receive external multimedia data. Each front camera and rear camera can be a fixed optical lens system or have focal length and optical zoom capabilities.
  • the audio component 810 is configured to output and/or input audio signals.
  • the audio component 810 includes a microphone (MIC), and when the electronic device 800 is in an operation mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal.
  • the received audio signal may be further stored in the memory 804 or transmitted via the communication component 816.
  • the audio component 810 further includes a speaker for outputting audio signals.
  • the I/O interface 812 provides an interface between the processing component 802 and a peripheral interface module.
  • the above-mentioned peripheral interface module may be a keyboard, a click wheel, a button, and the like. These buttons may include, but are not limited to: home button, volume button, start button, and lock button.
  • the sensor component 814 includes one or more sensors for providing the electronic device 800 with various aspects of state evaluation.
  • the sensor component 814 can detect the on/off status of the electronic device 800 and the relative positioning of the components.
  • the component is the display and the keypad of the electronic device 800.
  • the sensor component 814 can also detect the electronic device 800 or the electronic device 800.
  • the position of the component changes, the presence or absence of contact between the user and the electronic device 800, the orientation or acceleration/deceleration of the electronic device 800, and the temperature change of the electronic device 800.
  • the sensor component 814 may include a proximity sensor configured to detect the presence of nearby objects when there is no physical contact.
  • the sensor component 814 may also include a light sensor, such as a CMOS (Complementary Metal Oxide Semiconductor) or a CCD (Charge Coupled Device) image sensor for use in imaging applications.
  • CMOS Complementary Metal Oxide Semiconductor
  • CCD Charge Coupled Device
  • the sensor component 814 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.
  • the communication component 816 is configured to facilitate wired or wireless communication between the electronic device 800 and other devices.
  • the electronic device 800 can access a wireless network based on a communication standard, such as WiFi, 2G, 3G, 4G, or 5G, or a combination thereof.
  • the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel.
  • the communication component 816 further includes a near field communication (NFC) module to facilitate short-range communication.
  • the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.
  • RFID radio frequency identification
  • IrDA infrared data association
  • UWB ultra-wideband
  • Bluetooth Bluetooth
  • the electronic device 800 may be implemented by one or more application-specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPD), programmable logic devices (PLD), field-available A programmable gate array (FPGA), controller, microcontroller, microprocessor, or other electronic components are implemented to implement the above methods.
  • ASIC application-specific integrated circuits
  • DSP digital signal processors
  • DSPD digital signal processing devices
  • PLD programmable logic devices
  • FPGA field-available A programmable gate array
  • controller microcontroller, microprocessor, or other electronic components are implemented to implement the above methods.
  • a non-volatile computer-readable storage medium such as the memory 804 including computer program instructions, which can be executed by the processor 820 of the electronic device 800 to complete the foregoing method.
  • Fig. 15 is a block diagram showing an electronic device 900 according to an exemplary embodiment.
  • the electronic device 900 may be provided as a server.
  • the electronic device 900 includes a processing component 922, which further includes one or more processors, and a memory resource represented by a memory 932, for storing instructions that can be executed by the processing component 922, such as an application program.
  • the application program stored in the memory 932 may include one or more modules each corresponding to a set of instructions.
  • the processing component 922 is configured to execute instructions to perform the above-described methods.
  • the electronic device 900 may also include a power supply component 926 configured to perform power management of the electronic device 900, a wired or wireless network interface 950 configured to connect the electronic device 900 to a network, and an input output (I/O) interface 958 .
  • the electronic device 900 can operate based on an operating system stored in the memory 932, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM or the like.
  • a non-volatile computer-readable storage medium is also provided, such as a memory 932 including computer program instructions, which can be executed by the processing component 922 of the electronic device 900 to complete the foregoing method.
  • the present disclosure may be a system, method and/or computer program product.
  • the computer program product may include a computer-readable storage medium loaded with computer-readable program instructions for enabling a processor to implement various aspects of the present disclosure.
  • the computer-readable storage medium may be a tangible device that can hold and store instructions used by the instruction execution device.
  • the computer-readable storage medium may include, but is not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing, for example.
  • Non-exhaustive list of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM) Or flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, mechanical encoding device, such as a printer with instructions stored thereon
  • RAM random access memory
  • ROM read-only memory
  • EPROM erasable programmable read-only memory
  • flash memory flash memory
  • SRAM static random access memory
  • CD-ROM compact disk read-only memory
  • DVD digital versatile disk
  • memory stick floppy disk
  • mechanical encoding device such as a printer with instructions stored thereon
  • the computer-readable storage medium used here is not interpreted as the instantaneous signal itself, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (for example, light pulses through fiber optic cables), or through wires Transmission of electrical signals.
  • the computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to various computing/processing devices, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and/or a wireless network.
  • the network may include copper transmission cables, optical fiber transmission, wireless transmission, routers, firewalls, switches, gateway computers, and/or edge servers.
  • the network adapter card or network interface in each computing/processing device receives computer-readable program instructions from the network, and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing/processing device .
  • the computer program instructions used to perform the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or in one or more programming languages.
  • Source code or object code written in any combination, the programming language includes object-oriented programming languages such as Smalltalk, C++, etc., and conventional procedural programming languages such as "C" language or similar programming languages.
  • Computer-readable program instructions can be executed entirely on the user's computer, partly on the user's computer, executed as a stand-alone software package, partly on the user's computer and partly executed on a remote computer, or entirely on the remote computer or server carried out.
  • the remote computer can be connected to the user's computer through any kind of network-including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (for example, using an Internet service provider to connect to the user's computer) connection).
  • LAN local area network
  • WAN wide area network
  • an electronic circuit such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), can be customized by using the status information of the computer-readable program instructions.
  • FPGA field programmable gate array
  • PDA programmable logic array
  • the computer-readable program instructions are executed to realize various aspects of the present disclosure.
  • These computer-readable program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine that makes these instructions when executed by the processor of the computer or other programmable data processing device , A device that implements the functions/actions specified in one or more blocks in the flowcharts and/or block diagrams is produced. It is also possible to store these computer-readable program instructions in a computer-readable storage medium. These instructions make computers, programmable data processing apparatuses, and/or other devices work in a specific manner. Thus, the computer-readable medium storing the instructions includes An article of manufacture, which includes instructions for implementing various aspects of the functions/actions specified in one or more blocks in the flowcharts and/or block diagrams.
  • each block in the flowchart or block diagram may represent a module, program segment, or part of an instruction, and the module, program segment, or part of an instruction contains one or more components for realizing the specified logical function.
  • Executable instructions may also occur in a different order from the order marked in the drawings. For example, two consecutive blocks can actually be executed substantially in parallel, or they can sometimes be executed in the reverse order, depending on the functions involved.
  • each block in the block diagram and/or flowchart, and the combination of the blocks in the block diagram and/or flowchart can be implemented by a dedicated hardware-based system that performs the specified functions or actions Or it can be realized by a combination of dedicated hardware and computer instructions.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Marketing (AREA)
  • Business, Economics & Management (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Human Computer Interaction (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • General Health & Medical Sciences (AREA)
  • Databases & Information Systems (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
  • Information Transfer Between Computers (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

本公开涉及一种信息处理方法及装置、电子设备和存储介质，其中，该方法包括：对所播放的视频流中的第一视频图像进行人脸检测，得到所述第一视频图像的人脸信息；向服务器发送所述人脸信息；接收所述服务器发送的目标素材，其中，所述目标素材基于所述人脸信息得到；响应于检测到所述视频流播放至所述目标素材对应的开始播放时间，触发所述目标素材的显示。采用本公开，有利于实现第三方广告素材的自动插入。

Description

信息处理方法及装置、电子设备和存储介质 技术领域
本公开涉及信息处理技术领域，尤其涉及一种信息处理方法及装置、电子设备和存储介质。
背景技术
在信息处理的一些应用场景中，例如在播放流媒体信息(如视频流)场景中，还可以显示视频流之外的其他流媒体信息，比如，广告素材等。然而，对于第三方服务商提供的(over-the-top，OTT)广告素材来说，由于电视机和机顶盒无法获取第三方广告素材相关内容，无法实现广告素材的自动插入。
发明内容
本公开提出了一种信息处理的技术方案。
根据本公开的一方面，提供了一种信息处理方法，所述方法包括：对所播放的视频流中的第一视频图像进行人脸检测，得到所述第一视频图像的人脸信息；向服务器发送所述人脸信息；接收所述服务器发送的目标素材，其中，所述目标素材基于所述人脸信息得到；响应于检测到所述视频流播放至所述目标素材对应的开始播放时间，触发所述目标素材的显示。
可能的实现方式中，所述目标素材包括广告素材。
可能的实现方式中，触发所述目标素材的显示，包括：向终端设备发送所述目标素材，以使得所述终端设备显示所述目标素材；或者，在所述触发所述目标素材的显示之后，所述方法还包括：显示所述目标素材。
可能的实现方式中，所述目标素材的显示方式，包括：所述目标素材以角标形式显示于所述视频流播放界面的角落；或者，所述目标素材与所述播放界面的播放内容关联地以创可贴形式显示于所述视频流播放界面；或者，所述目标素材以横幅形式显示于所述视频流播放界面且横跨所述播放界面。
可能的实现方式中，所述方法还包括：接收所述服务器发送的所述目标素材的播放时间信息，所述播放时间信息包括所述目标素材的预设播放时间段对应的参考图像信息；基于所述播放时间信息，检测所述视频流是否播放到所述目标素材对应的开始播放时间。
可能的实现方式中，所述参考图像信息包括：所述目标素材的预设播放时间段对应的多帧参考图像的特征信息。
可能的实现方式中，所述多帧参考图像是通过对与所述预设播放时间段相邻的至少一个时间段进行多次采样得到的。
可能的实现方式中，基于所述播放时间信息，检测所述视频流是否播放到所述目标素材对应的开始播放时间，包括：将所述视频流待播放的第二视频图像与所述播放时间信息中包括的参考图像信息进行比对，得到比对结果；基于所述比对结果，确定所述视频流是否播放到所述目标素材对应的开始播放时间。在所述第一视频图像播放之后，播放所述第二视频图像。
可能的实现方式中，基于所述比对结果，确定所述视频流是否播放到所述目标素材对应的开始播放时间，包括：在所述比对结果为与所述多帧参考图像中的第一参考图像比对成功的情况下，确定所述视频流播放到所述目标素材对应的开始播放时间，其中，所述多帧参考图像中的前M个参考图像包括所述第一参考图像。
在一些实施例中，响应于检测到所述视频流播放至所述目标素材对应的开始播放时间，触发所述目标素材的显 示：响应于所述视频流的第二视频图像与所述多帧参考图像中的第一目标参考图像比对成功，触发所述目标素材的显示。
在一些实施例中，响应于所述视频流的第二视频图像与所述多帧参考图像中的第一目标参考图像比对成功，触发所述目标素材的显示，包括：响应于检测到所述第二视频图像与所述第一目标参考图像比对成功且所述视频流的第三视频图像与所述多帧参考图像中的第二目标参考图像比对成功，触发所述目标素材的显示，其中在所述第二视频图像播放之后，播放所述第三视频图像。
可能的实现方式中，所述方法还包括：在所述比对结果为与所述多帧参考图像中的第二参考图像比对成功的情况下，确定结束所述目标素材的播放，其中，所述多帧参考图像中的最后N个参考图像包括所述第二参考图像。在一些实施例中，所述多帧参考图像中目标参考图像之后的N个参考图像包括所述第二参考图像。
在一些实施例中，在视频流的第四视频图像与第二参考图像比对成功的情况下，触发所述目标素材的结束播放。
可能的实现方式中，所述方法还包括：响应于检测到所述符合预设拦截条件的广告素材，显示所述目标素材。
可能的实现方式中，所述预设拦截条件包括下列中的至少一项：
黑名单中列出的预设商家；
黑名单中列出的预设广告类型；
与所述目标素材具有不同来源或不同商家的广告素材。
根据本公开的一方面，提供了一种信息处理方法，所述方法包括：接收终端设备发送的至少一个第一视频图像的人脸信息；基于所述人脸信息与人脸模板库，确定所述至少一个第一视频图像所属的视频流信息，其中，所述人脸模板库包括多个预设人脸模板；根据所述视频流信息，获取目标素材；向所述终端设备发送所述目标素材。
可能的实现方式中，基于所述人脸信息与人脸模板库，确定所述至少一个第一视频图像所属的视频流信息，包括：基于所述人脸模板库包括的多个预设人脸模板，得到所述人脸信息对应的人物身份信息；基于所述人物身份信息查找视频库，得到所述至少一个第一视频图像所属的视频流信息。
可能的实现方式中，基于所述人脸模板库包括的多个预设人脸模板，得到所述人脸信息对应的人物身份信息，包括：确定所述人脸信息与所述多个预设人脸模板的第一比对结果；基于所述第一比对结果，得到所述人脸信息对应的人物身份信息。
可能的实现方式中，所述目标素材包括：广告素材。
可能的实现方式中，所述方法还包括：向所述终端设备发送所述目标素材对应的播放时间信息，其中，所述播放时间信息包括所述目标素材的预设播放时间段对应的参考图像信息。
可能的实现方式中，所述参考图像信息包括：所述目标素材的预设播放时间段对应的多帧参考图像的特征信息。
可能的实现方式中，所述方法还包括：在所述预设播放时间段的至少一个相邻时间段对所述视频流信息进行采样，得到多帧参考图像；对所述多帧参考图像进行特征提取处理，得到所述多帧参考图像的特征信息。
可能的实现方式中，所述方法还包括：向所述终端设备发送广告拦截指示，所述广告拦截指示用于指示当检测到所述终端设备所播放视频流中出现特定广告素材时，所述终端设备显示所述目标素材。
根据本公开的一方面，提供了一种信息处理装置，所述装置包括：第一检测单元，用于对所播放的视频流中的第一视频图像进行人脸检测，得到所述第一视频图像的人脸信息；第一发送单元，用于向服务器发送所述人脸信息；第一接收单元，用于接收所述服务器发送的目标素材，其中，所述目标素材基于所述人脸信息得到；第一响应单元，用于响应于检测到所述视频流播放至所述目标素材对应的开始播放时间，触发所述目标素材的显示。
根据本公开的另一方面，提供了一种信息处理装置，所述装置包括：第三接收单元，用于接收终端设备发送的至少一个第一视频图像的人脸信息；信息确定单元，用于基于所述人脸信息与人脸模板库，确定所述至少一个第一视频图像所属的视频流信息，其中，所述人脸模板库包括多个预设人脸模板；素材获取单元，用于根据所述视频流信息，获取目标素材；第二发送单元，用于向所述终端设备发送所述目标素材。
根据本公开的一方面，提供了一种电子设备，包括：处理器；用于存储处理器可执行指令的存储器；其中，所述处理器被配置为：执行上述信息处理方法。
根据本公开的一方面，提供了一种计算机可读存储介质，其上存储有计算机程序指令，所述计算机程序指令被处理器执行时实现上述信息处理方法。
在本公开实施例中，对所播放的视频流中的第一视频图像进行人脸检测，得到所述第一视频图像的人脸信息；向服务器发送所述人脸信息；接收所述服务器发送的目标素材，其中，所述目标素材基于所述人脸信息得到；响应于检测到所述视频流播放至所述目标素材对应的开始播放时间，触发所述目标素材的显示，从而有利于实现OTT广告素材的自动插入。
应当理解的是，以上的一般描述和后文的细节描述仅是示例性和解释性的，而非限制本公开。
根据下面参考附图对示例性实施例的详细说明，本公开的其它特征及方面将变得清楚。
附图说明
此处的附图被并入说明书中并构成本说明书的一部分，这些附图示出了符合本公开的实施例，并与说明书一起用于说明本公开的技术方案。
图1示出根据本公开实施例的信息处理方法的流程图。
图2a是根据本公开实施例示出的视频流中的一帧视频图像的示意图。
图2b示出根据本公开实施例中显示有目标素材的一帧视频图像的示意图。
图3-图5示出根据本公开实施例的素材显示方式的示意图。
图6示出根据本公开实施例的信息处理方法的流程图。
图7示出根据本公开实施例的视频搜索中的采样方式的示意图。
图8示出根据本公开实施例的视频流信息同步中的采样方式的示意图。
图9示出根据本公开实施例的信息处理方法中触发广告素材显示的时序关系示意图。
图10-图12示出根据本公开实施例的信息处理方法的多个用户界面示意图。
图13示出根据本公开实施例的信息处理装置的框图。
图14示出根据本公开实施例的电子设备的框图。
图15示出根据本公开实施例的电子设备的框图。
具体实施方式
以下将参考附图详细说明本公开的各种示例性实施例、特征和方面。附图中相同的附图标记表示功能相同或相似的元件。尽管在附图中示出了实施例的各种方面，但是除非特别指出，不必按比例绘制附图。
在这里专用的词“示例性”意为“用作例子、实施例或说明性”。这里作为“示例性”所说明的任何实施例不必解释为优于或好于其它实施例。
本文中术语“和/或”，仅仅是一种描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。另外，本文中术语“至少一种”表示多种中的任意一种或多种中的至少两种的任意组合，例如，包括A、B、C中的至少一种，可以表示包括从A、B和C构成的集合中选择的任意一个或多个元素。
另外，为了更好的说明本公开，在下文的具体实施方式中给出了众多的具体细节。本领域技术人员应当理解，没有某些具体细节，本公开同样可以实施。在一些实例中，对于本领域技术人员熟知的方法、手段、元件和电路未作详细描述，以便于凸显本公开的主旨。
图1示出根据本公开实施例的信息处理方法的流程图，该方法应用于信息处理装置，例如，该信息处理装置部署于终端设备，并可以播放视频流、进行视频同步检查及显示素材等。其中，终端设备可以包括用户设备(UE，User Equipment)、移动设备、蜂窝电话、无绳电话、个人数字处理(PDA，Personal Digital Assistant)、手持设备、计算设备、车载设备、可穿戴设备、电视等。在一些可能的实现方式中，该信息处理方法可以通过处理器调用存储器中存储的计算机可读指令的方式来实现。图1所示的信息处理方法的流程包括步骤S101-S104。
步骤S101、终端设备对所播放的视频流中的第一视频图像进行人脸检测，得到所述第一视频图像的人脸信息。
一示例中，该视频流可以包括电视剧或电影等的视频正片、短视频、综艺节目等，所述视频流可以由多帧视频图像组成。视频流的一帧或多帧视频图像中可以包含一个或多个人脸信息，比如，可以通过特征提取网络对第一视频图像进行特征提取，以得到多个特征信息，随后对该多个特征信息进行聚类，并根据聚类结果得到该人脸信息。
步骤S102、终端设备向服务器发送所述人脸信息。
一示例中，以视频流中包含的该第一视频图像为例，若该第一视频图像中包含多个人脸信息，可以将每个独立的人脸信息分别提取出来后发送给服务器，服务器可以将人脸信息与服务器的人脸模板库中的预设人脸模板进行比对，并根据比对结果得到与人脸信息所匹配的视频流信息，如，可以根据该比对结果搜索云端的视频流信息，以得到与终端设备所播放视频流匹配的云端剧集，并据此获取目标素材(如广告素材等)。
步骤S103、终端设备接收所述服务器发送的目标素材，其中，所述目标素材基于所述人脸信息得到。
一示例中，所述目标素材包括广告素材，可以通过服务器进行的人脸信息比对，得到该广告素材。
步骤S104、终端设备响应于检测到所述视频流播放至所述目标素材对应的开始播放时间，触发所述目标素材的显示。
一示例中，该开始播放时间可以为显示目标素材(如广告素材等)的广告点，可以根据该广告点来标识广告素材播放的起始时间。在该广告点触发广告素材的显示。图2a是根据本公开实施例示出的视频流中的一帧视频图像的示意图。图2b是根据本公开实施例示出的显示有目标素材的一帧视频图像的示意图，其中，所播放的视频流如视频正片11所示，触发显示的目标素材如广告素材12所示。
一示例中，服务器根据人脸信息比对得到目标素材后，可以向播放视频流的终端设备发送目标素材，以使得该终端设备显示所述目标素材。触发目标素材的显示之后，所述信息处理方法还包括：显示目标素材。
在播放视频流场景中，上述目标素材可以是OTT广告素材。OTT广告素材是第三方服务商通过互联网(如运营商)提供的。采用本申请公开的上述技术方案，实现了“终端设备从视频流的第一视频图像中提取人脸信息，并发送给服务器进行分析；服务器在云端视频流信息中进行搜索后得到对应的目标素材(如广告素材等)并将目标素材发送给终端设备；并在终端设备上完成目标素材的展示”的处理逻辑。通过该处理逻辑，可以将普通的非智能的 电视机变成高清的智能电视，即：基于该处理逻辑，普通的非智能的电视机，不仅可以通过互联网(如运营商提供的网络)得到各大视频影视内容的资源，还可以从服务器的大量第三方素材内容供应商得到更多样化的有效信息(如目标素材)。通过该处理逻辑，对所播放的视频流中的第一视频图像进行人脸检测，得到人脸信息后，将该人脸信息发送给服务器比对，以得到基于该人脸信息的对应目标素材，从而实现OTT广告的自动精准插入。
在一些可能的实现方式中，以目标素材为广告素材为例，该广告素材的显示方式可以包括角标、创可贴、横幅等形式，图3-图5示出根据本公开实施例的目标素材显示方式的示意图，阐述如下：
1、显示方式可以为：所述广告素材13以角标形式显示于所述视频流播放界面的角落，如图3所示。
2、显示方式可以为：所述广告素材14与播放界面的播放内容关联地以创可贴形式显示于所述视频流播放界面，如图4所示。比如，如果检测出播放内容中包括的关键信息“民族解放”等，则根据该关键信息查询对应该关键信息、或与该关键信息相关联的广告素材，并将查询到的广告素材显示到播放界面中，以与该播放内容互相呼应。比如，广告素材中包含的广告词“一生忠诚”，这与关键信息“民族解放”是相关联的，可以将该广告素材显示到播放界面中，以与该播放内容互相呼应。
3、显示方式可以为：所述广告素材15以横幅形式显示于所述视频流播放界面且横跨所述播放界面，如图5所示。
在一些可能的实现方式中，所述信息处理方法还包括：接收所述服务器发送的所述目标素材的播放时间信息，所述播放时间信息包括所述目标素材的预设播放时间段对应的参考图像信息；基于所述播放时间信息，检测所述视频流是否播放到所述目标素材对应的开始播放时间。其中，所述参考图像信息包括：所述目标素材的预设播放时间段对应的多帧参考图像的特征信息。该多帧参考图像是通过对与所述预设播放时间段相邻的至少一个时间段进行多次采样得到的。
在一些可能的实现方式中，基于所述播放时间信息，检测所述视频流是否播放到所述目标素材对应的开始播放时间，包括：将所述视频流待播放的第二视频图像与所述播放时间信息中包括的参考图像信息进行比对，得到比对结果；基于所述比对结果，确定所述视频流是否播放到所述目标素材对应的开始播放时间。第二视频图像的播放是在第一视频图像之后。
一示例中，将所述视频流待播放的第二视频图像与所述参考图像信息进行比对，得到比对结果，包括：对所述第二视频图像进行特征提取处理，得到所述第二视频图像的特征信息；基于所述第二视频图像的特征信息与所述参考图像信息中至少一帧参考图像的特征信息之间的相似度，得到比对结果。通过比对第二视频图像的特征信息与参考图像的特征信息，可以得到特征信息之间的相似度，从而得到比对结果。终端设备可以根据该比对结果，确定视频流是否播放到目标素材对应的开始播放时间，以便在该开始播放时间触发目标素材的显示，在触发目标素材的显示后，显示该目标素材。
在一些实施例中，当视频图像的特征信息与参考图像的特征信息之间的相似度大于或等于预设阈值时，可以确定该视频图像与该参考图像比对成功。预设阈值可以根据实际使用的确定相似度的方法而定。
在一些可能的实现方式中，在第二视频图像与所述多帧参考图像中的第一参考图像比对成功的情况下，确定所述视频流播放到所述目标素材对应的开始播放时间，其中，所述多帧参考图像中的前M个参考图像包括所述第一参考图像。
在一些实施例中，可以将视频流的视频图像与多帧参考图像进行比对，以确定是否触发目标素材的显示。例如，可以在第二视频图像与第一参考图像比对成功的情况下，即触发目标素材的显示，或者，在第二视频图像与第一参考图像比对成功且第三视频图像与第三参考图像比对成功的情况下，触发目标素材的显示，其中，第三视频图像是在第二视频图像之后播放的，该第一参考图像和第三参考图像可以是包括于多帧参考图像中的前M个参考图像之内 的任意参考图像，或者是预先设定的参考图像，具体的触发机制可以根据实际需求确定，本公开实施例对此不做限定。
在一些实施例中，在视频流的第二视频图像与所述多帧参考图像中的第二参考图像比对成功的情况下，确定结束所述目标素材的播放，和/或触发所述目标素材的结束播放，其中，所述多帧参考图像中的最后N个参考图像包括所述第二参考图像。
在一些实施例中，参考图像信息包括：与目标素材的开始播放时间关联的多帧参考图像的特征信息。在一个可选例子中，多帧参考图像是通过对与预设播放时间段相邻的至少一个时间段进行多次采样得到的。该预设播放时间段可以包含一个或多个预设时间点，例如，包含一个时间点(即：开始播放时间＝结束播放时间)、或者包含开始播放时间和结束播放时间、或者包含开始播放时间、结束播放时间和位于两者之间的至少一个时间点。在一个可选例子中，多帧参考图像是通过对与开始播放时间相邻的第一时间段和第二时间段进行多次采样得到的，其中，所述第一时间段的终点是所述开始播放时间，所述第二时间段的起点是所述开始播放时间。第二时间段的时长可以是该广告点位的广告时长或稍大于所述广告点位的广告时长，或者，第二时间段的起点为与所述开始播放时间相邻的下一时间点，此时，可选地，在以上例子中，第一参考图像是在第一时间段内被采样的。第二参考图像是在第二时间段内被采样的，但本公开实施例对此不做限定。
在一些实施例中，响应于检测到第二视频图像与第一参考图像比对成功，开始进行视频同步检查。当检测到视频流中的第三视频图像与所述多帧参考图像中预先设定的目标参考图像比对成功，触发目标素材的显示。
在一些可能的实现方式中，所述信息处理方法还包括：响应于检测到所述符合预设拦截条件的广告素材，显示所述目标素材。其中，所述预设拦截条件包括下列中的至少一项：黑名单中列出的预设商家；黑名单中列出的预设广告类型；或与所述目标素材具有不同来源或不同商家的广告素材。
一示例中，服务器向终端设备发送广告拦截指示，该广告拦截指示用于指示当检测到终端设备所播放视频流中出现符合预设拦截条件的广告时，终端设备显示目标素材。比如，所述视频流中播放的广告素材为耐克球鞋，终端设备根据广告拦截指示确定该耐克球鞋是黑名单中列出的商家，则将以角标、创可贴或横幅等形式显示目标素材，例如阿迪达斯球鞋的广告。另一示例中，所述视频流中播放的广告素材为奔驰广告，与目标素材“宝马广告”为竞品，则一旦发现，终端设备立马以角标、创可贴或横幅等形式显示宝马广告，以提高自身广告素材播放的精确率。采用本公开，出现预设拦截条件的广告素材的情况下，可以弹出自身匹配的广告素材，以对符合预设拦截条件的广告素材进行精准的拦截处理。
图6示出根据本公开实施例的信息处理方法的流程图，该方法应用于信息处理装置，该信息处理装置部署于服务器并可以执行视频流信息搜索及播放视频流与所显示的素材间的同步。如图6所示，该流程包括步骤S201-S204。
步骤S201、服务器接收终端设备发送的至少一个第一视频图像的人脸信息。
一示例中，终端设备播放的视频流中包含至少一个第一视频图像，对其中任一个视频图像而言，可以通过特征提取网络对该第一视频图像进行特征提取，以得到对应该视频图像的多个特征信息，对该多个特征信息进行聚类，并根据聚类结果得到人脸信息。对至少一个第一视频图像的所有第一视频图像进行人脸检测处理后，将至少一个第一视频图像的人脸信息发送给服务器，服务器接收该至少一个第一视频图像的人脸信息，进行人脸信息的比对，并根据比对结果得到对应视频流信息(如云端剧集)的目标素材。一示例中，云端剧集可以是电视剧或电影等的视频正片、短视频、综艺节目等。
步骤S202、服务器基于所述人脸信息与人脸模板库，确定所述至少一个第一视频图像所属的视频流信息，其中，所述人脸模板库包括多个预设人脸模板。
一示例中，基于该人脸模板库包括的多个预设人脸模板，可以得到人脸信息对应的人物身份信息。基于该人物 身份信息查找视频库，可以得到至少一个第一视频图像所属的视频流信息。以终端设备播放的视频流为电视剧为例，根据该人物身份信息(如霍建华)，可以从云端剧集搜索到与所播放电视剧匹配的可能剧集，比如，“甄嬛传”等。
其中，基于该人脸模板库包括的多个预设人脸模板，得到人脸信息对应的人物身份信息，可以包括：确定该人脸信息与多个预设人脸模板的第一比对结果；基于该第一比对结果，得到该人脸信息对应的人物身份信息(如霍建华)。
步骤S203、服务器根据所述视频流信息，获取目标素材。
一示例中，根据该人物身份信息查找视频，并得到至少一个第一视频图像所属的视频流信息后，还可以根据该视频流信息获取目标素材，其中，所述目标素材包括广告素材等。比如，可以根据该人物身份信息搜索云端(服务器)的视频流信息，得到与终端设备所播放视频流匹配的云端剧集“如甄嬛传第3集”，并据此获取对应的广告素材等，即获取对应“甄嬛传第3集”所植入的广告素材。假设待植入广告的云端剧集一共包括70集，每集植入8个广告点位，广告素材的植入可由运营人员完成。剧集的演员表、剧集名、剧集视频、剧集编号和演员表等也由运营人员提前上传到服务器。播放视频流与所显示的素材间的同步可提前完成。
步骤S204、服务器向所述终端设备发送所述目标素材。
一示例中，通过上述人脸信息比对，得到目标素材(如广告素材等)之后，服务器发送该目标素材给终端设备，则终端设备接收到该目标素材后，可以响应于检测到视频流播放至目标素材对应的开始播放时间，触发目标素材的显示。
采用本申请公开的技术方案，服务器接收终端设备发送的至少一个第一视频图像的人脸信息，将该人脸信息与人脸模板库中的多个预设人脸模板进行比对，以根据比对结果得到对应的视频流信息，及根据该视频流信息获取目标素材。其中，目标素材为广告素材的情况下，可以反馈广告素材给终端设备，以在终端设备上完成广告素材的显示。由于目标素材(如广告素材等)的加载和显示方式是通过所播放视频流中的人脸信息比对得到的，因此，该目标素材可以精准地匹配于该视频流，从而，用户可以实时得到与视频流相关的更多有效信息(如精准投放的诸如广告素材等)。
在一些可能的实现方式中，所述信息处理方法还包括：向所述终端设备发送所述目标素材对应的播放时间信息，其中，所述播放时间信息包括所述目标素材的预设播放时间段对应的参考图像信息。所述参考图像信息包括：所述目标素材的预设播放时间段对应的多帧参考图像的特征信息。
在一些可能的实现方式中，所述信息处理方法还包括：在所述预设播放时间段的至少一个相邻时间段对所述视频流信息进行采样，得到多帧参考图像，对所述多帧参考图像进行特征提取处理，以得到上述多帧参考图像的特征信息。
在一些可能的实现方式中，所述信息处理方法还包括：向所述终端设备发送广告拦截指示，所述广告拦截指示用于指示当检测到所述终端设备所播放视频流中出现特定广告素材时，所述终端设备显示所述目标素材。比如，视频流中播放的广告素材为耐克球鞋，终端设备根据该广告拦截指示，查询到该耐克球鞋是黑名单中列出的商家，则将以角标、创可贴或横幅等形式显示目标素材，例如阿迪达斯球鞋的广告。又如，视频流中播放的广告素材为奔驰广告，与目标素材“宝马广告”为竞品，则根据该广告拦截指示，一旦发现奔驰广告，终端设备立马以角标、创可贴或横幅等形式显示宝马广告，以提高自身广告素材播放的精确率。采用本公开，根据广告拦截指示，一旦查询到符合预设拦截条件的广告素材的情况下，可以弹出自身匹配的广告素材，以对符合预设拦截条件的广告素材进行精准的拦截处理。
本公开的一示例中，终端设备所播放的视频流中包含至少一个第一视频图像，在该第一视频图像中包含一个或 多个人脸信息，需要对人脸信息进行检测。目标素材可以为广告素材，服务器根据终端设备检测得到的人脸信息进行云端视频流信息(如云端剧集)的搜索。在确定当前播放的云端剧集以及当前播放时间点后，服务器可将广告素材以及参考图像信息发送给终端设备。终端设备进行参考图像特征信息比对以根据比对结果确定开始播放时间，以使得在该开始播放时间(或称广告点)触发该广告素材的显示，进而显示该广告素材。下面描述本申请实施例提供的视频流信息搜索、视频流信息同步以及广告素材触发显示的过程：
一：视频流信息搜索：包括剧集搜索和剧集中某一单集的搜索
就剧集搜索而言，需要搜索终端设备所播放的视频流是属于哪一个云端剧集，其中，云端剧集的演员表、剧集名、剧集视频、剧集编号、演员表等可以由运营人员提前上传到云端的服务器。终端设备开始播放视频流后，可以通过终端设备的SDK(Software Development Kit)或者APP(Application)检测播放3分钟内的视频，检测得到视频流一帧或多帧视频图像中独立的人脸信息作为待比对的人脸信息，并上传到服务端。服务端将该待比对的人脸信息与存储的云端剧集中演员比对，以根据比对结果识别出人物身份信息，如多个演员名或剧集的演员表，从而，可以根据多个演员名或剧集的演员表匹配出最可能的云端剧集(如甄嬛传)。
就单集搜索而言，需要在确认好终端设备所播放的视频流属于已知的云端剧集后，再搜索是云端剧集中的哪一集。检测到终端设备播放的视频流对应的云端剧集后，可以通过终端设备的SDK或者APP以某一频率，例如每分钟1次的频率上传视频流中的一帧视频图像到服务器，以进行单集搜索的处理。图7示出根据本公开实施例的视频搜索中的采样方式的示意图，如图7所示，服务器可以对云端剧集(如70集)的所有广告点进行采样。采样方式可以为每个广告点前1分钟，按照每秒采1帧，即：70集的剧集，每集8个广告位，一共采样70*8*60＝33600帧，得到33600帧的采样帧，这33600采样帧分别对应多个预设播放时间段。服务器将终端设备上传的一帧视频图像与云端剧集的所有采样帧(33600帧)进行比较，在采样帧中找出与这帧视频图像最相似的一帧采样视频图像，从而确定正在播放的该视频流为该帧采样视频图像对应的单集编号(如甄嬛传第3集)和播放时间(精确到秒)。
二：视频流信息同步
就视频流信息同步而言，可以通过服务器预先的视频流信息同步，以确保终端设备在同步的广告点位触发广告素材的显示。图8示出根据本公开实施例的视频流信息同步中的采样方式的示意图，如图8所示，在视频流信息同步过程中，服务器可以对每个广告点前后共8秒，按照每秒2帧进行采样得到16帧参考图像，然后对这16帧参考图像的每一帧分别提取特征信息，比如可以提取每一帧参考图像中的关键像素点的信息，从而压缩要处理的数据量，提高同步的处理效率。检测到终端设备正播放的视频流信息所对应的单集编号后(如甄嬛传第3集)和播放时间(精确到秒)后，服务器可以将该视频流信息播放点之后的所有后续广告点位的参考图像特征信息(比如，一集视频共16*8＝144条特征信息)、每帧参考图像时间点、广告素材下发到终端设备的SDK或APP。在终端设备的SDK或APP上继续播放视频流信息，可以将当前播放视频流信息与下载到本地的参考图像特征信息进行比对，可以得到特征信息之间的相似度，从而得到比对结果。终端设备根据该比对结果确认视频流是否播放到广告素材对应的开始播放时间，以便在该开始播放时间触发广告素材的显示，在触发广告素材的显示后，显示该广告素材。
需要指出的是，由于终端设备所播放视频流信息是目标剧集中的一集，且一集中不只一个广告点位，即有多个广告点位，需要多次的采样。视频流信息同步中的采样，是为了将该集中所有的广告点位都找到，将这些广告点位都同步到终端设备的SDK或APP，则在终端设备的SDK或APP上继续播放该视频流信息，将该视频流信息进行参考图像特征信息比对后可以得到与当前视频流信息的当前播放点最邻近的广告点位，以在该广告点位展现对应的广告素材。这与上述视频搜索中的采样是不同的，这里的采样，是为了将云端剧集中的某一集的参考图像特征信息与终端设备播放的视频流信息进行特征比较，以搜索得到当前播放视频流信息是对应该集中哪一个广告点位的广告素材的显示。
三：广告素材触发显示：
图9示出根据本公开实施例的信息处理方法中触发广告显示的时间节点间关系示意图，如图9所示，当终端设备的SDK或APP播放的视频流信息播放到t1结束时刻，且视频同步检查通过，则对于SDK而言，可以通知终端设备的APP显示广告内容；就APP而言，也可以直接通过终端设备的APP显示广告内容。广告播放过程中，SDK一直保持视频同步检查直到t2结束时刻。结合图8，最邻近的广告点位对应的参考图像信息包括16帧参考图像的特征信息，这16帧参考图像包括：第一帧参考图像、第二帧参考图像、…、第六帧参考图像、…、第十六帧参考图像。在视频流信息同步过程中，若当前播放点对应的第二视频图像与t1时间段(例如，广告点之前的3s)内的第一参考图像比对成功，假设与第三帧参考图像比对成功(也被称为第一次比对成功)，则终端设备可以确定在检测到当前视频流中的第三视频图像与第六帧参考图像比对成功时，触发广告素材的显示，并且在第一次比对成功后持续进行视频同步检查。在视频同步检查的过程中，检测到当前视频流中的第三视频图像与第六帧参考图像(上文中也被称为目标参考图像)比对成功时，触发广告素材的显示。在广告素材显示的过程中，终端设备继续进行视频同步检查，当检测到当前视频流中的第四视频图像与第十六帧参考图像比对成功时，该广告素材的显示结束。如果广告显示过程中，终端设备的SDK或APP播放的视频流信息与第七帧参考图像至第16帧参考图像中至少一帧无法比对成功，即：认为终端设备已经切换播放内容，停止广告素材的展现。同时，再次发起视频流搜索及视频流信息同步。在该实施例中，对于该广告点位，进行视频同步检查的时间段是从第二视频图像的播放点至t2结束时刻。如果视频同步检查的时间段内的视频同步检查均成功，则广告素材可播放到t2结束时刻。
假设该广告点位是该集视频中的第一个广告点位，在一些实施例中，该集视频一共40分钟，包括8个广告点位，第一个广告点位是第5分钟第0秒至第五分钟第5秒，第二个广告点位是第10分钟第0秒至第10分钟第5秒，以此类推，在第一个广告点位显示完广告素材后，终端设备会在第9分钟第57秒开始第二个广告点位的视频同步检查。在视频同步检查过程中，视频图像与参考图像比对的频率可以是每秒两次，与参考图像的采样频率一致。图10-图12示出根据本公开实施例的信息处理方法的多个用户界面示例示意图，其中，如图10所示，在终端设备播放视频流信息的过程中，获取该视频流信息的播放点为“00:16”，请求服务器查询视频资源，比如，当前播放的是哪个云端剧集及具体哪一集，还可以请求服务器查询广告点位，以便在该广告点位触发对应广告素材的显示。如图11所示，在服务器视频搜索完成并且终端设备接收到服务器发送的广告素材后，在终端设备上显示视频名称及广告点位“00:30-00:35”，此时播放点为“00:16”。当视频流播放到00:30且视频同步检查通过，则开始显示该广告素材。当如图12所示，显示广告素材的过程中，即在广告点位“00:30-00:35”区间内，显示当前播放点。
本领域技术人员可以理解，在具体实施方式的上述方法中，各步骤的撰写顺序并不意味着严格的执行顺序而对实施过程构成任何限定，各步骤的具体执行顺序应当以其功能和可能的内在逻辑确定。
本公开提及的上述各个方法实施例，在不违背原理逻辑的情况下，均可以彼此相互结合形成结合后的实施例，限于篇幅，本公开不再赘述。
此外，本公开还提供了信息处理装置、电子设备、计算机可读存储介质、程序，上述均可用来实现本公开提供的任一种信息处理方法，相应技术方案和描述和参见方法部分的相应记载，不再赘述。
图13示出根据本公开实施例的信息处理装置的框图，如图13所示，该处理装置，包括：第一检测单元51，用于对所播放的视频流中的第一视频图像进行人脸检测，得到所述第一视频图像的人脸信息；第一发送单元52，用于向服务器发送所述人脸信息；第一接收单元53，用于接收所述服务器发送的目标素材，其中，所述目标素材基于所述人脸信息得到；第一响应单元54，用于响应于检测到所述视频流播放至所述目标素材对应的开始播放时间，触发所述目标素材的显示。终端设备对播放的视频流中的第一视频图像进行人脸检测后，可以得到第一视频图像的人脸信息。所述目标素材包括广告素材的情况下，通过在服务器侧的人脸信息比对，可以根据人脸信息比对得到该广告素材并发送给终端，在终端设备侧响应于检测到视频流播放至该广告素材对应的开始播放时间，则触发该广告素材 的显示后，显示该广告素材。
由于如广告素材等素材的加载和显示方式是通过播放视频流中第一视频图像中的人脸信息的比对得到的，因此，终端设备收到该广告素材后，该广告素材的显示可以精准地匹配于所播放的视频流，从而，用户可以实时得到与视频流相关的更多有效信息(如精准投放的诸如广告素材等)。
在一些可能的实现方式中，所述第一响应单元，用于：向终端设备发送所述目标素材，以使得所述终端设备显示所述目标素材；或者，在所述触发所述目标素材的显示之后，显示所述目标素材。
在一些可能的实现方式中，所述目标素材的显示方式，包括如下至少一种：
1)目标素材以角标形式显示于所述视频流播放界面的角落；
2)所述目标素材与所述播放界面的播放内容关联地以创可贴形式显示于所述视频流播放界面；
3)所述目标素材以横幅形式显示于所述视频流播放界面且横跨所述播放界面。
在一些可能的实现方式中，所述装置还包括：第二接收单元，用于接收所述服务器发送的所述目标素材的播放时间信息，所述播放时间信息包括所述目标素材的预设播放时间段对应的参考图像信息；第二检测单元，用于基于所述播放时间信息，检测所述视频流是否播放到所述目标素材对应的开始播放时间。
在一些可能的实现方式中，所述参考图像信息包括：所述目标素材的预设播放时间段对应的多帧参考图像的特征信息。
在一些可能的实现方式中，所述多帧参考图像是通过对与所述预设播放时间段相邻的至少一个时间段进行多次采样得到的。
在一些可能的实现方式中，所述第二检测单元，用于：将所述视频流待播放的第二视频图像与所述播放时间信息中包括的参考图像信息进行比对，得到比对结果；基于所述比对结果，确定所述视频流是否播放到所述目标素材对应的开始播放时间。
在一些可能的实现方式中，所述第二检测单元，用于：对所述第二视频图像进行特征提取处理，得到所述第二视频图像的特征信息；基于所述第二视频图像的特征信息与所述参考图像信息中包括的至少一帧参考图像的特征信息之间的相似度，得到比对结果。
在一些可能的实现方式中，所述第二检测单元，用于：在所述比对结果为与所述多帧参考图像中的第一参考图像比对成功的情况下，确定所述视频流播放到所述目标素材对应的开始播放时间。
在一些可能的实现方式中，在所述比对结果为与所述多帧参考图像中的第二参考图像比对成功的情况下，确定结束所述目标素材的播放。
所述多帧参考图像中的前M个参考图像包括所述第一参考图像，所述多帧参考图像中的最后N个参考图像包括所述第二参考图像。
在一些可能的实现方式中，所述装置还包括素材显示单元，用于：响应于检测到所述符合预设拦截条件的广告素材，显示所述目标素材。
在一些可能的实现方式中，所述预设拦截条件包括下列中的至少一项：
黑名单中列出的预设商家；
黑名单中列出的预设广告类型；
与所述目标素材具有不同来源或不同商家的广告素材。
本公开提供了一种信息处理装置，所述装置包括：第三接收单元，用于接收终端设备发送的至少一个第一视频图像的人脸信息；信息确定单元，用于基于所述人脸信息与人脸模板库，确定所述至少一个第一视频图像所属的视频流信息，其中，所述人脸模板库包括多个预设人脸模板；素材获取单元，用于根据所述视频流信息，获取目标素材；第二发送单元，用于向所述终端设备发送所述目标素材。其中，所述目标素材可以包括广告素材，以在服务器侧根据人脸信息的比对，如基于人脸信息与人脸模板库来确定至少一个第一视频图像所属的视频流信息，从而，可以根据该视频流信息获取该广告素材。服务器将根据该人脸信息比对得到的该广告素材发送给终端设备，在终端侧响应于检测到视频流播放至该广告素材对应的开始播放时间，则触发该广告素材的显示，以显示该广告素材。
由于如广告素材等素材的加载和显示方式是通过播放视频流中第一视频图像中的人脸信息的比对得到的，因此，终端设备收到该广告素材后，该广告素材的显示可以精准地匹配于所播放的视频流，从而，用户可以实时得到与视频流相关的更多有效信息(如精准投放的诸如广告素材等)。
在一些可能的实现方式中，所述信息确定单元，用于：基于所述人脸模板库包括的多个预设人脸模板，得到所述人脸信息对应的人物身份信息；基于所述人物身份信息查找视频库，得到所述至少一个第一视频图像所属的视频流信息。
在一些可能的实现方式中，所述信息确定单元，用于：确定所述人脸信息与所述多个预设人脸模板的第一比对结果；基于所述第一比对结果，得到所述人脸信息对应的人物身份信息。
在一些可能的实现方式中，所述装置还包括第三发送单元，用于：向所述终端设备发送所述目标素材对应的播放时间信息，其中，所述播放时间信息包括所述目标素材的预设播放时间段对应的参考图像信息。
在一些可能的实现方式中，所述参考图像信息包括：所述目标素材的预设播放时间段对应的多帧参考图像的特征信息。
在一些可能的实现方式中，所述装置还包括：采样单元，用于在所述预设播放时间段的至少一个相邻时间段对所述视频流信息进行采样，得到多帧参考图像；特征提取单元，用于对所述多帧参考图像进行特征提取处理，得到所述多帧参考图像的特征信息。
在一些可能的实现方式中，所述装置还包括第四发送单元，用于：向所述终端设备发送广告拦截指示，所述广告拦截指示用于指示当检测到所述终端设备所播放视频流中出现特定广告素材时，所述终端设备显示所述目标素材。
在一些实施例中，本公开实施例提供的装置具有的功能或包含的模块可以用于执行上文方法实施例描述的方法，其具体实现可以参照上文方法实施例的描述，为了简洁，这里不再赘述。
本公开实施例还提出一种计算机可读存储介质，其上存储有计算机程序指令，所述计算机程序指令被处理器执行时实现上述方法。计算机可读存储介质可以是易失性计算机可读存储介质或非易失性计算机可读存储介质。
本公开实施例还提供了一种计算机程序产品，包括计算机可读代码，当计算机可读代码在设备上运行时，设备中的处理器执行用于实现如上任一实施例提供的信息处理方法的指令。
本公开实施例还提供了另一种计算机程序产品，用于存储计算机可读指令，指令被执行时使得计算机执行上述任一实施例提供的信息处理方法的操作。
该计算机程序产品可以具体通过硬件、软件或其结合的方式实现。在一个可选实施例中，所述计算机程序产品具体体现为计算机存储介质，在另一个可选实施例中，计算机程序产品具体体现为软件产品，例如软件开发包(Software Development Kit，SDK)等等。
本公开实施例还提出一种电子设备，包括：处理器；用于存储处理器可执行指令的存储器；其中，所述处理器被配置为上述方法。
电子设备可以被提供为终端设备、服务器或其它形态的设备。
图14是根据一示例性实施例示出的一种电子设备800的框图。例如，电子设备800可以是移动电话，计算机，数字广播终端，消息收发设备，游戏控制台，平板设备，医疗设备，健身设备，个人数字助理等终端设备。
参照图14，电子设备800可以包括以下一个或多个组件：处理组件802，存储器804，电源组件806，多媒体组件808，音频组件810，输入/输出(I/O)的接口812，传感器组件814，以及通信组件816。
处理组件802通常控制电子设备800的整体操作，诸如与显示，电话呼叫，数据通信，相机操作和记录操作相关联的操作。处理组件802可以包括一个或多个处理器820来执行指令，以完成上述的方法的全部或部分步骤。此外，处理组件802可以包括一个或多个模块，便于处理组件802和其他组件之间的交互。例如，处理组件802可以包括多媒体模块，以方便多媒体组件808和处理组件802之间的交互。
存储器804被配置为存储各种类型的数据以支持在电子设备800的操作。这些数据的示例包括用于在电子设备800上操作的任何应用程序或方法的指令，联系人数据，电话簿数据，消息，图片，视频等。存储器804可以由任何类型的易失性或非易失性存储设备或者它们的组合实现，如静态随机存取存储器(SRAM)，电可擦除可编程只读存储器(EEPROM)，可擦除可编程只读存储器(EPROM)，可编程只读存储器(PROM)，只读存储器(ROM)，磁存储器，快闪存储器，磁盘或光盘。
电源组件806为电子设备800的各种组件提供电力。电源组件806可以包括电源管理系统，一个或多个电源，及其他与为电子设备800生成、管理和分配电力相关联的组件。
多媒体组件808包括在所述电子设备800和用户之间的提供一个输出接口的屏幕。在一些实施例中，屏幕可以包括液晶显示器(LCD)和触摸面板(TP)。如果屏幕包括触摸面板，屏幕可以被实现为触摸屏，以接收来自用户的输入信号。触摸面板包括一个或多个触摸传感器以感测触摸、滑动和触摸面板上的手势。所述触摸传感器可以不仅感测触摸或滑动动作的边界，而且还检测与所述触摸或滑动操作相关的持续时间和压力。在一些实施例中，多媒体组件808包括一个前置摄像头和/或后置摄像头。当电子设备800处于操作模式，如拍摄模式或视频模式时，前置摄像头和/或后置摄像头可以接收外部的多媒体数据。每个前置摄像头和后置摄像头可以是一个固定的光学透镜系统或具有焦距和光学变焦能力。
音频组件810被配置为输出和/或输入音频信号。例如，音频组件810包括一个麦克风(MIC)，当电子设备800处于操作模式，如呼叫模式、记录模式和语音识别模式时，麦克风被配置为接收外部音频信号。所接收的音频信号可以被进一步存储在存储器804或经由通信组件816发送。在一些实施例中，音频组件810还包括一个扬声器，用于输出音频信号。
I/O接口812为处理组件802和外围接口模块之间提供接口，上述外围接口模块可以是键盘，点击轮，按钮等。这些按钮可包括但不限于：主页按钮、音量按钮、启动按钮和锁定按钮。
传感器组件814包括一个或多个传感器，用于为电子设备800提供各个方面的状态评估。例如，传感器组件814可以检测到电子设备800的打开/关闭状态，组件的相对定位，例如所述组件为电子设备800的显示器和小键盘，传感器组件814还可以检测电子设备800或电子设备800一个组件的位置改变，用户与电子设备800接触的存在或不存在，电子设备800方位或加速/减速和电子设备800的温度变化。传感器组件814可以包括接近传感器，被配置用来在没有任何的物理接触时检测附近物体的存在。传感器组件814还可以包括光传感器，如CMOS(Complementary Metal Oxide Semiconductor)或CCD(Charge Coupled Device)图像传感器，用于在成像应用中使用。在一些实施例中，该传感器组件814还可以包括加速度传感器，陀螺仪传感器，磁传感器，压力传感器或温度传感器。
通信组件816被配置为便于电子设备800和其他设备之间有线或无线方式的通信。电子设备800可以接入基于通信标准的无线网络，如WiFi、2G、3G、4G或5G等，或它们的组合。在一个示例性实施例中，通信组件816经 由广播信道接收来自外部广播管理系统的广播信号或广播相关信息。在一个示例性实施例中，所述通信组件816还包括近场通信(NFC)模块，以促进短程通信。例如，在NFC模块可基于射频识别(RFID)技术，红外数据协会(IrDA)技术，超宽带(UWB)技术，蓝牙(BT)技术和其他技术来实现。
在示例性实施例中，电子设备800可以被一个或多个应用专用集成电路(ASIC)、数字信号处理器(DSP)、数字信号处理设备(DSPD)、可编程逻辑器件(PLD)、现场可编程门阵列(FPGA)、控制器、微控制器、微处理器或其他电子元件实现，用于执行上述方法。
在示例性实施例中，还提供了一种非易失性计算机可读存储介质，例如包括计算机程序指令的存储器804，上述计算机程序指令可由电子设备800的处理器820执行以完成上述方法。
图15是根据一示例性实施例示出的一种电子设备900的框图。例如，电子设备900可以被提供为一服务器。参照图15，电子设备900包括处理组件922，其进一步包括一个或多个处理器，以及由存储器932所代表的存储器资源，用于存储可由处理组件922的执行的指令，例如应用程序。存储器932中存储的应用程序可以包括一个或一个以上的每一个对应于一组指令的模块。此外，处理组件922被配置为执行指令，以执行上述方法。
电子设备900还可以包括一个电源组件926被配置为执行电子设备900的电源管理，一个有线或无线网络接口950被配置为将电子设备900连接到网络，和一个输入输出(I/O)接口958。电子设备900可以操作基于存储在存储器932的操作系统，例如Windows ServerTM，Mac OS XTM，UnixTM,LinuxTM，FreeBSDTM或类似。
在示例性实施例中，还提供了一种非易失性计算机可读存储介质，例如包括计算机程序指令的存储器932，上述计算机程序指令可由电子设备900的处理组件922执行以完成上述方法。
本公开可以是系统、方法和/或计算机程序产品。计算机程序产品可以包括计算机可读存储介质，其上载有用于使处理器实现本公开的各个方面的计算机可读程序指令。
计算机可读存储介质可以是可以保持和存储由指令执行设备使用的指令的有形设备。计算机可读存储介质例如可以包括但不限于电存储设备、磁存储设备、光存储设备、电磁存储设备、半导体存储设备或者上述的任意合适的组合。计算机可读存储介质的更具体的例子(非穷举的列表)包括：便携式计算机盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、静态随机存取存储器(SRAM)、便携式压缩盘只读存储器(CD-ROM)、数字多功能盘(DVD)、记忆棒、软盘、机械编码设备、例如其上存储有指令的打孔卡或凹槽内凸起结构、以及上述的任意合适的组合。这里所使用的计算机可读存储介质不被解释为瞬时信号本身，诸如无线电波或者其他自由传播的电磁波、通过波导或其他传输媒介传播的电磁波(例如，通过光纤电缆的光脉冲)、或者通过电线传输的电信号。
这里所描述的计算机可读程序指令可以从计算机可读存储介质下载到各个计算/处理设备，或者通过网络、例如因特网、局域网、广域网和/或无线网下载到外部计算机或外部存储设备。网络可以包括铜传输电缆、光纤传输、无线传输、路由器、防火墙、交换机、网关计算机和/或边缘服务器。每个计算/处理设备中的网络适配卡或者网络接口从网络接收计算机可读程序指令，并转发该计算机可读程序指令，以供存储在各个计算/处理设备中的计算机可读存储介质中。
用于执行本公开操作的计算机程序指令可以是汇编指令、指令集架构(ISA)指令、机器指令、机器相关指令、微代码、固件指令、状态设置数据、或者以一种或多种编程语言的任意组合编写的源代码或目标代码，所述编程语言包括面向对象的编程语言—诸如Smalltalk、C++等，以及常规的过程式编程语言—诸如“C”语言或类似的编程语言。计算机可读程序指令可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中，远程计算机可以通过任意种类的网络—包括局域网(LAN)或广域网(WAN)—连接到用户计算机，或者，可以 连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。在一些实施例中，通过利用计算机可读程序指令的状态信息来个性化定制电子电路，例如可编程逻辑电路、现场可编程门阵列(FPGA)或可编程逻辑阵列(PLA)，该电子电路可以执行计算机可读程序指令，从而实现本公开的各个方面。
这里参照根据本公开实施例的方法、装置(系统)和计算机程序产品的流程图和/或框图描述了本公开的各个方面。应当理解，流程图和/或框图的每个方框以及流程图和/或框图中各方框的组合，都可以由计算机可读程序指令实现。
这些计算机可读程序指令可以提供给通用计算机、专用计算机或其它可编程数据处理装置的处理器，从而生产出一种机器，使得这些指令在通过计算机或其它可编程数据处理装置的处理器执行时，产生了实现流程图和/或框图中的一个或多个方框中规定的功能/动作的装置。也可以把这些计算机可读程序指令存储在计算机可读存储介质中，这些指令使得计算机、可编程数据处理装置和/或其他设备以特定方式工作，从而，存储有指令的计算机可读介质则包括一个制造品，其包括实现流程图和/或框图中的一个或多个方框中规定的功能/动作的各个方面的指令。
也可以把计算机可读程序指令加载到计算机、其它可编程数据处理装置、或其它设备上，使得在计算机、其它可编程数据处理装置或其它设备上执行一系列操作步骤，以产生计算机实现的过程，从而使得在计算机、其它可编程数据处理装置、或其它设备上执行的指令实现流程图和/或框图中的一个或多个方框中规定的功能/动作。
附图中的流程图和框图显示了根据本公开的多个实施例的系统、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段或指令的一部分，所述模块、程序段或指令的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个连续的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合，可以用执行规定的功能或动作的专用的基于硬件的系统来实现，或者可以用专用硬件与计算机指令的组合来实现。
在不违背逻辑的情况下，本申请不同实施例之间可以相互结合，不同实施例描述有所侧重，为侧重描述的部分可以参见其他实施例的记载。
以上已经描述了本公开的各实施例，上述说明是示例性的，并非穷尽性的，并且也不限于所披露的各实施例。在不偏离所说明的各实施例的范围和精神的情况下，对于本技术领域的普通技术人员来说许多修改和变更都是显而易见的。本文中所用术语的选择，旨在最好地解释各实施例的原理、实际应用或对市场中技术的技术改进，或者使本技术领域的其它普通技术人员能理解本文披露的各实施例。

Claims (21)

  1. 一种信息处理方法，包括：
    对所播放的视频流中的第一视频图像进行人脸检测，得到所述第一视频图像的人脸信息；
    向服务器发送所述人脸信息；
    接收所述服务器发送的目标素材，其中，所述目标素材基于所述人脸信息得到；
    响应于检测到所述视频流播放至所述目标素材对应的开始播放时间，触发所述目标素材的显示。
  2. 根据权利要求1所述的方法，还包括：
    接收所述服务器发送的所述目标素材的播放时间信息，所述播放时间信息包括所述目标素材的预设播放时间段对应的参考图像信息；
    基于所述播放时间信息，检测所述视频流是否播放到所述目标素材对应的开始播放时间。
  3. 根据权利要求2所述的方法，其中，所述参考图像信息包括：所述目标素材的预设播放时间段对应的多帧参考图像的特征信息。
  4. 根据权利要求3所述的方法，其中，所述多帧参考图像是通过对与所述预设播放时间段相邻的至少一个时间段进行多次采样得到的。
  5. 根据权利要求2至4中任一项所述的方法，其中，基于所述播放时间信息，检测所述视频流是否播放到所述目标素材对应的开始播放时间，包括：
    将所述视频流待播放的第二视频图像与所述参考图像信息进行比对，得到比对结果，其中在所述第一视频图像播放之后，播放所述第二视频图像；
    基于所述比对结果，确定所述视频流是否播放到所述目标素材对应的开始播放时间。
  6. 根据权利要求5所述的方法，其中，基于所述比对结果，确定所述视频流是否播放到所述目标素材对应的开始播放时间，包括：
    在所述比对结果为与所述多帧参考图像中的第一参考图像比对成功的情况下，确定所述视频流播放到所述目标素材对应的开始播放时间，其中，所述多帧参考图像中的前M个参考图像包括所述第一参考图像。
  7. 根据权利要求5所述的方法，还包括：
    在所述比对结果为与所述多帧参考图像中的第二参考图像比对成功的情况下，确定结束所述目标素材的播放，其中，所述多帧参考图像中的最后N个参考图像包括所述第二参考图像。
  8. 根据权利要求1至7中任一项所述的方法，还包括：
    响应于检测到所述符合预设拦截条件的广告素材，显示所述目标素材。
  9. 一种信息处理方法，包括：
    接收终端设备发送的至少一个第一视频图像的人脸信息；
    基于所述人脸信息与人脸模板库，确定所述至少一个第一视频图像所属的视频流信息，其中，所述人脸模板库包括多个预设人脸模板；
    根据所述视频流信息，获取目标素材；
    向所述终端设备发送所述目标素材。
  10. 根据权利要求9所述的方法，其中，基于所述人脸信息与人脸模板库，确定所述至少一个第一视频图像所属的视频流信息，包括：
    基于所述人脸模板库包括的多个预设人脸模板，得到所述人脸信息对应的人物身份信息；
    基于所述人物身份信息查找视频库，得到所述至少一个第一视频图像所属的视频流信息。
  11. 根据权利要求9或10所述的方法，还包括：
    向所述终端设备发送所述目标素材对应的播放时间信息，其中，所述播放时间信息包括所述目标素材的预设播放时间段对应的参考图像信息。
  12. 根据权利要求11所述的方法，其中，所述参考图像信息包括：所述目标素材的预设播放时间段对应的多帧参考图像的特征信息。
  13. 根据权利要求11或12所述的方法，还包括：
    在所述预设播放时间段的至少一个相邻时间段对所述视频流信息进行采样，得到多帧参考图像；
    对所述多帧参考图像进行特征提取处理，得到所述多帧参考图像的特征信息。
  14. 根据权利要求9-13中任一项所述的方法，还包括：
    向所述终端设备发送广告拦截指示，所述广告拦截指示用于指示当检测到所述终端设备所播放视频流中出现特定广告素材时，所述终端设备显示所述目标素材。
  15. 一种信息处理装置，包括：
    第一检测单元，用于对所播放的视频流中的第一视频图像进行人脸检测，得到所述第一视频图像的人脸信息；
    第一发送单元，用于向服务器发送所述人脸信息；
    第一接收单元，用于接收所述服务器发送的目标素材，其中，所述目标素材基于所述人脸信息得到；
    第一响应单元，用于响应于检测到所述视频流播放至所述目标素材对应的开始播放时间，触发所述目标素材的显示。
  16. 根据权利要求15所述的装置，还包括：
    第二接收单元，用于接收所述服务器发送的所述目标素材的播放时间信息，所述播放时间信息包括所述目标素材的预设播放时间段对应的参考图像信息；
    第二检测单元，用于基于所述播放时间信息，检测所述视频流是否播放到所述目标素材对应的开始播放时间。
  17. 根据权利要求16所述的装置，其中，所述参考图像信息包括：所述目标素材的预设播放时间段对应的多帧参考图像的特征信息。
  18. 根据权利要求17所述的装置，其中，所述多帧参考图像是通过对与所述预设播放时间段相邻的至少一个时间段进行多次采样得到的。
  19. 一种信息处理装置，包括：
    第三接收单元，用于接收终端设备发送的至少一个第一视频图像的人脸信息；
    信息确定单元，用于基于所述人脸信息与人脸模板库，确定所述至少一个第一视频图像所属的视频流信息，其中，所述人脸模板库包括多个预设人脸模板；
    素材获取单元，用于根据所述视频流信息，获取目标素材；
    第二发送单元，用于向所述终端设备发送所述目标素材。
  20. 一种电子设备，包括：
    处理器；
    用于存储处理器可执行指令的存储器；
    其中，所述处理器被配置为：执行权利要求1至8、权利要求9至14中任意一项所述的方法。
  21. 一种计算机可读存储介质，其上存储有计算机程序指令，所述计算机程序指令被处理器执行时实现权利要求1至8、权利要求9至14中任意一项所述的方法。
PCT/CN2020/087486 2019-12-11 2020-04-28 信息处理方法及装置、电子设备和存储介质 Ceased WO2021114552A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
KR1020217018772A KR20210090262A (ko) 2019-12-11 2020-04-28 정보 처리 방법과 장치, 전자 디바이스 및 기록 매체
JP2021534223A JP2022516701A (ja) 2019-12-11 2020-04-28 情報処理方法及び装置、電子デバイス並びに記録媒体

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201911267882.7A CN110996121A (zh) 2019-12-11 2019-12-11 信息处理方法及装置、电子设备和存储介质
CN201911267882.7 2019-12-11

Publications (1)

Publication Number Publication Date
WO2021114552A1 true WO2021114552A1 (zh) 2021-06-17

Family

ID=70092375

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/087486 Ceased WO2021114552A1 (zh) 2019-12-11 2020-04-28 信息处理方法及装置、电子设备和存储介质

Country Status (5)

Country Link
JP (1) JP2022516701A (zh)
KR (1) KR20210090262A (zh)
CN (1) CN110996121A (zh)
TW (1) TW202122989A (zh)
WO (1) WO2021114552A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114040248A (zh) * 2021-11-23 2022-02-11 维沃移动通信有限公司 一种视频处理方法、装置和电子设备
CN114845161A (zh) * 2022-05-10 2022-08-02 北京三快在线科技有限公司 基于视频结构信息的视频生成方法、装置、设备及介质

Families Citing this family (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110996121A (zh) * 2019-12-11 2020-04-10 北京市商汤科技开发有限公司 信息处理方法及装置、电子设备和存储介质
CN113055740A (zh) * 2020-04-26 2021-06-29 杜达亮 视频中物体识别、贴片方法及其系统
CN111667313A (zh) * 2020-06-08 2020-09-15 上海极链网络科技有限公司 广告展示方法、装置、客户端设备及存储介质
CN112312191A (zh) * 2020-10-30 2021-02-02 湖南快乐阳光互动娱乐传媒有限公司 一种视频广告素材的显示方法及装置
CN113766297B (zh) * 2021-05-27 2023-12-05 腾讯科技（深圳）有限公司 视频处理方法、播放终端及计算机可读存储介质
CN113240702A (zh) * 2021-06-25 2021-08-10 北京市商汤科技开发有限公司 图像处理方法及装置、电子设备和存储介质
CN113794836B (zh) * 2021-09-09 2023-10-13 北京字节跳动网络技术有限公司 一种子弹时间视频生成方法、装置、系统、设备和介质
CN114419699A (zh) * 2021-12-28 2022-04-29 青岛畅索科技有限公司 图像信息处理方法、系统、设备及存储介质
CN114679591A (zh) * 2021-12-30 2022-06-28 广州方硅信息技术有限公司 直播间的视频比例切换方法、装置、介质以及计算机设备
CN115033136B (zh) * 2022-06-28 2025-04-04 北京字跳网络技术有限公司 素材展示方法、装置、设备、计算机可读存储介质及产品
CN118333709A (zh) * 2024-03-06 2024-07-12 支付宝(杭州)信息技术有限公司 信息展示方法及装置

Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090125544A1 (en) * 2007-11-09 2009-05-14 Vibrant Media, Inc. Intelligent Augmentation Of Media Content
CN103024464A (zh) * 2011-12-31 2013-04-03 中国科学院计算技术研究所 提供与视频播放内容相关信息的系统和方法
CN104581430A (zh) * 2013-10-21 2015-04-29 华为技术有限公司 视频播放中的广告插播方法和设备
CN105117463A (zh) * 2015-08-24 2015-12-02 北京旷视科技有限公司 信息处理方法和信息处理装置
CN106656951A (zh) * 2016-09-14 2017-05-10 乐视控股（北京）有限公司 广告的投放方法、客户端、服务器及系统
CN108446390A (zh) * 2018-03-22 2018-08-24 百度在线网络技术（北京）有限公司 用于推送信息的方法和装置
CN108509611A (zh) * 2018-03-30 2018-09-07 百度在线网络技术（北京）有限公司 用于推送信息的方法和装置
CN109429078A (zh) * 2017-08-24 2019-03-05 北京搜狗科技发展有限公司 视频处理方法和装置、用于视频处理的装置
CN110880120A (zh) * 2018-09-06 2020-03-13 阿里巴巴集团控股有限公司 一种广告方法和广告系统以及识别装置
CN110996121A (zh) * 2019-12-11 2020-04-10 北京市商汤科技开发有限公司 信息处理方法及装置、电子设备和存储介质
CN110996156A (zh) * 2019-12-20 2020-04-10 杭州当虹科技股份有限公司 一种在视频播放过程中自动检测并推送广告的方法

Family Cites Families (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070101360A1 (en) * 2003-11-17 2007-05-03 Koninklijke Philips Electronics, N.V. Commercial insertion into video streams based on surrounding program content
CN101466024A (zh) * 2007-12-19 2009-06-24 上海亿动信息技术有限公司 用户设备上播放多媒体文件时插播广告信息的方法和装置
US8725560B2 (en) * 2009-02-02 2014-05-13 Modiface Inc. Method and system for simulated product evaluation via personalizing advertisements based on portrait images
US20110066944A1 (en) * 2009-09-14 2011-03-17 Barton James M Multifunction Multimedia Device
US8935719B2 (en) * 2011-08-25 2015-01-13 Comcast Cable Communications, Llc Application triggering
JP5808433B2 (ja) * 2011-12-16 2015-11-10 株式会社ソニー・コンピュータエンタテインメント 放送番組処理装置、放送番組処理方法、放送局装置、情報配信サーバ、プログラム及び情報記憶媒体
CN102868922B (zh) * 2012-09-11 2016-08-10 Tcl通力电子（惠州）有限公司 一种播放广告的方法及系统
CN103024479B (zh) * 2012-12-17 2016-03-02 深圳先进技术研究院 视频内容中自适应投放广告的方法及系统
US8572097B1 (en) * 2013-03-15 2013-10-29 FEM, Inc. Media content discovery and character organization techniques
CN105284122B (zh) * 2014-01-24 2018-12-04 Sk 普兰尼特有限公司 用于通过使用帧聚类来插入广告的装置和方法
WO2015119749A1 (en) * 2014-02-07 2015-08-13 Sony Computer Entertainment America Llc Scheme for determining the locations and timing of advertisements and other insertions in media
CN106303621A (zh) * 2015-06-01 2017-01-04 北京中投视讯文化传媒股份有限公司 一种视频广告的插入方法和装置
CN108259974A (zh) * 2018-03-07 2018-07-06 优酷网络技术（北京）有限公司 视频匹配方法及装置

Patent Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090125544A1 (en) * 2007-11-09 2009-05-14 Vibrant Media, Inc. Intelligent Augmentation Of Media Content
CN103024464A (zh) * 2011-12-31 2013-04-03 中国科学院计算技术研究所 提供与视频播放内容相关信息的系统和方法
CN104581430A (zh) * 2013-10-21 2015-04-29 华为技术有限公司 视频播放中的广告插播方法和设备
CN105117463A (zh) * 2015-08-24 2015-12-02 北京旷视科技有限公司 信息处理方法和信息处理装置
CN106656951A (zh) * 2016-09-14 2017-05-10 乐视控股（北京）有限公司 广告的投放方法、客户端、服务器及系统
CN109429078A (zh) * 2017-08-24 2019-03-05 北京搜狗科技发展有限公司 视频处理方法和装置、用于视频处理的装置
CN108446390A (zh) * 2018-03-22 2018-08-24 百度在线网络技术（北京）有限公司 用于推送信息的方法和装置
CN108509611A (zh) * 2018-03-30 2018-09-07 百度在线网络技术（北京）有限公司 用于推送信息的方法和装置
CN110880120A (zh) * 2018-09-06 2020-03-13 阿里巴巴集团控股有限公司 一种广告方法和广告系统以及识别装置
CN110996121A (zh) * 2019-12-11 2020-04-10 北京市商汤科技开发有限公司 信息处理方法及装置、电子设备和存储介质
CN110996156A (zh) * 2019-12-20 2020-04-10 杭州当虹科技股份有限公司 一种在视频播放过程中自动检测并推送广告的方法

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114040248A (zh) * 2021-11-23 2022-02-11 维沃移动通信有限公司 一种视频处理方法、装置和电子设备
CN114845161A (zh) * 2022-05-10 2022-08-02 北京三快在线科技有限公司 基于视频结构信息的视频生成方法、装置、设备及介质

Also Published As

Publication number Publication date
KR20210090262A (ko) 2021-07-19
CN110996121A (zh) 2020-04-10
JP2022516701A (ja) 2022-03-02
TW202122989A (zh) 2021-06-16

Similar Documents

Publication Publication Date Title
TW202122989A (zh) 訊息處理方法及裝置、電子設備和電腦可讀儲存介質
CN106028166B (zh) 直播过程中直播间切换方法及装置
EP3276976A1 (en) Method, apparatus, host terminal, server and system for processing live broadcasting information
US20170155958A1 (en) Method, Apparatus and System for Playing Multimedia Data, and Storage Medium
US20130205321A1 (en) Method and system for automatic content recognition protocols
CN108966025B (zh) 视频播放方法、装置及计算机可读存储介质
CN108259991B (zh) 视频处理方法及装置
CN106559712B (zh) 视频播放处理方法、装置及终端设备
CN108495168B (zh) 弹幕信息的显示方法及装置
WO2020135643A1 (zh) 目标角色视频片段播放方法、系统、装置及存储介质
CN104391711A (zh) 一种设置屏幕保护的方法及装置
US20170090684A1 (en) Method and apparatus for processing information
WO2017101345A1 (zh) 一种视频播放方法及装置
CN110719530A (zh) 一种视频播放方法、装置、电子设备及存储介质
CN112291631A (zh) 信息获取方法、装置、终端及存储介质
CN106375874A (zh) 视频处理方法、装置、终端设备及服务器
TW201837751A (zh) 搜索結果顯示方法及裝置
CN111182328B (zh) 一种视频剪辑方法、装置、服务器、终端及存储介质
CN112616053A (zh) 直播视频的转码方法、装置及电子设备
CN105242837B (zh) 应用页面获取方法及终端
CN106339908A (zh) 跟踪广告效果的方法和装置
CN107247794B (zh) 直播中的话题引导方法、直播装置及终端设备
CN113568551A (zh) 图片保存方法及装置
CN111629270A (zh) 一种候选项确定方法、装置及机器可读介质
TWI735797B (zh) 多媒體內容的推薦方法、裝置及非揮發性電腦可讀儲存介質

Legal Events

Date Code Title Description
ENP Entry into the national phase

Ref document number: 2021534223

Country of ref document: JP

Kind code of ref document: A

ENP Entry into the national phase

Ref document number: 20217018772

Country of ref document: KR

Kind code of ref document: A

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20899276

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20899276

Country of ref document: EP

Kind code of ref document: A1