JP2022516701A - Information processing methods and devices, electronic devices and recording media - Google Patents

Abstract

The present invention relates to an information processing method and device, an electronic device, and a recording medium, wherein the method performs face detection on a first video image in a video stream being played, and the first video. Obtaining the face information of the image, transmitting the face information to the server, receiving the target material transmitted by the server based on the face information, and the video stream said. It includes triggering the display of the target material in response to the detection that the target material has been reproduced until the start reproduction time corresponding to the target material. Adopting the present invention is beneficial for realizing automatic insertion of third-party advertising material. [Selection diagram] Fig. 1

Description

The present invention relates to the field of information processing technology, and more particularly to information processing methods and devices, electronic devices and recording media.

For example, in some information processing application scenarios, such as a scene playing streaming media information (eg, a video stream), other streaming media information other than the video stream, such as advertising material, can also be displayed. However, in the case of over-the-top (OTT) advertising materials provided by third-party service providers, automatic insertion of advertising materials is realized because TVs and set-top boxes cannot acquire the relevant content of third-party advertising materials. Can not do it.

The present invention provides a technical solution for information processing.

According to one aspect of the present invention, an information processing method is provided, in which face detection is performed on a first video image in a video stream being played, and the face information of the first video image is obtained. Obtaining, transmitting the face information to the server, receiving the target material transmitted by the server based on the face information, and the video stream corresponding to the target material. In response to the detection that reproduction has been performed up to the start reproduction time, the display of the target material is triggered.

In a possible implementation, the target material includes advertising material.

In a possible embodiment, triggering the display of the target material comprises causing the terminal device to display the target material by transmitting the target material to the terminal device, or the target. After triggering the display of the material, the method further comprises displaying the target material.

In a possible embodiment, the display method of the target material is to display the target material in the corner of the video stream playback interface in the form of a corner mark, or to associate the target material with the playback content of the playback interface. The term includes displaying the target material in the form of a band aid on the video stream playback interface, or displaying the target material in the form of a banner across the video stream playback interface.

In a possible embodiment, the method receives and replays the replay time information of the target material transmitted by the server, including reference image information corresponding to a predetermined replay time zone of the target material. Further including detecting whether or not the video stream has been played up to the start playback time corresponding to the target material based on the time information.

In a possible embodiment, the reference image information includes feature information of a plurality of frames of the reference image corresponding to a predetermined reproduction time zone of the target material.

In a possible embodiment, the plurality of frames of the reference image are obtained by performing sampling a plurality of times for at least one time zone adjacent to the predetermined reproduction time zone.

In a possible embodiment, detecting whether or not the video stream has been played up to the start playback time corresponding to the target material based on the playback time information is a second video image awaiting playback of the video stream. And the reference image information included in the reproduction time information are compared to obtain a comparison result, and based on the comparison result, whether or not the video stream is reproduced up to the start reproduction time corresponding to the target material. Includes determining or. After the first video image is reproduced, the second video image is reproduced.

In a possible embodiment, determining whether or not the video stream has been played up to the start playback time corresponding to the target material based on the comparison result is such that the comparison result is a reference image of the plurality of frames. If the comparison with the first reference image in the medium is successful, it includes determining that the video stream has been played up to the start playback time corresponding to the target material, and here, the reference image of the plurality of frames. The previous M reference images inside include the first reference image.

In some embodiments, triggering the display of the target material in response to the detection that the video stream has been played up to the start playback time corresponding to the target material is the video stream. It includes triggering the display of the target material in response to the successful comparison between the second video image and the first target reference image in the plurality of frames of the reference image.

In some embodiments, the display of the target material is triggered in response to a successful comparison between the second video image of the video stream and the first target reference image in the plurality of frames of the reference image. That is, the comparison between the second video image and the first target reference image was successful, and the third video image of the video stream and the second target reference image in the reference image of the plurality of frames were used. The third video image is played after the second video image has been played, including triggering the display of the target material in response to the detection that the comparison was successful.

In a possible embodiment, the method determines that the reproduction of the target material is terminated if the comparison result is successful in comparison with the second reference image in the plurality of frames of the reference image. The last N reference images in the plurality of frames of the reference image further include the second reference image. In some embodiments, the N reference images after the target reference image in the plurality of frames of the reference image include the second reference image.

In some embodiments, if the fourth video image of the video stream is successfully compared to the second reference image, it triggers the end of reproduction of the target material.

In a possible embodiment, the method further comprises displaying the target material in response to the detection of an advertising material satisfying a predetermined blocking condition.

In a possible embodiment, the predetermined blocking condition is

Being a blacklisted merchant and

Being a blacklisted ad type and

Includes at least one of having different sources or different merchant advertising material than the target material.

According to one aspect of the invention, an information processing method is provided that receives face information of at least one first video image transmitted by a terminal device, said face information, and a plurality of predetermined. Based on the face template library including the face template, determining the video stream information to which the at least one first video image belongs, and acquiring the target material based on the video stream information. Includes transmitting the target material to the terminal device.

In a possible embodiment, determining the video stream information to which the at least one first video image belongs, based on the face information and face template library, is a plurality of predetermined embodiments contained in the face template library. Based on the face template of, the person identification information corresponding to the face information is obtained, and the video library is searched based on the person identification information, and the video stream to which the at least one first video image belongs. Including getting information.

In a possible embodiment, obtaining the person identification information corresponding to the face information based on the plurality of predetermined face templates included in the face template library can be performed by the face information and the plurality of predetermined face templates. The first comparison result is determined, and the person identification information corresponding to the face information is obtained based on the first comparison result.

In a possible implementation, the target material includes advertising material.

In a possible embodiment, the method further comprises transmitting playback time information corresponding to the target material to the terminal device, wherein the playback time information is in a predetermined playback time zone of the target material. Includes corresponding reference image information.

In a possible embodiment, the reference image information includes feature information of a plurality of frames of the reference image corresponding to a predetermined reproduction time zone of the target material.

In a possible embodiment, the method comprises performing sampling on the video stream information in at least one adjacent time zone of the predetermined playback time zone to obtain a plurality of frames of reference images and said the plurality of frames. Further includes performing a feature extraction process on the reference image of the above to obtain feature information of the reference image of the plurality of frames.

In a possible embodiment, the method further comprises transmitting an ad blocking instruction to the terminal device, wherein the particular ad material appears in the video stream being played by the terminal device. If it is detected, the terminal device is instructed to display the target material.

According to one aspect of the present invention, an information processing apparatus is provided, in which the apparatus performs face detection on a first video image in a video stream being played and obtains face information of the first video image. A first detection unit for obtaining, a first transmission unit for transmitting the face information to the server, and a first for receiving the target material transmitted by the server and obtained based on the face information. A receiving unit and a first response unit for triggering the display of the target material in response to the detection that the video stream has been played up to the start playback time corresponding to the target material. Be prepared.

According to another aspect of the present invention, an information processing apparatus is provided, wherein the apparatus is a third receiving unit for receiving face information of at least one first video image transmitted by a terminal device, and the face information. And, based on a face template library including a plurality of predetermined face templates, an information determination unit for determining the video stream information to which the at least one first video image belongs, and the video stream information. A material acquisition unit for acquiring the target material based on the above, and a second transmission unit for transmitting the target material to the terminal device are provided.

According to one aspect of the invention, an electronic device is provided that comprises a processor and a memory for storing instructions that can be executed by the processor, the processor performing the information processing method described above. It is configured to do.

According to one aspect of the present invention, a computer-readable recording medium in which computer program instructions are stored is provided, and when the computer program instructions are executed by a processor, the above information processing method is realized.

According to an embodiment of the present invention, face detection is performed on a first video image in a video stream being played, face information of the first video image is obtained, and the face information is transmitted to a server. In response to receiving the target material transmitted by the server and obtained based on the face information, and detecting that the video stream has been played up to the start playback time corresponding to the target material. , It is useful to realize the automatic insertion of the OTT advertising material by triggering the display of the target material.

It should be understood that the general description above and the detailed description below are merely exemplary and interpretive and are not limitations to the present invention.

The other features and embodiments of the present invention will be clarified by explaining the exemplary embodiments in detail with reference to the following reference drawings.

The drawings herein are incorporated into the specification and form part of the specification. These drawings show examples conforming to the present invention and are used together with the specification to explain the examples of the present invention.
It is a flowchart which shows the information processing method which concerns on embodiment of this invention. It is a schematic diagram of the video image of one frame in the video stream which concerns on embodiment of this invention. It is a schematic diagram of the video image of one frame which displays the target material which concerns on embodiment of this invention. It is a schematic diagram which shows the material display method which concerns on embodiment of this invention. It is a schematic diagram which shows the material display method which concerns on embodiment of this invention. It is a schematic diagram which shows the material display method which concerns on embodiment of this invention. It is a flowchart which shows the information processing method which concerns on embodiment of this invention. It is a schematic diagram which shows the sampling method in the video search which concerns on embodiment of this invention. It is a schematic diagram of the sampling method during synchronization of the video stream information which concerns on embodiment of this invention. It is a schematic diagram which shows the timing relation which triggers the display of the advertisement material of the information processing method which concerns on embodiment of this invention. It is a schematic diagram which shows the plurality of user interfaces of the information processing method which concerns on embodiment of this invention. It is a schematic diagram which shows the plurality of user interfaces of the information processing method which concerns on embodiment of this invention. It is a schematic diagram which shows the plurality of user interfaces of the information processing method which concerns on embodiment of this invention. It is a block diagram which shows the information processing apparatus which concerns on embodiment of this invention. It is a block diagram which shows the electronic device which concerns on embodiment of this invention. It is a block diagram which shows the electronic device which concerns on embodiment of this invention.

Hereinafter, various exemplary examples, features, and embodiments of the present invention will be described in detail with reference to the drawings. The same reference numerals in the drawings indicate elements having the same or similar functions. Although the drawings show various aspects of the embodiments, it is not necessary to draw the drawings to scale unless exclusively shown.

The term "exemplary" here means "exemplary, exemplary, or descriptive." No embodiment described herein as "exemplary" is to be construed as superior or better than any other embodiment.

The term "and / or" in the text merely describes the relationship of the related object and indicates that three types of relationships can exist. For example, A and / or B can represent three cases: A exists alone, A and B exist at the same time, and B exists alone. Further, the term "at least one kind" in the text indicates any one of a plurality of kinds or any combination of at least two kinds of a plurality of kinds. For example, including at least one of A, B, and C may indicate selecting any one or more elements from the set composed of A, B, and C.

Also, in order to better explain the invention, a large amount of specific details are given in the specific embodiments below. As will be appreciated by those skilled in the art, the invention can be implemented as well without some specific details. In some embodiments, methods, means, elements and circuits well known to those of skill in the art are not described in detail so that the gist of the invention is more prominent.

FIG. 1 is a flowchart showing an information processing method according to an embodiment of the present invention, and the method is applied to an information processing apparatus. For example, the information processing device is arranged in a terminal device, plays a video stream, performs a video synchronization check, material display, and the like. Here, the terminal device is a user device (UE, User Equipment), a mobile device, a mobile phone, a wireless telephone, a personal digital assistant (PDA, Personal Digital Assistant), a handheld device, a computing device, an in-vehicle device, a wearable device, or the like. You may. In some possible implementations, the information processing method may be implemented by a method of calling computer-readable instructions stored in memory by a processor. The flow of the information processing method shown in FIG. 1 includes steps S101 to S104.

In step S101, the terminal device executes face detection on the first video image in the video stream being played, and obtains the face information of the first video image.

In one example, the video stream may include a positive film such as a television drama or movie, a short video, a variety show, etc., and the video stream may be composed of a plurality of frames of video images. A one-frame or multiple-frame video image of a video stream may contain one or more facial information, for example, a feature extraction network may be used to perform feature extraction on a first video image to perform feature extraction. After obtaining the information, clustering can be performed on the plurality of feature information, and the face information can be obtained based on the clustering result.

In step S102, the terminal device transmits the face information to the server.

In one example, to give an example of the first video image included in the video stream, if the first video image contains a plurality of face information, each independent face information is extracted first. It is sent to the server, and the server compares the face information with a predetermined face template in the face template library of the server, and obtains video stream information matched with the face information based on the comparison result. For example, search the video stream information in the cloud based on the comparison result to obtain a cloud drama that matches the video stream being played on the terminal device, and obtain the target material (advertising material, etc.) based on this. can do.

In step S103, the terminal device receives the target material transmitted by the server and obtained based on the face information.

In one example, the target material includes an advertising material, and the advertising material can be obtained through a face information comparison executed by a server.

In step S104, the terminal device triggers the display of the target material in response to the detection that the video stream has been played up to the start playback time corresponding to the target material.

In one example, the start playback time may be an advertising point displaying a target material (advertising material, etc.). The start time for playing the advertising material can be identified based on the advertising point. Trigger the display of advertising material at the advertising point. FIG. 2a is a schematic diagram showing a one-frame video image in a video stream according to an embodiment of the present invention. FIG. 2b is a schematic diagram showing a one-frame video image of the target material according to the embodiment of the present invention, wherein the video stream being reproduced seems to be shown on the positive film 11. The target material that was triggered and displayed is as shown in the advertising material 12.

In one example, the server can make the terminal device display the target material by transmitting the target material to the terminal device that plays the video stream after obtaining the target material through the face information comparison. .. After triggering the display of the target material, the information processing method further comprises displaying the target material.

In the video stream playback scenario, the above target material can be OTT advertising material. OTT advertising material is provided by a third-party service provider via the Internet (operator, etc.). Adopting the above technical solution of the present invention, "the terminal device extracts face information from the first video image of the video stream, sends it to the server for analysis, and the server performs the cloud video stream. The processing logic of "searching for the corresponding target material (advertising material, etc.) from the information, transmitting the target material to the terminal device, and executing the exhibition of the target material by the terminal device" has been realized. With this processing logic, a normal non-smart TV can be transformed into an HD smart TV. That is, the processing logic not only allows ordinary non-smart TVs to obtain resources for major video film content over the Internet (such as networks provided by operators), but also a number of third-party materials on the server. More diversified and effective information (target material, etc.) can be obtained from content providers. The processing logic executes face detection on the first video image in the video stream being played, obtains face information, and then sends the face information to a server for comparison. It is possible to obtain the corresponding target material based on the information and realize the automatic precision insertion of the OTT advertisement.

In some possible realizations, to give an example of an advertising material as a target material, the display method of the advertising material may include a method such as a corner mark, a band-aid, a banner, and FIGS. 3 to 5 are books. It is a schematic diagram which shows the target material display method which concerns on embodiment of an invention, and is as follows.

The display method 1 may display the advertising material 13 in the form of a corner mark in the corner of the video stream reproduction interface, as shown in FIG.

The display method 2 may display the advertising material 14 on the video stream playback interface in the form of a band-aid in relation to the playback content of the playback interface, as shown in FIG. For example, when key information such as "ethnic liberation" included in the reproduced content is detected, the corresponding key information or the advertising material associated with the key information is searched and searched based on the key information. The displayed advertising material is displayed on the playback interface so that the playback content can be used in response. For example, if the ad material contains the ad word "lifelong loyalty", it is associated with the key information "ethnic liberation", so the ad material is displayed on the playback interface and the playback content is used. Try to respond.

The display method 3 may display the advertising material 15 in the form of a banner over the video stream playback interface on the video stream playback interface, as shown in FIG.

In some possible implementations, the information processing method receives playback time information of the target material transmitted by the server, including reference image information corresponding to a predetermined playback time zone of the target material. It further includes detecting whether or not the video stream has been played up to the start playback time corresponding to the target material based on the playback time information. Here, the reference image information includes feature information of a plurality of frames of reference images corresponding to a predetermined reproduction time zone of the target material. The reference image of the plurality of frames is obtained by performing sampling a plurality of times for at least one time zone adjacent to the predetermined reproduction time zone.

In some possible implementations, detecting whether or not the video stream has been played up to the start play time corresponding to the target material based on the play time information is the first awaiting play of the video stream. 2 The video image is compared with the reference image information included in the playback time information to obtain a comparison result, and based on the comparison result, the video stream is played back until the start playback time corresponding to the target material. Includes determining whether or not it has been done. The second video image is played after the first video image.

In one example, comparing the second video image waiting to be played back in the video stream with the reference image information to obtain a comparison result is performed by executing a feature extraction process on the second video image. Based on the degree of similarity between obtaining the feature information of the second video image and the feature information of the second video image and the feature information of at least one frame reference image in the reference image information, the comparison result is obtained. To get and include. By comparing the feature information of the second video image with the feature information of the reference image, the similarity between the feature information can be obtained and the comparison result can be obtained. Based on the comparison result, the terminal device triggers the display of the target material at the start playback time by determining whether or not the video stream has been played until the start playback time corresponding to the target material, and the target material is displayed. After triggering the display, the target material is displayed.

In some embodiments, if the similarity between the feature information of the video image and the feature information of the reference image is greater than or equal to a predetermined threshold, it is determined that the comparison between the video image and the reference image is successful. Can be done. The predetermined threshold can be determined according to the method of determining the similarity actually used.

In some possible implementations, if the second video image is successfully compared to the first reference image in the multi-frame reference image, the video stream plays until the start playback time corresponding to the target material. Here, the previous M reference images in the reference images of the plurality of frames include the first reference image.

In some embodiments, the video image of the video stream and the multi-frame reference image can be compared to determine whether or not to trigger the display of the target material. For example, if the comparison between the second video image and the first reference image is successful, the display of the target material can be triggered. Alternatively, when the comparison between the second video image and the first reference image is successful, and the comparison between the third video image and the third reference image is successful, the display of the target material can be triggered. Here, the third video image is played after the second video image. The first reference image and the third reference image are arbitrary reference images included in the previous M reference images in the reference images of a plurality of frames, or are preset reference images. possible. The specific trigger mechanism is determined according to the actual requirements, and the embodiments of the present invention are not limited thereto.

In some embodiments, if the comparison between the second video image of the video stream and the second reference image in the multi-frame reference image is successful, it is determined that the reproduction of the target material is finished, and / Alternatively, it further includes triggering the end of reproduction of the target material, where the last N reference images in the plurality of frames of the reference image include the second reference image.

In some embodiments, the reference image information includes feature information of the multi-frame reference image related to the start playback time of the target material. In one optional example, the multi-frame reference image is obtained by performing multiple samplings for at least one time zone adjacent to a predetermined reproduction time zone. The predetermined playback time zone may include one or more predetermined time points, for example, one time point (that is, start playback time = end playback time), start playback time and end playback time, or start playback. Includes time, end replay time, and at least one time point between the two. In one optional example, the multi-frame reference image may be obtained by performing multiple samplings for the first and second time zones adjacent to the start playback time. Here, the end point of the first time zone is the start reproduction time, and the start point of the second time zone is the start reproduction time. The time in the second time zone can be the advertising time of the advertising point or the advertising time slightly larger than the advertising point, or the starting point of the second time zone is the next adjacent to the starting playback time. It can be at one point in time. In this case, selectively, in the above example, the first reference image is sampled within the first time zone. The second reference image is sampled within the second time zone, but the embodiment of the present invention is not limited thereto.

In some embodiments, the video synchronization check begins to be performed in response to the detection of successful comparison between the second video image and the first reference image. When it is detected that the comparison between the third video image in the video stream and the preset target reference image in the reference images of the plurality of frames is successful, the display of the target material is triggered.

In some possible implementations, the information processing method further comprises displaying the target material in response to the detection of an advertising material satisfying a predetermined blocking condition. Here, the predetermined blocking condition is a predetermined merchant listed in the blacklist, a predetermined advertisement type listed in the blacklist, or a source or different from the target material. Includes having at least one of the merchant's advertising material.

In one example, the server sends an ad blocking instruction to the terminal device, which is the terminal device when an ad that meets a predetermined blocking condition appears while the terminal device is playing a video stream. Instructs you to display the target material. For example, if the ad material being played in the video stream is a Nike ball shoe, the terminal device determines that the Nike ball shoe is a blacklisted merchant based on the ad blocking instructions and corners. Display target material in the form of marks, band-aids, or banners, such as ads for adidas ball shoes. In another example, if the advertising material played in the video stream is a Mercedes-Benz advertisement, it competes with the target material of the "BMW advertisement". Therefore, once discovered, the terminal device has a corner mark and a band. By displaying the BMW advertisement in the form of an aid or a banner, the accuracy of reproduction of the own advertisement material is improved. When the present invention is adopted, when an advertisement material having a predetermined blocking condition appears, an advertisement material matching with itself can be popped up to execute an accurate blocking process for the advertisement material satisfying the predetermined blocking condition. can.

FIG. 6 is a flowchart showing an information processing method according to an embodiment of the present invention, the method is applied to an information processing apparatus, the information processing apparatus is arranged in a server, video stream information retrieval and video stream reproduction. You can perform synchronization with the displayed material. As shown in FIG. 6, the flow includes steps S201 to S204.

In step S201, the server receives the face information of at least one first video image transmitted by the terminal device.

In one example, the video stream being played by the terminal device contains at least one first video image, and in the case of any one video image in the video stream, the feature extraction network is used to make the first video image. Feature extraction can be performed on one video image to obtain multiple feature information of the corresponding video image, clustering can be performed on the plurality of feature information, and face information can be obtained based on the clustering result. Can be obtained. After performing face detection processing on all the first video images of at least one first video image, the face information of at least one first video image is transmitted to the server, and the server sends the face information of the at least one first video image to the server. The face information of the image can be received, the face information can be compared, and the target material of the corresponding video stream information (cloud drama, etc.) can be obtained based on the comparison result. In one example, the cloud drama can be a positive film such as a television drama or movie, a short video, a variety show, and the like.

In step S202, the server determines the video stream information to which the at least one first video image belongs, based on the face information and a face template library including a plurality of predetermined face templates.

In one example, the person identification information corresponding to the face information can be obtained based on a plurality of predetermined face templates included in the face template library. The person identification video library can be searched to obtain video stream information to which at least one first video image belongs. For example, the video stream being played by the terminal device is a TV drama, such as "Empresses in the Palace", which is matched to the TV drama playing from the cloud drama based on the person's identity information (for example, Wallace Huo). You can search for dramas.

Here, obtaining the person identification information corresponding to the face information based on the plurality of predetermined face templates included in the face template library is the first comparison result between the face information and the plurality of predetermined face templates. It may include the determination of the above and the acquisition of the person identification information (such as Kasumi Jianhua) corresponding to the face information based on the first comparison result.

In step S203, the server acquires the target material based on the video stream information.

In one example, the video is searched based on the person identification information to obtain the video stream information to which at least one first video image belongs, and then the target material is further acquired based on the video stream information. Here, the target material includes advertising material and the like. For example, by searching the video stream information of the cloud (server) based on the person's identity information, a cloud drama (Tsukikanden episode 3) matched with the video stream being played by the terminal device is obtained, and based on this. It is possible to acquire the corresponding advertising material and the like, that is, it is possible to acquire the advertising material to be input corresponding to the "cloud computing episode 3". There are a total of 70 episodes in the cloud drama of the waiting advertisement, and 8 advertisement spots are put into each episode, and the advertisement material is put in by the operation staff. The drama cast list, drama name, drama video, drama number, cast list, etc. are also uploaded to the server in advance by the operation staff. Synchronization between the playback of the video stream and the displayed material can be performed in advance.

In step S204, the server transmits the target material to the terminal device.

In one example, when the target material (advertising material, etc.) is obtained through the above face information comparison and then the target material is transmitted to the terminal device by the server, the video stream is generated after the target material is received by the terminal device. Triggers the display of the target material in response to the detection that it has been played up to the start playback time corresponding to the target material.

When the technical solution of the present invention is adopted, the server receives the face information of at least one first video image transmitted by the terminal device, and the face information and a plurality of predetermined face templates in the face template library are combined with the face information. By comparing, the corresponding video stream information is obtained based on the comparison result, and the target material is acquired based on the video stream information. Here, when the target material is an advertisement material, the advertisement material is displayed on the terminal device by feeding back the advertisement material to the terminal device. Since the loading and display method of the target material (advertising material, etc.) is obtained by comparing the face information in the video stream being played, the target material is accurately matched to the video stream and the user. Can get more effective information related to the video stream (such as exactly the ad material you put in) in real time.

In some possible implementations, the information processing method further comprises transmitting to the terminal device the reproduction time information corresponding to the target material, wherein the reproduction time information is a predetermined of the target material. Includes reference image information corresponding to the playback time zone of. The reference image information includes feature information of a plurality of frames of the reference image corresponding to a predetermined reproduction time zone of the target material.

In some possible implementations, the information processing method performs sampling on the video stream information in at least one adjacent time zone of the predetermined playback time zone to obtain a multi-frame reference image. Further, the feature extraction process is executed on the reference image of the plurality of frames to obtain the feature information of the reference image of the plurality of frames.

In some possible embodiments, the information processing method further comprises transmitting an ad blocking instruction to the terminal device, wherein the ad blocking instruction is specific in the video stream being played by the terminal device. When it is detected that the advertising material has appeared, the terminal device is instructed to display the target material. For example, if the ad material being played in the video stream is a Nike ball shoe, the terminal device will be searched for the Nike ball shoe as a blacklisted merchant based on the ad blocking instructions. Then, the target material is displayed in the form of a corner mark, band-aid, banner, etc., and an advertisement for adidas ball shoes is displayed, for example. Also, for example, if the advertising material being played in the video stream is a Mercedes-Benz advertisement, it will compete with the target material of the "BMW advertisement", so once discovered, the terminal device will have a corner mark, band-aid, etc. Alternatively, by displaying the BMW advertisement in the form of a banner or the like, the accuracy of reproduction of the own advertisement material is improved. When the present invention is adopted, when an advertisement material having a predetermined blocking condition appears, an advertisement material matching with itself can be popped up to execute an accurate blocking process for the advertisement material satisfying the predetermined blocking condition. can.

In one example of the invention, the video stream being played by the terminal device contains at least one first video image, which contains one or more facial information. , It is necessary to detect face information. The target material can be an advertising material, and the server performs a search for cloud video stream information (such as a cloud drama) based on the face information detected and obtained by the terminal device. After determining the cloud drama of the current playback and the current playback time point, the server can send the advertising material and the reference image information to the terminal device. The terminal device performs a reference image feature information comparison and determines the start playback time based on the comparison result, thereby triggering the display of the advertisement material at the start playback time (or called an advertisement point), and further. Display the advertising material. Hereinafter, the process of video stream information retrieval, video stream information synchronization, and advertising material display trigger provided by the embodiment of the present invention will be described.

1. 1. Video Stream Information Retrieval: Includes drama search and single episode search within the drama.

In the case of drama search, it is necessary to search which cloud drama the video stream being played by the terminal device belongs to, where the cast list, drama name, drama video, drama number, cast list, etc. of the cloud drama are listed. , Can be uploaded to the cloud server in advance by the operation staff. After the terminal device starts playing the video stream, the terminal device detects and plays the video within 3 minutes by using the SDK (Software Development Kit) or APP (Upload) of the terminal device, and the video stream 1 Independent face information in a frame or a multi-frame video image can be detected as face information awaiting comparison and uploaded to the service side. The service side compares the face information waiting for comparison with the actor in the stored cloud drama, and recognizes the person identification information based on the comparison result. For example, recognize multiple actor names or drama cast lists. Therefore, it is possible to match the most probable cloud dramas (such as Empresses in the Palace) based on multiple actor names or cast lists of dramas.

In the case of a single episode search, it is necessary to confirm that the video stream being played by the terminal device belongs to a known cloud drama, and then search for which episode in the cloud drama. After detecting the cloud drama corresponding to the video stream being played by the device, one frame in the video stream at a specific frequency, such as once a minute, using the device's SDK or APP. Performs a single episode search process by uploading the video image of. FIG. 7 is a schematic diagram showing a sampling method in the video search according to the embodiment of the present invention, and as shown in FIG. 7, the server is used for all the advertising points of the cloud drama (for example, 70 episodes). Sampling can be performed. The sampling method samples one frame per second within one minute before each advertisement point, that is, it is a drama of 70 episodes, and if there are eight advertisement positions in each episode, a total of 70 * 8 * 60 = 33600 frames are sampled to obtain 33600 sampling frames, where each of these 33600 sampling frames corresponds to a plurality of predetermined playback time zones. The server compares the one frame video image uploaded by the terminal device with all the sampling frames (33600 frames) of the cloud drama, and searches the sampling frames for the one frame sampled video image most similar to these frame video images. Thereby, the single episode number (for example, episode 3 of the song ring transmission) and the reproduction time (in seconds) of the sampled video image of the frame corresponding to the video stream being reproduced are determined.

2. 2. Video stream information synchronization

In the case of video stream information synchronization, the server is used to pre-synchronize the video stream information to ensure that the terminal device triggers the display of the ad material at the synchronized ad spot. FIG. 8 is a schematic diagram of a sampling method for synchronizing video stream information according to an embodiment of the present invention. As shown in FIG. 8, in the video stream information synchronization process, the server is used before and after each advertising point. Within a total of 8 seconds, 2 frames are sampled per second to obtain 16 frames of reference images, and then feature information is extracted for each frame in these 16 frames of reference images. For example, by extracting the information of the key pixel in each frame reference image, the amount of data to be processed is compressed, and the processing efficiency of synchronization is improved. After discovering a single episode number (such as Rinkanden Episode 3) and playback time (in seconds) corresponding to the video stream information being played by the terminal device, the server will do everything after that video stream information playback point. Reference image feature information of subsequent advertising spots (for example, one episode video contains a total of 16 * 8 = 144 feature information), the time of the reference image of each frame, and the advertising material, the SDK of the terminal device or Deliver to APP. Continue to play the video stream information on the SDK or APP of the terminal device, download the currently playing video stream information locally and compare it with the reference image feature information to get the similarity with the feature information. , Get the comparison result. The terminal device triggers the display of the advertisement material at the start playback time by confirming whether or not the video stream is played up to the start playback time corresponding to the advertisement material based on the comparison result, and displays the advertisement material. Is triggered and then the advertising material is displayed.

Since the video stream information being played by the terminal device is one episode in the target drama and only one advertising spot is included in one episode, that is, there are multiple advertising spots, sampling is performed multiple times. You need to explain what you need to do. Synchronous sampling of video stream information finds all the ad spots in the episode, synchronizes all of these ad spots to the SDK or APP of the terminal device, and the video stream information on the SDK or APP of the terminal device. Continue to play, perform a reference image feature information comparison against the video stream information, get the ad spot closest to the current play point of the current video stream information, and display the corresponding ad material at the ad spot. Because. The sampling is different from the sampling during the video search described above, and the sampling here is for the reference image feature information of a specific episode in the cloud drama and the video stream information being played by the terminal device. This is because the feature comparison is performed to search for and obtain which video stream information currently being played corresponds to the display of the advertising material of which advertising spot in the episode.

3. 3. Advertising material trigger display

FIG. 9 is a schematic diagram of the relationship between time nodes that triggers advertisement display in the information processing method according to the embodiment of the present invention, and is reproduced by the SDK or APP of the terminal device as shown in FIG. When the video stream information is played up to the end of t1 and the video synchronization check is passed, in the case of SDK, the APP of the terminal device can be notified to display the advertisement content, and the APP of the terminal device can be notified. In this case as well, the advertisement content can be displayed directly via the APP of the terminal device. During the ad playback process, the SDK maintains a video sync check until the end of t2. Referring to FIG. 8, the reference image information corresponding to the nearest advertising spot includes the feature information of the 16-frame reference image, and these 16-frame reference images are the reference image of the first frame and the reference of the second frame. The image, ..., The reference image of the 6th frame, ..., The reference image of the 16th frame is included. In the process of synchronizing video stream information, we succeeded in comparing the second video image corresponding to the current playback point with the first reference image in the t1 time zone (for example, 3s before the advertisement point), and with the third frame reference image. Assuming a successful comparison (also known as the first successful comparison), the terminal device was detected to have successfully compared the third video image currently in the video stream with the reference image in frame 6. Occasionally, it confirms that it triggers the display of the ad material, and continues to run the video sync check after the first comparison is successful. In the process of the video synchronization check, when it is detected that the comparison between the 3rd video image currently in the video stream and the reference image of the 6th frame (also referred to as the target reference image above) is successful, the advertising material Trigger the display. In the process of displaying the advertising material, the terminal device continues to perform the video synchronization check and when it is detected that the 4th video image currently in the video stream is successfully compared with the reference image of the 16th frame. , Ends the display of the advertising material. In the process of displaying an advertisement, if the video stream information played by the SDK or APP of the terminal device cannot be successfully compared with at least one frame from the reference image of the 7th frame to the reference image of the 16th frame, that is, Stop displaying advertising material when it is considered that the playback content has already been switched by the terminal device. At the same time, resume video stream search and video stream information synchronization. In the embodiment, the time zone for executing the video synchronization check for the advertising spot is from the reproduction point of the second video image to the end point of t2. If all the video synchronization checks within the video synchronization check time zone are successful, the advertisement material can be played until the end of t2.

Assuming that the ad spot is the first ad spot in the video of the episode, in some embodiments, the video of the episode has a total of 40 minutes and contains 8 ad spots, 1 The second advertising spot is from the 5th minute 0th second to the 5th minute 5th second, and the second advertising spot is from the 10th minute 0th second to the 10th minute 5th second. After displaying the ad material in the first ad spot, the terminal device will start the video synchronization check of the second ad spot at 9 minutes 57 seconds. In the video synchronization check process, the frequency of comparison between the video image and the reference image can be twice per second, which is consistent with the sampling frequency of the reference image. 10 to 12 are schematic views showing an example of a plurality of user interfaces of the information processing method according to the embodiment of the present invention, and here, as shown in FIG. 10, the video stream information is reproduced by the terminal device. In the process of being processed, the playback point from which the video stream information was acquired is "00:16", and the server is requested to search for the video resource. For example, you can request to find out which cloud drama is currently playing, and specifically which episode, and even ask the server to search for ad spots. Can trigger the display of the corresponding advertising material at the advertising spot. As shown in FIG. 11, after the search for the server video is completed and the advertising material transmitted by the server is received by the terminal device, the video name and the advertising spot "00: 30-00: 35" on the terminal device. Is displayed, and at this time, the reproduction point is "00:16". If the video stream plays until 00:30 and passes the video sync check, it will start displaying the ad material. As shown in FIG. 12, the current reproduction point is displayed in the process of displaying the advertisement material, that is, in the section of the advertisement spot "00: 30-00: 35".

Those skilled in the art will appreciate that in the above method of the specific embodiment, the description order of each step does not mean a strict execution order and constitutes a restriction on the implementation process, but the specific execution order of each step. , Understand that it should be determined by its function and possible internal logic.

The embodiments of each of the above methods referred to in the present invention can be combined together to form combined embodiments without violating principles and logic, and due to space limitations, the present invention I will not explain it repeatedly.

The present invention further provides an information processing device, an electronic device, a computer-readable recording medium, and a program, all of which can realize any information processing method provided by the present invention. Applicable technical solutions can be referred to in the corresponding description in the method section and are not described repeatedly herein.

FIG. 13 is a block diagram showing an information processing apparatus according to an embodiment of the present invention, and as shown in FIG. 13, the processing apparatus has a face with respect to a first video image in a video stream being reproduced. The first detection unit 51 for executing detection to obtain the face information of the first video image, the first transmission unit 52 for transmitting the face information to the server, and the said transmission by the server. In response to the first receiving unit 53 for receiving the target material obtained based on the face information and the detection that the video stream has been played up to the start playback time corresponding to the target material. , A first response unit 54 for triggering the display of the target material. The terminal device can perform face detection on the first video image in the video stream being played and then obtain the face information of the first video image. When the target material includes the advertisement material, the advertisement material obtained by the face information comparison is transmitted to the terminal through the face information comparison on the server side, and the start playback time in which the video stream corresponds to the advertisement material by the terminal device side. In response to the detection that the advertisement material has been played up to, the display of the advertisement material is triggered to display the advertisement material.

Since the loading and display method of the material such as the advertisement material is obtained by comparing the face information in the first video image in the playback video stream, the display of the advertisement material is reproduced after the advertisement material is received by the terminal device. By accurately matching to the video stream you are playing, you can get more effective information related to the video stream (for example, exactly the ad material you put in) in real time.

In some possible embodiments, the first response unit causes the terminal device to display the target material or triggers the display of the target material by transmitting the target material to the terminal device. After that, the target material is displayed.

In some possible implementations, the display method of the target material is

1) Displaying the target material in the form of a corner mark in the corner of the video stream playback interface.

2) Displaying the target material in the form of a band-aid on the video stream playback interface in relation to the playback content of the playback interface.

3) Displaying the target material in the form of a banner over the video stream playback interface on the video stream playback interface, including at least one of the following.

In some possible embodiments, the device is for receiving reproduction time information of the target material transmitted by the server, including reference image information corresponding to a predetermined reproduction time zone of the target material. A second receiving unit and a second detection unit for detecting whether or not the video stream has been reproduced up to the start reproduction time corresponding to the target material based on the reproduction time information are further provided.

In some possible implementations, the reference image information includes feature information of a plurality of frames of the reference image corresponding to a predetermined reproduction time zone of the target material.

In some possible implementations, the multi-frame reference image is obtained by performing sampling multiple times for at least one time zone adjacent to the predetermined reproduction time zone.

In some possible implementations, the second detection unit compares the second video image awaiting playback of the video stream with the reference image information contained in the playback time information to obtain a comparison result. Based on the comparison result, it is determined whether or not the video stream has been played up to the start playback time corresponding to the target material.

In some possible implementations, the second detection unit performs feature extraction processing on the second video image to obtain feature information of the second video image and features of the second video image. A comparison result is obtained based on the degree of similarity between the information and the feature information of at least one frame reference image included in the reference image information.

In some possible implementations, if the comparison result is that the comparison result is successful in comparison with the first reference image in the plurality of frames of the reference image, the video stream is said to be said. It is confirmed that the image has been played until the start playback time corresponding to the target material.

In some possible implementations, if the comparison result is successful in comparison with the second reference image in the plurality of frames of the reference image, it is determined that the reproduction of the target material is terminated.

The previous M reference images in the plurality of frame reference images include the first reference image, and the last N reference images in the plurality of frame reference images include the second reference image. include.

In some possible implementations, the device further comprises a material display unit, which displays the target material in response to the detection of advertising material satisfying a predetermined blocking condition.

In some possible implementations, the predetermined blocking condition is

Being a blacklisted merchant and

Being a blacklisted ad type and

Includes at least one of having different sources or different merchant advertising material than the target material.

The present invention provides an information processing device, wherein the device includes a third receiving unit for receiving face information of at least one first video image transmitted by a terminal device, the face information, and a plurality of predetermined pieces. Based on a face template library containing the face template, an information determination unit for determining the video stream information to which the at least one first video image belongs, and a target material based on the video stream information. It includes a material acquisition unit for acquisition and a second transmission unit for transmitting the target material to the terminal device. Here, by including the advertising material, the target material includes a video to which at least one first video image belongs on the server side based on the comparison of the face information, for example, based on the face information and the face template library. In order to determine the stream information, the advertising material can be acquired based on the video stream information. The server sends the advertisement material obtained by comparing the face information to the terminal device, and responds that the terminal side detects that the video stream has been played up to the start playback time corresponding to the advertisement material. Then, the display of the advertisement material is triggered to display the advertisement material.

Since the loading and display method of the material such as the advertisement material is obtained by comparing the face information in the first video image in the playback video stream, the display of the advertisement material is reproduced after the advertisement material is received by the terminal device. By accurately matching to the video stream you are playing, you can get more effective information related to the video stream (for example, exactly the ad material you put in) in real time.

In some possible implementations, the information determination unit obtains person identification information corresponding to the face information based on a plurality of predetermined face templates included in the face template library, and obtains the person identification information corresponding to the face information. Based on, the video library is searched to obtain video stream information to which the at least one first video image belongs.

In some possible implementations, the information determination unit determines a first comparison result between the face information and the plurality of predetermined face templates, and responds to the face information based on the first comparison result. Get the identity information of the person to do.

In some possible embodiments, the apparatus further comprises a third transmission unit for transmitting reproduction time information corresponding to the target material to the terminal device, wherein the reproduction time information is the target. Includes reference image information corresponding to a given playback time zone of the material.

In some possible implementations, the reference image information includes feature information of a plurality of frames of the reference image corresponding to a predetermined reproduction time zone of the target material.

In some possible embodiments, the device performs sampling on the video stream information in at least one adjacent time zone of the predetermined playback time zone to obtain a multi-frame reference image. Further includes a unit and a feature extraction unit for executing a feature extraction process on the reference image of the plurality of frames to obtain feature information of the reference image of the plurality of frames.

In some possible embodiments, the device further comprises a fourth transmission unit for transmitting an ad blocking instruction to the terminal device, wherein the ad blocking instruction is in a video stream being played by the terminal device. When it is detected that a specific advertising material has appeared, the terminal device is instructed to display the target material.

In some embodiments, the features or modules included in the apparatus provided by the embodiments of the present invention are capable of performing the methods described in the embodiments of the above methods and the specific implementation thereof. Can refer to the description of the embodiment of the above method and will not be repeated here for the sake of brevity.

The embodiments of the present invention further provide a computer-readable recording medium in which computer program instructions are stored, and when the computer program instructions are executed by a processor, the above method is realized. The computer-readable recording medium can be a volatile computer-readable recording medium or a non-volatile computer-readable recording medium.

An embodiment of the present invention further provides a computer program product comprising a computer readable code, the information processing method provided by the above embodiment by the processor in the device when the computer readable code is operated on the device. The instruction to realize it is executed.

The embodiments of the present invention further provide another computer program product for storing computer readable instructions, the information processing method provided by the computer according to any of the above embodiments when the instructions are executed. To execute the operation of.

The computer program product can be specifically realized by a method of hardware, software, or a combination of both. In one example of the option, the computer program product is embodied as a computer recording medium, and in another example of the option, the computer program product is, for example, a software development kit (SDK). It can be embodied as a software product.

An embodiment of the invention further provides an electronic device, wherein the electronic device comprises a processor and a memory for storing instructions that can be executed by the processor, wherein the processor comprises the above method. Configured to run.

The electronic device may be provided as a terminal device, a server, or other form of device.

FIG. 14 is a block diagram of the electronic device 800 according to the embodiment of the present invention. For example, the electronic device 800 may be a terminal device such as a mobile phone, a computer, a digital broadcasting terminal, an information transmitting / receiving device, a game console, a tablet device, a medical device, a fitness device, or a personal digital assistant.

Referring to FIG. 14, the electronic device 800 includes processing component 802, memory 804, power supply component 806, multimedia component 808, audio component 810, input / output (I / O) interface 812, sensor component 814 and communication component 816. It may include one or more of these components.

The processing component 802 typically controls operations related to general operations of the electronic device 800, such as display, telephone calls, data communications, camera operations and recording operations. The processing component 802 may include one or more processors 820 that execute instructions to perform all or part of the steps of the above method. Also, the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and the other components. For example, the processing component 802 may include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

The memory 804 is configured to store each type of data to support operations on the electronic device 800. Examples of these data include instructions of any application program or method for operating on the electronic device 800, contact data, telephone directory data, messages, pictures, video and the like. The memory 804 may include any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory. (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk may be realized.

The power supply component 806 supplies power to various components of the electronic device 800. The power component 806 may include a power management system, one or more power sources, as well as other components for generating, managing, and distributing power for the electronic device 800.

The multimedia component 808 includes a screen that supplies one output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). When the screen includes a touch panel, it may be realized as a touch screen so as to receive an input signal from the user. The touch panel includes one or more touch sensors to sense touches, slides and gestures on the touch panel. The touch sensor can not only sense the boundaries of the touch or slide motion, but also further detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes one front camera head and / or a back camera head. When the electronic device 800 is in an operating mode, eg, shooting mode or video mode, the front camera head and / or the back camera head can receive external multimedia data. Each of the front camera head and the back camera head may be one fixed optical lens system or may have focal length and optical zoom capability.

The audio component 810 is configured to output and / or input an audio signal. For example, the audio component 810 comprises a microphone (MIC), and the microphone is configured to receive an external audio signal when the electronic device 800 is in operating mode, eg, calling mode, recording mode and voice recognition mode. To. The received audio signal can be further stored in memory 804 or transmitted via the communication component 816. In some embodiments, the audio component 810 further includes a speaker for outputting an audio signal.

The I / O interface 812 supplies an interface between the processing component 802 and the peripheral interface module, and the peripheral interface module may be a keyboard, a click wheel, a button, or the like. These buttons may include, but are not limited to, a home page button, a volume button, a start button and a lock button.

The sensor component 814 includes one or more sensors for supplying the electronic device 800 with state estimation in various aspects. For example, the sensor component 814 can detect the on / off state of the electronic device 800 and the relative position of the component, for example, the component is a display and a numeric keypad of the electronic device 800. The sensor component 814 also further repositions the electronic device 800 or the component of the electronic device 800, the presence or absence of contact between the user and the electronic device 800, the orientation or acceleration / deceleration of the electronic device 800, and the temperature change of the electronic device 800. Can be further detected. Sensor component 814 may include a proximity sensor to detect the presence of nearby objects in the absence of any physical contact. The sensor component 814 may further include an optical sensor used for imaging applications, such as a CMOS or CCD image sensor. In some embodiments, the sensor component 814 may further include an accelerometer, gyro sensor, magnetic sensor, pressure sensor or temperature sensor.

The communication component 816 is configured to facilitate wireless or wired communication between the electronic device 800 and other user equipment. The electronic device 800 can access a wireless network based on a communication standard, for example, WiFi, 2G or 3G, or a combination thereof. In one exemplary embodiment, the communication component 816 receives broadcast signals or broadcast-related notification information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication component 816 may further include a Near Field Communication (NFC) module to facilitate short range communication. For example, NFC modules may be implemented by radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth® (BT) technology and other technologies.

In an exemplary embodiment, the electronic device 800 is one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic for performing the above methods. It may be implemented by a device (PLD), a field programmable gate array (FPGA), a controller, a microcontroller, a microprocessor or other electronic component.

An exemplary embodiment further provides a non-volatile computer readable storage medium, eg, a memory 804 containing computer program instructions. The computer program instruction may be executed by the processor 820 of the electronic device 800 to carry out the method.

FIG. 15 is a block diagram of the electronic device 900 according to the embodiment of the present invention. For example, the electronic device 900 may be provided as a server. Referring to FIG. 7, the electronic device 900 includes a processing component 922, which further comprises one or more processors and a memory resource typified by the memory 932, the memory 932 being processed. Stores instructions that can be executed by component 922, such as application programs. The application program stored in memory 932 may include one or more modules, each corresponding to a set of instructions. Further, the processing component 922 is configured to execute the above method by executing an instruction.

The electronic device 900 includes a power component 926 configured to perform power management of the electronic device 900, a wired or wireless network interface 950 configured to connect the electronic device 900 to a network, and inputs / outputs (I). / O) An interface 958 may be further provided. The electronic device 900 may operate an operating system stored in memory 932, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, or the like.

An exemplary embodiment further provides a non-volatile computer readable storage medium, eg, a memory 932 containing computer program instructions. The method can be implemented by executing the computer program instruction by the processing component 922 of the electronic device 900.

The present invention provides systems, methods and / or computer program products. Computer program products may include computer readable storage media. The computer-readable storage medium contains computer-readable program instructions for causing the processor to perform each aspect of the present invention.

The computer-readable storage medium may be a tangible device that maintains and stores the instructions used by the instruction executing device. The computer-readable storage medium may be, for example, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination described above, but is not limited thereto. More specific examples of computer-readable storage media (non-exhaustive list) are portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory). ), Static random access memory (SRAM), portable compressed disk read-only memory (CD-ROM), digital multifunction disk (DVD), memory stick, floppy disk, mechanical coding device, for example, instructions are stored. Includes punch cards or in-groove protrusion structures, and any suitable union as described above. The computer-readable storage medium used herein is an instantaneous signal itself, eg, a radio wave or other freely propagating electromagnetic wave, a waveguide or an electromagnetic wave propagating through another transmission medium (eg, an optical fiber cable). It is not interpreted as an optical signal transmitted through an optical pulse) or an electric wire.

The computer-readable program instructions described herein are downloaded from a computer-readable storage medium to each arithmetic / processing device, or via a network such as the Internet, local area network, wide area network and / or wireless network to an external computer or external. You may download it to a storage device. The network may include copper transmission cables, fiber optic transmissions, wireless transmissions, routers, firewalls, switches, gateway computers and / or edge servers. The network adapter or network interface in each arithmetic / processing device receives a computer-readable program instruction from the network and transfers the computer-readable program instruction to store the computer-readable program instruction in the computer-readable storage medium in each arithmetic / processing apparatus.

The computer program instructions for performing the operations of the present invention are assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcodes, firmware instructions, state setting data, or one or more types. It may be source code or target code created by any combination of programming languages. The programming language includes an object-oriented programming language (eg, Smalltalk, C ++, etc.) and a conventional process programming language (eg, "C" wording) or a similar programming language. Computer-readable program instructions may be executed entirely on the user computer, partially on the user computer, or as one independent software package, some on the user computer. The other part may be executed on the remote computer, or may be entirely executed on the remote computer or the server. In the case of a remote computer, the remote computer may be connected to the user computer or to an external computer via any type of network (including local area network (LAN) or wide area network (WAN)). (For example, connect to the Internet using an Internet service provider). In some embodiments, electronic circuits such as programmable logic circuits, field programmable gate arrays (FPGAs) or programmable logic arrays (PLAs) are personalized and customized using state information from computer-readable program instructions. The electronic circuit can implement each aspect of the present invention by executing computer-readable program instructions.

Here, each aspect of the present invention has been described with reference to the flowchart and / or block diagram of the method, apparatus (system) and computer program product according to the embodiment of the present invention. As can be understood, each block of the flowchart and / or the block diagram and each combination of the blocks in the flowchart and / or the block diagram can be realized by a computer-readable program instruction.

These computer-readable program instructions can be fed to the processor of a general purpose computer, a dedicated computer or other programmable data processing device to create a single machine. Thus, a device that, when executed by the processor of a computer or other programmable data processing device, realizes the function / operation specified by one or more blocks in the flowchart and / or block diagram. , Generated. These computer-readable program instructions may be stored in a computer-readable storage medium. With these instructions, computers, programmable data processing equipment and / or other equipment operate in a particular manner. As described above, the computer-readable medium in which the instruction is stored includes one manufactured product, which is an instruction of each aspect that realizes the function / operation specified by one or more blocks in the flowchart and / or the block diagram. including.

Computer-readable program instructions may be loaded into a computer, other programmable data processor, or other equipment. In this way, a series of operational steps are performed on a computer, other programmable data processor or other device, and the process of realization by the computer is created. Thereby, an instruction executed on a computer, other programmable data processing device, or other device realizes a function / operation defined by one or more blocks in a flowchart and / or a block diagram.

Flow charts and block diagrams in the drawings show feasible systematic architectures, functions and operations of systems, methods and computer program products according to a plurality of embodiments of the present invention. In this regard, each block in the flowchart or block diagram may represent part of a module, program segment or instruction. A part of the module, program segment or instruction includes one or more executable instructions for performing a defined logical function. In some alternative embodiments, the functions marked in the block may occur in a different order than shown in the drawings. For example, two consecutive blocks may, in fact, be executed essentially in parallel, and may be executed in reverse order, depending on such function. It should be noted that each block in the block diagram and / or flowchart, and the combination of blocks in the block diagram and / or flowchart, is realized in a system based on dedicated hardware that performs a specified function or operation. It may be realized by a combination of dedicated hardware and computer instructions.

If the logic is not violated, different embodiments of the invention can be combined with each other, emphasizing the description of the different embodiments. See the records of other examples for some of the explanations.

In the above, each embodiment of the present invention has been described. The above description is exemplary, not exhaustive, and is not limited to each disclosed embodiment. Many modifications and changes will be apparent to those skilled in the art, provided that they do not deviate from the scope and spirit of each of the embodiments described. The terms in the text have been selected to best interpret the principles of each embodiment, actual applications, or technological improvements to technology in the market, or to allow one of ordinary skill in the art to understand each embodiment disclosed in the text. To.

Claims (21)

It is an information processing method
Performing face detection on the first video image in the video stream being played to obtain the face information of the first video image,
Sending the face information to the server and
Receiving the target material transmitted by the server and obtained based on the face information, and
An information processing method comprising: triggering the display of the target material in response to the detection that the video stream has been played up to the start playback time corresponding to the target material. Receiving the reproduction time information including the reference image information corresponding to the predetermined reproduction time zone of the target material of the target material transmitted by the server.
The information processing method according to claim 1, further comprising detecting whether or not the video stream has been played up to the start playback time corresponding to the target material based on the playback time information. .. The information processing method according to claim 2, wherein the reference image information includes feature information of a plurality of frames of reference images corresponding to a predetermined reproduction time zone of the target material. The information according to claim 3, wherein the reference image of the plurality of frames is obtained by performing sampling a plurality of times for at least one time zone adjacent to the predetermined reproduction time zone. Processing method. It is possible to detect whether or not the video stream has been played up to the start playback time corresponding to the target material based on the playback time information.
The second video image waiting to be played back in the video stream, and the second video image played after the first video image is played back, is compared with the reference image information to obtain a comparison result. When,
One of claims 2 to 4, wherein, based on the comparison result, it is determined whether or not the video stream has been played up to the start playback time corresponding to the target material. The information processing method described in the section. Based on the comparison result, it is possible to determine whether or not the video stream has been played up to the start playback time corresponding to the target material.
If the comparison result is that the comparison with the first reference image in the reference image of the plurality of frames is successful, it is determined that the video stream has been played up to the start playback time corresponding to the target material. Including,
The information processing method according to claim 5, wherein the previous M reference images in the plurality of frames of the reference images include the first reference image. When the comparison result is that the comparison with the second reference image in the reference image of the plurality of frames is successful, it further includes confirming that the reproduction of the target material is finished.
The information processing method according to claim 5, wherein the last N reference images in the plurality of frames of the reference images include the second reference image. The information processing method according to any one of claims 1 to 7, further comprising displaying the target material in response to the detection of an advertising material satisfying a predetermined blocking condition. .. It is an information processing method
Receiving the face information of at least one first video image transmitted by the terminal device,
Determining the video stream information to which the at least one first video image belongs, based on the face information and a face template library containing a plurality of predetermined face templates.
Acquiring the target material based on the video stream information,
An information processing method comprising transmitting the target material to the terminal device. Determining the video stream information to which the at least one first video image belongs is based on the face information and face template library.
Obtaining the person identification information corresponding to the face information based on a plurality of predetermined face templates included in the face template library, and
The information processing according to claim 9, wherein the information processing according to claim 9, comprising searching a video library based on the person identification information to obtain video stream information to which the at least one first video image belongs. Method. Further including transmitting the reproduction time information corresponding to the target material to the terminal device.
The information processing method according to claim 9 or 10, wherein the reproduction time information includes reference image information corresponding to a predetermined reproduction time zone of the target material. The information processing method according to claim 11, wherein the reference image information includes feature information of a plurality of frames of reference images corresponding to a predetermined reproduction time zone of the target material. Sampling the video stream information in at least one adjacent time zone of the predetermined playback time zone to obtain a reference image of a plurality of frames.
The information processing method according to claim 11 or 12, further comprising performing a feature extraction process on the plurality of frames of the reference image to obtain feature information of the plurality of frames of the reference image. .. Further including sending an ad blocking instruction to the terminal device,
The advertisement blocking instruction is characterized in that when it is detected that a specific advertisement material appears in a video stream being played by the terminal device, the terminal device instructs the terminal device to display the target material. The information processing method according to any one of claims 9 to 13. It is an information processing device
A first detection unit for performing face detection on a first video image in a video stream being played and obtaining face information of the first video image, and
The first transmission unit for transmitting the face information to the server,
A first receiving unit for receiving the target material transmitted by the server and obtained based on the face information, and
It is characterized by comprising a first response unit for triggering the display of the target material in response to the detection that the video stream has been played up to the start playback time corresponding to the target material. Information processing device. A second receiving unit for receiving reproduction time information including reference image information corresponding to a predetermined reproduction time zone of the target material of the target material transmitted by the server, and
15. The information processing device described. The information processing apparatus according to claim 16, wherein the reference image information includes feature information of a plurality of frames of reference images corresponding to a predetermined reproduction time zone of the target material. The information according to claim 17, wherein the reference image of the plurality of frames is obtained by performing sampling a plurality of times for at least one time zone adjacent to the predetermined reproduction time zone. Processing equipment. It is an information processing device
A third receiving unit for receiving the face information of at least one first video image transmitted by the terminal device, and
An information determination unit for determining the video stream information to which the at least one first video image belongs, based on the face information and a face template library including a plurality of predetermined face templates.
A material acquisition unit for acquiring the target material based on the video stream information, and
An information processing device including a second transmission unit for transmitting the target material to the terminal device. It ’s an electronic device,
With the processor
With memory for storing instructions that can be executed by the processor,
An electronic device, wherein the processor is configured to perform the method of any one of claims 1-8 and 9-14. A computer-readable recording medium that stores computer program instructions.
A computer-readable recording medium, wherein when the computer program instruction is executed by a processor, the method according to any one of claims 1 to 8 and claims 9 to 14 is realized.

Images