WO2015196749A1

WO2015196749A1 - Scenario recognition-based information recommendation method and device

Info

Publication number: WO2015196749A1
Application number: PCT/CN2014/093401
Authority: WO
Inventors: 孔建华; 朱兴昌
Original assignee: 中兴通讯股份有限公司
Priority date: 2014-06-27
Filing date: 2014-12-09
Publication date: 2015-12-30
Also published as: CN105227973A

Abstract

Disclosed are a scenario recognition-based information recommendation method and device. The scenario recognition-based information recommendation method comprises: matching a video stream with a preset scenario feature to obtain a scenario feature corresponding to the video stream; according to a correlation between the scenario feature and recommended information, acquiring the recommended information corresponding to the video stream; and presenting the recommended information corresponding to the video stream. By means of the present invention, the information recommendation accuracy is increased, so that information recommendation can be realized according to a video scenario.

Description

Information recommendation method and device based on scene recognition

Technical field

The present invention relates to the field of communications, and in particular to a method and apparatus for recommending information based on scene recognition.

Background technique

Currently, in the Internet Protocol Television (IPTV)/Over The Top TV (OTT-TV) system, traditional business recommendations (such as advertising, shopping) are either The service recommendation link is directly configured on the Electronic Program Guide (EPG) interface for recommendation, and the user can enter the specific page to view the recommendation information; or the recommendation information is pushed to the EPG interface through some message systems, and the scheduled display is performed. To the end user.

However, these business recommendations are relatively independent, operating as a stand-alone business form, and may be pushed by some users with no interest in recommendation information.

Summary of the invention

In view of the problem of inaccurate information recommendation in the related art, the present invention provides a method and device for recommending information based on scene recognition to solve at least the above problems.

According to an embodiment of the present invention, a method for information recommendation based on scene recognition is provided, including: matching a video stream with a preset scene feature to obtain a scene feature corresponding to the video stream; and according to the scene feature and the recommendation information Corresponding relationship, obtaining recommendation information corresponding to the video stream.

Preferably, the matching the video stream with the preset scene feature comprises: encoding the video stream to obtain a feature object to be matched; and matching the to-be-matched feature object with the scene feature object to obtain the video stream. Corresponding scene feature, wherein the scene feature object is obtained by encoding the preset scene feature in advance.

Preferably, matching the to-be-matched feature object with the scene feature object to obtain the scene feature corresponding to the video stream includes: matching the to-be-matched feature object with the scene feature object; if the to-be-matched The similarity between the feature object and the scene feature object satisfies a preset condition, and the scene feature corresponding to the scene feature object is used as the scene feature corresponding to the video stream.

Preferably, encoding the preset scene feature in advance comprises: encoding the preset scene feature according to a scene mode corresponding to the preset scene feature.

Preferably, the scene mode of the preset scene feature comprises at least one of the following: a video scene, a voice scene, a subtitle scene, and a scenario scene.

Preferably, the format of the preset scene feature includes at least one of the following: a picture, a text, and a voice.

Preferably, the preset scene feature includes: a first scene feature and/or a second scene feature, wherein the first scene feature is a general scene feature, and the second scene feature is the video stream Dedicated scene features.

Preferably, the presenting the recommendation information corresponding to the video stream comprises: presenting the recommendation information corresponding to the video stream at a preset time.

Preferably, the method further includes: presenting recommendation information corresponding to the video stream.

Preferably, the video stream is a cached video stream to be played, and the recommendation information corresponding to the video stream is: when the video stream is played, the recommended information corresponding to the video stream is presented.

Preferably, the recommendation information is set with a recommendation identification code, and the recommendation identification code is used to distinguish recommendation information; the correspondence between the scene feature and the recommendation information is a correspondence between the scene feature and the recommendation identification code.

According to still another embodiment of the present invention, an information recommendation device based on scene recognition is provided, including: a matching module, configured to match a video stream with a preset scene feature to obtain a scene feature corresponding to the video stream; The obtaining module is configured to obtain the recommendation information corresponding to the video stream according to the corresponding relationship between the scene feature and the recommendation information.

Preferably, the matching module includes: an encoding unit configured to encode the video stream to obtain a feature object to be matched; and a matching unit configured to match the to-be-matched feature object with a scene feature object to obtain a The scene feature corresponding to the video stream, wherein the scene feature object is obtained by encoding the preset scene feature in advance.

Preferably, the coding unit is further configured to perform coding on the preset scene feature according to the scene mode corresponding to the preset scene feature, to obtain the scene feature object.

Preferably, the presentation module is configured to present recommendation information corresponding to the video stream at a preset time.

Preferably, the method further includes: a presentation module, configured to present recommendation information corresponding to the video stream

Preferably, the video stream is a cached video stream to be played, and the presentation module is configured to present recommendation information corresponding to the video stream when the video stream is played.

According to the present invention, the video stream is matched with the preset scene feature to obtain the scene feature corresponding to the video stream; the recommendation information corresponding to the video stream is obtained according to the corresponding relationship between the scene feature and the recommendation information; and the recommendation information corresponding to the video stream is presented. , based on the video scene can be more accurate recommendation information.

DRAWINGS

The drawings described herein are intended to provide a further understanding of the invention, and are intended to be a part of the invention. In the drawing:

1 is a flowchart 1 of a method for recommending information based on scene recognition according to an embodiment of the present invention;

2 is a structural block diagram of an information recommendation apparatus based on scene recognition according to an embodiment of the present invention;

3 is a second flowchart of a method for recommending information based on scene recognition according to an embodiment of the present invention;

4 is a structural block diagram 1 of a first device according to an embodiment of the present invention;

FIG. 5 is a structural block diagram 2 of a first device according to an embodiment of the present invention; FIG.

6 is a flowchart 3 of a method for recommending information based on scene recognition according to an embodiment of the present invention;

7 is a structural block diagram 1 of a second device according to an embodiment of the present invention;

FIG. 8 is a structural block diagram 2 of a second device according to an embodiment of the present invention; FIG.

9 is a schematic diagram of a preferred scene recognition based information recommendation system according to an embodiment of the present invention;

10 is a structural block diagram of a preferred scene recognition based information recommendation system according to an embodiment of the present invention;

FIG. 11 is a flowchart of scene feature delivery according to an embodiment of the present invention; FIG.

FIG. 12 is a flowchart of scene feature generation according to an embodiment of the present invention; FIG.

FIG. 13 is a flowchart of scene feature matching according to an embodiment of the present invention; FIG.

14 is a flow chart of information recommendation according to an embodiment of the present invention;

15 is a flowchart of an information recommendation method according to Embodiment 1 of the present invention;

16 is a flowchart of an information recommendation method according to Embodiment 2 of the present invention;

17 is a flowchart of an information recommendation method according to Embodiment 3 of the present invention;

Figure 18 is a flowchart of an information recommendation method according to Embodiment 4 of the present invention.

detailed description

The invention will be described in detail below with reference to the drawings in conjunction with the embodiments. It should be noted that the embodiments in the present application and the features in the embodiments may be combined with each other without conflict.

In the following embodiments, the first device may be various types of devices, such as a set top box, a mobile terminal, a home server, a router, and the like. The second device may be a server, a cluster with servers, or the like. It is to be expected that the methods and apparatus of the following embodiments can be implemented by a computer program.

FIG. 1 is a flowchart 1 of a method for recommending information based on scene recognition according to an embodiment of the present invention. As shown in FIG. 1, the method includes steps S102 to S104.

In step S102, the video stream is matched with the preset scene feature to obtain a scene feature corresponding to the video stream.

Step S104: Acquire recommendation information corresponding to the video stream according to the correspondence between the scene feature and the recommendation information.

In the embodiment of the present invention, the recommendation information corresponding to the video stream may also be presented.

In the embodiment of the present invention, the scene mode of the preset scene feature may include at least one of the following: a video scene, a voice scene, a subtitle scene, and a scenario scene. The format of the preset scene feature may include at least one of the following: a picture, a text, and a voice.

In the embodiment of the present invention, the scene feature is related to the content of the video, and may be information such as an image, a dialogue, a plot, an actor, a movie release time, or a theme included in the video content. When the user chooses to watch a certain video, the scene of the video content is effectively and intelligently recognized, and then the interactive recommendation based on the images, dialogues, plots, actors and the like included in the video content can well solve the recommended targeting. , the relevance problem, enhance the user's interaction, and truly achieve specific services such as recommending advertisements and shopping according to the user's manipulation of a certain content.

In an embodiment of the present invention, in the foregoing step S102, the video stream may be encoded to obtain a feature object to be matched, and the feature object to be matched is matched with the scene feature object to obtain a scene feature corresponding to the video stream, where The scene feature object is obtained by pre-coding the preset scene features. Of course, different coding methods are used, and different coding modes are used, and the coding may not be performed.

In the embodiment of the present invention, the preset feature features are encoded, and the scene features in the form of pictures and voice texts may be encoded to obtain a vector that is convenient for comparison. Correspondingly, the video stream is encoded by extracting images, subtitles, voices, etc. from the video stream, and encoding the information into corresponding vectors.

In an embodiment of the present invention, in step S102, the feature object to be matched may be matched with the scene feature object; if the similarity between the feature object to be matched and the scene feature object satisfies a preset condition, the scene feature object is correspondingly The scene feature is used as the scene feature corresponding to the video stream.

In an embodiment of the present invention, the preset scene feature is encoded in advance, and the preset scene feature may be encoded according to the preset scene mode corresponding to the scene feature. For example, different encoding methods are used for images and speech.

In an embodiment of the present invention, the preset scene feature may include: a first scene feature and/or a second scene feature, where the first scene feature is a general scene feature, and the second scene feature is a video stream. Dedicated scene features. The above-mentioned general scene features may be scene features applicable to all videos, or may be scene features applicable to a type of video. The dedicated scene features described above may be scene features set for a single video, or may be scene features set for certain videos.

In the embodiment of the present invention, the recommendation information corresponding to the presentation video stream may be a link for displaying the recommendation information on the screen, or may directly display the recommendation information.

In an implementation manner of the embodiment of the present invention, the presentation time of the recommendation information may be set, and the recommendation information corresponding to the video stream is presented at a preset time.

In an embodiment of the present invention, the video stream may be a cached video stream to be played, and the recommended information corresponding to the video stream may be presented when the video stream is played. For example, performing scene matching on the cached video stream to be played, and obtaining recommendation information corresponding to the video stream, and presenting the recommendation information when the video is in the private room. For example, in the cached video stream, when a conference scene appears in a certain period of time, the conference-related recommendation information, such as office equipment advertisement, conference hotel information, etc., may be presented while playing the video of the time period.

In an embodiment of the present invention, the recommendation information may be set with the recommendation identifier, and the recommendation identifier is used to distinguish the recommendation information; the correspondence between the scene feature and the recommendation information is the correspondence between the scene feature and the recommendation identifier.

2 is a structural block diagram of an information recommendation device based on scene recognition according to an embodiment of the present invention. As shown in FIG. 2, the device mainly includes: a matching module 10 and an obtaining module 20. The matching module 10 is configured to match the video stream with the preset scene feature to obtain a scene feature corresponding to the video stream. The obtaining module 20 is configured to obtain the recommendation information corresponding to the video stream according to the corresponding relationship between the scene feature and the recommendation information.

In an embodiment of the present invention, the matching module 10 includes: an encoding unit configured to encode a video stream to obtain a feature object to be matched; and a matching unit configured to match the to-be-matched feature object with the scene feature object Obtaining a scene feature corresponding to the video stream, wherein the scene feature object is obtained by encoding the preset scene feature in advance.

In an embodiment of the present invention, the coding unit is further configured to perform coding on a preset scene feature according to a preset scene mode corresponding to the scene feature, to obtain a scene feature object.

In an embodiment of the present invention, the scene mode of the preset scene feature includes at least one of the following: a video scene, a voice scene, a subtitle scene, and a scenario scene.

In an embodiment of the embodiment of the present invention, the format of the preset scene feature includes at least one of the following: a picture, a text, and a voice.

In an embodiment of the present invention, the preset scene feature includes: a first scene feature and/or a second scene feature, where the first scene feature is a common scene feature, and the second scene feature is a video stream. Dedicated scene features.

In an embodiment of the present invention, the method may further include: a presentation module, configured to present recommendation information corresponding to the video stream.

In an embodiment of the embodiment of the present invention, the presentation module is configured to present recommendation information corresponding to the video stream at a preset time.

In an embodiment of the present invention, the video stream is a cached video stream to be played, and the presentation module is configured to present the recommended information corresponding to the video stream when the video stream is played.

In the embodiment of the present invention, the recommendation information may be set with a recommendation identification code, and the recommendation identification code is used to distinguish the recommendation information; the correspondence between the scene feature and the recommendation information is a correspondence between the scene feature and the recommendation identification code.

The foregoing method and apparatus of the embodiments of the present invention may be implemented in one device or may be implemented through multiple devices. The modules of the above devices may be located in one device or distributed on different devices. This embodiment of the present invention does not limit this.

FIG. 3 is a second flowchart of a method for recommending information based on scene recognition according to an embodiment of the present invention. As shown in FIG. 3, the method includes steps S302 to S306.

Step S302: The first device matches the video stream with the preset scene feature to obtain a scene feature corresponding to the video stream.

Step S304: The first device acquires the recommendation information corresponding to the video stream from the second device according to the scene feature corresponding to the video stream.

Step S306, the first device presents the recommendation information corresponding to the video stream.

In the embodiment of the present invention, the foregoing first device may be a device such as a set top box or a mobile terminal. In the foregoing step S302, the first device may locally match the video stream with preset scene features. In addition, the first device may perform the foregoing step S302 by using the second device, where the first device may encode the video stream, obtain the feature object to be matched, send the feature object to be matched to the second device, and receive the second device according to the to-be-matched The feature object matches the scene feature corresponding to the obtained video stream. By implementing matching on the second device, the performance requirements of the first device can be reduced, and the first device does not need to perform complicated processing.

In an embodiment of the present invention, the first device performing local matching may include: the first device encodes the video stream to obtain a feature object to be matched; and the first device matches the to-be-matched feature object with the scene feature object. Obtaining a scene feature corresponding to the video stream, wherein the scene feature object is obtained by pre-coding the preset scene feature. For the coding, reference may be made to the foregoing description of the embodiments of the present invention, and details are not described herein.

Before the step S302, the first device may acquire a preset scene feature from the second device, where the preset scene feature includes a first scene feature and/or a second scene feature, where the first scene feature is a common scene feature The second scene feature is a dedicated scene feature of the video stream.

The first device may send a request message to the second device, where the request message carries a first update time of a pre-stored scene feature in the first device, where the first device receives the preset scene feature. And the second device sends the preset scene feature in the case that the first update is not later than the second update time of the scene feature on the second device. By carrying the update time, the information interaction between the first device and the second device can be reduced, and if the scene feature is updated, the updated scene feature is sent.

In a preferred embodiment of the present invention, the first device pre-stores the foregoing first scene feature, and the first device may acquire the second scene feature from the second device. The foregoing first scenario may be that the first device requests to be sent from another device, or the other device may be actively configured to the first device, and the other device may include the foregoing second device.

In an embodiment of the present invention, in the foregoing step S304, the first device may send the scenario feature corresponding to the video stream to the second device, and receive the recommendation information corresponding to the video stream sent by the second device, where the second device The device is configured to obtain the recommendation information corresponding to the video stream according to the corresponding relationship between the scene feature and the recommendation information.

In another embodiment of the present invention, in the foregoing step S304, the first device may determine, according to the correspondence between the scene feature and the recommendation identifier of the recommendation information, the recommended identifier corresponding to the scene feature corresponding to the video stream, and Sending the determined recommendation identifier to the second device, and receiving the recommendation information corresponding to the video stream sent by the second device, where the second device is configured to obtain the recommendation information corresponding to the video stream according to the correspondence between the recommendation identifier and the recommendation information, where The recommended identification code is used to identify the recommended information.

In the embodiment of the present invention, in step S306, the first device may present the recommendation information according to a preset time. The above video stream may be a cached video to be played, and the recommended information may be presented when the video stream is played. In addition, the recommendation information may be presented on the first device, or the first device may present the recommendation information through other devices. For example, in the case where the first device is a set top box, the first device can present recommendation information on the television terminal of the set top box.

FIG. 4 is a structural block diagram of a first device according to an embodiment of the present invention. As shown in FIG. 4, the first device mainly includes: a matching module 402, a first obtaining module 404, and a rendering module 406. The matching module 402 is configured to match the video stream with the preset scene feature to obtain a scene feature corresponding to the video stream. The first obtaining module 404 is connected to the matching module 402 and configured to be based on the scene feature corresponding to the video stream. The recommendation information corresponding to the video stream is obtained from the second device; and the presentation module 406 is connected to the first acquisition module 404 and configured to present the recommendation information corresponding to the video stream.

In an embodiment of the present invention, the matching module 402 is configured to match the video stream with the preset scene feature on the first device; and/or, the matching module 402 may include: a coding unit, set to be The video stream is encoded to obtain a feature object to be matched; the sending unit is configured to send the feature object to be matched to the second device, and the receiving unit is configured to receive the scene feature corresponding to the video stream obtained by the second device according to the matching feature object to be matched.

In an embodiment of the present invention, the first device may further include: a second acquiring module, configured to acquire a preset scene feature from the second device, where the preset scene feature includes the first scene feature and And a second scene feature, wherein the first scene feature is a general scene feature and the second scene feature is a dedicated scene feature of the video stream.

In an embodiment of the present invention, the second obtaining module includes: a sending unit, configured to send a request message to the second device, where the request message carries a feature of the scene feature pre-stored in the first device An update time; the receiving unit is configured to receive the second device, and send the preset scene feature in a case that the first update time is not later than the second update time of the scene feature on the second device.

In the embodiment of the present invention, the first device may pre-store the first scene feature, and the second acquiring module is configured to acquire the second scene feature from the second device.

The first acquiring module 404 may include: a first acquiring unit, configured to send a scene feature corresponding to the video stream to the second device, and receive recommendation information corresponding to the video stream sent by the second device, where the second device is configured according to Corresponding relationship between the scene feature and the recommendation information acquires recommendation information corresponding to the video stream. Or the second obtaining unit is configured to: according to the correspondence between the scene feature and the recommendation identifier of the recommendation information, determine a recommendation identifier corresponding to the scene feature corresponding to the video stream, and send the determined recommendation identifier to the second device, and receive The recommendation information corresponding to the video stream sent by the second device, wherein the second device is configured to obtain the recommendation information corresponding to the video stream according to the correspondence between the recommendation identifier and the recommendation information, where the recommendation identifier is used to identify the recommendation information.

5 is a structural block diagram 2 of a first device according to an embodiment of the present invention. As shown in FIG. 2, the first device includes: a communication interface 502 configured to communicate with a second device and at least one third device; at least one process The device 504 is configured to match the video stream with the preset scene feature to obtain a scene feature corresponding to the video stream, and obtain, according to the scene feature corresponding to the video stream, the recommendation information corresponding to the video stream from the second device; The recommended information corresponding to the video stream.

The above-mentioned method of the embodiment of the present invention may be performed by the at least one processor 504, and details are not described herein again.

FIG. 6 is a flowchart 3 of a method for recommending information based on scene recognition according to an embodiment of the present invention. As shown in FIG. 6, the method includes steps S602 to S604.

Step S602: The second device, in response to the first request of the first device, matches the video stream requested by the first device with a preset scene feature, to obtain a scene feature corresponding to the video stream.

Step S604: The second device sends the recommendation information corresponding to the video stream to the first device according to the corresponding feature of the scene and the recommendation information according to the corresponding feature of the video stream.

In an embodiment of the present invention, in the foregoing step S602, the first request carries a to-be-matched feature object of the video stream, where the to-be-matched feature object is encoded by the first device.

In the above step S604, the second device may send the recommendation information corresponding to the video stream to the first device according to the corresponding relationship between the scene feature and the recommendation information; or the second device may be based on the correspondence between the scene feature and the recommendation information. Acquiring the recommendation information corresponding to the video stream; the second device sends the recommended identifier of the obtained recommendation information to the first device, where the recommendation identifier is used to distinguish the recommendation information.

In an embodiment of the present invention, the method may further include: the second device responding to the second request of the first device, and sending, according to the recommended identifier carried in the second request, the carried identifier code to the first device. Recommended information. In the embodiment of the present invention, when receiving the multiple recommended identifiers, the first device may request the second device to obtain the recommended information, or may receive the recommended identifier to obtain the corresponding recommended information.

In an embodiment of the present invention, the method may further include: responding to the third request of the first device by the second device, according to the first update time carried in the third request and the feature of the scene on the second device The second update time is sent to the first device, where the first update time is an update time of the scene feature stored on the first device; wherein the first update time is earlier than the second update time, and the second device is The first device sends a scene feature.

In the embodiment of the present invention, the preset scene feature may include: a first scene feature and/or a second scene feature, where the first scene feature is a general scene feature, and the second scene feature is a dedicated scene feature of the video stream. . For the first scene feature and the second scene feature, refer to the foregoing description of the embodiments of the present invention, and details are not described herein again.

Preferably, the third message is used to request the second scene feature, wherein the first device stores the first scene feature in advance.

FIG. 7 is a structural block diagram 1 of a second device, including: a matching module 702 and a first sending module 704, according to an embodiment of the present invention. The matching module 702 is configured to: in response to the first request of the first device, match the video stream requested by the first device with the preset scene feature to obtain a scene feature corresponding to the video stream; the first sending module 704, and the matching The module 702 is connected to be configured to send the recommendation information corresponding to the video stream to the first device according to the corresponding relationship between the scene feature and the recommendation information according to the corresponding feature of the video stream.

In the embodiment of the present invention, the first request carries the to-be-matched feature object of the video stream, where the to-be-matched feature object is obtained by encoding the video stream by the first device.

In an embodiment of the present invention, the first sending module 704 may include: a first sending unit, configured to send the recommended information corresponding to the video stream to the first device according to the corresponding relationship between the scene feature and the recommendation information; or The second sending unit is configured to acquire the recommended information corresponding to the video stream according to the corresponding relationship between the scene feature and the recommendation information, and send the recommended identifier of the obtained recommended information to the first device, where the recommended identifier is used to distinguish the recommended information.

In an embodiment of the present invention, the second device may further include: a second sending module, configured to send the carried to the first device according to the second identifier that is sent in response to the second request The recommended information corresponding to the identification code.

In an embodiment of the present invention, the second device may further include: a third sending module, configured to respond to the third request of the first device, according to the first update time carried in the third request, and the second device The second update time of the scene feature is sent to the first device, where the first update time is an update time of the scene feature stored on the first device; wherein, if the first update time is earlier than the second The update time, the third sending module sends the scene feature to the first device.

In the embodiment of the present invention, the preset scene feature includes: a first scene feature and/or a second scene feature, where the first scene feature is a common scene feature, and the second scene feature is a dedicated scene feature of the video stream. . For the first scenario feature and the second scenario feature, refer to the foregoing description of the embodiments of the present invention, and details are not described herein again.

Preferably, the third message is used to request the second scene feature, wherein the first device may pre-store the first scene feature.

FIG. 8 is a block diagram showing the structure of a second device according to an embodiment of the present invention. As shown in FIG. 8, the second device may include: a communication interface 802 connected to at least one processor 804, configured to be associated with the first device and at least a third device communication; the at least one processor 804 is configured to: in response to the first request of the first device, match the video stream requested by the first device with a preset scene feature, to obtain a scene feature corresponding to the video stream; The stream corresponds to the scene feature, and the recommendation information corresponding to the video stream is sent to the first device according to the corresponding relationship between the scene feature and the recommendation information.

The foregoing method of the embodiment of the present invention may be performed by at least one processor 804, and details are not described herein again.

According to the embodiment of the present invention, the video stream is matched with the preset scene feature to obtain the scene feature corresponding to the video stream; and the recommendation information corresponding to the video stream is obtained according to the corresponding relationship between the scene feature and the recommendation information; and the video stream corresponding to the video stream is presented. Recommended information, based on the video scene can be more accurate recommendation information.

Preferred embodiments of the embodiments of the present invention are described below.

9 is a schematic diagram of a preferred scene recognition based information recommendation system, as shown in FIG. 9, in accordance with an embodiment of the present invention. The system mainly includes: a recommendation server (corresponding to the second device described above) 1 , which is responsible for presetting the scene mode and scene feature information (also referred to as a scene feature in the above embodiment), and giving a specific service recommendation according to the recommendation identification code. Form; set top box (equivalent to the first device described above) 2, responsible for generating and storing scene feature information, code stream buffering and identification, feature matching and recommendation identification code generation; IPTV/OTT system 3, mainly providing basic video playback Function; third-party system 4, mainly provides specific presentation of various recommended services.

The above-mentioned system mainly includes the following interaction process: the scenario is sent, and the recommendation server 1 delivers the scene feature according to the preset scene mode; the scenario generation, the set top box 2 encodes the scene feature, and generates the generated scene feature object and recommendation recognition. The code is stored in the scene feature library; the scene recognition 2, the set top box 2 performs code stream buffering when playing the video from the IPTV/OTT system 3, pre-analyzes the cached code stream, and analyzes the result to perform intelligent matching. If the matching is successful, the recommended feature is output. The service recommendation, the set top box 2 initiates a request to the recommendation server according to the recommended identification code, and displays the recommendation page according to the returned recommendation information, thereby redirecting to the third party system.

For example, when the user selects to watch a certain video, the scene of the video content in the IPTV system is effectively and intelligently recognized, and then the interactive recommendation based on the images, dialogues, plots, actors and the like included in the video content is good. Solve the targeted and relevant issues of recommendation, enhance the user's interaction, and truly achieve specific services such as recommending advertisements and shopping according to the user's manipulation of a certain content.

FIG. 10 is a structural block diagram of a preferred scene recognition based information recommendation system according to an embodiment of the present invention. As shown in FIG. 10, the system includes a set top box 2, a recommendation server 1, an IPTV/OTT system 3, and a third party system 4. The set top box 2 includes a scene generation module 210, a scene feature library 220, a code stream identification module 230, a feature matching module 240, and a service recommendation module 250.

(1) Maintaining scene feature information in the recommendation server, such as a video scene, a voice scene, a subtitle scene, a scenario scene, etc., wherein the video scene is mainly a picture information of actors, costumes, articles, foods, etc. that need to be matched in the recorded video; The scene is mainly the keywords in the dialogue and the narration that need to be matched in the recorded video; the subtitle scene is mainly the subtitle keyword that needs to be matched in the recorded video, and the scenario of the scenario is mainly the preset plot time point that needs to be matched in the recorded video;

(2) A recommendation identifier is bound to each scene feature information in the recommendation server to establish an association relationship between the scene feature and the recommendation identifier; and the relationship between the recommendation identifier and the recommendation information is maintained for identification of the recommendation information;

(3) The scene generation module in the set top box accepts scene feature information (including the recommendation identification code) from the recommendation server, and encodes and generates specific semantics according to different scene modes, such as a video scene, a voice scene, a subtitle scene, and a scenario scene. Scene feature objects and save to the scene feature library;

(4) The code stream identification module in the set top box is responsible for pre-resolving the code stream in the cache to generate a feature object to be matched;

(5) The feature matching module in the set top box performs the operation matching between the feature object to be matched and the scene feature database. If the feature to be matched includes a certain scene feature in the scene feature database, the corresponding identification code corresponding to the scene feature is obtained.

(6) After obtaining the recommended identification code, the service recommendation module in the set top box is responsible for requesting the recommendation server to obtain the recommendation information, and displaying the recommendation information to the user in a predetermined format, and the user can jump to the third party system by displaying the recommendation information. To obtain more detailed business information and complete the entire process of business recommendation.

Based on the implementation principle shown in FIG. 1 , the method for implementing service recommendation based on scene intelligent identification in the embodiment of the present invention includes the following steps:

Step A: The server is recommended to maintain and deliver the scene features.

Step B: The set top box receives the scene features from the recommendation server to generate scene features.

Step C: The set top box receives the video stream from the IPTV/OTT system and performs scene recognition matching.

Step D: The set top box generates recommendation information according to the matching result.

FIG. 11 is a flowchart of scene feature delivery according to an embodiment of the present invention, which is executed in the above step A, and the implementation steps are as follows:

Step S1102: The scene information of the scene is included in the recommendation server, including but not limited to a video scene, a voice scene, a subtitle scene, a scenario scene, and the like; the scene feature information may be bound to a specific video, or may be a full video adaptation.

Step S1104: The service recommendation information is entered in the recommendation server, and a unique recommendation identification code is generated for each service recommendation information.

Step S1106: The recommendation server binds the scene feature information and the service recommendation information, and the binding relationship is established by the recommendation identification code, so that bidirectional access can be implemented.

Step S1108: Before the set-top box reads the video, the set-top box obtains the request for the common scene feature information from the recommendation server, and carries the user information and the timestamp. The behavior can be obtained according to the actual demand and is obtained when the set-top box is started up to reduce the viewing. Frequently acquired repeated scene feature information before the video.

Step S1110: The recommendation server detects the validity of the user. If the detection is passed and the latest timestamp on the server is updated with the comparison of the timestamp field, the video feature information and the recommendation identifier are sent to the set-top box, and the delivered protocol can be based on Not limited to the HTTP protocol.

Step S1112: The set-top box obtains a request for the specific video scene feature information from the recommendation server before playing the video, and carries the user information, the content number, and the delivery time stamp.

Step S1114: The recommendation server detects the validity of the user. If the detection is passed and the latest timestamp on the server is updated with the comparison of the timestamp field, the specific video scene feature information and the recommended identification code are sent to the set top box, and the issued protocol may be Based on, but not limited to, the HTTP protocol.

FIG. 12 is a flowchart of scene feature generation according to an embodiment of the present invention, which is executed in the above step B, and the implementation steps are as follows:

Step S1202: The set top box receives the scene feature information from the recommendation server.

Step S1204: The scene generation module of the set top box encodes the scene feature information according to the scene mode, and generates a scene feature object of a specific semantic.

Step S1206: The scene generation module of the set top box establishes an association relationship between the scene feature object and the recommendation identifier.

Step S1208: The set top box stores the generated scene feature object and the association relationship with the recommended identification code into the scene feature database, and stores the time stamp.

FIG. 13 is a flowchart of scene feature matching according to an embodiment of the present invention, which is executed in the above step C, and the implementation steps are as follows:

Step S1302: The set top box plays the video code stream from the IPTV/OTT system, and buffers the code stream.

Step S1304: The code stream identification module of the set top box identifies the cached code stream, and can identify each frame of the video, a piece of voice, a subtitle, and a play time, and encode to form a specific semantics. Match the feature object and pass it to the feature matching module. Among them, voice and subtitle mainly form text, the playing time is mainly absolute time and relative time, the absolute time format adopts year, month, day, hour, minute and second, and the relative time format adopts hour and minute.

Step S1306: The feature matching template in the set top box performs an operation matching operation between the to-be-matched feature object and the scene feature library.

Step S1308: If the feature to be matched includes a certain scene feature in the scene feature library, such as the feature included in the image and the image feature object in the scene feature library reach a threshold, the voice and subtitle in the code stream are included in the scene feature library. Corresponding to the text keyword, if the preset story time point arrives, the matching is successful, then step S1305 is performed, and if the matching fails, the processing is directly discarded.

Step S1310: The feature matching template acquires the corresponding feature identifier of the scene feature from the scene feature database.

Step S1312: At the same time, the feature matching template records the playing time of the video code stream, and sets the recommended triggering time to the service recommendation module, and prepares to trigger the service recommendation process.

FIG. 14 is a flowchart of information recommendation according to an embodiment of the present invention, which is performed in the above step D, and the implementation steps are as follows:

Step S1402: The service recommendation module in the set top box determines whether the recommended trigger time arrives; if yes, proceeds to step S1404.

Step S1404: The service recommendation module assembly request message in the set top box initiates a request to the recommendation server, where the request message carries the user information and the recommendation identification code.

Step S1406: After receiving the request of the set top box, the recommendation server detects the validity of the user. If the detection passes, the specific recommendation information is returned according to the recommended identification code, and generally includes the interface display information and the URL thereof.

Step S1408: After receiving the recommendation information, the service recommendation module in the set top box parses and assembles according to a predetermined template format.

Step S1410: After the service recommendation information is parsed and assembled, the service recommendation page is formed, and the service recommendation page is displayed to the user.

Step S1412: After viewing the recommendation page, the user may select whether to jump from the recommendation page to the third-party system for shopping or the like according to his or her hobbies.

In order to facilitate a better understanding of the aspects and principles of the embodiments of the present invention, the present invention will be described in detail by way of several specific embodiments.

Example 1

Based on the local scene recognition of the set top box, the scene generation module, the scene feature library, the code stream identification module, the feature matching library, and the service recommendation module of the set top box in the schematic diagram are all built in the set top box to perform scene recognition matching, as shown in FIG. The steps are as follows:

Step S1502: The user clicks on the video, and the set top box requests the recommendation server to deliver the scene feature.

Step S1504: The set top box performs scene feature coding and stores it in the scene feature library.

Step S1506: When the user watches the video, the set top box requests the video in the IPTV/OTT system and receives the video code stream.

Step S1508: The set top box caches the code stream, and identifies and codes the cache code stream to generate an identifiable feature object to be matched.

Step S1510: The set top box identifies and matches the to-be-matched feature object and the feature matching library, and obtains the recommended identification code.

Step S1512: The set top box carries the recommended identification code to obtain recommendation information from the recommendation server.

Step S1514: The set top box parses the recommendation information returned by the recommendation server and presents it to the user.

Example 2

Based on the cloud-based scene recognition, the scene generation module, the scene feature library, and the feature matching module of the set top box are moved to the recommendation server in the schematic diagram, and the recommendation server is used as the scene recognition processing in the cloud, and the code stream identification module and the service recommendation module continue to be placed. The report and recommendation request are reported in the set top box feature, as shown in FIG. 16, and the steps include:

Step S1602: The recommendation server encodes the scene features and stores them in the scene feature library.

Step S1604: When the user watches the video, the set top box requests the video in the IPTV/OTT system and receives the video code stream.

Step S1606: The set top box caches the code stream, and identifies and codes the cache code stream to generate an identifiable feature object to be matched.

Step S1608: The set top box sends the to-be-matched feature object to the recommendation server for cloud matching.

Step S1610: The recommendation server identifies and matches the to-be-matched feature object and the feature matching library, and obtains the recommended identification code.

Step S1612: The recommendation server returns the recommendation identification code to the set top box.

Step S1614: The set top box carries the recommended identification code to obtain recommendation information from the recommendation server.

Step S1616: The set top box parses the recommendation information returned by the recommendation server and presents it to the user.

Example 3

Based on the scene recognition of the mobile terminal (such as a mobile phone, a Pad, etc.), all the corresponding modules in the set top box can be transplanted to the mobile terminal to implement scene recognition and service recommendation of the mobile terminal. The mobile terminal is mainly accessed through the wireless network and is connected to the Wifi mode. The process of the specific embodiment can be performed by referring to the first embodiment, and the scenario is matched in the mobile terminal; when accessing the 3G/4G mode, more needs to be considered. The network interaction and the data traffic are reduced. Therefore, in combination with the cloud processing mode of the embodiment 2, the video segment is identified on the cloud server side, and all the recommended identification codes of the video segment are returned after the matching is identified, as shown in FIG. :

Step S1702: The recommendation server encodes the scene features and stores them in the scene feature library.

Step S1704: When the user watches the video, the mobile terminal requests the video in the IPTV/OTT system and receives the video code stream.

Step S1706: The mobile terminal initiates a feature matching request to the recommendation server, where the request is for a video code stream time period.

Step S1708: The recommendation server performs matching identification on the requested video code stream time period in the cloud, and generates multiple recommendation identification codes.

Step S1710: The recommendation server returns multiple recommendation identification codes to the mobile terminal.

Step S1712: The mobile terminal records each recommendation identification code and its recommended trigger time.

Step S1714: After the recommended trigger time arrives, the mobile terminal carries the recommended identification code to obtain recommendation information from the recommendation server.

Step S1716: The mobile terminal parses the recommendation information returned by the recommendation server and presents it to the user.

Step S1718: The mobile terminal starts the feature matching request of the next video stream according to the video viewing progress, and repeats step S1706.

Example 4

Based on the scene recognition mode of the video pre-processing mode, this mode mainly performs matching matching on the video in advance in the cloud. The main advantage is that the matching is fast, the requirements on the terminal are not high, and the versatility is good, because the video is released when the IPTV/OTT system is released. All the scene matching has been pre-processed, and the recommended identifier of the video and the recommended trigger time have been generated. However, due to the pre-processing, it is impossible to pay attention to the matching of the personality characteristics of the user. As shown in Figure 18, the steps include:

Step S1802: The recommendation server performs batch pre-processing on the video in the IPTV/OTT system, matches the scene feature library, and identifies the videos one by one.

Step S1804: The recommendation identifier generated by the recommendation server according to the result of the matching identification and the recommended triggering time are stored in correspondence with the video.

Step S1806: When the user watches the video, the set top box mobile terminal requests the video in the IPTV/OTT system and receives the video code stream.

Step S1808: The set top box/mobile terminal initiates a feature matching request to the recommendation server.

Step S1810: The recommendation server queries the stored pre-processing matching result, and if it hits, returns the result.

Step S1812: The recommendation server returns all the recommended identification codes of the video to the set top box/mobile terminal.

Step S1814: The set top box/mobile terminal records each recommendation identification code and its recommended trigger time.

Step S1816: After the recommended trigger time arrives, the set top box/mobile terminal carries the recommended identification code to obtain recommendation information from the recommendation server.

Step S1818: The mobile terminal parses the recommendation information returned by the recommendation server and presents it to the user.

From the above description, it can be seen that the present invention achieves the following technical effects to solve the existing information and service recommendation information and association and interaction in the video, and the plurality of video sharing service recommendation information, so that the user can watch During the video process, relevant recommendation information is obtained as the video scene changes, the user's interaction is enhanced, and the pertinence and versatility of the service recommendation are enhanced; in the IPTV/OTT system, the intelligent recognition based on the video content is The field of business recommendation has universal practical application significance.

It will be apparent to those skilled in the art that the various modules or steps of the present invention described above can be implemented by a general-purpose computing device that can be centralized on a single computing device or distributed across a network of multiple computing devices. Alternatively, they may be implemented by program code executable by the computing device, thereby Storing them in a storage device is performed by a computing device, and in some cases, the steps shown or described may be performed in an order different than that herein, or separately fabricated into individual integrated circuit modules, or Multiple of these modules or steps are fabricated as a single integrated circuit module. Thus, the invention is not limited to any specific combination of hardware and software.

The above description is only the preferred embodiment of the present invention, and is not intended to limit the present invention, and various modifications and changes can be made to the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and scope of the present invention are intended to be included within the scope of the present invention.

Industrial applicability

According to the foregoing technical solution provided by the embodiment of the present invention, the video stream is matched with the preset scene feature to obtain a scene feature corresponding to the video stream; and the recommendation information corresponding to the video stream is obtained according to the corresponding relationship between the scene feature and the recommendation information; The recommendation information corresponding to the video stream is presented, and the information can be more accurately recommended based on the video scene.

Claims

A method for recommending information based on scene recognition, comprising:

Matching a video stream with a preset scene feature to obtain a scene feature corresponding to the video stream;

The recommendation information corresponding to the video stream is obtained according to the correspondence between the scene feature and the recommendation information.
The method of claim 1, wherein matching the video stream with preset scene features comprises:

Encoding the video stream to obtain a feature object to be matched;

Matching the to-be-matched feature object with the scene feature object to obtain a scene feature corresponding to the video stream, wherein the scene feature object is obtained by encoding the preset scene feature in advance.
The method of claim 2, wherein the matching the feature object to be matched with the scene feature object to obtain the scene feature corresponding to the video stream comprises:

Matching the to-be-matched feature object with the scene feature object;

If the similarity between the to-be-matched feature object and the scene feature object meets a preset condition, the scene feature corresponding to the scene feature object is used as the scene feature corresponding to the video stream.
The method of claim 2, wherein pre-coding the preset scene feature comprises:

And classifying the preset scene feature according to the scene mode corresponding to the preset scene feature.
The method according to claim 1 or 4, wherein the scene mode of the preset scene feature comprises at least one of the following: a video scene, a voice scene, a subtitle scene, and a scenario scene.
The method according to claim 1, wherein the format of the preset scene feature comprises at least one of the following: a picture, a text, a voice.
The method according to claim 1, wherein the preset scene feature comprises: a first scene feature and/or a second scene feature, wherein the first scene feature is a general scene feature, and the second The scene feature is a dedicated scene feature of the video stream.
The method of claim 1, further comprising: presenting recommendation information corresponding to the video stream.
The method of claim 1, wherein the video stream is a cached video stream to be played, and the presenting the recommended information corresponding to the video stream comprises: when the video stream is played, presenting the video stream Recommended information.
The method according to claim 1, wherein the recommendation information is provided with a recommendation identification code, wherein the recommendation identification code is used to distinguish recommendation information; and the corresponding relationship between the scene feature and the recommendation information is a scene feature and a recommendation identifier. Correspondence relationship.
An information recommendation device based on scene recognition, comprising:

a matching module, configured to match a video stream with a preset scene feature to obtain a scene feature corresponding to the video stream;

The obtaining module is configured to obtain the recommendation information corresponding to the video stream according to the corresponding relationship between the scene feature and the recommendation information.
The apparatus of claim 11, wherein the matching module comprises:

a coding unit, configured to encode the video stream to obtain a feature object to be matched;

The matching unit is configured to match the to-be-matched feature object with the scene feature object to obtain a scene feature corresponding to the video stream, where the scene feature object is obtained by encoding the preset scene feature in advance.
The device according to claim 12, wherein the encoding unit is further configured to perform encoding on the preset scene feature according to a scene mode corresponding to the preset scene feature, to obtain the scene feature object. .
The apparatus of claim 11, further comprising: a presentation module configured to present recommendation information corresponding to the video stream.