CN104754413A - Image search based television signal identification and information recommendation method and device - Google Patents

Image search based television signal identification and information recommendation method and device Download PDF

Info

Publication number
CN104754413A
CN104754413A CN201310746405.5A CN201310746405A CN104754413A CN 104754413 A CN104754413 A CN 104754413A CN 201310746405 A CN201310746405 A CN 201310746405A CN 104754413 A CN104754413 A CN 104754413A
Authority
CN
China
Prior art keywords
information
user
image frame
image
picture
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201310746405.5A
Other languages
Chinese (zh)
Other versions
CN104754413B (en
Inventor
文伟
史晓宇
窦则胜
曹宇宁
张文文
王炎
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Samsung Telecom R&D Center
Beijing Samsung Telecommunications Technology Research Co Ltd
Samsung Electronics Co Ltd
Original Assignee
Beijing Samsung Telecommunications Technology Research Co Ltd
Samsung Electronics Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Samsung Telecommunications Technology Research Co Ltd, Samsung Electronics Co Ltd filed Critical Beijing Samsung Telecommunications Technology Research Co Ltd
Priority to CN201310746405.5A priority Critical patent/CN104754413B/en
Publication of CN104754413A publication Critical patent/CN104754413A/en
Application granted granted Critical
Publication of CN104754413B publication Critical patent/CN104754413B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/45—Management operations performed by the client for facilitating the reception of or the interaction with the content or administrating data related to the end-user or to the client device itself, e.g. learning user preferences for recommending movies, resolving scheduling conflicts
    • H04N21/466—Learning process for intelligent management, e.g. learning user preferences for recommending movies
    • H04N21/4668—Learning process for intelligent management, e.g. learning user preferences for recommending movies for recommending content, e.g. movies
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
    • H04N21/258—Client or end-user data management, e.g. managing client capabilities, user preferences or demographics, processing of multiple end-users preferences to derive collaborative data
    • H04N21/25866—Management of end-user data
    • H04N21/25891—Management of end-user data being end-user preferences
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/45—Management operations performed by the client for facilitating the reception of or the interaction with the content or administrating data related to the end-user or to the client device itself, e.g. learning user preferences for recommending movies, resolving scheduling conflicts
    • H04N21/466—Learning process for intelligent management, e.g. learning user preferences for recommending movies
    • H04N21/4667—Processing of monitored end-user data, e.g. trend analysis based on the log file of viewer selections
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
    • H04N21/4728—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for selecting a Region Of Interest [ROI], e.g. for requesting a higher resolution version of a selected region
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/478—Supplemental services, e.g. displaying phone caller identification, shopping application
    • H04N21/4788—Supplemental services, e.g. displaying phone caller identification, shopping application communicating with other users, e.g. chatting

Landscapes

  • Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • General Engineering & Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Computer Graphics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

The invention provides an image search based television signal identification and information recommendation method. The image search based television signal identification and information recommendation method comprises the following steps of determining a currently-played picture according to a television program played on a television of a user; conducting image analysis on the picture and determining image features corresponding to the picture; searching relevant information of the picture according to the image features and determining the television program viewed by the user currently; analyzing the relevant information and recommending the relevant information of the television program played currently for the user on the television. By means of the scheme, the information of the television program being viewed by the user is obtained from mass video program data through image analysis on the television programs in a searching mode. After the relevant information of the program viewed by the user is obtained, the relevant information can be further recommended for the user.

Description

Based on picture search identification TV signal and the method and apparatus of recommendation information
Technical field
The present invention relates to multimedia technology field, specifically, the present invention relates to based on picture search identification TV signal and the method and apparatus of recommendation information.
Background technology
In recent years, along with the develop rapidly of electronic industry and the communication technology, the new business development based on data, speech, video is swift and violent.The fast development of microelectric technique, computer hardware technique, the work become increasingly complex for terminal equipment process lays the foundation, for terminal equipment personalization provide realize may, terminal is broken away from the to a certain extent restriction of network, can possess function from strength to strength.In addition, user itself also has urgent demand to terminal, wish termination function more powerful, more flexibly, simpler and more direct.The development of information technology, makes terminal technology move towards intellectuality, mobile, multifunction.
Such as, along with popularizing gradually of Intelligent television terminal, television terminal will develop into the main source of the acquisition of information of many people gradually.TV user is no longer simple meeting unidirectional reception and watch TV programme, and user also expects the acquisition of active participate TV programme relevant information.At present, television operator does not also have a set of perfect in user behavior, the solution of intelligently pushing user interest information.
In TV identification field, the existing identification for TV programme and search technique are mainly through identifying the station symbol of TV station, again in conjunction with other relevant informations, such as TV guide, the GPS region location information of user and temporal information etc., the television channel that comprehensive acquisition active user is watching and TV programme.Along with the development of intelligent television and universal, existing TV retrieval technique, owing to depending on TV station's station symbol information and other third party's information unduly, cannot meet the demand that user is various.Especially use program request or recorded broadcast function user, or when the multiple station symbol of live middle appearance, prior art has no way of obtaining the information relevant to TV programme at all.
And in video search field more widely, existing technology is mainly for Internet video or other video datas.Internet video is for same video, larger image deformation or other conversion can't be produced, and the design of TV program information due to Set Top Box and the difference of user habit, often various change is produced to same video, also make prior art no matter in accuracy or on search efficiency, all can not meet the demand of intelligent television application.
Therefore, be necessary to propose the more rational scheme that information is provided based on video image, to solve the problem that user is difficult to obtaining information from video image.
Summary of the invention
The embodiment of the present invention provides a kind of based on picture search identification TV signal and the method and apparatus of recommendation information, by analyzing the picture image of TV, thereafter intelligently for user provides the information be associated.
For solving in prior art the one or more defects existed, the embodiment of the present invention proposes a kind of based on picture search identification TV signal and the method for recommendation information on the one hand, comprises the following steps:
According to the TV programme that user plays, determine the image frame of current broadcasting on TV;
Graphical analysis is carried out to described image frame, determines the characteristics of image that described image frame is corresponding;
According to described characteristics of image, search for the information be associated with described image frame, determine the TV programme that active user watches;
The information be associated described in analysis, described TV recommends the information relevant to the TV programme of current broadcasting to user.
The embodiment of the present invention also proposed on the other hand a kind of based on picture search identification TV signal and the equipment of recommendation information, comprising:
Select module, for the TV programme play according to user, determine the image frame of current broadcasting on TV;
Analysis module, for carrying out graphical analysis to described image frame, determines the characteristics of image that described image frame is corresponding;
Retrieval module, for according to described characteristics of image, searches for the information be associated with described image frame, determines the TV programme that active user watches;
Recommending module, for the information be associated described in analyzing, described TV recommends the information relevant to the TV programme of current broadcasting to user.
The such scheme that the present invention proposes, to go forward side by side the mode of line search by carrying out graphical analysis to TV programme, can obtain the video program information that user is watching in massive video program data.Unlike previous technologies be, the technical scheme that the present invention proposes carries out Treatment Analysis to the image of each frame of video frequency program, without the need to specific mark, such as TV station symbol information, by the image processing algorithm of design, in the data with various image deformation, the program that active user watches can be located at a high speed accurately.After the relevant information obtaining the program that user watches, the information of being correlated with can be recommended to user further.
Accompanying drawing explanation
The present invention above-mentioned and/or additional aspect and advantage will become obvious and easy understand from the following description of the accompanying drawings of embodiments, wherein:
Fig. 1 for implement functional schematic of the present invention in television system;
Fig. 2 to provide the method flow diagram of information for the embodiment of the present invention based on video image;
Fig. 3 is the flow chart carrying out analyzing retrieval to image frame;
Fig. 4 to provide the device structure schematic diagram of information for the embodiment of the present invention based on video image;
Fig. 5 for use system flow chart of the present invention in television system;
Fig. 6 is for providing the schematic diagram of information during movie;
Fig. 7 is for providing the schematic diagram of information during playing video advertisement;
The schematic diagram of Fig. 8 for providing based on the user request information information of carrying out;
Fig. 9 is the schematic diagram setting up intelligent social network.
Embodiment
Be described below in detail embodiments of the invention, the example of described embodiment is shown in the drawings, and wherein same or similar label represents same or similar element or has element that is identical or similar functions from start to finish.Being exemplary below by the embodiment be described with reference to the drawings, only for explaining the present invention, and can not limitation of the present invention being interpreted as.
Those skilled in the art of the present technique are appreciated that unless expressly stated, and singulative used herein " ", " one ", " described " and " being somebody's turn to do " also can comprise plural form.Should be further understood that, the wording used in specification of the present invention " comprises " and refers to there is described feature, integer, step, operation, element and/or assembly, but does not get rid of and exist or add other features one or more, integer, step, operation, element, assembly and/or their group.Should be appreciated that, when we claim element to be " connected " or " coupling " to another element time, it can be directly connected or coupled to other elements, or also can there is intermediary element.In addition, " connection " used herein or " coupling " can comprise wireless connections or wirelessly to couple.Wording "and/or" used herein comprises one or more whole or arbitrary unit listing item be associated and all combinations.
Those skilled in the art of the present technique are appreciated that unless otherwise defined, and all terms used herein (comprising technical term and scientific terminology), have the meaning identical with the general understanding of the those of ordinary skill in field belonging to the present invention.It should also be understood that, those terms defined in such as general dictionary, should be understood to that there is the meaning consistent with the meaning in the context of prior art, unless and by specific definitions as here, otherwise can not explain by idealized or too formal implication.
Those skilled in the art of the present technique are appreciated that, here used " terminal ", " terminal equipment " had both comprised the equipment of wireless signal receiver, it only possesses the equipment of the wireless signal receiver without emissivities, comprise again the equipment receiving and launch hardware, it has and on bidirectional communication link, can carry out the reception of two-way communication and launch the equipment of hardware.This equipment can comprise: honeycomb or other communication equipments, its honeycomb or other communication equipment of having single line display or multi-line display or not having multi-line display; PCS(Personal Communications Service, PCS Personal Communications System), it can combine voice, data processing, fax and/or its communication ability; PDA(PersonalDigital Assistant, personal digital assistant), it can comprise radio frequency receiver, beep-pager, the Internet/intranet access, web browser, notepad, calendar and/or GPS(Global PositioningSystem, global positioning system) receiver; Conventional laptop and/or palmtop computer or other equipment, it has and/or comprises the conventional laptop of radio frequency receiver and/or palmtop computer or other equipment.Here used " terminal ", " terminal equipment " can be portable, can transport, be arranged in the vehicles (aviation, sea-freight and/or land), or be suitable for and/or be configured at local runtime, and/or with distribution form, any other position operating in the earth and/or space is run.Here used " terminal ", " terminal equipment " can also be communication terminal, access terminals, music/video playback terminal, can be such as PDA, MID(Mobile Internet Device, mobile internet device) and/or there is the mobile phone of music/video playing function, also can be the equipment such as intelligent television, Set Top Box.
In ensuing introduction, the section Example mode that the present invention proposes can be described for the video of televising.Should be appreciated that technical scheme disclosed by the invention is equally applicable to other video techniques.Therefore, the technical scheme that information is provided based on video image that the present invention proposes, blanket technical scheme when being a kind of process video image, is described can not be interpreted as limitation of the present invention for the video of televising, just in order to set forth the present invention better.
In order to the object of real the present invention, the embodiment of the present invention proposes a kind of based on picture search identification TV signal and the method for recommendation information, comprises the following steps:
According to the TV programme that user plays, determine the image frame of current broadcasting on TV;
Graphical analysis is carried out to described image frame, determines the characteristics of image that described image frame is corresponding;
According to described characteristics of image, search for the information be associated with described image frame, determine the TV programme that active user watches;
The information be associated described in analysis, described TV recommends the information relevant to the TV programme of current broadcasting to user.
The said method that the present invention proposes, to go forward side by side the mode of line search by carrying out graphical analysis to TV programme, can obtain the video program information that user is watching in massive video program data.Unlike previous technologies be, the technical scheme that the present invention proposes carries out Treatment Analysis to the image of each frame of video frequency program, without the need to specific mark, such as TV station symbol information, by the image processing algorithm of design, in the data with various image deformation, the program that active user watches can be located at a high speed accurately.After the relevant information obtaining the program that user watches, the information of being correlated with can be recommended to user further.
As shown in Figure 1, for implementing functional schematic of the present invention in television system, forming primarily of four partial functions of this system: one. ustomer premises access equipment, is embodied as Intelligent television terminal in video television system; Two. data acquisition server cluster is the server cluster being embodied as Real-time Collection TV station video data in video television system; Three. video database, is embodied as the database storing the video that Real-time Collection arrives and the database storing recorded video in video television system.Four. data processing server, is embodied as data processing centre in video television system.In concrete application, video database and data processing server can be integrated on ustomer premises access equipment, namely, and can integrated above-mentioned other three partial functions modules except data acquisition server cluster on television terminal; Or, video database and data processing server on one server integrated, mutual by server and ustomer premises access equipment, realizes the technical scheme that the present invention proposes; Or above-mentioned four partial functions are positioned on four or multiple independently equipment.Should be appreciated that concrete integration mode does not affect realization of the present invention; Below this tetrameric function is introduced.
One. ustomer premises access equipment
Intelligent television terminal is that user watches the collection terminal of image information and the display end of pushed information reception.Embedded image characteristic extracting module in Intelligent television terminal.Image characteristics extraction module, according to the video of the current broadcasting of user, determines the image frame of current broadcasting, extracts the characteristics of image of token image content, can also comprise the audio-frequency information feature that present image is corresponding.As main feature after image content features and audio frequency characteristics merge, by Internet Transmission to data processing server, be used for searching for the information be associated with this main feature.In addition, all right recording user behavioral data of Intelligent television terminal, watches information for analyzing user and forms corresponding user model for training.
Two. data acquisition server cluster
There is television acquisition module, the information of this module Real-time Collection television channel used in television acquisition servers cluster, and by image characteristics extraction module acquires individual features.The image of all channels collected and/or sound characteristic, be sent to thereafter in video database.
Three. video database
Video database includes but not limited to the real-time video information that television acquisition servers cluster sends, the video information etc. recorded.By setting up index to video information and the image extracted from video information and/or sound characteristic, depositing in index database, retrieving further for data processing server.Secondly, as mentioned above, in actual applications, video database can be embedded in user terminal, or deposits in particular server, by network delivery data.
Four. data processing server
Data processing server carries out matched and searched, to obtain the relevant information of user terminal video picture by the video information of user side collection being retrieved in video database.Meanwhile, data processing server by adding up and data mining extensive TV station data and a large amount of user watched information, thus obtains corresponding unique user model and group of subscribers behavior model.Various model will be used for the Intelligent Services such as last information recommendation and social networks.This part mainly comprises single user rating analysis module, group of subscribers behaviour classification, behavioural analysis module, and for the large scale database of data analysis.Specifically, data processing server has: (1) TV information retrieval module and (2) intellectual analysis and recommending module.
Wherein, (1) TV information retrieval module is responsible for main TV information retrieval and relevant recommendation, can be divided into multiple functional module, be respectively the logic flow of process:
(a) images match module
This module in charge is according to the characteristics of image of the TV information of the user's viewing obtained from network on Intelligent television terminal, carry out searching and mating in television image index database (can be integrated in video database), thus obtain all information that active user watches television image.Such as, these information include but not limited to: the channel that user watches, the system time of TV picture, the id information of user.These information can be used for analyze user watch behavior.
(b) range image analysis module
This module according to Client-initiated graphical analysis request and further customer interaction information, by image processing techniquess such as object identifications, can be analyzed user's request, and this request is sent to intellectual analysis and recommending module further.
(2) intellectual analysis and recommending module
Intellectual analysis and recommending module can perform following three major functions:
(a) establishing target image/video data bank
By passing through characteristics of image from video database or the Internet, and/or inquire about based on Client-initiated graphical analysis request and further customer interaction information, store all relevant informations wanting content recommendation, these information comprise image, video, the raw information of text and the organizational information of all these metamessages, and set up corresponding data directory structure.
B the identification of () target is mated
Image Feature Matching is passed through to the television image of active user, in target image video data database, carries out search coupling.The correlated results searched, is used as the candidate information recommended user and carry out showing.
C () intellectual analysis is recommended
After obtaining the strategy of user model and some recommendations, consider the concrete image frame watching behavior pattern and group behavior custom and current viewing of single user, intelligence according to above information, carry out the candidate information shown from recommending user, choose the interested content of user's possibility, be pushed on Intelligent television terminal and show.
As shown in Figure 2, for the embodiment of the present invention to provide the method flow diagram of information based on video image, comprise step S210 to S240, below in conjunction with specific embodiment, each step in method flow diagram is described.
S210: the TV programme play on TV according to user, determines the image frame of current broadcasting.
In step S210, when user is by TV watching TV programme, by obtaining frame of video, determines the image frame of current broadcasting and analyzing, various services based on video image can be provided for user.
In addition, when user is interested in the some image frames in the video play, can directly by the solicited message of terminal input user, solicited message is used to indicate the information that user expects to search for.Usually, user expects that the information of searching for is relevant to the video pictures of current broadcasting.Such as, if there is user to ask, terminal enters picture capturing state, freezes current picture and at terminal demonstration, enters man-machine interaction state, waits for the solicited message of user's input.After user chooses its area-of-interest, will enter the analysis of step S220, the area-of-interest chosen based on the picture intercepted and user carries out analyzing and information recommendation.After user's input, terminal carries out picture unblock, recovers the normal play of terminal.
When user not input request information time, terminal can select the image frame of current broadcasting voluntarily, for analysis afterwards and information recommendation.
S220: carry out graphical analysis to image frame, determines the characteristics of image that image frame is corresponding.
In step S220, according to the analysis to image frame, obtain the attribute information of user, and obtain the content information on current picture image.TV recognition technology of the prior art, normally for the comparison of TV guide and the identification of TV station's station symbol; Be different from the process means of prior art, the present invention proposes to analyze image frame, obtains relevant information.
Specifically, graphical analysis is carried out to image frame, determines the characteristics of image that image frame is corresponding, comprise following any one or various ways:
Temporal sequence of images signature analysis is carried out to image frame, determines the characteristics of image that image frame is corresponding;
Regional characteristics analysis is carried out to image frame, determines the characteristics of image that image frame is corresponding;
Based on fixed group, recognition of face analysis is carried out to image frame, determines the characteristics of image that described image frame is corresponding.
As shown in Figure 3, for carrying out the flow chart analyzing retrieval to image frame.
Specifically, can be divided into:
1. temporal sequence of images signature analysis
Temporal sequence of images signature analysis is carried out to image frame, determines to comprise following handling process by the characteristics of image that image frame is corresponding:
Feature buffer memory: the characteristics of image buffer memory before present image time point is got off, the time point of the characteristics of image of buffer memory be divided into current point in time before continuous multiple frames and and current point in time there is the multiframe at certain hour interval;
Set up temporal sequence of images feature: feature based buffer memory, set up temporal sequence of images feature;
Set up temporal sequence of images index database: based on temporal sequence of images feature, set up temporal sequence of images index database.
In a particular application, open after using temporal sequence of images feature, by extract present frame continuous time series characteristics of image and interval time sequential image feature, obtain the temporal sequence of images feature of present frame, and search in respective index storehouse with this.
Continuous time series image characteristics extraction:
The continuous multiple frames image special medical treatment adjacent with current time dot image in buffer memory and present image feature are merged, forms the new characteristics of image with time span, as a part for the time series feature of present image.
Interval time, sequential image feature extracted:
By in buffer memory and current time dot image there is the characteristics of image of the cache image at certain hour interval, merge with present image feature, form another part of the time series feature of present image.
For example, current video just divides y second at the x of displaying video A, before, has play video B.The partial video image of feature buffer memory will divide under buffer memory y second adjacent video A with current x, and and present image there is the image at certain hour interval, the parts of images such as, in A in image comparatively early or video B more early.
Characteristics of image adjacent with the time for present image merges by continuous time series characteristics of image, and interval time, the characteristics of image of buffer memory in the characteristics of image of current video A and video B merged by sequential image feature.
2. regional characteristics analysis
Regional characteristics analysis is carried out to image frame, determines the characteristics of image that image frame is corresponding, the corresponding audio frequency of current picture is analyzed simultaneously, and the different modalities such as sound, image feature is merged, comprise the following steps:
If there is the optimum size parameter fed back and obtain, only extract the feature of optimum size image frame, otherwise, extract the characteristics of image that multiple sized image region is corresponding.
The optimum size pictorial feature of the audio frequency characteristics that present image picture extraction obtained is corresponding and acquisition merges, or audio frequency characteristics and multiple characteristics of image is merged respectively, obtains multi-modal high dimensional feature.
High dimensional feature compressed and encodes, obtaining the coding of described image frame;
Specifically, as shown in Figure 3, following handling process is comprised:
Regional choice:
Namely according to whether having the optimum size determining present image picture from the region parameter of index database feedback.If there is no feedback parameter, then multizone/many sized image analysis is carried out to image.
Owing to being subject to the setting of the equipment different parameters such as Set Top Box, television image can produce various image change, such as, increase black surround, with captions, and cutting out etc. of image.As previously described, technology in the past often adopts and identifies television icon or in video analysis, adopt the many variations extracted specific local feature region and produce to resist video image.But identify that television icon receives the restriction of the problems such as many icons, the solution of local feature region often needs longer signature analysis extraction time, cannot meet high speed needs.
When there is not feedback, regional choice take picture centre as regional center by image frame, according to different length-width ratio, cuts out the subgraph of multiple present image picture.Respectively feature is extracted to this multiple subgraph, line retrieval of going forward side by side, after obtaining result for retrieval, the optimized image region of TV equipment is fed back in region selection module, so that during later retrieval, only extract the picture of best region size.
Fusion Features:
Exactly characteristics of image and audio frequency characteristics are merged, obtain the high dimensional feature of picture.Specifically, comprising:
By the color of the picture of described optimum size, texture, local feature processes, and the audio frequency characteristics synchronous with image frame is weighted combination, obtains the high dimensional feature of current corresponding picture.
Feature Compression:
Dimension compression carried out to described high dimensional feature and encodes, obtaining the coding of described image frame.Specifically, be exactly by the mode such as non-supervisory or semi-supervised, high dimensional feature be mapped as the numerical characteristics of lower dimension, and by quantifying, numerical characteristics be quantified as binary coding.Thereafter, according to the coding of image frame, determine the characteristics of image that image frame is corresponding.
In addition, according to the coding of image frame, determine the characteristics of image that image frame is corresponding, before this or afterwards, also comprise:
Carry out parameter model coupling and/or parameter feedback to described characteristics of image coding, parameter comprises following one or more parameter:
Region parameter, search Neighbourhood parameter, neighbor searching parameter.
Specifically, as shown in Figure 3, following handling process is also comprised:
Multi-parameters model is estimated
Parameter comprises: region parameter, search Neighbourhood parameter, neighbor searching parameter etc.Wherein, region parameter be used for many sized image are retrieved after, feed back to intelligent television, in order to determine current optimum size parameter.
Search Neighbourhood parameter is the neighbour's quantity in many neighborhoods Hash, namely the image index position of carrying out Hash is carried out to the figure place of 01 upset.Distance threshold used when neighbor searching parameter is the distance of the Hash coding of final calculating two images, in order to define the similitude of image.
3. face supplemental characteristic is analyzed
As shown in Figure 3, specifically, recognition of face analysis is carried out to image frame, determines to comprise following handling process by the characteristics of image that image frame is corresponding:
Face cluster:
By the method that machine learning, statistical learning or the degree of depth learn, the extensive facial image obtained is carried out cluster, set up the faceform of different personage from TV or other media.
Face datection and recognition of face:
By the method for template matches, or using the face of different angles as different objects, adopt the method determination face particular location in the picture of object detection.Similarity Measure between the star's face re-using different faceform's graders or obtained by current face and cluster, identifies current face.
Star's face judges:
By identifying star's face in star's face index database, searching for star's relevant information in a database, after being convenient to, being recommended user.
S230: according to characteristics of image, searches for the information be associated with image frame, determines the TV programme that active user watches.
In step S230, retrieved by characteristics of image from video database or the Internet.Namely, based on the characteristics of image obtained in step S220, and/or based on Client-initiated graphical analysis request and further customer interaction information, inquire about in video database or the Internet.
Usually, in specific implementation, the analytic process in the server by disclosing in step S220, has carried out data acquisition and image characteristic analysis to video data.
For video television system, the information of server Real-time Collection television channel used, and by image characteristics extraction module acquires individual features.The image of all channels collected and/or sound characteristic are sent in TV index database, set up index and store, form video database.
For different signature analysis schemes, set up different index databases, specifically, as shown in Figure 3, comprising:
Index database to the multiple index of the characteristics of image that regional choice extracts:
Multiple index contains the multiple model such as multilayer index, Hash neighbour.After the binary coding being obtained feature by Feature Compression, set up multi-modal aspect indexing storehouse by search methods such as Hash.
The method of Hash neighborhood search, obtains neighbour's sequence by overturning specific Hash position.Such as, the binary of the characteristics of image obtained is encoded to 10010001110101011 ..., after use Hash sets up index, the Hash sequence obtained is 10010, its 1 neighborhood Hash neighbour is 00010,11010,10110,10000,10011,2 neighborhoods are then upset two Hash positions.The quantity of neighborhood depends on the query capability of system and different application scenarioss.
Neighborhood Hash solves the quantization error problem occurred in feature coding process, expands the query context of Hash, adds the recall rate of system.Further, by the method for statistical learning, significance level and the quantization error length of different Hash position can be determined, thus determine the Hash position needing to carry out neighborhood upset accurately.Such as in example just now, Hash sequence after quantification is 10010, be 5 Hash, find in the statistics of large data sets, 2nd and the 5th occurs that the probability of quantization error is larger, thus in the generation of 1 neighborhood Hash neighbour, only for the 2nd and the 5th, its 1 neighborhood Hash neighbour is that 11010 and 10011. same statistical methods are also applied and manyly to be overturn with the Hash of 1 neighborhood.
According to the temporal sequence of images feature of S220, set up temporal sequence of images index.Index have employed the hash index mode of standard, is similar to the multiple hash modes such as local sensitivity Hash for setting up index to temporal sequence of images feature.
Obtain the feature representation of different star's face according to the extensive face cluster described in S220, by the method for dimensionality reduction and/or Hash, be the feature of certain length by star's face Feature Mapping of higher-dimension, then set up effective star's face index by tree structure.
Above-mentioned three different index databases will be retrieved respectively to feature corresponding to it, and afterwards using regional choice feature as main feature, the result for retrieval of time series feature and star's face feature, as auxiliary, comprehensively generates ranking results.
Such as, the major part of current picture is the face of certain personage, and after regional choice characteristic key, candidate result may be multiple different face image, now just can sort in screening, by the TV information that star's face feature determination present image is corresponding at the second layer.Again such as, when all there is identical picture or video in multiple television channel, cannot judge the information such as its television channel watched only by present image, time series feature, in second layer screening, will determine that user watches channel.
Through above-mentioned analysis, can determine the attribute information of user, attribute information comprises following one or more information: user ID information, the GPS information of user, the channel that user watches, the temporal information of user's viewing, the historical record that user watches, the preference information that user watches.
In addition, the content information on current picture image can also be determined, such as: current advertisement of playing certain commodity, or currently play certain film etc.
Therefore, through step S210 to S230, graphical analysis is carried out to image frame, just can obtain TV programme, television image that active user is watching.
S240: analyze the information be associated, recommends the information relevant to the TV programme of current broadcasting to user on TV.
In step S240, analyze and obtain according to step S230 the information be associated, in terminal, recommend the information relevant to the video of current broadcasting to user thereafter.
In addition, when in step before, received the solicited message of user's input, solicited message is used to indicate the information that user expects to search for; And/or having obtained the attribute information of user, attribute information comprises following one or more information: user ID information, the GPS information of user, the channel that user watches, the temporal information of user's viewing, the historical record that user watches, the preference information that user watches; Then according to these information, the information be associated is obtained to step S230 and does to analyze further and screening, thereafter the result of screening is recommended to user in terminal.
Such as, terminal recommends the information relevant to the video of current broadcasting to user, comprising:
Search for from the information be associated step S230, search the information with solicited message and/or attribute information with matching fields, the information with matching fields is recommended to user.
For video television system, data processing centre by adding up and data mining extensive TV station data and a large amount of user watched information, thus obtains corresponding unique user behavior model and group of subscribers behavior model.Various model will be used for the Intelligent Services such as last information recommendation and social networks.After the strategy of the user model obtained and some recommendations, consider the concrete image frame watching behavior pattern and group behavior custom and current viewing of single user, intelligence according to above information, carry out the candidate information shown from recommending user, choose the interested content of user's possibility, be pushed on Intelligent television terminal and show.
Specifically, when using the treatment step in S210-S230 as user, database will record the video information that each user watches, these information include but not limited to: the television channel of user's viewing, the program category of user's viewing, user watches and saves object time etc.Sole user's model by the statistics recorded to active user, or adopts the method for machine learning, and training obtains the preference degree of active user to variety classes video and variety classes commodity, in order to carry out personalized recommendation in the recommendation below.And group of subscribers model is by carrying out statistical classification to the custom of all users, obtain entirety custom and the preference of a certain class user, when unique user belongs to a certain types of populations, more effective recommendation is carried out in the preference alignment according to this types of populations.
In addition, based on the above-mentioned disclosed technical scheme of the present invention, user can set up individual social networks or social colony based on the video play.
Such as, as embodiments of the invention, also comprise:
Search the attribute information of the user in identical period viewing same video;
User in identical period viewing same video is constructed as an interest group;
Receive the message that the user within same interest group sends, and the message of user's transmission other users to same interest group are sent.
Wherein, user can arrange the attribute information of user as message filtering condition, receive the message of other users transmission meeting the attribute information that user is arranged, attribute information comprises following any one or more information: user ID information, the GPS information of user, the channel that user watches, the preference information that user watches.
Furthermore, user inputs information in terminal, realizes carrying out information interaction with other users of same interest group.
For video television system, user, when seeing certain TV programme, such as, when watching the football game match, according to the technical scheme that the present invention proposes, inquiring about the user watching the football match of same field in current time, these users being constructed as an interest group.Intelligent television terminal can receive the information that user sends, and these information can be sent to other users of interest group.In addition, user can according to personal like, and the attribute information according to user screens, and such as, can select same sex, age bracket is close, and other users being in same city exchange.Meanwhile, when Intelligent television terminal has camera, user and user directly except can carry out Word message mutual before, directly can also carry out video calling, etc.
For reach video image retrieval to retrieval rate and under multiple image conversion to the requirement of accuracy, the present invention proposes a kind of image characteristic analysis method being improved search accuracy rate by multiple features model and feedback mechanism.First multiple characteristic model is set up to TV image, utilize the continuation feature of video image, after database side confirms active user's television characteristics model, then reduce client multiple features model amount of calculation, to ensure the high efficiency of system by feedback.
In addition, the invention allows for the complementarity utilizing the multi-modal features such as audio-video images sequence and star's face that may occur, carry out auxiliary method of carrying out video image search, avoid the interference of height similar image, improve the accuracy of search.
Furthermore, the present invention watches programme information by collecting user, sets up personal behavior model, thus the system of carrying out intelligent recommendation and the program watched by search subscriber, for watching that the user of same program sets up the method for social networks.
In addition, based on the technical scheme that the present invention proposes, video frequency program operation field can have a set of perfect in user behavior, the solution of intelligently pushing user interest information.To personal user, Consumer's Experience can be improved; To advertiser, precisely can throw in advertisement, increase advertising income; To operator, can net income increase pattern.
As shown in Figure 4, for the embodiment of the present invention to provide the device structure schematic diagram of information based on video image.Based on picture search identification TV signal and the equipment 400 of recommendation information, comprising: select module 410, analysis module 420, retrieval module 430 and recommending module 440.
Wherein, selecting the TV programme of module 410 for playing according to user on TV, determining the image frame of current broadcasting;
Analysis module 420, for carrying out graphical analysis to image frame, determines the characteristics of image that image frame is corresponding;
Retrieval module 430, for according to characteristics of image, is searched for the information be associated with image frame, is determined the TV programme that active user watches;
Recommending module 440, for analyzing the information be associated, recommends the information relevant to the TV programme of current broadcasting to user on TV.
As the embodiment of the said equipment 400, analysis module 420, for carrying out graphical analysis to image frame, determines the characteristics of image that image frame is corresponding, comprises following any one or various ways:
Analysis module 420, for carrying out temporal sequence of images signature analysis to image frame, determines the characteristics of image that image frame is corresponding;
Analysis module 420, for carrying out regional characteristics analysis to image frame, determines the characteristics of image that image frame is corresponding;
Analysis module 420, for carrying out recognition of face analysis to image frame based on fixed group, determines the characteristics of image that image frame is corresponding.
As the embodiment of the said equipment 400, analysis module 420, for carrying out regional characteristics analysis to image frame, is determined to comprise the following steps the characteristics of image that image frame is corresponding:
Analysis module 420, for carrying out the analysis of many sized image to image frame, determines the optimum size of image frame;
Analysis module 420, for extracting characteristics of image to the picture of optimum size and carry out Fusion Features, obtains the high dimensional feature of the picture of optimum size;
Analysis module 420, for compressing high dimensional feature and encoding, obtains the coding of image frame;
Analysis module 420, for the coding according to image frame, determines the characteristics of image that image frame is corresponding.
As the embodiment of the said equipment 400, analysis module 420, for carrying out Fusion Features to the picture of optimum size, obtains the high dimensional feature of the picture of optimum size, comprising:
Analysis module 420 is for the color of the picture by optimum size, and texture, local feature processes, and the audio frequency characteristics synchronous with image frame is weighted combination, obtains the high dimensional feature of the picture of optimum size.
As the embodiment of the said equipment 400, analysis module 420 is for before or after determining the characteristics of image that image frame is corresponding according to the coding of characteristics of image, also comprise: analysis module 420 is for carrying out parameter model coupling and/or parameter feedback to characteristics of image coding, and parameter comprises following one or more parameter:
Region parameter, search Neighbourhood parameter, neighbor searching parameter.
As the embodiment of the said equipment 400, also comprise: receiver module (not illustrating in Fig. 4),
Before the information that analysis module 420 analysis is associated,
Receiver module is for receiving the solicited message of user's input, and solicited message is used to indicate the information that user expects to search for; And/or
Receiver module is for obtaining the attribute information of user, and attribute information comprises following one or more information: user ID information, the GPS information of user, the channel that user watches, the temporal information of user's viewing, the historical record that user watches, the preference information that user watches.
As the embodiment of the said equipment 400, recommending module 440, for recommending the information relevant to the video of current broadcasting to user in terminal, comprising:
Retrieval module 430, for searching for from the information be associated, searches the information with solicited message and/or attribute information with matching fields, and recommending module 440 is for recommending the information with matching fields to user.
As the embodiment of the said equipment 400, also comprise:
Retrieval module 430 is for searching the attribute information of the user in identical period viewing same video;
Analysis module 420 is for being constructed as an interest group by the user in identical period viewing same video;
The message that receiver module sends for receiving user within same interest group, recommending module 440 for and the message that user is sent send to other users of same interest group.
As the embodiment of the said equipment 400, also comprise:
The attribute information that receiver module is arranged for receiving user, using attribute information as message filtering condition, receive the message of other users transmission meeting the attribute information that user is arranged, attribute information comprises following any one or more information:
User ID information, the GPS information of user, the channel that user watches, the preference information that user watches.
As the embodiment of the said equipment 400, receiver module, for receiving user's input information in terminal, realizes carrying out information interaction with other users of same interest group.
The said equipment that the present invention proposes, to go forward side by side the mode of line search by carrying out graphical analysis to TV programme, can obtain the video program information that user is watching in massive video program data.Unlike previous technologies, the technical scheme that the present invention proposes carries out Treatment Analysis to the image of each frame of video frequency program, without the need to specific mark, and such as TV station symbol information.By the image processing algorithm of design, in the data with various image deformation, the program that active user watches can be located at a high speed accurately.After the relevant information obtaining the program that user watches, the information of being correlated with can be recommended to user further.
Based on the said equipment that the present invention proposes, video frequency program operation field can have a set of perfect in user behavior, the solution of intelligently pushing user interest information.To personal user, Consumer's Experience can be improved; To advertiser, precisely can throw in advertisement, increase advertising income; To operator, can net income increase pattern.
In order to better understand, inventive concept of the present invention is described, below in conjunction with concrete application scenarios, the invention will be further described.
As shown in Figure 5, for using system flow chart of the present invention in television system.
1. the automatic retrieval of user's TV information and relevant information are recommended
This system branch, by watching that the information such as the video/audio of program are analyzed to user is current, extracts above-mentioned various feature, retrieves in a database, carry out corresponding recommendation by the information retrieved.After subsystem function is opened, Television clients will send TV frame feature to server end automatically continuously, and obtains corresponding recommendation information.
In particular to handling process have:
Open intelligent recommendation system to judge:
Judge whether user opens intelligent recommendation system.If unlatching intelligent recommendation, then system enters and automatically detects and recommended flowsheet, waits for user's active request simultaneously; If do not opened, then system is in wait state all the time.
Graphical analysis:
This is for analyzing a Core Feature of retrieval, and concrete handling process is shown in Fig. 3.Its function is analysis image, obtains the multi-modal feature that image frame is corresponding.For next step image retrieval prepares individual features.
Indexed search
This is for analyzing another Core Feature of retrieval, and concrete formation is shown in Fig. 3.Its function is the characteristics of image exported according to graphical analysis, utilizes the index of the index information determination picture frame of image data base.
Index database
Index database stores the index structure of image data base, and this structure is used for carrying out realizing rapid image coupling in a database based on characteristics of image and searching.
Database module
Database module stores all channels that television acquisition servers collects, the data of all frames.These data comprise the multi-modal feature of picture frame, the channel information of picture frame, the temporal information of picture frame, multichannel information etc.
Video information analysis module
The function of this module is the result for retrieval according to frame of video, and the user model data stored, and integrated treatment result for retrieval extracts the relevant information of present analysis frame.
User model module
This module saves the personal behavior model of registered user, this model have recorded and comprises user ID information, user's viewing channel information, user's viewing time information, user watches historical information, and user watches the analysis integrated user's using forestland information out of preference information etc.
User's active request judge module
This module determines whether user's active request, after the service of startup intelligent recommendation, monitors whether useful family active request all the time.After receiving user's active retrieval request, intercept requests picture also enters further man-machine interaction.
Automatic result for retrieval detection module
The function of this module is, when if not user's active request, processes further the result of automatically retrieval, and detecting in a period of time user may interested information, and ensures to ask recommendation server to push this information within one stable period.
Recommendation server module
The function of this module is the result according to retrieval analysis, corresponding propelling movement request, is pushed to the information that client is relevant.
Client chooses module
The function of this module is, according to the information that push server pushes, choosing and push destination accordingly, can be television terminal, also can be mobile terminal.
The flow process of its work, as shown in Figure 5:
Television terminal obtains frame of video, and this picture frame enters automatic retrieval and the relevant information commending system of user's TV information.After unlatching intelligent mode recommendation, system enters retrieval and recommended flowsheet automatically, monitors whether useful householder moves solicited message simultaneously.In automatic retrieval and recommended flowsheet, picture frame enters image analysis module, the multi-modality images feature of abstract image frame, then indexed search module is entered, the retrieving information of this CMOS macro cell characteristics of image, according to the index database set up on the database, in image library, carry out the retrieval frame by frame based on picture frame.The result of retrieval, enters video information analysis module, according to the user model stored, comprehensively analyzes and extracts the information (channel, temporal information, programme information) needed.The information extracted, enters user's active request judge module, according to the result judged, if not user's active request, enters automatic result for retrieval detection module, detects Detection Information, obtains stable propelling movement and asks; If user's active request, directly enter recommendation request flow process.Finally, recommendation server, according to recommendation request, is recommended, and chooses module by client, selects the target terminal pushed to be Television clients or mobile phone terminal or other smart machines.
2. the retrieval request that user initiatively initiates responds and range image analysis
The random interest that this system branch produces article in certain two field picture for solving user.User obtains current playing frames by sending initiatively retrieval request to TV, and is retrieved special article in current picture by further man-machine interaction.This function provides a kind of new user and the mutual mode of TV, by existing object recognition technique and search technique, meets the depth search request of user.
In particular to module have:
The function of this module of user's active request judge module is for determining whether user's active request;
The function of this module of picture capturing module is, temporary freeze present frame, and the picture freezed is maintained certain area on screen, waits for the interactive operation of user;
Human-computer interaction module, this module is supplied to the interface of user one input information.User can choose interested region on the picture freezed, using the region chosen as the input analyzed further;
This functions of modules of range image analysis request judge module is for judge whether user asks to carry out range image analysis;
The function of this module of picture unlocked state is unlock the picture freezed, and makes Television clients enter normal smooth play mode;
The function of this module of range image analysis module is, according to the result of man-machine interaction, the regional area interested of image is analyzed, picture material (the specific target be used in identified region, promotion item, star's face, mark building terrestrial reference), in data bank, search coupling, and feed back to the relevant information of recognition result;
This module of library module houses needs the particular content of the target of carrying out depth analysis (specific target, promotion item, star's face, mark building terrestrial reference), the various information relevant with content therewith.
The flow process of its work, as shown in Figure 5:
Television clients is by frame of video acquisition module, under the pattern of intelligent recommendation system, enter retrieval request response and the range image analysis flow process of user's active initiation: system wait user asks, if there is user to ask, enter picture capturing module, freeze current picture and at terminal demonstration, enter human-computer interaction module, waiting for the input of user.After user chooses its area-of-interest, penetration depth graphical analysis request judge module.If carry out depth analysis, system enters picture unlocked state, recovers the normal play of terminal, the area-of-interest penetration depth image analysis module simultaneously chosen.Range image analysis module, the content of analysis image, carry out searching coupling with the identification target in data bank, identify the particular content of image, and the relevant information of contents extraction target according to image, feed back, the content of feedback, by recommendation server, is chosen different clients and is recommended;
If do not carry out range image analysis, system enters initiatively retrieval and recommended flowsheet branch, processes according to the automatic retrieval of 1. user's TV information above and the flow process of relevant information recommendation.
3. based on television video search and the intelligent social network of user's GPS position information
Different Individual to be coupled together composition user network by excavating common ground between different people by social networks.At internet arena, social networks has become the application that each Internet firm competitively fights for, and at TV category, social networks is still blank out.This function proposes a kind of novel social networks and sets up mode.Watch same program, especially the user of sports cast has certain identical characteristic.This function, by the television video identification that proposes and retrieval technique, in conjunction with user's GPS information, recommends the crowd with identical rating characteristic in certain region above to user, thus helps user to set up the social networks of oneself.
In particular to module have:
Open intelligent social network judge module
The function of this module opens intelligent social network for judging whether.
Image retrieval module
This module is image retrieval nucleus module, and its concrete way of realization is identical with the retrieval module in 1, for simplifying here only with a square frame representative.Its function is, the frame of video according to input is retrieved in image library, thus obtains the viewing information of user.The GPS information of this user's viewing information and client, together as the foundation of community information analysis below.
Community information analysis module
The function of this module is, according to user's viewing information that GPS information and the image retrieval module of client draw, analyzes the community relations of each user, carries out community to unique user.
Enter social networks judge module
The function of this module is for judge whether user has a mind to enter social networks
Social networking service device module
The function of this module is for providing social networking service.
The flow process of its work, as shown in Figure 5:
Television clients obtains frame of video by frame of video acquisition module, under the pattern of opening intelligent social network, enter the intelligent social flow through a network based on television video search and GPS: enter image retrieval module, obtain the viewing information of user, be sent to community information analysis module together with GPS information, separate out the community relations of each user.If user selects to enter social networks, then provide social networking service by social networking service device.
As shown in Figure 6, for providing the schematic diagram of information during movie.When televising, opening intelligent recommendation function and automatically retrieving and information recommendation.While content is televised in display, utilize the widescreen characteristic of advance TV, show the relevant information of the actual program retrieved simultaneously, such as, just in movie Pearl Harbor in Fig. 6, background server retrieval goes out the film information that active user is watching, the personal information recommending region to recommend out performer in film, relevant historical event, relevant tourist destination information, and similar film etc. information.As shown in Figure 7, for providing the schematic diagram of information during playing video advertisement.In figure, television image plays Commdity advertisement, and background server obtains program subject matter by detection, recommends current just in the dependent merchandise information of broadcast advertisement to user.
Furthermore, when user initiatively provides solicited message, information will be provided for user's specific aim.Such as, user is after sending active retrieval request, and terminal intercepts current request picture, and allows user send accurate search request by further man-machine interaction, and request can be certain commodity in advertisement, also can be certain star in program etc.
The schematic diagram of Fig. 8 for providing based on the user request information information of carrying out, specifically, Fig. 8 is the advertisement of the broadcasting woman style high-heeled shoes identical with Fig. 7, and user can retrieve with mutual further, to the content of interest that server transmission user irises out by active request.Such as, in Fig. 8, the solicited message of user is inquired about the clothes of model, now, based on the solicited message of user, the solicited message sent user by technology such as object identifications or picture material are analyzed, and provide and ask relevant information to user, and be no longer and program-related information.Now, what Fig. 8 showed is the information relevant to the clothes of model, instead of the information of the woman style high-heeled shoes of the advertising theme of current broadcasting.In like manner, under different video pictures scenes, user can send active request information according to the demand of oneself, such as, when watching star's concert, user is interested in the star that in program, certain occurs, can obtain further introduction and the relevant information of star by sending initiatively retrieval request.
As shown in Figure 9, for setting up the schematic diagram of intelligent social network.In figure, by the search of program watched user and the GPS position information of user, obtain the list that certain territorial scope adduction sees the user of same program, user can initiate social networks request to other users, connect after obtaining response, add network.Meanwhile, also according to the behavioural habits of customer group watching different program, corresponding colony information on services can be recommended to same class customer group.
Those skilled in the art of the present technique are appreciated that the one or more equipment that the present invention includes and relate to for performing in operation described in the application.These equipment for required object and specialized designs and manufacture, or also can comprise the known device in all-purpose computer.These equipment have storage computer program within it, and these computer programs optionally activate or reconstruct.Such computer program can be stored in equipment (such as, computer) in computer-readable recording medium or be stored in and be suitable for store electrons instruction and be coupled in the medium of any type of bus respectively, described computer-readable medium includes but not limited to that the dish of any type (comprises floppy disk, hard disk, CD, CD-ROM, and magneto optical disk), ROM(Read-Only Memory, read-only memory), RAM(Random Access Memory, memory immediately), EPROM(Erasable Programmable Read-Only Memory, Erarable Programmable Read only Memory), EEPROM(Electrically Erasable ProgrammableRead-Only Memory, EEPROM (Electrically Erasable Programmable Read Only Memo)), flash memory, magnetic card or light card.Namely, computer-readable recording medium comprises and being stored or any medium of transmission information with the form that can read by equipment (such as, computer).
Those skilled in the art of the present technique are appreciated that the combination that can realize the frame in each frame in these structure charts and/or block diagram and/or flow graph and these structure charts and/or block diagram and/or flow graph with computer program instructions.Those skilled in the art of the present technique are appreciated that, the processor that these computer program instructions can be supplied to all-purpose computer, special purpose computer or other programmable data processing methods realizes, thus is performed the scheme of specifying in the frame of structure chart disclosed by the invention and/or block diagram and/or flow graph or multiple frame by the processor of computer or other programmable data processing methods.
Those skilled in the art of the present technique are appreciated that various operations, method, the step in flow process, measure, the scheme discussed in the present invention can be replaced, changes, combines or delete.Further, there is various operations, method, other steps in flow process, measure, the scheme discussed in the present invention also can be replaced, change, reset, decompose, combine or delete.Further, of the prior art have also can be replaced with the step in operation various disclosed in the present invention, method, flow process, measure, scheme, changed, reset, decomposed, combined or deleted.
The above is only some embodiments of the present invention; it should be pointed out that for those skilled in the art, under the premise without departing from the principles of the invention; can also make some improvements and modifications, these improvements and modifications also should be considered as protection scope of the present invention.

Claims (20)

1., based on picture search identification TV signal and a method for recommendation information, it is characterized in that, comprise the following steps:
According to the TV programme that user plays, determine the image frame of current broadcasting on TV;
Graphical analysis is carried out to described image frame, determines the characteristics of image that described image frame is corresponding;
According to described characteristics of image, search for the information be associated with described image frame, determine the TV programme that active user watches;
The information be associated described in analysis, described TV recommends the information relevant to the TV programme of current broadcasting to user.
2. according to claim 1 based on picture search identification TV signal and the method for recommendation information, it is characterized in that, graphical analysis is carried out to described image frame, determine the characteristics of image that described image frame is corresponding, comprise following any one or various ways:
Temporal sequence of images signature analysis is carried out to described image frame, determines the characteristics of image that described image frame is corresponding;
Regional characteristics analysis is carried out to described image frame, determines the characteristics of image that described image frame is corresponding;
Based on fixed group, recognition of face analysis is carried out to described image frame, determines the characteristics of image that described image frame is corresponding.
3. according to claim 2 based on picture search identification TV signal and the method for recommendation information, it is characterized in that, regional characteristics analysis is carried out to described image frame, determine to comprise the following steps the characteristics of image that described image frame is corresponding:
The analysis of many sized image is carried out to described image frame, is estimated by parameter feedback, determine the optimum size of described image frame;
The picture of described optimum size is extracted to characteristics of image and carries out Fusion Features, obtains the high dimensional feature of the picture of described optimum size;
Described high dimensional feature compressed and encodes, obtaining the coding of described image frame;
According to the coding of described image frame, determine the characteristics of image that described image frame is corresponding.
4. according to claim 3 based on picture search identification TV signal and the method for recommendation information, it is characterized in that, Fusion Features is carried out to the picture of described optimum size, obtains the high dimensional feature of the picture of described optimum size, comprising:
By the color of the picture of described optimum size, texture, local feature processes, and the audio frequency characteristics synchronous with described image frame is weighted combination, obtains the high dimensional feature of the picture of described optimum size.
5. according to claim 3 based on picture search identification TV signal and the method for recommendation information, it is characterized in that, before or after determining according to the coding of described characteristics of image the characteristics of image that described image frame is corresponding, also comprise: carry out parameter model coupling and/or parameter feedback to described characteristics of image coding, described parameter comprises following one or more parameter:
Region parameter, search Neighbourhood parameter, neighbor searching parameter.
6. according to claim 1 based on picture search identification TV signal and the method for recommendation information, it is characterized in that, before the information be associated described in analysis, also comprise:
Receive the solicited message of user's input, described request information is used to indicate the information that user expects to search for; And/or
Obtain the attribute information of user, described attribute information comprises following one or more information: user ID information, the GPS information of user, the channel that user watches, the temporal information of user's viewing, the historical record that user watches, the preference information that user watches.
7. according to claim 6 based on picture search identification TV signal and the method for recommendation information, it is characterized in that, described terminal recommends the information relevant to the video of current broadcasting to user, comprising:
Search for from the described information be associated, search the information with described request information and/or described attribute information with matching fields, the information with matching fields is recommended to user.
8. according to claim 6 based on picture search identification TV signal and the method for recommendation information, it is characterized in that, also comprise:
Search the attribute information of the user in identical period viewing same video;
User in identical period viewing same video is constructed as an interest group;
Receive the message that user within same interest group sends, and the message described user sent sends to other users of same interest group.
9. according to claim 8 based on picture search identification TV signal and the method for recommendation information, it is characterized in that, also comprise:
Receive the attribute information that described user is arranged, using described attribute information as message filtering condition, receive the message of other users transmission meeting the attribute information that described user is arranged, described attribute information comprises following any one or more information:
User ID information, the GPS information of user, the channel that user watches, the preference information that user watches.
10. according to claim 8 based on picture search identification TV signal and the method for recommendation information, it is characterized in that, receive described user input information in described terminal, realize carrying out information interaction with other users of same interest group.
11. 1 kinds based on picture search identification TV signal and the equipment of recommendation information, is characterized in that, comprising:
Select module, for the TV programme play according to user, determine the image frame of current broadcasting on TV;
Analysis module, for carrying out graphical analysis to described image frame, determines the characteristics of image that described image frame is corresponding;
Retrieval module, for according to described characteristics of image, searches for the information be associated with described image frame, determines the TV programme that active user watches;
Recommending module, for the information be associated described in analyzing, described TV recommends the information relevant to the TV programme of current broadcasting to user.
12. is according to claim 11 based on picture search identification TV signal and the equipment of recommendation information, it is characterized in that, described analysis module is used for carrying out graphical analysis to described image frame, determines the characteristics of image that described image frame is corresponding, comprises following any one or various ways:
Described analysis module is used for carrying out temporal sequence of images signature analysis to described image frame, determines the characteristics of image that described image frame is corresponding;
Described analysis module is used for carrying out regional characteristics analysis to described image frame, determines the characteristics of image that described image frame is corresponding;
Described analysis module is used for carrying out recognition of face analysis to described image frame based on fixed group, determines the characteristics of image that described image frame is corresponding.
13. is according to claim 12 based on picture search identification TV signal and the equipment of recommendation information, it is characterized in that, described analysis module is used for carrying out regional characteristics analysis to described image frame, determines to comprise the following steps the characteristics of image that described image frame is corresponding:
Described analysis module is used for carrying out the analysis of many sized image to described image frame, determines the optimum size of described image frame;
Described analysis module is used for the picture of described optimum size is extracted to characteristics of image and carries out Fusion Features, obtains the high dimensional feature of the picture of described optimum size;
Described analysis module is used for compressing described high dimensional feature and encoding, and obtains the coding of described image frame;
Described analysis module is used for the coding according to described image frame, determines the characteristics of image that described image frame is corresponding.
14. is according to claim 13 based on picture search identification TV signal and the equipment of recommendation information, it is characterized in that, described analysis module is used for carrying out Fusion Features to the picture of described optimum size, obtains the high dimensional feature of the picture of described optimum size, comprising:
Described analysis module is used for the color of the picture of described optimum size, and texture, local feature processes, and the audio frequency characteristics synchronous with described image frame is weighted combination, obtains the high dimensional feature of the picture of described optimum size.
15. is according to claim 13 based on picture search identification TV signal and the equipment of recommendation information, it is characterized in that, before or after described analysis module is used for determining according to the coding of described characteristics of image the characteristics of image that described image frame is corresponding, also comprise: described analysis module is used for carrying out parameter model coupling and/or parameter feedback to described characteristics of image coding, and described parameter comprises following one or more parameter:
Region parameter, search Neighbourhood parameter, neighbor searching parameter.
16. is according to claim 11 based on picture search identification TV signal and the equipment of recommendation information, it is characterized in that, also comprise: receiver module,
Before the information be associated described in described analysis module analysis,
Described receiver module is for receiving the solicited message of user's input, and described request information is used to indicate the information that user expects to search for; And/or
Described receiver module is for obtaining the attribute information of user, and described attribute information comprises following one or more information: user ID information, the GPS information of user, the channel that user watches, the temporal information of user's viewing, the historical record that user watches, the preference information that user watches.
17. is according to claim 16 based on picture search identification TV signal and the equipment of recommendation information, it is characterized in that, described recommending module, for recommending the information relevant to the video of current broadcasting to user in described terminal, comprising:
Described retrieval module is used for searching for from the described information be associated, and searches the information with described request information and/or described attribute information with matching fields, and described recommending module is used for the information with matching fields to recommend to user.
18. is according to claim 16 based on picture search identification TV signal and the equipment of recommendation information, it is characterized in that, also comprise:
Described retrieval module is for searching the attribute information of the user in identical period viewing same video;
Described analysis module is used for the user in identical period viewing same video to be constructed as an interest group;
The message that described receiver module sends for receiving user within same interest group, described recommending module is used for and the message described user sent sends to other users of same interest group.
19. is according to claim 18 based on picture search identification TV signal and the equipment of recommendation information, it is characterized in that, also comprise:
The attribute information that described receiver module is arranged for receiving described user, using described attribute information as message filtering condition, receive the message of other users transmission meeting the attribute information that described user is arranged, described attribute information comprises following any one or more information:
User ID information, the GPS information of user, the channel that user watches, the preference information that user watches.
20. is according to claim 18 based on picture search identification TV signal and the equipment of recommendation information, it is characterized in that, described receiver module, for receiving described user input information in described terminal, realizes carrying out information interaction with other users of same interest group.
CN201310746405.5A 2013-12-30 2013-12-30 Method and apparatus for identifying television signals and recommending information based on image search Active CN104754413B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201310746405.5A CN104754413B (en) 2013-12-30 2013-12-30 Method and apparatus for identifying television signals and recommending information based on image search

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201310746405.5A CN104754413B (en) 2013-12-30 2013-12-30 Method and apparatus for identifying television signals and recommending information based on image search

Publications (2)

Publication Number Publication Date
CN104754413A true CN104754413A (en) 2015-07-01
CN104754413B CN104754413B (en) 2020-04-21

Family

ID=53593409

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201310746405.5A Active CN104754413B (en) 2013-12-30 2013-12-30 Method and apparatus for identifying television signals and recommending information based on image search

Country Status (1)

Country Link
CN (1) CN104754413B (en)

Cited By (28)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105912249A (en) * 2015-12-15 2016-08-31 乐视网信息技术（北京）股份有限公司 Method and device for realizing video recommendation
CN106096520A (en) * 2016-06-02 2016-11-09 乐视控股（北京）有限公司 A kind of information-pushing method and device
WO2017008498A1 (en) * 2015-07-13 2017-01-19 中兴通讯股份有限公司 Method and device for searching program
CN106454442A (en) * 2016-11-03 2017-02-22 Tcl集团股份有限公司 Advertisement putting method and advertisement receiving end
CN106657188A (en) * 2015-11-02 2017-05-10 天脉聚源（北京）科技有限公司 WeChat shake terminal content pushing method and system
WO2017096801A1 (en) * 2015-12-09 2017-06-15 乐视控股（北京）有限公司 Information processing method and device
CN107124648A (en) * 2017-04-17 2017-09-01 浙江德塔森特数据技术有限公司 The method that advertisement video is originated is recognized by intelligent terminal
CN107436944A (en) * 2017-07-31 2017-12-05 福州瑞芯微电子股份有限公司 A kind of method and system of video search
CN107566858A (en) * 2017-09-05 2018-01-09 北京奇艺世纪科技有限公司 One kind recommends method, apparatus and electronic equipment
CN108111603A (en) * 2017-12-21 2018-06-01 广东欧珀移动通信有限公司 Information recommendation method, device, terminal device and storage medium
CN108337531A (en) * 2017-12-27 2018-07-27 北京酷云互动科技有限公司 Method for visualizing, device, server and the system of video feature information
CN108334518A (en) * 2017-01-20 2018-07-27 中国移动通信有限公司研究院 A kind of advertisement loading method and device
CN109040776A (en) * 2018-07-27 2018-12-18 北京感动无限科技有限公司 Recognition methods and device for broadcasting content
CN109063149A (en) * 2018-08-07 2018-12-21 四川长虹电器股份有限公司 A kind of method of personalized push screen protection picture
CN109495784A (en) * 2018-11-29 2019-03-19 北京微播视界科技有限公司 Information-pushing method, device, electronic equipment and computer readable storage medium
CN110020113A (en) * 2017-09-28 2019-07-16 南京无界家居科技有限公司 A kind of family product prediction technique and device based on characteristic matching
CN110267102A (en) * 2018-03-12 2019-09-20 腾讯科技（深圳）有限公司 Processing method and processing device, medium and the electronic equipment of video data
CN110689964A (en) * 2019-09-12 2020-01-14 银江股份有限公司 Health data sample searching method and system
CN110781388A (en) * 2019-10-12 2020-02-11 微梦创科网络科技（中国）有限公司 Information recommendation method and device for image information
WO2020049666A1 (en) * 2018-09-05 2020-03-12 日本電気株式会社 Time-series data processing device
CN110909651A (en) * 2019-11-15 2020-03-24 腾讯科技（深圳）有限公司 Video subject person identification method, device, equipment and readable storage medium
CN112073764A (en) * 2019-06-10 2020-12-11 海信视像科技股份有限公司 Display device
CN112733593A (en) * 2021-03-18 2021-04-30 成都中科大旗软件股份有限公司 Method and system for realizing image information identification based on image position
CN112788379A (en) * 2019-11-04 2021-05-11 海信视像科技股份有限公司 Display apparatus and method
CN112866762A (en) * 2020-12-31 2021-05-28 北京达佳互联信息技术有限公司 Processing method and device for acquiring video associated information, electronic equipment and server
CN113672769A (en) * 2021-08-03 2021-11-19 康佳集团股份有限公司 Multi-window automatic recommendation method, device, terminal and medium based on image recognition
WO2022142650A1 (en) * 2020-12-29 2022-07-07 深圳Tcl数字技术有限公司 Video recording-based information pushing method, smart television, and storage medium
CN117177014A (en) * 2023-11-03 2023-12-05 帕科视讯科技(杭州)股份有限公司 IPTV-based children anti-addiction early warning method, storage medium and server

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080271078A1 (en) * 2007-04-30 2008-10-30 Google Inc. Momentary Electronic Program Guide
CN102868934A (en) * 2012-08-01 2013-01-09 青岛海信传媒网络技术有限公司 Method and device for retrieving video object information based on smart television
CN102892031A (en) * 2012-09-26 2013-01-23 西南石油大学 Method for intelligently identifying television program by mobile phone
CN103458275A (en) * 2013-09-02 2013-12-18 深圳市茁壮网络股份有限公司 Real-time interaction digital television information recommendation system and method
CN103475943A (en) * 2013-09-11 2013-12-25 马清虎 Interaction method and system of television program

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080271078A1 (en) * 2007-04-30 2008-10-30 Google Inc. Momentary Electronic Program Guide
CN101690181A (en) * 2007-04-30 2010-03-31 谷歌公司 Momentary electronic program guide
CN102868934A (en) * 2012-08-01 2013-01-09 青岛海信传媒网络技术有限公司 Method and device for retrieving video object information based on smart television
CN102892031A (en) * 2012-09-26 2013-01-23 西南石油大学 Method for intelligently identifying television program by mobile phone
CN103458275A (en) * 2013-09-02 2013-12-18 深圳市茁壮网络股份有限公司 Real-time interaction digital television information recommendation system and method
CN103475943A (en) * 2013-09-11 2013-12-25 马清虎 Interaction method and system of television program

Cited By (41)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2017008498A1 (en) * 2015-07-13 2017-01-19 中兴通讯股份有限公司 Method and device for searching program
CN106713973A (en) * 2015-07-13 2017-05-24 中兴通讯股份有限公司 Program searching method and device
CN106657188A (en) * 2015-11-02 2017-05-10 天脉聚源（北京）科技有限公司 WeChat shake terminal content pushing method and system
WO2017096801A1 (en) * 2015-12-09 2017-06-15 乐视控股（北京）有限公司 Information processing method and device
CN105912249A (en) * 2015-12-15 2016-08-31 乐视网信息技术（北京）股份有限公司 Method and device for realizing video recommendation
CN106096520A (en) * 2016-06-02 2016-11-09 乐视控股（北京）有限公司 A kind of information-pushing method and device
CN106454442A (en) * 2016-11-03 2017-02-22 Tcl集团股份有限公司 Advertisement putting method and advertisement receiving end
CN108334518A (en) * 2017-01-20 2018-07-27 中国移动通信有限公司研究院 A kind of advertisement loading method and device
CN107124648A (en) * 2017-04-17 2017-09-01 浙江德塔森特数据技术有限公司 The method that advertisement video is originated is recognized by intelligent terminal
CN107436944A (en) * 2017-07-31 2017-12-05 福州瑞芯微电子股份有限公司 A kind of method and system of video search
CN107566858A (en) * 2017-09-05 2018-01-09 北京奇艺世纪科技有限公司 One kind recommends method, apparatus and electronic equipment
CN110020113B (en) * 2017-09-28 2021-04-20 南京无界家居科技有限公司 Home product prediction method and device based on feature matching
CN110020113A (en) * 2017-09-28 2019-07-16 南京无界家居科技有限公司 A kind of family product prediction technique and device based on characteristic matching
CN108111603A (en) * 2017-12-21 2018-06-01 广东欧珀移动通信有限公司 Information recommendation method, device, terminal device and storage medium
CN108337531A (en) * 2017-12-27 2018-07-27 北京酷云互动科技有限公司 Method for visualizing, device, server and the system of video feature information
CN110267102A (en) * 2018-03-12 2019-09-20 腾讯科技（深圳）有限公司 Processing method and processing device, medium and the electronic equipment of video data
CN110267102B (en) * 2018-03-12 2022-05-03 腾讯科技（深圳）有限公司 Video data processing method and device, medium and electronic equipment
CN109040776B (en) * 2018-07-27 2021-11-05 联达时代(南京)智能科技有限公司 Identification method and device for playing content
CN109040776A (en) * 2018-07-27 2018-12-18 北京感动无限科技有限公司 Recognition methods and device for broadcasting content
CN109063149A (en) * 2018-08-07 2018-12-21 四川长虹电器股份有限公司 A kind of method of personalized push screen protection picture
CN109063149B (en) * 2018-08-07 2021-06-22 四川长虹电器股份有限公司 Method for pushing screen saver pictures individually
WO2020049666A1 (en) * 2018-09-05 2020-03-12 日本電気株式会社 Time-series data processing device
JP7131616B2 (en) 2018-09-05 2022-09-06 日本電気株式会社 Time-series data processor
JPWO2020049666A1 (en) * 2018-09-05 2021-05-13 日本電気株式会社 Time series data processing device
CN109495784A (en) * 2018-11-29 2019-03-19 北京微播视界科技有限公司 Information-pushing method, device, electronic equipment and computer readable storage medium
CN112073764A (en) * 2019-06-10 2020-12-11 海信视像科技股份有限公司 Display device
US11706485B2 (en) 2019-06-10 2023-07-18 Hisense Visual Technology Co., Ltd. Display device and content recommendation method
CN110689964B (en) * 2019-09-12 2022-08-26 银江技术股份有限公司 Health data sample searching method and system
CN110689964A (en) * 2019-09-12 2020-01-14 银江股份有限公司 Health data sample searching method and system
CN110781388A (en) * 2019-10-12 2020-02-11 微梦创科网络科技（中国）有限公司 Information recommendation method and device for image information
CN112788379A (en) * 2019-11-04 2021-05-11 海信视像科技股份有限公司 Display apparatus and method
CN110909651B (en) * 2019-11-15 2023-12-26 腾讯科技（深圳）有限公司 Method, device and equipment for identifying video main body characters and readable storage medium
CN111310731B (en) * 2019-11-15 2024-04-09 腾讯科技（深圳）有限公司 Video recommendation method, device, equipment and storage medium based on artificial intelligence
CN111310731A (en) * 2019-11-15 2020-06-19 腾讯科技（深圳）有限公司 Video recommendation method, device and equipment based on artificial intelligence and storage medium
CN110909651A (en) * 2019-11-15 2020-03-24 腾讯科技（深圳）有限公司 Video subject person identification method, device, equipment and readable storage medium
WO2022142650A1 (en) * 2020-12-29 2022-07-07 深圳Tcl数字技术有限公司 Video recording-based information pushing method, smart television, and storage medium
CN112866762A (en) * 2020-12-31 2021-05-28 北京达佳互联信息技术有限公司 Processing method and device for acquiring video associated information, electronic equipment and server
CN112733593A (en) * 2021-03-18 2021-04-30 成都中科大旗软件股份有限公司 Method and system for realizing image information identification based on image position
CN113672769A (en) * 2021-08-03 2021-11-19 康佳集团股份有限公司 Multi-window automatic recommendation method, device, terminal and medium based on image recognition
CN117177014A (en) * 2023-11-03 2023-12-05 帕科视讯科技(杭州)股份有限公司 IPTV-based children anti-addiction early warning method, storage medium and server
CN117177014B (en) * 2023-11-03 2024-01-30 帕科视讯科技(杭州)股份有限公司 IPTV-based children anti-addiction early warning method, storage medium and server

Also Published As

Publication number Publication date
CN104754413B (en) 2020-04-21

Similar Documents

Publication Publication Date Title
CN104754413A (en) Image search based television signal identification and information recommendation method and device
CN112565825B (en) Video data processing method, device, equipment and medium
JP4062908B2 (en) Server device and image display device
CN106156360B (en) A kind of application method of multimedia player
DE112011103903B4 (en) Method for receiving a particular service and video display device
CN100493195C (en) Time-shifted television video matching method combining program content metadata and content analysis
CN102341765B (en) For the Method and Technology of adaptable search
CN104903892B (en) Object-based image retrieval system and search method
CN103384883B (en) Top-K process is utilized to make semantic abundant
US11409817B2 (en) Display apparatus and method of controlling the same
US11070851B2 (en) System and method for providing image-based video service
US8478759B2 (en) Information presentation apparatus and mobile terminal
US20130111514A1 (en) Second screen interactive platform
US20150070506A1 (en) Event triggered location based participatory surveillance
CN107241622A (en) video location processing method, terminal device and cloud server
CN107846561A (en) The video segment recognition methods of network television and context targeted content display methods
CN102084361A (en) Media asset management
CN101711392A (en) Video detection system and methods
CN106415546A (en) Systems and methods for locally detecting consumed video content
CN104216956A (en) Method and device for searching picture information
US20170134810A1 (en) Systems and methods for user interaction
CN110750719A (en) IPTV-based information accurate pushing system and method
US20130132842A1 (en) Systems and methods for user interaction
CN100508594C (en) Commercial information searching method and apparatus, and provid method and apparatus accompanying information
CN106713973A (en) Program searching method and device

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant