CN101536513B - Methods and apparatus for detecting on-screen media sources - Google Patents

Abstract

Methods and apparatus for detecting on-screen media sources are disclosed. An example method includes monitoring a region of interest of a video signal for a first image, comparing the first image to a stored image, and confirming the media device change when the stored image differs from the first image and is associated with a secondary criterion.

Description

Be used to detect the method and apparatus of on-screen media sources

Technical field

Present invention relates in general to audience measurement system, relate more specifically to be used to detect the method and apparatus of on-screen media sources.

Background technology

The television program designing people utilizes televiewer's demography to improve the marketability (marketability) of their television program layout usually, and confirms during this Promgramming, to do the optimum cost of advertisement.In addition, accurate TV watching demography makes the advertiser can be with the crowd of commercial content to quasiexpectation.

In order to confirm these vital statistics, audience measurement companies can be enlisted a plurality of televiewers as the respondent.Collect watching of the spectators that enlist and be accustomed to and the consensus data relevant, and be used to confirm televiewer's demography with adding up with enlisting spectators.

Relating in one aspect to of audience measurement confirmed the sign that shows content on TV.Conventional audience measurement system utilizes channel detection techniques to discern TV by tuning channel (channel information just).Can this channel information be combined with Promgramming information (like listing) then, with confirm TV by be tuned to detect in time of channel and show program on TV.

In addition, audience members can use TV consume from one or more alternatives source (as numeral general/video disc (DVD) player, video tape recorder (VCR), game console, personal computer etc.) medium.For example, audience members can use that TV is played games, gambled, shopping and video request program or the like.Though the alternative source of media possibly not be to derive from broadcaster (such as the source of DVD player, VCR, video game console etc.), can strengthen enlisting spectators' demographic characteristic and watch the measurement of custom the measurements in these alternative sources.

Except detect TV by be tuned to channel, the channel detection system also can be used for detecting channel change event, in channel change event, TV stops to show the program relevant with first channel, and begins to show the program relevant with second channel.Because channel change event corresponding to the change of the program that is shown to program audience, is used to begin the triggering signal that one or more obtain the method for the identification data relevant with the new program that is shown to the audient so channel change event can be used as.

In addition, therefore channel change event and/or switch to one or more alternative source of media (after this being called " media change incident ") often corresponding to the change of audience members qualification also can be used for triggering a kind of method of gathering the information relevant with spectators.For example, audience measurement system can comprise have one group the input button device (for example, personnel's measuring instrument), wherein each the input button all assignment be used to represent single spectators.Personnel's measuring instrument can be represented them out front through for example pressing the suitable input button that is arranged on personnel's measuring instrument through the prompting audience members, and is suitable for gathering the information relevant with the audient.With the triggering signal that the media change incident is pointed out as personnel's measuring instrument, make it possible to accurately to write down and to cause the audience size that maybe possibly cause the media change incident and/or the change of membership qualification by the media change incident.

Description of drawings

Fig. 1 is the block diagram that is connected to the exemplary audience measuring system of example television systems.

Fig. 2 is the block diagram of the employed first exemplary on-screen media sources detector (OSMSD) in the exemplary audience measuring system among Fig. 1.

Fig. 3 is the block diagram of the employed second exemplary OSMSD in the exemplary audience measuring system among Fig. 1.

Fig. 4 A illustration the exemplary region-of-interest of exemplary television of Fig. 1.

Fig. 4 B-4E illustration the picture on a series of TVs that may be displayed on Fig. 1 and Fig. 4 A, illustration TV watching, title browse (banner surfing), video request program and TV guide characteristic.

Fig. 5 illustration be used for confirming the flow chart of spectators' movable (activity) exemplary process.

Fig. 6 A illustration be used to have determined whether to take place the flow chart of first exemplary process of channel change event.

Fig. 6 B illustration be used to have determined whether to take place the flow chart of second exemplary process of channel change event.

Fig. 7 is for possibly be presented at the example content in the exemplary region-of-interest by what the frame collector among Fig. 1 collected.

Fig. 8 illustration the example content among Fig. 7 in the situation after greyscale image transitions is black and white image.

Fig. 9 illustration one group of illustrative properties of the example content shown in Fig. 8.

Figure 10 illustration form one group of exemplary digital assembly of one group of numerical digit through arranging.

Figure 11 A-11C illustration be used to discern the flow chart of the exemplary process of institute's indicated channel number.

Figure 12 A-12E has described a series of pictures, illustration by the exemplary series of the detected region-of-interest of exemplary OSMSD.

Figure 13 illustration exemplary video program request picture.

Figure 14 A illustration the example state diagram of media apparatus.

Figure 14 B illustration be used for confirming the flow chart of the exemplary process of spectators' active state according to the screen order.

Figure 15 has described exemplary screen, illustration have the video-game that can discern region-of-interest.

Figure 16 has described exemplary many pictures display frame.

Figure 17 has described series of displays and has restarted the exemplary screen of characteristic.

Figure 18 A is that exemplary prior art screen is gone up the block diagram that shows reader (OSDR) system.

Figure 18 B is the block diagram that has adopted the exemplary OSDR system of OSMSD front end filter.

Figure 19 illustration with the corresponding one group of exemplary process timeline of exemplary OSDR and OSMSD system of Figure 18 A-18B.

Embodiment

Though following specification discloses the example system that except that miscellaneous part, also is included in the software of carrying out on the hardware, should be noted that this system only is illustrative, not will be understood that it is restrictive.For example, can imagine, in the disclosed hardware and software component any one or all can only be embodied as specialized hardware, only be embodied as software, only be embodied as firmware or be embodied as the combination in any of hardware, firmware and/or software.

In addition,, should be appreciated that the media system that can disclosed system easily be applied to many other types, include but not limited to television system, media recording device and/or video game apparatus though carried out following discloses to example television systems.Therefore, though following specification has been described specific example system and processing, it is readily appreciated by a person skilled in the art that disclosed example is not the unique channel that realizes these systems.

Media environment

In the exemplary environment for use of Fig. 1 illustration the block diagram of exemplary audience measuring system 100 that can the monitoring television spectators.The application number that the further details of exemplary audience measuring system 100 was submitted on April 19th, 2004 is to provide during the PCT of PCT/US2004/012272 applies for, by reference the description in its entirety of this PCT application is incorporated into here.Exemplary audience measuring system 100 comprises demultiplexer 106, frame collector 108 and on-screen media sources detector (OSMSD) 112.In the illustrative example of institute; Audience measurement system 100 is connected to exemplary media center 101, and media center 101 comprises and is used for receiving STB 104, the video game console 105 of signals and being connected to the TV 110 of STB 104 from TV service supplier 102.These assemblies of television system 101 can couple together through any-mode (comprising mode shown in Figure 1).

TV 110 can be the TV or the TV pick-up attacnment of any type.For example; TV 110 can be to support NTSC (NTSC) standard, line-by-line inversion (PAL) standard, transmit the TV and/or the display unit of standard of standard (for example HDTV (HDTV)), the exploitation of DVB (DVB) project of colored and storage (SECAM) standard, advanced television system committee (ATSC) exploitation in order, perhaps can be multi-media computer system etc.

TV service supplier 102 can be any TV service supplier, for example cable TV service supplier, telecommunications TV service supplier, satellite television service provider and/or radio frequency (RF) TV service supplier.TV service supplier 102 for example can provide simulation and/or digital television signal to media center 101 through coaxial cable or any wireless connections.

STB 104 can be any STB, for example cable television converter, telecommunications TV transducer, digital radio broadcast transducer, direct broadcasting satellite (DBS) decoder, personal digital video recorder (for example TiVo), digital video disc (DVD) player, video tape recorder (VCR) etc.STB 104 receives a plurality of television channels from TV service supplier 102.Usually, STB 104 is imported of selecting in a plurality of television channels and/or is exported the vision signal that on selected television channel, transmits based on the user.Under the situation of analog signal, STB 104 be tuned to the program of special channels to obtain on this channel, to transmit.For digital signal, 104 pairs of specific packets of STB are decoded, with the program that obtains on selected television channel, to transmit.

Video game console 105 can be any video game system; The for example GameCube (registered trade mark) of the Xbox (registered trade mark) of any version of Microsoft (registered trade mark), Nintendo (registered trade mark) and/or Wii (registered trade mark), and/or the PlayStation (registered trade mark) of any version of Sony (registered trade mark).Under hard-core situation, also can video-game be offered the user, and be presented on the TV 110 through STB 104 (for example passing through the Internet).The user can the video game content of access stored in STB 104, and/or from for example TV service supplier receives video-game content.

TV service supplier can also provide the other forms of medium that are not called as broadcasted content usually, for example gambling, internet content, Weather information and/or shopping.STB 104 can also allow the user to watch the display frame of many pictures, program guide picture and/or video request program (VOD) service of a plurality of channels, and their (if being detected) all can provide the abundant habits information resource of watching for advertisement measurement company.

The output of STB 104 and/or video game console 105 feeds back to the demultiplexer 106 of audience measurement system 100.Demultiplexer 106 can be a simulation Y separator.In exemplary audience measuring system 100, demultiplexer has produced two signals of the output of sign STB 104.Certainly, it is readily appreciated by a person skilled in the art that demultiplexer 106 can produce the signal of any amount and/or the signal of arbitrary resolution (HDTV signal for example.) in these two signals one is fed back to TV 110, and another signal is sent to frame collector 108.

Frame collector 108 can utilize the frame collector of any type that can buy on the market to realize, and is used for converting standard television signal into numerical data.For example, frame collector 108 can be a digital bitmap with the NTSC conversion of signals.The every frame of ntsc television image has 525 horizontal lines.Scan from left to right and from the top down or " collection " these lines, wherein whenever skip (promptly staggered) at a distance from one.As staggered result, need carry out the scanning of two subframes and could accomplish entire frame.Each field picture scanning approximately expends 1/60 second, thus complete frame of scanning in per 1/30 second.The alternative or additionally, frame collector 108 can be a digital bitmap with PAL (line-by-line inversion) signal and/or SECAM (transmitting color and storage in order) conversion of signals.No matter what signal is by formative television broadcasting standard at first, the bitmap of gained can be arbitrary dimension, and can encode to the shade and/or the color of any amount.For example, frame collector 108 can be gathered 320x240 gray-scale pixels, 640x480 gray-scale pixels, high definition pixel, coloured image etc.

Frame collector 108 periodically (for example, every 120ms) is gathered and is presented at the image corresponding " screenshotss " on the TV 110 in particular moment, and the DID of gained is sent to on-screen media sources detector (OSMSD) 112.112 pairs of a series of images acquired of OSMSD scan, to determine whether to have taken place channel change event and/or source of media change incident (hereinafter the two all being called " media change incident ").Below will combine the description of Fig. 2 and Fig. 3 to discuss OSMSD112 in more detail.

Exemplary media center 101 can also comprise the remote control 114 that is used to send control information, and this control information can be by any one or all receive among STB 104, video game console 105, TV 110, frame collector 108 and/or the OSMSD112.Persons of ordinary skill in the art will recognize that remote control 114 can adopt the technology (including but not limited to infrared (IR) transmission, radio frequency transmission, wired/cable connection etc.) of any type to send this information.

Exemplary audience measuring system 100 can also comprise the personnel's measuring instrument 116 that is used to gather the information relevant with the audient.Exemplary personnel's measuring instrument 116 can have one group of input button; Each input button all assignment is used to represent corresponding spectators, and exemplary personnel's measuring instrument 116 can point out audience members to represent that through pressing suitable input button they are whether in the middle of spectators.Personnel's measuring instrument 116 can also receive an instruction or the one group of instruction that makes personnel's measuring instrument 116 prompting audience members from OSMSD112.Prompting to audience members can include but not limited on the screen that points to one or more audience members, present one or more figure superimposed images (graphic overlayimage).For example, personnel's measuring instrument 116 can generate the graphical information of the text that superposeed, and this information requirements audience members is pressed their button of sign.This figure superimposed image can be in one or more ROI, detected, thereby the activity of personnel's measuring instrument 116 can be confirmed.In addition, OSMSD112 can to revise the operation of OSMSD112, for example, make OSMSD112 give central processor equipment 118 with report information from personnel's measuring instrument 116 reception information.As those skilled in the art will appreciate that, the technology of personnel's measuring instrument 116 any kind capable of using (including but not limited to infrared (IR) transmission, radio frequency transmission, wired/cable connection etc.) receives and/or transmission information.

Fig. 2 is the block diagram of exemplary OSMSD 200 that can be used for realizing the OSMSD112 of Fig. 1.Exemplary OSMSD 200 receives corresponding to screenshotss or its a part of image and/or the view data relevant with TV 110 from frame collector for example 108.As following further go through, if satisfy certain image condition, then OSMSD 200 with the image data storage that receives in memory 202.Memory 202 also can be used for storing intermediate image and/or through to handle one or more extractions zone of the one or more images that produce from the view data of frame collector 108.

For the image that receives is handled, exemplary OSMSD 200 will comprise region-of-interest (ROI) extractor 204.ROI extractor 204 reads the image that is stored in the memory 202, and from the image that is being processed, extracts one or more region-of-interests (ROI).As following go through more, the ROI that extracts can be presented at the zone on the TV 110 explicitly corresponding to the demonstration with channel change event process intermediate frequency Taoist monastic name.From the image that is being processed, extract after the ROI, the ROI extractor can be stored back the ROI image of gained in the memory 202.

Image management/processing

Before storage ROI image, ROI extractor 204 can be passed to image quantization device 206 with the ROI image.Through using 206 pairs of ROI images of image quantization device to quantize, can reduce the needed treating capacity of subsequent blocks/function among memory space (just producing lower memory footprint) and/or the OSMSD 200 significantly.For example, the memory storage space of second order black and white (for example, binary system) ROI image needs maybe be little of 1/8 of 256 rank gray scale ROI images.

With reference to Fig. 2, image quantization device 206 with first image with a plurality of colors and/or strength grade as input.Then; Image quantization device 206 is second image based on one or more quantizer thresholds 208 with this first image transitions; This second image than original first image color still less (for example has; The chroma threshold value), strength grade that replaces (for example, pixel intensity/luminance threshold) and/or higher or lower picture element density.For example, can image quantization device 206 be embodied as, first image transitions that will have 256 gray scales based on single threshold value 208 is second black and white image with two gray scales.In this example, can be the white in second image greater than the pixel transitions of threshold value 208 with gray value in first image.Similarly, can be the black in second image less than the pixel transitions of threshold value 208 with gray value in first image.In addition, can (for example manually or automatically) quantizer threshold 208 be adjusted, to improve the definition of second image based on the result of the subsequent treatment that image is carried out.After the conversion, image quantization device 206 can be stored in the quantized image (second image just) of gained and supply subsequent treatment to use in the memory 202.

For to handling from the image of the memory 202 quantification ROI image of image quantization device 206 (for example from), exemplary OSMSD 200 will comprise object detector 210 and edge detector 212.Object detector 210 can be used for detecting the one or more objects in the ROI image according to a group objects characteristic that is stored in the memory 202.For example, can object detector 210 be configured to, detect the numerical digit in the ROI image according to a stack features relevant (for example height, width, interdigital interval etc.) with numerical digit.In addition, can object detector 210 be configured to, detect and when the user gets into the broadcasting guide pattern of STB 104, possibly appear at the symbol on the TV 110, for example Channel icons (like " peacock " of NBC) and/or service icon.As following further go through, can confirm source of media based on the image, numeral and/or the symbol that appear on the video screen.Source of media can include but not limited to that live tv is watched, postponed to watch, video request program is watched, pay watch, the demonstration of many pictures, internet browsing, DVD and/or VCR record and/or playback, Time Warner (registered trade mark), StartOver (registered trade mark) service and/or interactive application (for example shopping online, customer service, guide pattern and/or instant message transmit).As long as be activated; Video game console 105 (the for example Xbox of any version of Microsoft company) will significantly show (splash) " Xbox " speech in the zone line of video screen 110, and in the zone, middle and lower part, significantly shows " Microsoft " speech.Edge detector 212 can be used for detecting the one or more edges (the for example transformation/difference between the strength grade) in the ROI image.For example, edge detector 212 can be used for confirming that whether the ROI image is corresponding to static region (quiet area), the zone that just has the constant background color/intensity.

In exemplary OSMSD 200, when detecting expectation object (for example channel, symbol, trade mark, Channel icons) or group of objects, object detector 210 all can count-up counter 214 at every turn.On the contrary, if in the ROI image, detected the edge corresponding to static region, then edge detector 212 can for example reset counter 214.The behavior and the purposes of object detector 210 and edge detector 212 will become clearer through the following subsequent descriptions to Fig. 4 A-11C that provides.

Continue to describe Fig. 2, counter 214 can be used for detecting in one or more images of denoted object detector 210 in being stored in memory 202 number of times of expectation object or group of objects.Counter comparator 216 can be used for the value of counter 214 and comparator threshold 218 are compared, and to confirm for example whether in the image of sufficient amount, detected expectation object or group of objects, thereby express possibility monitored incident has taken place.For example; Whether in the images acquired of sufficient amount, detect channel; The channel change having taken place to express possibility, and/or whether in the images acquired of sufficient amount, detected " Xbox " and " Microsoft " image, has just opened game console 105 with the expression user.The behavior of counter 214 sum counter comparators 216 and purposes will become clearer through the following subsequent descriptions to Fig. 4 A-11C that provides.

For more two or more images, exemplary OSMSD 200 will comprise Image Comparator 220.Image Comparator 220 can be used for two images of comparison, and for example ROI image and institute's reference image stored/template are similar still different to confirm that these two images (or its content) come down to.For example, detecting under the situation of channel change event, Image Comparator 220 can be with comparing corresponding to the ROI image of current channel displayed number and reference picture/template corresponding to previous channel displayed number.If it is identical in fact that Image Comparator 220 is determined two images, then channel change event possibly not take place also.Yet, if two images are different in essence, channel change event possibly take place.Can based on for example detected the sufficient amount object from the expression of counter comparator 216 and/or start Image Comparator 220 from the triggering signal that the expression of object detector 210 has detected expectation object or group of objects.The behavior of Image Comparator 220 and purposes will become clearer through the following subsequent descriptions to Fig. 4 A-11C that provides.

In addition, exemplary OSMSD 200 can comprise object identifiers 222, is used for discerning the one or more objects that are present in the image (for example ROI image) that is being processed.For example, object identifiers 222 can be discerned the one or more objects in the ROI image based on a group objects characteristic that is stored in the memory 202.As stated, object can include but not limited to symbol, Channel icons and/or trade mark.In another example, the result of object identifiers 222 Image Comparators 220 capable of using discerns the one or more objects in the ROI image.In a back example, Image Comparator 220 can with the ROI image with compare corresponding to the reference picture/template of known object.If it is identical in fact with reference picture/template to determine the ROI image, then object identifiers 222 can draw the conclusion that the ROI image comprises known object.The behavior of object identifiers 222 and purposes will become clearer through the following subsequent descriptions to Fig. 4 A-17 that provides.

Exemplary OSMSD 200 can also comprise remote signal detector 224, is used for detecting the signal from remote control (the for example remote control 114 of Fig. 1).OSMSD 200 can use remote signal detector 224 to confirm that whether remote control 114 has sent one or more signals in frame collector 108 has collected the time window of the current image that is being processed.The output of remote signal detector 224 can be used for confirming whether the image that is being processed maybe be corresponding to the media change incident.As stated, the media change incident can include, but are not limited to the startup of channel change, VCR player, DVD player and/or game console.In addition, the media change incident can comprise electronic program guides use, VOD, home shopping, gamble and/or watch the display frame of many pictures of multichannel.For example; If remote signal detector 224 is determined remote control 114 and is not sent signal (do not show media center 101 is applied the Any user input); Then remote signal detector 224 can be reset counter 214, with represent current detection for example to all numerical digits all do not correspond to channel change event.Similarly, although occurred trade mark on the screen, lack corresponding remote signal and may let the people expect that this sign occurs as the result that advertisement rather than user/spectators call TiVo (registered trade mark) DVR such as TiVo (registered trade mark) sign.The behavior of remote signal detector 224 and purposes will become clearer through the following subsequent descriptions to Fig. 4 A-17 that provides.

Can the image that object identifiers 222 identifies be stored in the video memory 202.In addition, as shown in Figure 1, can also this image that identifies be transmitted to central processor equipment 118.Central equipment 118 can further be handled these images, for example to confirm whether detected specific image changes or specific media apparatus (for example, set-top box VOD pattern, PPV pattern, VCR, DVD player etc.) corresponding to channel in specific ROI.In addition or alternatively, exemplary OSMSD200 is the image that identifies in the processing memory 202 further, confirming that spectators are using which media apparatus and/or which kind of media apparatus pattern, as following further go through.

Fig. 3 is the block diagram of exemplary OSMSD 250, and OSMSD 250 can be used for realizing the OSMSD 112 of Fig. 1 or the exemplary OSMSD 200 of Fig. 2.Exemplary OSMSD 250 can be personal computer (PC) or other processors or calculation element arbitrarily.Specifically, exemplary OSMSD250 comprises the Main Processor Unit 252 by power supply 253 power supplies.Main Processor Unit 252 can comprise the processor 254 that is electrically connected to host memory device 258 and/or one or more interface circuit 260 through system interconnection 256.For example, system interconnection 256 can be an address/data bus.Certainly, it is readily appreciated by a person skilled in the art that the interconnection that also can use except that bus is connected to host memory device 258 with processor 254.For example, can use one or more special circuits and/or cross bar switch (crossbar) that processor 254 is connected to host memory device 258.

Processor 254 can comprise the CPU (CPU) of any type, for example from the microprocessor of IntelPentium (registered trade mark) microprocessor family, Intel Itanium (registered trade mark) microprocessor family, Intel Centrino (registered trade mark) processor family and/or Intel XScale (registered trade mark) processor family.Processor 254 can comprise the cache memory of any type, for example static random access memory (SRAM).Host memory device 258 can comprise dynamic random access memory (DRAM), but also can comprise nonvolatile memory.Host memory device 258 can be used for storing will be by the software program of processor unit 254 with the any-mode execution.

The interface standard of any type capable of using (for example dummycable interface, digital cable interface, satellite-signal interface, Ethernet interface and/or general-purpose serial bus USB interface) realizes interface circuit 260.Can one or more input units 262 be connected to interface circuit 260, so that to Main Processor Unit 252 input data and/or orders.For example, input unit 262 can be keyboard, mouse, touch-screen, tracking plate, tracking ball, isopoint and/or speech recognition system.In addition, interface circuit 260 can be handled numerical data input that the frame collector 108 among Fig. 1 supplied, input that remote control detector 263 is supplied and corresponding to the input and/or the output of personnel's measuring instrument 264.Remote control detector 263 detects the signal that the remote control 114 among Fig. 1 is sent.For example; Can remote control detector 263 be designed to detection signal and (for example send (signaling) incident; Detect remote control 114 and sent signal) or the signal that remote control 114 is sent decoded, to confirm just sending to the control information of receiving system (for example STB 104).Similarly, can with personnel's measuring instrument interface 264 be designed to from and/or to personnel's measuring instrument 116 reception/transmission signals of Fig. 1.For example, personnel's measuring instrument interface 264 is used under the situation that detects the channel change and sends signals to personnel's measuring instrument 116.Persons of ordinary skill in the art will recognize that and can all or part of of frame collector 108, remote control detector 263 and/or personnel's measuring instrument interface 264 be incorporated among the OSMSD 250.

Can also one or more displays, printer, loud speaker and/or other output devices 265 be connected to Main Processor Unit 252 through one or more in the interface circuit 260.For example, output device can be used for observing the data of collecting or is used for diagnostic purpose.Display 265 can be the display of cathode ray tube (CRT), LCD (LCD) or any other types.Display 265 can be provided in the vision indication of the data that produce in Main Processor Unit 252 runnings.In addition, Visual Display can comprise the input of prompter's type of operator, evaluation, detection data etc.

OSMSD 250 can also comprise one or more storage devices 266.For example, OSMSD 250 can comprise one or more hard disk drives, compact disk (CD) driver, DVD driver and/or other computer media I/O (I/O dress) device.

OSMSD 250 can also pass through to the connection 267 and other device swap datas of network 268.It can be the network connection of any type that network connects 267; For example Ethernet connection, Digital Subscriber Line (DSL), telephone wire, coaxial cable etc.; Network 268 can be the network of any type, for example the wireless network of the Internet, telephone network, cable network and/or any type.For example, OSMSD250 can be through network 268 and central processor equipment 270 swap datas.Central processor equipment 270 can receive data from OSMSD 250, and central processor equipment 270 can be confirmed to watch for one group and adds up or discern one group and watch behavior according to these data.In addition, OSMSD 250 can be from central processor equipment 270 reception information, and to revise the operation of OSMSD 250, for example OSMSD 250 provides data frequency to central processor equipment 270.

Fig. 4 A illustration may be displayed on image and the type of ROI on the exemplary television 300 (the for example exemplary television 110 of Fig. 1).Exemplary television 300 comprises display unit 302, for example can be in viewing area 304 CRT, plasma or the LCD of display video/picture material.When the channel that is received when TV 300 changes (for example through Fig. 1 STB 104), new channel is presented in the zone 306, and this zone 306 for example is positioned near the upper right corner of viewing area 304.Zone 306 comprises it for example can being pure color (for example blue), transparent, translucent or opaque background area 308.One group of numerical digit 310 of the expression new channel number that can on background 308, superpose.After this also zone 308 is called first region-of-interest (ROI) 306, and this zone can be associated with the channel demonstration.Similarly, when spectators call different media apparatus, for example during TiVo (registered trade mark) DVR, can in the left side of viewing area 304, show TiVo (registered trade mark) marks 311 in zone 313.

In some instances, be presented at programme content among the ROI 306 can comprise the characteristic that is similar to channel in fact (for example may flash or the situation of the sports of other digital informations or contest performance under).In order to reduce by error this displaying contents to be confirmed as the possibility of channel, can adopt second formation (indicating device), for example check other zones of viewing area 304, to confirm whether to have shown channel.For example, many exemplary television 300 and/or STB 104 can also synchronously show the title guide with in a ROI 306, showing new channel in Title area 312, and for example 304 bottom is provided with this Title area 312 along the viewing area.The title guide can comprise such as the description of the title of current channel and/or catchword, current time, the programme content of broadcasting through selected channel, the information such as duration of programme content.Usually, under the situation that has shown the title guide, the pure background colour of the display message that superposeed above Title area 312 is mainly (for example blue).Therefore, can in Title area 312, define 314 (just having shown the zone that has constant Show Color/intensity usually under the situation of title) of known quiescent state zone, and static region 314 can be used to indicate the current title guide that whether showing.Therefore after this also static region 314 is called the 2nd ROI 314, and this zone can be associated with the demonstration of title guide.

For many exemplary television 300 and/or STB 104, possibly be enough to indicate the detection of channel not have mistake to the detection of channel among the ROI 306 and the title guide in the Title area 312 (for example through detecting the static region among the 2nd ROI314).Yet in other exemplary television 300 and/or STB 104, the title guide in the Title area 312 shows maybe be corresponding to the incident of non-channel change.Under these situations, can check other zones of viewing area 304, to confirm to exist and channel change event or the corresponding indicated channel of other incidents number.For example, the title guide of the user's current channel information corresponding that can ask STB 104 to provide to comprise and watching.This request can also cause and in a ROI306, shows current channel number simultaneously.Alternatively, the user can for example ask STB 104 to produce the title guides, this title guide comprise with the current channel of watching outside the channel information corresponding.Under latter event; The user can get into " title is browsed " pattern; Under this pattern; The user can still show in the programme content of current channel in viewing area 304, overviews and one group of available channel information corresponding content (for example, title guide information) beyond the current channel.In the title navigation process, the channel of the channel that can title be browsed (rather than the new channel after current channel or the channel change event) is presented among the ROI 306.Simultaneously, can current channel number be presented in the zone 316, zone 316 for example is positioned near the upper left corner of viewing area 304.Therefore after this also zone 316 is called the 3rd ROI 316, and this zone can be in one or more title guide operator scheme processes be associated with the demonstration of current channel number.Therefore, can check the displaying contents among the 3rd ROI 316 for example, confirming detected channel among the ROI 306, or confirm to be presented at channel among the ROI 306 corresponding to the title browsing event corresponding to channel change event.

Fig. 4 B comprises exemplary screen 320, its illustration title browse.Different STBs can adopt different ROI in watching channel ROI 324, to show the current channel of being watched 322 and have and show title channel 326 among the title channel ROI 328 of associated description information 328.Fig. 4 B-4E also illustration the user can be with four kinds that STB 104 calls the exemplary pattern of watching.Shown in Fig. 4 B, TV (" TV ") watches that mode 3 20 provides title 322 for spectators, title 322 comprises network icon " HBO " 324, be used to represent current one group of numerical digit 326 of watching channel with the current relevant descriptor 328 of channel 326 of watching.Fig. 4 C illustration comprise the program guide mode 330 of " TV guide " icon 332 and programme information type 334.Fig. 4 D illustration the exemplary screen shot of title browse mode 336, these screenshotss comprise that is used to represent a current numerical digit 338 of watching channel.Title browse mode 336 also comprises title 340, network icon 342, channel 344 and descriptive information 346, each in them all with current content of watching outside programme content relevant.Fig. 4 E illustration comprise the VOD mode 3 48 of exemplary title 349 and exemplary video program request (VOD) icon 350.

In television viewing mode 320, guide mode 3 30, title browse mode 336 and/or the VOD mode 3 48 each all comprises unique ROI, icon, symbol and/or its combination.For example, though television viewing mode 320 all comprises title (being respectively 322 and 340) with title browse mode 336, also exist one group of numerical digit 338 just to represent that the user is browsing title rather than only watching TV simultaneously.Can monitor specific ROI and discern through training OSMSD 112 exists and/or does not exist one or more icons, symbol and/or numeral to discern each watching mode.As a result, can make through video and/or remote control to be used for confirming spectators' behavior, comprise and confirm that spectators are using one or more media apparatus (for example DVD player, VCR, video game console, TiVo (registered trade mark) etc.).

It will be appreciated by the skilled addressee that above-mentioned region-of-interest (for example ROI 306,314 and/or 316) can adapt to particular set-top box 104 and/or the TV 110 that in television system 101, adopts.For example, another exemplary set-top box 104 can make channel be presented to be arranged in a ROI306 in 304 upper left corners, viewing area (rather than the upper right corner that the exemplary television 300 of Fig. 4 A is described).Alternative or additionally, Title area 312 can be along the tops of viewing area 304 (rather than in exemplary television 300, describe along the bottom) is provided with.And in the title navigation process, the independent position that some STBs 104 and/or TV 110 can be in Title areas 312 rather than in viewing area 304 (with respect to the title browsing channel number that is presented in the exemplary television 300 among the ROI 306) shows the channel that title is browsed.Usually, quantity and the characteristic to one or more region-of-interests calibrated in the initial configuration process of audience measurement system 100.For example can new parts (for example new STB 104) upgraded corresponding calibration setting when being installed in the television system 101.Also can detect the reliability of channel change event with raising by any or all come calibration setting is upgraded in central processor equipment 222, OSMSD 112 and/or the frame collector 108.

Fig. 5 shows a flow chart, this flow chart illustration be used for confirming the exemplary process 500 of spectators' active state.Handling 500 can be carried out by OSMSD (for example exemplary OSMSD 112 or 200), and can be implemented as the one or more software programs that are stored in one or more memories and carried out with any-mode by one or more processors.Yet, can be manually and/or carry out through one or more hardware units and to handle 500 some or whole frames.

In general; The relevant one or more ROI of the frame of video of the predetermined quantity that exemplary process 500 scannings and the continuous acquisition arrive (ROI 306,314 and 316 of Fig. 4 A for example; Or the ROI 322,324,326,332 and 338 of Fig. 4 B), with confirm the user whether watching broadcast medium 320, watch guide equipment 330, carry out title and browse 336 and/or consuming VOD service 348.Especially, when the user calls any in the above-mentioned watching mode, specific ROI combination will comprise numeral, symbol and/or icon.

Exemplary process 500 starts from frame 502, and wherein the frame collector among Fig. 1 108 is gathered screenshotss, and these screenshotss are corresponding to the video signal frame that is transmitting to TV 110 from for example STB 104.The screenshotss that collect can be corresponding to the whole video frame, or only corresponding to the part of frame.OSMSD112 extracts at least one ROI (frame 504) from the screenshotss that collect.For example, the ROI that OSMSD 112 can extract and storage is relevant with title, zone 322,340 and 349 for example, and/or the relevant ROI of extraction and Guide characteristic 332.As following further go through, can make amendment to the numeral, symbol and/or the icon that from one or more ROI, extract, detect and/or save memory with convenient.

If OSMSD 112 does not detect the characteristic of screen heading (for example 322,340 and 348), then whether OSMSD 112 inspections exist " TV guide " symbol 332 (frame 508).If detected Guide symbol 332, the current media device that then is being used is a STB 104, and present mode is guide mode 3 30 (frame 510).In order to reduce to misread media apparatus and/or the movable possibility of spectators that is being used, OSMSD 112 can assess other ROI.For example; If broadcasting the television advertising that comprises TV guide symbol 332; And this symbol just in time rest on screen position identical shown in Fig. 4 C on, then possible errors ground detects and is guide mode 3 30, and in fact this moment, the user only watched radio and television.Can be used to indicate one or more other ROI to avoid this problem through analysis such as one or more characteristics of guidance features 330.Specifically, OSMSD 112 can check the programme information 334 of grid format, and is shown in Fig. 4 B, effective with verification guide pattern.If there is not title (frame 506), and do not occur TV guide symbol (frame 508) among the predetermined ROI of title, then possibly can't know current media source and/or watching mode (frame 512).Can continue other frame of video collections, with the media apparatus of searching spectators' pattern and/or be used (frame 502).

If have title (frame 506) on the screen, then in the illustrative example of institute, any pattern in three exemplary patterns all maybe be effectively.Specifically, under the situation that has title (322,340,349), television viewing mode 320, title browse mode 336 or VOD mode 3 48 possibly be effective.OSMSD 112 can confirm whether the upper right corner of screen exists one group of numerical digit 338, shown in Fig. 4 B, if exist, then confirms spectators' browsing title (frame 516).Though different STBs can show title through different modes and browse, when the user looked back alternative channel 344 and corresponding programme information 346, the current channel of being watched usually remained on the screen.Yet,, whether have VOD symbol (frame 518) in the picture that the OSMSD112 inspection collects, for example the VOD symbol 350 shown in Fig. 4 B if the upper right corner of screen does not comprise one group of numerical digit 338 (frame 514).In the illustrative example of institute, the upper right corner of video screen exists VOD symbol 350, title 349 and does not have one group of numerical digit 338, and these expression spectators have called VOD (frame 520).On the other hand, if there is not VOD symbol 350, then spectators possibly watch broadcast program, maybe possibly supress information button to obtain to comprise the title picture 322 (frame 522) of programme information 328.Handle 500 and can repeat (frame 502), use to continue monitoring spectators and/or media apparatus.

Use frame collector 108 and OSMSD 112 can detect one or more spectators' activities and corresponding resource, include, but are not limited to that the channel that spectators initiate with STB 104 changes, title is browsed, VOD, TV guide service, video-game and/or the demonstration of many pictures carried out through STB 104 and/or independent video game console watch.

The OSMSD that channel changes detects

To pass activity on the screen of monitoring one or more channels change indicating devices through discussion, use frame collector 108 and OSMSD 112 to detect the situation that television channel changes based on the previous description of Fig. 1 now to Fig. 4 E.Specifically, the frame collector 108 that receives in fact the signal of telecommunication identical with TV 110 is periodically gathered " screenshotss " that are presented at the image on the TV 110.Screenshotss are the numerals that are presented at the image on the TV 110 in particular moment.These screenshotss comprise one or more region-of-interests, and for example the ROI 306,314 and 316 of Fig. 4 A wherein shields upward movable (or not upward activity of screen) and can be used for confirming whether television channel change has taken place.As stated, many TVs 110 that possibly receive STB 104 control are suitable for when channel changes channel is presented at temporarily the upper right corner (for example ROI 306) (or some other presumptive areas) of TV viewing area 304.Therefore in this example, can ROI 306 be used as a ROI, and can carry out level and vertical scanning, and be converted into black and white image ROI 306.Can check the view data of gained then, to confirm whether to have shown in this zone one or more numerical digits of expression channel.Whether represent channel and therefore can indicate the channel change has taken place in order to confirm to be presented at content among the ROI 306, one or more characteristics (for example comprising height, width and content center) of displaying contents are detected and measure.These characteristics are organized in TV 110 or the STB 104 in manufacture process in advance, and make the mode that is used for controlling channel demonstration on screen by TV 110 or STB 104.Therefore, show that in a ROI 306 have in advance the content of the channel characteristic of establishment can be used for indication the channel change possibly take place, and be used to trigger the prompting on personnel's measuring instrument 116 for example.

According to the running status of TV 110 and/or STB 104, indicated channel number possibly be not enough to proof the channel change has taken place in a ROI 306.For example, the detected numeral that is presented among the ROI 306 maybe be relevant with the programme content that is showing, and it doesn't matter with the channel change.In this case, the numeral of demonstration shows that with those and actual channel number the numeral that is associated maybe be luckily similar.For example, television advertising can show contact phone number on screen, and this number rests on the same ROI that is used for showing tuning in to channels.In another example; Different TVs 110 and/or STB 104 can carry out an activity that is called " title is browsed "; This activity allows spectators to continue to watch first channel; Make TV (for example in the Title area 312 of Fig. 4 A) show the title guide simultaneously, the title guide has comprised and this moment or other times available program-associated information on one or more other channels.The information that is included in the title guide can comprise channel number information and/or program identification information (for example programm name).Term " is browsed " through being usually used in describing this behavior because many television systems 101 allow spectators through repeat to press in the remote control 114 designated button or through designated button is overviewed keeping the predetermined minimum period depressed position fast with a plurality of channels in each relevant information.In addition; As the response that channel is changed, can carry out some TVs 110 and/or STB 104 that title browses and also will show that position on the same screen of channel (for example ROI 306) shows the channel (the title guide is providing the channel of its information) that title is browsed.In addition, can utilize with being used to of being stored and show that changing the relevant identical characteristic of channel with actual channel shows the channel that title is browsed.Therefore, in a ROI 306, detect numerical digit and possibly not correspond to the channel change, but possibly represent to browse the channel that obtains its information through title.

After this will miss measured channel changes to confirm to be defined as in a ROI 306, to detect and does not correspond to actual channel and change incident and be identified as the numerical digit of having represented that channel changes by error.Change the possibility of confirming in order to reduce the mistake measured channel, can check other region-of-interests and/or can monitor empty screen.For example, can check the 2nd ROI, for example background area or static region (the for example ROI 314 of Fig. 2).Static region ROI314 can constitute the part of video screen, when among the ROI 306 during indicated channel, shows pure color in this part.Therefore, if static region ROI 314 does not show solid background when having detected the channel change, it possibly be that mistake is surveyed that then detected channel changes.Therefore yet as stated, when spectators' browsing title and when static region ROI314 shows solid background, some TVs 110 and/or STB 104 are presented at channel among the one ROI306.In this title navigation process, be presented at channel among the ROI 306 and do not represent the channel of the current channel of being watched, also do not represent to have taken place the channel change.On the contrary, outside the current channel of being watched of institute's channel displayed number expression and spectators seek the channel of its information.

Can do not thought to represent that actual channel changes the channel of incident in order to ensure this title number of browsing, can also check the 3rd ROI (the for example ROI 316 of Fig. 4 A) by mistake.The 3rd region-of-interest can be positioned at the for example upper left corner of video screen; Because when many TVs 110 and/or STB 104 all are configured to showing the title browsing channel in the upper right corner (for example ROI 306) when viewing area 304, actual channel number is presented at the upper left corner (for example ROI 316) of viewing area 304.Therefore, if in a ROI 306 and the 3rd ROI 316, all detected channel, then detected channel is not represented channel change event in a ROI 306.

In case in a ROI 306, detected channel, just can it and channel displayed before current channel displayed number number have been compared.If channel does not then take place and changes with current just number identical in channel displayed in channel displayed number as yet before.In fact most possibly be to show current channel of being watched in order to obtain information.Comparatively speaking, if the channel change then possibly take place with current just number different in channel displayed in channel displayed number before.

Fig. 6 A provides a flow chart, this flow chart illustration be used to determine whether to take place the exemplary process 600 of channel change event.Handling 600 can be by OSMSD, and for example exemplary OSMSD112 or 200 carries out, and can be implemented as the one or more software programs that are stored in one or more memories and carried out with any-mode by one or more processors.Yet, can manually carry out and/or carry out and handle 600 some or whole frames through one or more hardware units.Though with reference to illustrative flow chart description in Fig. 6 A handle 600, one ordinarily skilled in the art will readily appreciate that and can use many other to carry out the method for processing 600.For example, can change the order of the many frames in these frames, can change the operation of one or more frames, can make up the operation of one or more frames, and/or can remove one or more frames.

In general, exemplary process 600 scannings and continuous acquisition to one or more ROI (the for example ROI 306,314 and 316 of Fig. 4 A) of being associated of the frame of video of predetermined quantity, with confirm continuous acquisition to frame whether all shown the numerical digit that the expression channel changes.In the illustrative example of institute, the predetermined quantity of frame still also can adopt other frame numbers corresponding in the quantity that the successive frame of channel takes place to have shown when channel changes.Especially, when channel taking place changing, TV 110 and/or STB 104 show scheduled time length with the channel of reselecting usually, and this scheduled time length is enough to make spectators to see and channel-identification number.This scheduled time length will be controlled the quantity of the successive frame of wherein wanting indicated channel number, and will be according to just changing in the setting of monitored TV 110 and/or STB 104.If in a ROI (for example ROI 306), detected this numerical digit, then can also check one or more other region-of-interests that are associated with same frame of video, will cause mistake to survey the possibility of confirming to reduce detected numerical digit.

Exemplary process 600 starts from frame 602, and wherein the frame collector 108 of Fig. 1 is gathered and the corresponding screenshotss of video signal frame that are delivered to TV 110 from for example STB 104.The screenshotss that collect can be corresponding to the whole video frame, or only corresponding to the part of frame.Next, OSMSD112 extracts one or more ROI (frame 604) from the screenshotss that collect.For example, OSMSD 112 can extract three independent ROI (the for example ROI 306,314 and 316 of Fig. 4 A) and be stored in the memory.In leaching process, the method through any type converts ROI into gray level image.Can the gray level image of gained further be converted into many group digital datas of representing gray level image with the arbitrary format type.Control proceeds to frame 606 then, at these OSMSD 112 levels and vertical scanning the one ROI 306, determines whether to exist numerical digit (frame 606).Specifically, through individual element ground from left to right (or opposite) read corresponding screenshotss data and carry out the horizontal sweep of a ROI 306, and through individual element ground from top to bottom (or on the contrary) read frame data and carry out vertical scanning.In scanning process, will be black and white (for example binary system) image corresponding to the greyscale image transitions of ROI 306 based on one or more quantization thresholds.Discuss this conversion process in more detail below in conjunction with Fig. 7 and Fig. 8.

Whether there is expression to have the characteristic of one or more numerical digits in the inspection black and white image data.Specifically, with reference to Fig. 9, can each numerical digit be divided into one or more parts (component).Through detecting one or more in these parts, and one or more characteristics of these parts of measurement/record, OSMSD 112 can detect whether there are one or more numerical digits that formed by this part.For example; According to the one or more parameter/characteristics that are stored in TV 110 and/or the STB 104; The channel and the part thereof that are presented among the ROI 306 will have predetermined width/or width/scope; Can not be regarded as potential part thereby any width/that shows is not equal to the content that preset width/height or width/exceed outside the preset range, therefore can it be abandoned.Similarly, Arabic numerical digit (just 1,2,3,4 etc.) does not have horizontal clearance (horizontalgap), and the content that therefore has this gap arbitrarily can not be regarded as potential numeral, therefore can it be abandoned.Horizontal clearance is defined as the zone that comprises horizontally disposed pixel column, and this pixel column is extending on the whole width of numerical digit and is being arranged between the outermost end vertical edge (the just top of numerical digit and bottom) of numerical digit.The value of the pixel in the horizontal clearance is equal to each other in fact, but in fact with corresponding to demonstration numerical digit pixel value different.

Similarly, can content mode placed in the middle in a ROI 306 be compared with the character characteristic placed in the middle of being stored.If content be according to the character of being stored characteristic placed in the middle and placed in the middle, then this content can be represented numerical digit.Carrying out horizontal sweep or vertical scanning or the two; And checked after the mapping of numerical data of gained that OSMSD 112 uses check results to increase progressively numerical digit detection counter (under the situation that has detected numerical digit) or removes Digital Detecting counter (under the situation that does not detect numerical digit as yet).

As those skilled in the art will appreciate that, being confirmed to be quantity that numerical digit (and then causing the Digital Detecting counter to increase progressively) before must detected characteristic in institute's content displayed can handle 600 the needed certainty of system and become according to carrying out.For example, along with the increase of the feature detection consistent with the numerical digit demonstration, the certainty that displaying contents is actually numerical digit also increases.Along with the inspection to region-of-interest is more comprehensive, processing complexity also possibly increase.For example, handling 600 possibly require the mapping of numerical data to comprise and the corresponding to content of height, width, spacing and/or character characteristic placed in the middle of being stored.

As those skilled in the art will appreciate that; Can the one ROI306 and any ROI thus be scanned/check by the method for any type of the arbitrary characteristics that can monitor one or more numerical digits part and/or numerical digit part, to determine whether to exist numerical digit: this method can comprise following technology: check the height of the location/position at any edge of these parts, any part, any partial width, with the quantity of the isolated any down suction of arbitrary portion and/or width (be used for confirming whether these gaps equal or the scope of expection and preset distance between two digital numerical digits of expression multidigit of dropping on interior), do not have any horizontal clearance etc.One ordinarily skilled in the art will readily appreciate that and to use STB 104 and/or TV 110 peculiar other features/parameters to confirm whether the numerical digit that is presented in the channel viewing area (for example ROI 306) should be regarded as channel.

In the example of Fig. 6 A; If OSMSD 112 is because the characteristic (if any) of detected part is inconsistent and to determine the content that is presented among the ROI 306 can not be numerical digit (frame 608) with the characteristic of being stored, then whether OSMSD 112 inspections can use another quantization threshold that the ROI image quantization is the black and white form.That kind as discussed in detail below, the different quantized threshold value can produce the black and white ROI with better marginal definition.If can use another threshold value (frame 609), then frame 606 is returned in control, and wherein OSMSD 112 converts gray scale ROI image into black and white image once more, once more image is scanned to determine whether to exist numerical digit.Control then according to aforesaid way and carry out.Yet if cannot use another threshold value (frame 609), OSMSD 112 removes numerical digit detection counter (frame 610), and frame 602 is returned in control.On the contrary; If the characteristic consistent (frame 608) of the characteristic that OSMSD 112 determines the display part and the numerical digit that is used for indicated channel number of being stored; Then OSMSD 112 can increase progressively continuous detecting counter (frame 612), with in the frame that shows current collection just at indicated channel number.

Detecting numerical digit and then making under the situation that the numerical digit detection counter increases progressively, OSMSD 112 determines whether to scan the 2nd ROI, for example the background area 314 of Fig. 4 A (frame 614).For example, OSMSD 112 can read certain position in the memory 208, to determine whether scanning the 2nd ROI 314.In the example of Fig. 6 A, detected numerical digit if be among the ROI 306 at frame 608, then control arrives frame 614.Frame 614 can be used for starting through checking whether shown the program that will cause detecting in the effective or invalid information of the detected numerical digit in frame 608 places the mistake survey among the 2nd ROI 314.When in fact having shown the numerical digit of expression channel among the ROI 306; Can the 2nd ROI 316 be defined as a part that comprises Title area 312; In this part, the title guide is compiled in response to the user prompt through for example remote control 114 inputs and occurs.As stated, can with the title guide be configured to provide with available on the current channel displayed or on other channels available program-associated information.In order to represent whether there is the title guide, for example, can the 2nd ROI 316 be defined as the zone that under the effective situation of title guide, will keep static (just pure background colour) in the Title area 312 as the result of channel change event.

Therefore; Because in an illustrative example; The 2nd ROI 314 will only comprise solid background when a ROI 306 comprises channel; So scan the edge (frame 616) of the 2nd ROI 314, exist the edge just to show that the 2nd ROI 314 is only filled by background color, therefore be illustrated among the ROI 306 detected digit representation mistake survey and confirm.Similar with the processing of a ROI 306, in scan process, will corresponding to the greyscale image transitions of the 2nd ROI 314 black and white image based on one or more quantization thresholds.If there is not edge (frame 618) among the 2nd ROI 314, and then show that the background area is static (as desired when the indicated channel), then OSMSD 112 decision that can confirm/confirm to make (just just indicated channel number) at frame 608 places.Therefore, if detected the edge at frame 618 places, then control proceeds to frame 610 and frame thereafter (as stated); In these frames; The numerical digit detection counter is not detected channel as yet with expression, and gathers another frame of video and check (frame 602) by zero clearing in present frame.As those skilled in the art will appreciate that, there are many methods to carry out rim detection, OSMSD 112 can use in these methods any one to confirm whether the 2nd ROI 314 comprises the edge.Be similar to the processing of a ROI 306 in addition; Under the situation that does not detect the edge; If there are a plurality of quantization thresholds to use, then OSMSD 112 can confirm and should not have the edge to attempt conversion once more and/or scanning and the 2nd ROI 314 corresponding black and white images once more before in the zone.

If 618 places do not detect the edge at frame, then OSMSD 112 determines whether to scan the 3rd ROI, for example the ROI 316 of Fig. 3 (frame 620).For example can scan the 3rd ROI 316, with further confirmation detected digit representation channel and corresponding to channel change event in a ROI 306.The user can browse the title guide usually and obtain the information relevant with available programs, and need not in fact to change current tuning in to channels.When having shown the title guide, can TV 110 and/or STB 104 be configured in a ROI 306, show the channel (seeking its information) browsed number, and in the 3rd ROI 316, show the current channel watched number.Therefore, exist numerical digit possibly be illustrated among the ROI 306 detected numerical digit among the 3rd ROI 316 and do not represent that channel changes, but relevant with the title browsing event.

Do not have edge (that is to say that the background area is static) among the 2nd ROI 314 if determine at frame 618 OSMSD of place 112, if perhaps can not scan the 2nd ROI 314, then OSMSD 112 can determine whether to scan the 3rd ROI 316 (frame 620).OSMSD 112 can] reading certain position in the memory 208 once more, whether can carry out to confirm this scanning.If can carry out, then OSMSD 112 scannings the 3rd ROI 316 is to confirm whether numerical digit is shown.Similar with the processing of a ROI 306, in scan process, will corresponding to the greyscale image transitions of the 3rd ROI 316 black and white image based on one or more quantization thresholds.As stated, some TVs 110 and/or STB 104 are configured to when browsing the guide channel and be presented among the ROI 306 (the for example upper right corner of viewing area 304), channel is presented at (the for example upper left corner of viewing area 304) among the 3rd ROI 316.Therefore, if shown the numerical digit with channel indicating characteristic among the 3rd ROI 316, then detected numerical digit can be represented title browsing channel number rather than actual channel (frame 622) in a ROI 306.If a ROI 306 comprises channel and the 3rd ROI 316 comprises channel, the decision of then making to a ROI 306 can be represented the mistake survey.In order to detect the demonstration of numerical digit in the 3rd ROI 316, can use above to inspection the one ROI 306 described any one method/tests.

Therefore, if in the 3rd ROI 316, detected one or more numerical digits (frame 624), then 112 pairs of continuous detecting counter O resets of OSMSD (frame 610).If do not detect numerical digit (frame 624), maybe can not scan the 3rd ROI 316 (frame 620), then will in a ROI 306, detected channel not be regarded as mistake survey (just they being regarded as effective detection).(should be noted that: be similar to the processing of a ROI 306; Under the situation that does not detect numerical digit; And if have a plurality of quantization thresholds to use, then OSMSD 112 can attempt changing once more with the 3rd ROI 316 corresponding black and white images and/or scanning once more before confirming not have numerical digit in should the zone.) in this case, OSMSD 112 is through confirming whether the channel whether the continuous detecting counter is confirmed to be presented among the ROI 306 above predetermined threshold has shown minimum at least frame number (frame 626) continuously.

If the continuous detecting counter does not surpass predetermined threshold (frame 626), then detected channel does not show enough video frame numbers as yet in a ROI 306, and therefore control turns back to frame 602 and frame thereafter, to gather another video as stated.If the continuous detecting counter has surpassed predetermined threshold (frame 626), then detected channel has shown enough frame numbers in a ROI 306, and control proceeds to frame 628.At frame 628 places, the channel whether current channel that is presented among the ROI 306 of OSMSD 112 inspections collects with combining nearest channel to change is complementary.The current ROI image that collect of OSMSD 112 through will comprising current channel number compares with reference picture that comprises nearest channel displayed number or template and carries out this operation.If the current channel image that collects is complementary with the reference radio channel image/template that collects before, channel does not then take place as yet change.And in fact, spectators' most probable shows the channel of the current channel of being watched for acquired information.Therefore, 112 pairs of continuous detecting counter O resets of OSMSD (frame 610), and collection another frame of video (frame 602) is returned in control.If current channel displayed number and channel displayed before number do not match, the channel change has then taken place.In this case, OSMSD 112 stores current channel ROI image, as showing the reference picture/template (frame 630) that compares with follow-up channel.OSMSD 112 reports channel change event (frame 632) then.In addition, as stated, the channel change event of being reported (frame 632) can make the personnel measuring instrument 116 transmission information of OSMSD 112 to Fig. 1, thereby makes personnel's measuring instrument 116 prompting audience members show that it is present in the middle of the spectators.

Except changing, handle 600 and just can also for example preserve and/or report in channel displayed number from the purpose of channel identification at frame 632 places reports channel.It will be appreciated by the skilled addressee that the method for many preservations and/or report channel.For example, can gather the digitized image of channel and it is stored in the memory 208, so that send central processor equipment 222 to through network 218 after a while.As another example, OSMSD 112 can convert the digitized image of channel into numerical value by the arbitrary image recognition technology.Can be with this value storage in memory 208, so that send central processor equipment 222 to through network 218 after a while.In another example, OSMSD 112 can with the channel image that collects and the expression of being stored might channel a series of templates of image compare, till detecting coupling.Can the digital information that identify the channel of being represented by matching template be stored in the memory 208 then, and/or transmit, and owing to be used for current channel number is reported to central processor equipment 222 through network 218.But the channel of central processor equipment 222 operation reports confirm with for example monitored spectators, by the channel watched, the program watched etc. relevant watch statistics.Description below in conjunction with Figure 11 A, 11B and 11C discusses channel identification in more detail.

Illustrated in the flow chart of Fig. 6 B and be used to determine whether to take place another exemplary process 650 that channel changes.This exemplary process is similar to exemplary process 600, but comprises that also use reduces the frame that mistake is surveyed the possibility of confirming from the information of remote control 114.Because have much overlappingly between the flow chart among Fig. 6 A and the 6B, institute thinks for simplicity, will can not be described among two figure the in fact similarly frame that occurs simultaneously here once more.In fact, interested reader can obtain the complete description of corresponding frame with reference to the above description of figure 6A.In order in this process, to help the reader, come in fact similarly frame of mark with identical label in the drawings.In addition, in order to reduce the complexity of Fig. 6 B, in exemplary process 650, omitted the retry routine frame 609 among Fig. 6 A.Yet, persons of ordinary skill in the art will recognize that can identical retry routine be inserted into handle in 650 with Fig. 6 A in processing 600 similar positions, or be inserted into any other appropriate positions.

Fig. 6 B and Fig. 6 A are compared, thereby cause under the situation that the Digital Detecting counter increases progressively (frame 612) detecting numerical digit, the OSMSD 112 of the processing 650 of execution graph 6B confirms whether remote control detectors 213 can use (frame 602).For example, OSMSD 112 can read certain position in the memory 208, to determine whether the using detection signal from remote control 114 to determine whether to have taken place channel change event.If remote control detector 213 can be with (frame 652), then whether OSMSD 112 definite remote control detectors 213 receive in detecting numerical digit time window before and/or have decoded from the signal (frame 654) of remote control 114.If remote control detector 213 receives and/or the signal of having decoded (frame 654), perhaps remote control detector 213 can not be used (frame 652), and then control proceeds to frame 614, and the operation of frame 614 was described in the above.If remote control detector 213 does not receive signal (frame 654); Then control proceeds to frame 610 and frame thereafter (as stated); Wherein the Digital Detecting counter is not detected channel by zero clearing to be illustrated in as yet in the present frame, and gathers another frame of video and check.

It will be appreciated by the skilled addressee that and to assist channel-identification to change incident in the information that the use of the optional position in exemplary process 600 and 650 receives from remote control.For example, can in a ROI 306, detect numerical digit after (for example after frame 612) use the frame 652 of Fig. 6 B and 654 remote control trace routine.Alternatively, can check the second or the 3rd ROI (being respectively 314 or 316) (for example frame 618 or 624) use remote control afterwards trace routine, or the remote control trace routine is being checked a ROI 306 initial step before with dealing with in 600 or 650.Similarly, detecting remote information can be as carrying out the triggering signal of handling the one or more frames in 600 or 650.

Image quantization

Though above example and discussion relate to the detection of channel, OSMSD 112 is not restricted, also can detected symbol and/or icon.Specifically, OSMSD 112 can detected symbol, " TV guide " symbol 332 shown in Fig. 4 B for example, and/or detect Web broadcast icon, for example " HBO " icon 324 shown in Fig. 4 B.Among Fig. 7 illustration ROI 306 and/or 316 the exemplary ROI 660 of representative graph 4A.As stated, ROI 660 can be extracted from the screenshotss that frame collector 108 collects by OSMSD 112.In order to reduce amount of memory storage and/or processing requirements, can gray scale ROI 660 be quantified as black and white shown in Figure 8 (for example binary system) image 700.In the example of Fig. 7, ROI 660 comprises channel 664, solid background (static region) 670 and the zone between two numerical digits 672.In Fig. 8, will be illustrated as ROI 700 through the same exemplary ROI of OSMSD 112 after gradation conversion is black and white.Can find out that from Fig. 7 and Fig. 8 the channel 664 of light shade is converted into white channel 704.Zone 672 between background 670 sum digits of dark-shaded is converted into black region 710 and 712 respectively.In order to carry out the conversion to ROI 700 from ROI 660, can use light color and/or dark threshold value to set strength grade, all pixels that are higher than this strength grade all are converted into white, and all pixels that are lower than this strength grade all are converted into black.Use this quantization threshold that gray scale (or colored) image transitions has been simplified rim detection and/or channel comparison program simultaneously as black and white image.For rim detection (for the channel in ROI 306 and/or 316 detect or static region ROI 314 in feature detection) for, become pixel column and/or the pixel column of white (or the display polaritic that uses according to particular set-top box 104 becomes black from white) when corresponding program is identified in image by vertical and/or horizontal sweep from black.In addition, can optional polarity parameter be offered OSMSD 112, come the initialization edge detection procedure with leading background color (for example white or black) by the expection of particular set-top box 104.Discuss in more detail below and be used for channel corresponding program relatively.

According to the characteristic of the background area in one or more region-of-interests (for example ROI 306 or 316) be presented at the programme content in the viewing area 304, single quantization threshold possibly be not enough to produce robust performance.For example, if the background area 308 among the ROI 306 is transparent or semitransparent, then channel displayed numbers 310 and the intensity difference between the background 308 will depend on the intensity of the display program content that overlaps with a ROI306.For example, if represent channel 310 by light color (for example white), then intensity in a ROI 306 less relatively coincidence programme content maybe with intensity bigger content different quantized threshold value in this zone.Therefore the exemplary process 600 of Fig. 6 A is supported a plurality of quantization thresholds of use of frame 609 expressions.

In exemplary process 600, can select and use quantization threshold as getting off.At first, based on the indicating characteristic of STB 104 and/or TV 110 baseline threshold is calibrated.Many STBs 104 can be provided baseline (reference) gray scale numerical value by inquiry with TV 110.Next, can with image quantization each some place of black and white form in 600 handling, carry out first by first knowledge quantization threshold and quantize to attempt.If obtained positive result (for example detecting numeral under the situation of ROI 306 and/or 316 or under the situation of ROI 314, detecting the edge) based on quantized image, then control proceeds to subsequent point suitable in the program.On the contrary, if obtained negative decision, then attempt another threshold value.For example, can confirm other threshold values (for example depart from initial threshold+/-4% ,+/-10% etc. threshold values) with respect to initial threshold.Can repeatedly attempt, so that image is quantized, till one group of quantization threshold is by limit, at this moment, the corresponding path of the negative decision of respective point in 600 will followed and handle to control.Persons of ordinary skill in the art will recognize that and to use many technology to confirm that one or more quantization thresholds, aforementioned content are exactly an example of these technology.

Characteristics of image

Fig. 9 illustration the one group example feature relevant with the ROI of Fig. 8 700, this stack features can be to be detected and/or inspection to confirm whether institute's content displayed has constituted the channel demonstration.For example, the distance between last horizontal edge 802 and the following horizontal edge 804 defines character height 818, when determining whether to exist on the screen channel, can with character height 818 with above and/or under expectation character height threshold value compare.Opposing vertical edge 805,806,808,810,812, the distance between 814 define character duration 824,828,832; When determining whether to exist on the screen channel, can with these character durations with above and/or under expectation character duration threshold value compare.In addition; Opposing vertical edge 805,806,808,810,812, the distance between 814 can be used for limiting the interval 826,830 between the character; When determining whether to exist on the screen channel, can with these at interval with above and/or under expectation character pitch threshold value compare.In addition, suitably the distance between the edge can be used for limiting the character placed in the middle 816,820,822,834 that shows numerical digit, when determining whether to exist on the screen channel, can with these characters placed in the middle with above and/or under expectation character threshold value placed in the middle compare.As those skilled in the art will appreciate that, can use one or more edge detecting technology to discern one or more marginal positions, can utilize the distance between these edges to limit numerical digit height, width etc. then.Alternatively, can and go up one group of pixel with same color characteristic of extending at specific direction (for example horizontal direction) and count setting adjacent one another are, to confirm the width of numerical digit.Similarly, can count, to confirm the height of numerical digit the neighbor that extends in vertical direction with public color.

Change detection processing (the for example previous processing of describing 600 or 650) at channel and in suitable ROI (for example ROI 306), detect channel based on for example above-mentioned characteristic; And definite numeral that is shown does not also correspond to after the title browsing event, and the processing meeting compares detected channel and previous channel (the for example frame 628 of Fig. 6 A).Though can use any technology to carry out this comparison, method for optimizing be based on calculate and comprise current channel number ROI quantized image and stored and previous channel number corresponding reference picture/template between dot product.As stated, the ROI that comprises current channel number is gathered by frame collector 108, and is black and white image by conversion (quantification).Similarly, stored with previous channel number corresponding reference picture/template also be black and white image, based on the characteristic of STB 104 and/or TV 110, this black and white image can corresponding to the ROI that had for example before collected or store the numeral ideal represent.In either case, all the respective pixel through will quantizing the ROI image and institute's reference image stored/template multiply each other and calculate dot product, thus in the middle of each location of pixels forms one group inner product.Then inner products in the middle of these are added up and form dot product.Say that from mathematics dot product is provided by following expression formula:

$\mathrm{dotproduct}=\underset{r}{\mathrm{\Σ}}\underset{c}{\mathrm{\Σ}}\mathrm{image}(r,c)\×\mathrm{template}(r,c)$

Wherein image (r, c) the ROI image that collects of expression is in location of pixels (r, the value of c) locating; Template (r, c) expression institute reference image stored/template is in location of pixels (r, the value of c) locating; And index (r c) refers to the location of pixels of the capable and c row of the r of correspondence image.

For black and white image, white pixel can be represented with the value that equals 1, and black picture element can be represented (noting: can use relative arbitrarily or other representations that are equal to) with the value that equals-1.So the inner product of two same pixel values has obtained value 1, and the inner product of two different pixel values has obtained value-1, just:

White x white=1x1=1;

Black x black=-1x-1=1;

White x black=1x-1=-1;

Black x white=-1x1=-1.

Therefore, if the ROI image that collects and institute's reference image stored/template class are seemingly, then mutually summation longways of inner product, the dot product of gained will become bigger on the occasion of.On the contrary, if two image dissmilarities, then inner product with summation mutually with disappearing, and the dot product of gained will drop on from less on the occasion of in the scope the bigger negative value.

To the definition of dot product and the expression of respective image, can use a large amount of useful standards previous channel that the channel of the ROI that collects and institute's reference image stored/template is represented number to compare based on previous.A kind of preferred this standard is quantity equivalent standard (number-equivalencecriterion), and this standard is defined as the ratio between the sum of all pixels in dot product and the image, just:

$\mathrm{criterion}=\frac{\mathrm{dotproduct}}{\mathrm{numpixels}}=\frac{\mathrm{dotproduct}}{\mathrm{RC}}$

Wherein numpixels is the sum of all pixels in the image, and it equals the pixel count of R in capable and the product of the pixel count in the C row.Suppose that pixel value is 1 or-1 for black and white image, then the equivalent expression of the quantity equivalent standard of above definition comprises:

$\mathrm{criterion}=\frac{(\mathrm{num\; same\; pixels}-\mathrm{num\; diff\; pixels})}{\mathrm{numpixels}};$

$\mathrm{Criterion}=\frac{(\mathrm{Num\; same\; pixels}-\mathrm{Num\; diff\; pixels})}{(\mathrm{Num\; same\; pixels}+\mathrm{Num\; diff\; pixels})};$ Or

criterion＝ratio?of?same?pixels-ratio?of?different?pixels

The quantity of the location of pixels that wherein the num same pixels ROI image representing to collect is identical with the value of the template of being stored, and num diff pixels representes to be worth the quantity of pixels with different position.The amount of ratio of same pixels equals the sum (numpixels) of num same pixels divided by location of pixels, and the amount of ratio of different pixels equals num diff pixels divided by numpixels.According to performance, one in these previous expression formulas maybe be more more preferred than other expression formulas.

Channel change to detect handles that the represented previous channel of the represented channel of ROI image that (for example handling 600 or 650) can use the quantity equivalent standard of above definition to confirm to collect and institute's reference image stored/template is number similar in fact still to be different in essence.For example, if standard has the threshold criteria value more than or equal to 95%, then this processing can confirm that channel comes down to similarly, therefore channel does not take place as yet change.If the value of standard is less than 95%, then this processing can confirm that channel is different in essence, and therefore confirms to have taken place the channel change.

With reference to Figure 10,, can check that displaying contents is to confirm whether it comprises the part of representing numerical digit in order to reduce to be used for confirming whether displaying contents representes the complexity of any particular algorithms of numerical digit.For example; Can numeral 1 be regarded as having the numerical digit of single part; This part comprise one group adjacent and have a pixel 950 of public colored/shaded; These pixel 950 vertical extent the location of pixels of predetermined quantity, wherein based on the altitude feature that is stored in associated television or the STB, the location of pixels of this predetermined quantity is represented desired numeral height.Through further for example, can numeral 3 be regarded as having three horizontally disposed parts 954,956,958 and single vertically disposed part 952.If detected part has proper characteristics (characteristic that the TV 110 that just shows with control figure and/or the indicating characteristic of STB 104 are complementary), then OSMSD 112 can show that having detected screen goes up channel and show.For example, OSMSD 112 can compare the one or more position in one group of detected part and one group of section construction that shows corresponding to numerical digit of being stored.If confirm coupling, then OSMSD 112 can show that having detected screen goes up the channel demonstration.OSMSD 112 can further use and mate the relevant information of structure and just discern the value in channel displayed number.In another example, OSMSD 112 can be not compare the arrangement of a plurality of parts each other and/or compares with the section construction information that is stored in the memory.In fact, OSMSD 112 can only check the characteristic of part, shows the desired stack features of numerical digit part if this part has one group, and then OSMSD 112 can be identified as this part and represent upward numerical digit demonstration of screen.

Channel change to detect is handled (for example handling 600 or 650) and can be expanded to and not only detect the channel that channel change event but also identification are reselected.Among Figure 11 A illustration a kind of exemplary process 1000 that is used to discern indicated channel number.Other that possibly need at given channel-identification number are handled under resources and the situation of time, and exemplary process 1000 can be carried out background process concurrently and change detection processing 600 or 650 with the prospect channel.In order to realize such execution, 1000 unsettled channel ROI images to be identified such as adopt that formations are stored are handled in example channel identification.With reference to Figure 11 A, exemplary process 1000 comprises the son processing 1002 and 1004 (although also can be configured to carry out with serial mode with handling 1002 and 1004) of two executed in parallel.The purpose that son is handled 1002 (queue managements) is that channel ROI image is added in the image queue.The purpose that son is handled 1004 (queue processing) is to handle to be stored in the ROI image in the image queue.

Sub-processing 1002 has been shown among Figure 11 B, and wherein control starts from frame 1005, and wherein OSCCD112 determines whether the new channel ROI image output of exemplary process 600 or 650 (for example from).If there is not new image, then frame 1005 is returned in control, and OSMSD 112 waits for and new image occurs.On the contrary, if new image is arranged, the state of OSMSD 112 check image formations then.If image queue is less than (frame 1006), then OSMSD 112 adds new channel ROI image in the formation (frame 1008).After adding to image in the formation, control turns back to frame 1005, and OSMSD 112 waits for and new channel ROI image occurs.Yet, if OSMSD 112 confirms that at frame 1006 places formation is full, possibly cause error condition, thereby cause exemplary process 1002 to withdraw from.

Sub-processing 1004 has been shown among Figure 11 C, and wherein control starts from frame 1010, and wherein OSMSD112 confirms whether image queue includes channel ROI image.If formation is empty, then frame 1010 is returned in control, and OSMSD 112 waits for that image is added in the formation.On the contrary, if formation is not empty, then control proceeds to frame 1012, and OSMSD 112 takes out under formation is the situation of first in first out (FIFO) configuration and is stored in the oldest image in the formation.According to the application-specific of channel recognizer, the configuration of can using also that last in, first out (LIFO).In either case, obtain next image (frame 1012) to be identified afterwards at OSMSD 112, control proceeds to frame 1014, and wherein OSMSD 112 takes out the channel image template from memory.The channel image template is corresponding to one in one group of template of one group of known channel.OSMSD 112 compares (frame 1016) with the channel ROI image that collects and this template image then.If two images do not match, then control proceeds to frame 1018, and wherein OSMSD 112 determines whether the template of another storage.If another template is arranged, then control turns back to frame 1014, and wherein from memory, takes out should next one template for OSMSD 112, and controls and as above-mentioned, carry out.If there is not another template (frame 1018), then OSMSD 112 generates and shows that channel discerns unsuccessful report.Control is returned frame to 1010 subsequently and as above-mentioned, is carried out.

Yet if OSMSD 112 determines the channel ROI image that collects and the channel image template of being stored coupling (for example, calculating and the quantity equivalent standard based on above-mentioned dot product) at frame 1016 places, control proceeds to frame 1022.At frame 1022 places, OSMSD 112 generates the channel in the ROI image that collects is identified as the report corresponding to the channel of matching template.Control can turn back to frame 1010 subsequently and as above-mentioned, carry out, and perhaps control can proceed to the one group of frame that is included in the frame 1024 alternatively.

The processing that frame 1024 is carried out can be used for revising channel change detection processing (for example handling 600 or 650) through replacing with the storing template with known features based on the channel image that had before collected (for example in the frame 628 of Fig. 6 A, using).If frame 1024 ability usefulness are then controlled from frame 1022 beginnings and are carried out as follows.After identifying channel based on selected template (frame 1022), control proceeds to frame 1026, and wherein OSMSD 112 confirms whether the channel that identifies still is used as channel change event and detects the reference radio channel number of handling (for example handling 600 or 650).For example; OSMSD 112 can check the ROI image that collects whether to confirm it corresponding to the reference radio channel image that collects, and this reference radio channel image has been used to determine whether to take place channel change event (for example based on label in the numeral that is included in each image and/or header information).If the channel that identifies is not used (frame 1028), then frame 1010 is returned in control, and as above-mentioned, carries out.On the contrary, if the channel that identifies still in usefulness, is then controlled and proceeded to frame 1028, wherein OSMSD 112 confirms whether the reference picture or the storing template that collect are used as the reference radio channel image.If storing template is using (frame 1028), then frame 1010 is returned in control, and as above-mentioned, carries out.Yet if the image that collects is used as reference radio channel image (frame 1028), OSMSD 112 can replace with the corresponding templates of confirming at frame 1022 places with the reference picture that collects.Control turns back to frame 1010 subsequently, and as above-mentioned, carries out.

Can with channel identification handle for example be loaded into companion's processor (companion processor) or even central processor equipment 222 be used for handling.In this configuration, the ROI image that collects (for example through interface circuit 210) is sent to companion's processor in OSMSD 112 meetings and/or (for example through network 218) sends to central processor equipment 222.

Except channel change event detection and channel identification, also can illustrative methods of describing in the literary composition and equipment be applied to other application.For example, can be to one or more the making amendment in ROI, the template of being stored and/or the fixed reference feature, thus make OSMSD (for example OSMSD200) can detect the variation of the letter/text that is shown, and/or also discern this letter/text.In another example, can detect and/or discern specific shape/object based on suitable selection to ROI, the template of being stored and fixed reference feature.

Also can be with OSMSD (for example OSMSD 200,250) as the image processing of any amount and/or recognizer the front end filter or the gate of (showing reader on the screen of the corresponding institute of the frame of video images acquired that for example is used to handle and receive).In this application, can the image that OSMSD200,250 (and/or being similar to the processing that channel changes detection processing 600 or 650) is used for only selecting to comprise expectation content (for example optional network specific digit, letter, object etc.) be handled.As stated, can OSMSD 200,250 (or similar processing) be configured to based on the numeral that the suitable selection of ROI, the template of being stored and/or fixed reference feature is detected expectation, letter, object etc.Like this, OSMSD 200,250 (or similar processing) just can reduce relevant image processing and/or the recognizer demand to processor and memory significantly.

Can also confirm watching mode and media apparatus source through the ROI order.The different characteristics of watching can and/or be called the order that shows unique discernible image, numeral, symbol and/or icon in the characteristic procedure after the user calls characteristic.For example; If the user has called paying and watched (PPV) characteristic, then can one or more affirmation pictures, one or more request spectators retrieved picture and PPV selective listing, corresponding viewing time, screenshotss, movie promos, the program description of waiting for when PPV selects and/or watch price to present to audience.Because whether each picture all occurs with predefined procedure, be this predefined procedure so OSMSD 112 can monitor one or more ROI, with identification spectators behavior.Though following example has comprised the ROI sequence detection to STB PPV, this is also nonrestrictive, also can similarity ought to be used VCR, DVD player and/or game console.Specifically, the manufacturer of different media apparatus can show different picture orders usually when media apparatus starts.For example, game console can show the trade mark (the for example Xbox (registered trade mark) of Microsoft (registered trade mark)) of the title and/or the game console of manufacturer before the peculiar picture of video-game that shows any specific.In addition, VCR can show unique screenshotss order to it when the user arranges VCR to come the recorded announce program.

Figure 12 A-12E illustration the exemplary screen sequence that after the user calls exemplary PPV characteristic, is shown.Figure 12 A illustration for example press the initial picture that is shown behind the PPV button on remote control or the STB spectators.OSMSD 112 can monitor title ROI 1202, message box ROI1204, and/or the one or more sub-ROI in title ROI 1202 and/or the message box ROI 1204.OSMSD 112 can monitor trade (brand) name (for example, " iCTRL " 1208) and/or the certain tones Taoist monastic name 1210 whether PPV symbol 1206, PPV characteristic are arranged in the title 1202 particularly.In addition, whether OSMSD112 can have identical brand name 1212 and/or phrase " just a moment,please a moment " 1214 in the supervisory messages frame 1204.Next exemplary PPV characteristic can show the picture that has wherein only kept message box 1204 and do not had title 1202, shown in Figure 12 B.When the PPV characteristic continues initialization and/or obtains available PPV the rendition list, simple text based may occur and " please wait for " picture 1216, shown in Figure 12 C.Because the PPV program data maybe be very big, so disturbed for satellite television subscribers and weather conditions under the situation that satellite-signal receives to delay, especially user that the user provides information to have the certain hour amount.The information that receives can comprise another " please wait for " picture 1218, and this picture 1218 comprises one group of abundanter figure, shown in Figure 12 D.At last, exemplary PPV picture sequence can end at the narrative description 1226 of one or more screenshotss 1222, pricing information 1224 and/or the PPV program of the rendition list, available PPV program.

In addition or alternatively, whether watching mode identification can exist special symbol to confirm through monitoring by OSMSD 112.Broadcaster and/or set-top box manufacturer realize watching function with measurable mode usually.Therefore, state diagram can the suitable ROI of illustration, and this ROI should be monitored movable to have determined whether follow-up spectators.It is movable to confirm spectators not need OSMSD 112 scannings and handle all ROI, and selectivity ROI scanning needs less OSMSD 112 disposal abilities and the response time of increase.Figure 13 illustration exemplary screen 1303, spectators have selected the VOD characteristic in this picture.This exemplary screen comprises title 1304 and symbol 1306, and symbol 1306 only just appears on the picture in the limited quantity watching mode process of exemplary set-top box 104.As a result, when OSMSD 112 detects symbol 1306, know that any limited quantity state next can occur just allows to monitor specific ROI.

Figure 14 A illustration the example state diagram of exemplary set-top box 104.Have no restriction, other set-top box manufacturer can have the watching mode configuration that is different in essence.Though illustrative example state diagram is a figure in Figure 14 A, this diagrammatic representation is from the illustration purpose, and state diagram can be machine readable format, and has no restriction.VOD picture 1302 among " selection VOD " state 1408 expression Figure 13.The state diagram of state 1408 in Figure 14 A is known in shining upon because " select VOD ", so can detect any succeeding state through the ROI and/or the possibility symbol of monitoring limited quantity.For example, according to " select VOD " pattern 1408, OSMSD 112 oneself need not the detection paying close attention to symbol " B " 1410, symbol " C " 1412 or distributed any ROI of these symbols.Because next possible state is limited to " guide/information " 1414, " watching TV " 1416 or " watching VOD " 1418; Disappearance when so OSMSD 112 only need detect title 1304 and symbol " A " 1306 (expression " guide/information " state 1414); The disappearance of detection symbol " A " 1306 when title 1304 is retained on the screen (expression " watching TV " pattern 1416), or the disappearance of detection title 1304 when symbol " A " 1306 is retained on the screen (expression " watching VOD " pattern 1418).

One routine figure has been shown among Figure 14 B, illustration be used for confirming the exemplary process 1450 that spectators are movable according to picture sequence.Exemplary process 1450 starts from frame 1452, and wherein the frame collector 108 of Fig. 1 is gathered screenshotss, and these screenshotss are corresponding to the frame of video signal that passes to TV 110 from for example STB 104.If OSMSD 112 did not carry out at least exemplary process 1450 (frame 1454) as yet before definite spectators' present mode, then OSMSD 112 must extract all ROI and analyze (frame 1456) from frame of video.On the other hand, if the next one of media apparatus (for example exemplary set-top box 104) possibly state be known, then OSMSD 112 only need extract the subclass (frame 1458) of ROI from frame of video.OSMSD 112 can know current state and all possible succeeding state by means of known state diagram (the for example state diagram shown in Figure 14 A).The state diagram of media apparatus is normally static, and is the function of the firmware program on the media apparatus.Can this state diagram be published is user's manual, or can in being installed to spectators house, confirm through experience before, and/or is stored among the OSMSD 112.

If there is not known symbol (frame 1460) in the frame of video that collects, then handle repetition (frame 1452) to attempt confirming the pattern of exemplary set-top box 104.On the other hand, if detected known symbol, then OSMSD 112 confirm this symbols existence whether with state diagram in the next one in maybe pattern any one consistent.For example, if for " watching TV " 1416 and detected symbol " A " 1306 and do not have title 1304, then there is candidate's next pattern (frame 1462) in preceding mode, and this candidate pattern is identified as " watching VOD " (frame 1464).Yet, if detected the symbol of expectation, but be not complementary with (for example in the state diagram of Figure 14 A) state mapping, repeat exemplary process 1450 (frame 1452).If OSMSD 112 also not exclusively knows the possible state of the institute of media apparatus, just this result may appear.Alternatively, if having the television advertising of known symbol, just this result may appear.Yet,, think that the generation state does not change if known symbol does not appear among the ROI of expectation.

As stated, confirm that spectators' activity and/or the used one or more media apparatus of identification spectators are not limited to change channel and/or STB.Figure 15 illustration exemplary game picture 1500, these pictures can be produced by independent game console, or as the part of set-top box functionality.After collecting the frame of video of TV, OSMSD 112 can be based on the detection of particular game icon 1504,1506,1508 and 1510 being discerned " selecting recreation " picture 1502.In addition or alternatively, can discern " selecting recreation " picture 1502 by other picture symbols (for example top water riglet 1512 in the specific ROI 1516 and/or bottom water riglet 1514).All can comprise with recreation corresponding to game icon 1504,1506,1508 and each solitary play of 1510 and to carry out other relevant picture frame layers.Though OSMSD memory 216,252 can be filled with the exhaustive details relevant with the pattern layout of each recreation, can utilize some public characteristic of each recreation to come conserve memory demand and/or disposal ability.For example, each among recreation " A " 1518, recreation " B " 1520 and recreation " C " 1522 all comprises the public menu graphical 1524 that is arranged in lower right corner ROI.Therefore, OSMSD 112 need not extract ROI information from whole watch the picture, thereby has saved considerable processing resource and/or memory and be used for other purposes.If public menu graphical 1524 once disappeared from the ROI of expectation, then OSMSD can reference with Figure 13 B in the similar state diagram of the state diagram discussed the focus ROI of alternative is analyzed.

OSMSD 112 also can by interaction content (for example the demonstration of many pictures, weather, divine by astrology, lottery ticket and/or help screen) detect User Activity.Active area (active screen) generally includes certain user input, the channel of for example in single many pictures show, preferentially watching, region-of-interest weather data, the birthday by information of importing for the result that obtains divining by astrology, local lottery number and/or in order to obtain the help topic that the result imports from help screen.But each active area all comprises the peculiar distinguished symbol of special characteristic, literal and/or icon usually.For example, Figure 16 illustration exemplary many pictures display frame 1602, it comprises seven news broadcast programs that are shown to the user simultaneously.OSMSD 112 can discern this many pictures by the monogram that is spelled as " NewsMix " among near the ROI the top center of picture 1602 and show 1604.

OSMSD 112 can have no restrictedly to detect other characteristics and/or the operational mode of one or more media apparatus.Though some media apparatus is at predetermined a plurality of characteristics and/or the state (for example being embedded in the firmware) of just having packed into during shipment from manufacturer there, other media apparatus (for example STB) but can become the time spent presents wide range for spectators service and/or characteristic at them.Broadcaster not only provides one group of abundant content of multimedia to the user, but also can supply the diversification service, comprises that home shopping, the demonstration of many pictures, gambling, recreation, VOD and/or other medium strengthen characteristic.Along with the new feature of media apparatus and/or state diagram (for example new DVD player, VCR, STB, video game console and the recreation of carrying out on it) become availablely, can this state diagram be uploaded in the memory 208 and/or other storage devices 216 of OSMSD 112.

A kind of additional features comprises the service of restarting (SOS).The spectators that miss the broadcast program beginning can call SOS and restart this broadcast program, refund and/or temporaty suspension.SOS makes the spectators of the unforeseen delay that spectators can't watch in the regular program time of particular broadcast particularly useful for having received.Figure 17 illustration the exemplary screen shot 1700 of SOS, it comprises symbol, icon and/or the title that supplies OSMSD 112 monitorings and identification among the predetermined ROI.For example, the user can select the SOS characteristic at the push button panel on remote control and/or the STB during the living broadcast of programs 1702.The selection of SOS characteristic caused the SOS characteristic in the ROI of the upper left corner of live 1702 pictures, occurred and confirm title 1704.In addition, lower banner 1706 is shown to spectators, with provide programm name 1708, scheduled broadcasts time 1710 (for example, 8:00-8:30pm) with the current time 1712 (for example, 8:15pm).Before calling SOS, spectators can confirm the degree that broadcast program is missed, and under this exemplary cases, half of program broadcasted.If spectators' decision is made sure response to SOS characteristic affirmation title 1704, and restarts program, then can the user be presented in the remarkable display frame 1714 of SOS.The remarkable display frame 1714 of SOS comprises the SOS symbol 1716 that is arranged in center ROI, and this symbol can be detected by OSMSD 112.Significantly display frame 1714 makes the user be sure of before the beginning of display program 1718, to have started SOS.

OSMSD 112 can detect SOS according to being similar to the mode that combines Figure 12 A-12E to be discussed, and wherein OSMSD 112 detects VOD through the sequence of ROI.Very similar with the example state diagram among Figure 13 B, SOS also can show can be by OSMSD 112 detected measurable ROI sequences.

Among Figure 18 A illustration show the block diagram of reader (OSDR) system 1800 on the exemplary screen.Exemplary OSDR system 1800 comprise can be in fact with Fig. 1 in the similar or identical frame collector 1804 of frame collector 108.The vision signal that with video frame rate was 30 frame/seconds offers frame collector 1804 as input.Frame collector 1804 with set rate (for example every at a distance from 120 milliseconds carry out once or be equal to the speed of 8.3 of per seconds) screenshotss of the vision signal imported of collection.The screenshotss that collect are offered OSDR 1808 to be handled.In this example, OSDR 1808 analyzes each screenshots according to aforementioned screenshotss speed, to confirm whether for example given screenshotss comprise the concern content, and if comprise that just processing comprises the screenshotss of this content.Therefore, the processing carried out of 1808 pairs first screenshotss of OSDR must be accomplished before second screenshotss arrive.Therefore OSDR 1808 possibly need a large amount of processor resources to satisfy this real-time treatment limits.

Figure 18 B illustration adopt the block diagram of OSMSD 1854 as the exemplary OSDR system 1850 of front end filter.OSMSD 1854 in fact can be similar or identical with the OSMSD 112 among Fig. 1.Exemplary OSDR system 1850 comprises the OSDR 1808 that frame collector 1804 and the exemplary OSDR system 1800 that combines among Figure 18 A are discussed.Yet in this example, OSMSD1854 handled the screenshotss that receive from frame collector 1804 with predetermined screenshotss speed (for example, whenever carrying out screenshotss one time at a distance from 120 milliseconds) before this screenshotss being offered OSDR 1808.Therefore, whether OSMSD 1854 can be configured to analyze each input screenshots has the concern content, and the screenshotss that only will comprise this content offer OSDR 1808.Like this, just can reduce screenshotss are offered the Mean Speed (for example significantly less than every at a distance from 120 milliseconds of beginning speed of carrying out screenshotss) of the OSDR 1808 of example system 1850 significantly with respect to the predetermined screenshotss speed of beginning.Through reducing on average to import screenshotss speed, can be illustrated in fig. 19 shown below the demand that reduces 1808 pairs of processors of OSDR significantly.In addition, can OSMSD 1854 be configured to based on given demands of applications unprocessed form and/or the selected screenshotss that quantize form (for example black and white) offered OSDR 1808.

Figure 19 illustration the exemplary process timeline of example system 1850 of example system 1800 and Figure 18 B of Figure 18 A.For each the processing time line 1904,1908 among Figure 19, for 1912; The trunnion axis express time, and vertical axis is illustrated in the needed processor speed of execution handled in the represented time quantum (for example measuring with 1,000,000 instructions of per second or MIPS).Therefore, the area of each processing time line 1904,1908,1912 below all representes to carry out the needed instruction number of handled (MIPS for example ^*Second=1,000,000 instructions).

The sequence of four (4) the individual screenshotss 1916,1920,1924,1928 that the frame collector 1804 by Figure 18 A or Figure 18 B collects has been shown among Figure 19.For the example system among Figure 18 A 1800, represent processing time line corresponding to OSDR 1808 with timeline 1904.OSDR 1808 among Figure 18 A is configured to handle each input screenshots, and as expectation, timeline 1904 shows three processing regions 1932,1936 and 1940 that correspond respectively to screenshotss 1916,1920 and 1924.Because the real-time treatment limits of the OSDR 1808 of Figure 18 A (for example must before next screenshotss (for example screenshotss 1920) arrive, accomplish for the processing of certain screenshots (for example screenshotss 1916)), so OSDR 1808 may need (for example represented by the height of processing region 1932,1936 and 1940) a large amount of processor resources.

The processing time line that OSMSD (the for example OSMSD 1854 of Figure 18 B) handles each screenshots 1916,1920,1924 is represented with timeline 1908.That kind as discussed above can be configured to OSMSD 1854 only in the screenshotss 1916,1920,1924 of black and white (binary system) version, to search for special object (for example, channel, ROI, symbol, icon).Therefore, the processing requirements of OSMSD1854 can be significantly less than the processing requirements of OSDR 1808 (for example represented) by compare with OSDR processing region 1932,1936, the 1940 respectively height and the width that have reduced of OSMSD processing region 1944,1948,1952.

For illustration with OSMSD 1854 (for example; Have the lower reason device demand shown in the timeline 1908) and OSDR 1808 is (for example; Have the higher processor demand shown in the timeline 1904) the potential benefit used together, represent processor timeline with timeline 1912 corresponding to the example system 1850 of Figure 18 B.In example system 1850, shown in processing region 1956,1960 and 1964 difference, OSMSD 1854 is configured to handle each input screenshots 1916,1920,1924 etc.Under the example scenario of Figure 19, screenshotss 1916 comprise the concern content, and screenshotss 1920 and 1924 then do not comprise the concern content.Therefore, can use OSMSD 1954 to select screenshotss 1916 to handle, and abandon screenshotss 1920 and 1924 by OSDR 1808.Therefore, at this moment the 1808 processing 1916 required times of screenshotss of OSDR can expand to the time that is associated with screenshotss 1920 and 1924.In other words, under the situation of the OSDR 1808 of system 1850,, then still can satisfy the real-time restriction that OSDR 1808 is faced if the 1808 required original processing regions 1932 of the OSDR in the system 1800 expand to processing region 1968,1972 and 1976.Therefore; The processor demand of the OSDR 1808 that in the system 1850 that with OSMSD 1854 is the front end filter, uses obviously than the OSDR 1808 of use in example system 1800 low (for example, by processing region 1968,1972,1976 reduce highly represented).

Though described some method, equipment and goods in the literary composition, the coverage of present patent application is not limited to this.On the contrary, present patent application has covered on literal or has fallen into all devices, method and goods in the appended claim scope in the doctrine of equivalents situation.

The name that present patent application requires on June 20th, 2006 to submit to is called the U.S. Provisional Patent Application the 60/815th of " Methods andApparatus for Detecting On-Screen Media Sources "; No. 122 priority is incorporated into this with this temporary patent application integral body by reference.

Claims (33)

1. the method for an identification activity home media source apparatus, this method may further comprise the steps:

Receiving video signals;

Gather the part of said vision signal at region-of-interest; And

Part that collects and the state diagram that is designed are compared never to provide said vision signal in a plurality of home media source apparatus of display device, to discern said movable family source of media device; The state diagram that is wherein designed comprises the multiple arrangement pattern, and each device pattern all is associated corresponding to an image and with next pattern of candidate.

2. method according to claim 1, wherein said state diagram comprise the expectation video image that is positioned at said region-of-interest.

3. method according to claim 2, this method is further comprising the steps of: when in said region-of-interest, detecting the expectation video image, said movable family source of media device and said vision signal are associated.

4. method according to claim 2, wherein said expectation video image comprises at least one in digital picture, letter image, symbol or the broadcasting station identifier.

5. method according to claim 1, wherein said movable family source of media device comprise that STB, video tape recorder, device for reproducing digital video disc, game console, television set, delay viewing apparatus, video request program viewing apparatus, many pictures show at least one in viewing apparatus, internet browsing device or the personnel's measuring instrument.

6. method according to claim 1, the step of wherein gathering the part of said vision signal comprises that this part to said vision signal quantizes, with the generating quantification vision signal.

7. method according to claim 6; The step that wherein this part of said vision signal is quantized comprises colour-video signal converted into the greyscale video signal, converts colour-video signal into black-and-white video signal, or is at least one in the black-and-white video signal with the greyscale video conversion of signals.

8. method according to claim 6, the memory footprint of wherein said quantitation video signal is lower than said vision signal.

9. method according to claim 6, this method is further comprising the steps of: use quantization threshold and generate said quantitation video signal, this threshold value has been specified at least one in chroma, pixel intensity or the picture element density.

10. method according to claim 1, the step of wherein gathering the part of said vision signal comprises this a part of edge that detects said vision signal.

11. method according to claim 10, a part of edge of this of wherein said vision signal comprise this a part of static region around said vision signal.

12. method according to claim 1; This method also is included in the step that second region-of-interest is gathered the second portion of said vision signal; This second region-of-interest is a static region; And this method is further comprising the steps of: if said static region does not exist, then ignore this part of said vision signal.

13. discern the method that the home media source apparatus changes for one kind, this method may further comprise the steps:

Whether the region-of-interest of monitoring video signal exists first image;

First image and the state diagram that is designed are compared; The state diagram that is wherein designed comprises the multiple arrangement pattern; Each device pattern all is associated corresponding to an image and with next pattern of candidate, and one in first image and a plurality of home media source apparatus is associated; And

When the image that is associated with next pattern of said candidate is different from first image and is associated with second standard; Confirm that said home media source apparatus changes, in a plurality of home media source apparatus of display device, discern replacement home media source apparatus so that said vision signal never to be provided.

14. method according to claim 13, wherein second standard comprises second image, static region, existence the 3rd region-of-interest that are arranged in second region-of-interest, does not have at least one of the 4th region-of-interest or remote signal.

15. method according to claim 13, the image that wherein is associated with next pattern of said candidate comprises the first pixel total value, and first image comprises the second pixel total value.

16. method according to claim 15, this method is further comprising the steps of: the dot product that calculates the first pixel summation and the second pixel summation.

17. method according to claim 16, wherein when the image that is associated with next pattern of said candidate and first images category like the time, said dot product has produced positive output.

18. method according to claim 16, the image that wherein ought be associated with next pattern of said candidate and first image are not simultaneously, said dot product has produced negative output.

19. method according to claim 16; Wherein when the image that is associated with next pattern of said candidate and first images category like the time; Said dot product addition longways mutually, and when the image that is associated with next pattern of said candidate is different from first image, said dot product disappear mutually addition.

20. method according to claim 13, this method is further comprising the steps of: calculating is used for confirming whether first image is different from the standard of the image that is associated with next pattern of said candidate.

21. method according to claim 20, wherein said standard comprise first image and the dot product of the image that is associated with next pattern of said candidate and the ratio between the pixel count in first image.

22. method according to claim 20, wherein said standard comprise that difference between similar pixel count and the different pixels number is divided by the sum of all pixels in first image.

23. method according to claim 20, wherein said standard comprise poor between the ratio of ratio and different pixels of similar pixel.

24. method according to claim 20, this method is further comprising the steps of: said standard and threshold value are compared, to confirm whether first image is different from the image that is associated with next pattern of said candidate.

25. a method that is used to discern the media apparatus pattern, this method may further comprise the steps:

Region-of-interest in very first time monitoring video signal;

With the region-of-interest of monitoring with comprise that the state diagram of multiple arrangement pattern compares, confirming media apparatus pattern candidate, each device pattern all corresponding to an image and with said media apparatus pattern candidate at least one be associated;

At the said region-of-interest of second time supervision; And

The region-of-interest and the said state diagram of monitoring are compared, said media apparatus pattern candidate is confirmed as the media apparatus pattern and said vision signal identification activity home media source apparatus in a plurality of home media source apparatus of display device never is provided.

26. method according to claim 25, wherein said state diagram comprise at least one the multiple arrangement pattern in STB, television set, video tape recorder, device for reproducing digital video disc, personnel's measuring instrument or the game console.

27. method according to claim 25, wherein said device pattern comprise that channel changes pattern, poster browse mode, video request program pattern, paying watching mode or guide pattern.

28. method according to claim 25, this method is further comprising the steps of: in the 3rd time detecting second media apparatus pattern.

29. an equipment that is used for monitoring video signal, this equipment comprises:

The region-of-interest extractor is used for extracting picture signal from the zone of vision signal;

The image quantization device is used for the picture signal of extracting is quantized; With

The object comparator; Be used for the picture signal that quantizes and the mode sequences state diagram of home media device are compared, availablely watch pattern and said vision signal identification activity home media source apparatus in a plurality of home media source apparatus of display device never is provided with what discern said home media device.

30. equipment according to claim 29, wherein said home media device comprises at least one in STB, television set, video tape recorder, device for reproducing digital video disc, personnel's measuring instrument or the game console.

31. equipment according to claim 29, this equipment also comprises: the quantizer threshold module is used for to said image quantization device quantization threshold being provided, to realize reducing picture element density, change pixel intensity or to remove at least one purpose in the pixel color.

32. an equipment that is used to handle vision signal, this equipment comprises:

Frame collector is used for gathering vision signal with first rate;

The source of media detector is used for said vision signal is quantized to improve the image detection to first region-of-interest and second region-of-interest; With

Show reader on the screen, be used for comparing never to provide said vision signal in a plurality of home media source apparatus of display device, to discern the home media source apparatus with the mode sequences state diagram at first region-of-interest and the detected image of second region-of-interest.

33. equipment according to claim 32, wherein said source of media detector also comprises: quantizer threshold is used for adjusting at least one of picture element density, pixel color or pixel intensity.

Images