WO2012143596A1

WO2012143596A1 - Detection of graphics added to a video signal

Info

Publication number: WO2012143596A1
Application number: PCT/FI2011/050350
Authority: WO
Inventors: Vesa Suontama
Original assignee: Supponor Oy
Priority date: 2011-04-18
Filing date: 2011-04-18
Publication date: 2012-10-26
Also published as: CA2831816A1; US20140036093A1; US8878999B2; CN103493498A; EP2700035A1; EP2700035A4; RU2013149995A

Abstract

The invention allows detecting which graphics, if any,have been added to the video signal at any given time by utilizing only the various signals provided by prior art hardware. A first video signal, graphics signal(s), and a second video signal are received. It is then determined which graphics, if any, are present in a given frame of the received second video signal based on com- paring actual values of pixels of the received second video signal and calculated values of pixels derived from the first video signal, and the graphics signal(s),and the combination of graphics signal(s)which produces the best match is searched for.

Description

TITLE OF THE INVENTION:

DETECTION OF GRAPHICS ADDED TO A VIDEO SIGNAL

BACKGROUND OF THE INVENTION:

Field of the Invention:

The invention relates to video signal pro^¬ cessing. In particular, the invention relates to methods, computer programs and apparatuses for detecting the presence of additional graphics in a video signal.

Description of the Related Art:

Today, it is common for television broadcasts to add information in the form of graphics to the video signal (s) being shot by the television cam- era(s) . For example, television broadcasts of sports events are typically produced so that additional in^¬ formation in the form of graphics is provided to the viewer to help him/her to understand the game. This additional information is added on top of the video images, and the additional information may be e.g. a game clock, a logo of the broadcasting television station (typically added to the upper right corner) , a player's name, etc.

In a broadcast environment, the additional graphics are typically formed by a graphics system and mixed by a vision mixer. A vision mixer (also called video switcher, video mixer or production switcher) is a device that mixes different video sources to output feeds. Typically a vision mixer can be found in a pro- fessional television production environment such as a television studio, a cable broadcast facility, a com^¬ mercial production facility, a remote truck/outside broadcast van (OB van), or a linear video editing bay. E.g. a vision engineer located in an outside broadcast van is listening to instructions from a director of a sports broadcast and selecting the camera to be shown. The director also instructs the vision engineer to turn certain graphics on or off in order to produce an aesthetic experience for the viewer and/or to make the program more informative to the viewer. For example, the game clock is typically hidden during replay after a goal, and the name of a player is displayed together with the image of the player.

The vision mixer gets different inputs, such as camera inputs, recorder inputs, and feeds from a graphics system. These inputs to the vision mixer are typically independent of the director's instructions. For example, on one of the inputs, the clock is on all the time.

Typically, graphics to be added on top of the video images is partly transparent, and the graphics is arranged on top of the video images by means of two auxiliary signals: a graphics signal and a mask sig^¬ nal. The graphics signal includes the graphics to be added (such as a logo of a broadcasting station, the name of a player in a sports broadcast, or a game clock) , and the mask signal (also known as a key sig^¬ nal) defines the transparency (also known as alpha) of pixels of its associated graphics signal. A mask sig- nal is typically a monochrome signal, where completely black areas correspond to completely transparent, and completely white areas correspond to opaque. Areas be^¬ tween completely black and completely white correspond to various degrees (e.g. in percentages) of transpar- ency. Typically but not always, each graphics sig^¬ nal/mask signal -pair corresponds to a single graphics t, and there may be several of these graphics sig^¬ nal/mask signal -pairs per one video signal. At any given time, one or more of these graphics signal/mask signal -pairs may be mixed on or off the video signal (e.g. by the vision engineer using the vision mixer), so that the corresponding graphics will or will not be visible as required. In the art, the video signal without the added graphics is often called a clean feed, and the video signal with the added graphics is called a dirty feed.

Typically, a prior art vision mixer outputs only the clean feed and the dirty feed, and at least the dirty feed is then forwarded in the broadcast sig^¬ nal transmission chain until it finally reaches the viewers. That is, prior art vision mixers are not con- figured to output any specific information about which combination of the input graphics signals is on (i.e. mixed into the clean feed to create the dirty feed) at any given time. The presence of transparency (i.e. the mask signals) means that one cannot just compare the clean feed and the dirty feed pixel-by-pixel to try to determine the added graphics based on the differences, since it is not trivial to determine the color and the transparency of a pixel in such a case.

Yet, there are situations in which it would be useful to be able to detect which graphicsare added to a video signal, such as a television broadcast sig^¬ nal at any given time. For example, the present appli^¬ cant's earlier patent application WO 2009/074710 de^¬ scribes a method for modifying the content of a tele- vision image by inserting substitutive content into specific areas of a television image. Information about which graphics are added to the television image at any given time facilitates such insertion of the substitutive content.

As described above, to obtain this infor^¬ mation about which graphics are added to the video signal at any given time, in prior art one has had to e.g. modify conventional vision mixers so that they can provide this information. However, this is a major disadvantage since it requires the owner of the vision mixer to do this, and modifications to expensive ex^¬ isting systems are risky. Therefore, an object of the present invention is to alleviate the problems described above and to introduce a solution that allows detecting which graphics are added to the video signal at any given time by utilizing only the various signals provided by e.g. a vision mixer and/or a graphics system, i.e. without requiring any modifications to existing conventional hardware. SUMMARY OF THE INVENTION:

A first aspect of the present invention is a method of detecting the presence of additional graphics in a video signal. The following are re^¬ ceived: a first video signal, at least one graphics signal, and a second video signal comprising said first video signal mixed at least periodically with the at least one graphics signal. Calculated values of pixels are derived from a frame of the received first video signal and a corresponding frame of the received at least one graphics signal. The calculated values are compared with actual values of pixels of a corre^¬ sponding frame of the received second video signal. A presence status of the at least one graphics signal in the frame of the received second video signal is de- termined based on the performed comparison.

A second aspect of the present invention is an apparatus for detecting the presence of additional graphics in a video signal. The apparatus comprises a receiver configured to receive a first video signal, at least one graphics signal, and a second video sig^¬ nal comprising said first video signal mixed at least periodically with the at least one graphics signal. The apparatus further comprises a calculator configured to derive calculated values of pixels from a frame of the received first video signal and a corre^¬ sponding frame of the received at least one graphics signal. The apparatus further comprises a comparator configured to compare the calculated values with actu^¬ al values of pixels of a corresponding frame of the received second video signal. The apparatus further comprises a selector configured to determine a pres- ence status of the at least one graphics signal in the frame of the received second video signal based on the performed comparison.

A third aspect of the present invention is a computer program comprising code adapted to cause the following when executed on a data-processing system:

receiving a first video signal, at least one graphics signal, and a second video signal comprising said first video signal mixed at least periodically with the at least one graphics signal;

deriving calculated values of pixels from a frame of the received first video signal and a corre^¬ sponding frame of the received at least one graphics signal ;

comparing the calculated values with actual values of pixels of a corresponding frame of the re^¬ ceived second video signal; and

determining a presence status of the at least one graphics signal in the frame of the received second video signal based on the performed comparison.

A fourth aspect of the present invention is an apparatus for detecting the presence of additional graphics in a video signal. The apparatus comprises a receiving means for receiving a first video signal, at least one graphics signal, and a second video signal comprising said first video signal mixed at least pe^¬ riodically with the at least one graphics signal. The apparatus further comprises a calculating means for deriving calculated values of pixels from a frame of the received first video signal and a corresponding frame of the received at least one graphics signal. The apparatus further comprises a comparing means for comparing the calculated values with actual values of pixels of a corresponding frame of the received second video signal. The apparatus further comprises a se^¬ lecting means for determining a presence status of the at least one graphics signal in the frame of the re- ceived second video signal based on the performed com^¬ parison .

In an embodiment of the invention, the receiving further comprises receiving transparency information indicating transparency of at least a portion of a frame of an associated graphics signal, and the de^¬ riving further comprises deriving the calculated values of pixels from the received transparency information in addition to the frame of the received first video sig^¬ nal and the corresponding frame of the received at least one graphics signal.

In an embodiment of the invention, the deriving further comprises deriving the calculated values for a number of presence combinations of the at least one graphics signal.

In an embodiment of the invention, the deter^¬ mination of the presence status is performed by select^¬ ing - as indicative of the presence status of the at least one graphics signal in the frame of the received second video signal - the presence combination the cal- culated values of which best match with the compared actual values according to a statistical criterion.

In an embodiment of the invention, an effec^¬ tive result of used graphics information and/or associ^¬ ated resulting transparency information is produced, based on the determined presence status.

In an embodiment of the invention, the pro^¬ duced used graphics information and its associated used transparency information are applied to one or more ad^¬ ditional incoming video signals to produce one or more additional output video signals.

In an embodiment of the invention, the comput^¬ er program of the third aspect of the present invention is stored on a computer-readable medium. In an embodi^¬ ment of the invention, the computer-readable medium on^¬ ly includes a non-transitory medium.

It is to be understood that the aspects and embodiments of the invention described above may be used in any combination with each other. Several of the aspects and embodiments may be combined together to form a further embodiment of the invention. A method, an apparatus, or a computer program which is an aspect of the invention may comprise at least one of the em^¬ bodiments of the invention described above.

The invention allows detecting which graphics are added to the video signal at any given time by utilizing only the various signals readily provided by e.g. a vision mixer and/or a graphics system, i.e. without requiring any modifications to existing conventional hardware. This way, the detection of the graphics can be performed in a significantly more cost-effective and less time- and effort-consuming manner. Furthermore, the invention allows detecting the presence of the added graphics even when there are rounding errors or noise present.

BRIEF DESCRIPTION OF THE DRAWINGS:

The accompanying drawings, which are included to provide a further understanding of the invention and constitute a part of this specification, illus^¬ trate embodiments of the invention and together with the description help to explain the principles of the invention. In the drawings:

Fig. 1 is a block diagram illustrating an apparatus according to an embodiment of the invention as deployed in connection with a prior art vision mixer; and

Fig. 2 is a flow diagram illustrating a method according to an embodiment of the invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS:

Reference will now be made in detail to the embodiments of the invention, examples of which are illustrated in the accompanying drawings.

In the following, the invention is described with examples related to television broadcasting. However, it is to be understood that the present inven^¬ tion is not limited to such implementations. Instead, the present invention can as well be used in any ap- plications in which a first video signal, graphics signal (s) , and a second video signal are available. The first video signal may be e.g. a clean video sig^¬ nal or a clean feed, and the second video signal may be e.g. a dirty video signal or a dirty feed. The first and/or second video signal may be a live signal being shot with e.g. a television camera, or the first and/or second video signal may be e.g. a previously recorded video signal stored e.g. in a file or several files .

Furthermore, herein the term "signal" is used as a logical concept. I.e. each signal is not neces^¬ sarily transferred in a separate physical channel. Ra^¬ ther, several signals may be transferred in a common physical channel shared by the signals.

Figure 1 is a block diagram illustrating an apparatus 100 according to an embodiment of the inven^¬ tion as deployed in connection with a prior art vision mixer 180.

The vision mixer 180 is a device that mixes different video sources to output feeds. Typically a vision mixer can be found in a professional television production environment such as a television studio, a cable broadcast facility, a commercial production fa^¬ cility, a remote truck/outside broadcast van, or a linear video editing bay.

As depicted in Figure 1, inputs of the vision mixer 180 include a first graphics signal Gl and a se- cond graphics signal G2. The first graphics signal Gl may comprise first graphics or graphical element (s) (e.g. a television station logo), and the second graphics signal G2 may comprise second graphics or graphical element (s) (e.g. a game clock) . The inputs of the vision mixer 180 may further include a first mask signal Ml and a second mask signal M2. The first mask signal Ml may comprise transparency information that indicates transparency of at least a portion of a frame of its associated graphics signal Gl, such as e.g. pixel-specific transparency values (also known as alpha values) for the first graphics signal Gl, and the second mask signal M2 may comprise transparency information that indicates transparency of at least a portion of a frame of its associated graphics signal G2, such as e.g. pixel-specific transparency values for the second graphics signal G2. The transparency values may vary e.g. from 0 to 1, with 0 corresponding to e.g. totally transparent, 1 corresponding to e.g. totally opaque, and values between 0 and 1 correspond^¬ ing to e.g. various degrees of transparency. In the art, the term "mask" is also known as "key".

The inputs of the vision mixer 180 may fur^¬ ther include a first camera signal CI, a second camera signal C2, and a third camera signal C3, each being fed from e.g. a respective television camera (not il^¬ lustrated in Figure 1) .

At any given time, one of the camera signals CI, C2, C3 is selected as active (by e.g. a vision en- gineer operating the vision mixer 180), and one or more of the graphics signals (or graphics or graphical elements in the graphics signals) may be added/mixed (again by e.g. the vision engineer operating the vision mixer 180) into the selected camera signal with the transparency of each added graphics sig^¬ nal/graphics/graphical element being defined by its respective transparency information or transparency values. Accordingly, each graphics/graphical element may be opaque, partially transparent, or wholly trans^¬ parent. In the following, the camera signal selected as active without any added graphics/graphical element will be referred to as a first or "clean" video signal (also known as clean feed) , and the camera signal se^¬ lected as active with at least periodically added graphics/graphical element (s) will be referred to as a second or "dirty" video signal (also known as dirty feed) . The clean video signal CF and the dirty video signal DF are outputs from the vision mixer 180.

It is to be understood that the amount of the various signals depicted in Figure 1 is provided as an example only, and that any number of signals may be used.

Conventionally, at least the dirty video sig^¬ nal DF and optionally the clean video signal CF would next be forwarded in the broadcast signal transmission chain in order to transmit the dirty video signal DF as a television broadcast signal to the viewers.

However, the present invention introduces the apparatus 100 for detecting which of the graphics/graphical elements are added to the dirty video signal DF at a given time, and the clean video signal CF and the dirty video signal DF are therefore first input into this apparatus 100.

In the non-limiting example of Figure 1, the inputs of the apparatus 100 further include the first graphics signal Gl, the second graphics signal G2, the first mask signal Ml (carrying first transparency in^¬ formation) , and the second mask signal M2 (carrying second transparency information) . The apparatus 100 may obtain the first graphics signal Gl, the second graphics signal G2, the first mask signal Ml, and the second mask signal M2 e.g. from a conventional graphics system (not illustrated in Figure 1) arranged in connection with the vision mixer 180 in a broadcasting environment.

As used herein, the term "graphics system" refers to a prior art graphics machine or entity that generates graphics signals and mask signals and pro^¬ vides them to a vision mixer.

In other words, the apparatus 100 comprises a receiver 110 that is configured to receive a first or clean video signal CF, graphics signals Gl and G2, and a second or dirty video signal DF that comprises the first video signal Gl mixed at least periodically with the graphics signals Gl and/or G2. In an embodiment of the invention, the receiver 110 may be further configured to receive transparency information (comprised e.g. the mask signals Ml, M2) indicating transparency of at least a portion of a frame of an associated graphics signal.

As is known in the art, the mixing of a clean video signal and a graphics signal may be expressed with the following Equation 1 (calculated e.g. sepa^¬ rately for each pixel and each color component (e.g. in RGB space as used herein as an example, but any other color space such as YUV space can be used instead, since conversion between color spaces is trivi- al for those skilled in the art) ) :

DFi = (1-Mi) * CF + Mi * Gi (Eq. 1)

In the Equation 1, the Mi represents the transparency value (e.g. from 0 to 1) of the first added graphics signal, Gi represents the intensity of the first added graphics signal, CF represents the in^¬ tensity of the clean video signal, and DFi represents the intensity of the dirty video signal after mixing the first added graphics signal into it. Herein, the intensity of a pixel consists of intensities or intensity measures of the color compo^¬ nents of the pixel.

The equation examples below are for a linear RGB color system. As is known to those skilled in the art, equations for other color systems may also be de^¬ rived .

If there is also a second graphics signal to be added, the following Equation 2 represent the situ- ation after adding the first and the second graphics signals :

DF₂ = (1-M₂) * DFi + M₂ * G₂ (Eq. 2) In the Equation 2, the M₂ represents the transparency value (from 0 to 1) of the second added graphics signal, G₂ represents the intensity of the second added graphics signal, DFi represents the in^¬ tensity of the dirty video signal after mixing the first added graphics signal into it, and DF₂ repre^¬ sents the intensity of the dirty video signal after mixing the second added graphics signal into it.

Combining Equation 1 and Equation 2 produces the following Equation 3:

DF₂ = (1 - M₂) * ((1-Mi) * CF + Mi * Gi) + M₂ * G₂

(Eq. 3)

Generalizing this for n inputs produces the following Equation 4:

DF_N = (1 - M_N) * ( DF_N-I ) + M_N * G_N (Eq. 4)

The problem of the present invention, i.e. finding the added graphics, can be transformed to an equation so that we try to find multipliers or coeffi^¬ cients (K_N) for each of the mask values. Each coeffi- cient K can have as its value e.g. either 0 or 1, of which e.g. 0 indicates that the corresponding graphical element is not present, and e.g. 1 indicates that it is present.

For the example of Figure 1 of two graphics signals and two mask signals, this produces the fol^¬ lowing Equation 5:

DF₂ = (1 - K₂ * M₂ ) * ((1-Ki * Mi ) * CF + Ki * Mi * Gi ) + K₂ * M₂ * G₂ (Eq. 5)

Equation 5 can be obtained from Equation 3 by substituting M_N with K_N * M_N.

Obviously, Equation 5 can be extended to a general case, but for the purposes of facilitating the understanding of the invention, the case of two graphics signals and two mask signals will be used herein to illustrate the invention. Similarly, other means of finding which graphics are on the air can be used.

The apparatus 100 further comprises a calcu^¬ lator 120 that is configured to derive calculated val^¬ ues from a frame of the received first video signal CF, from a corresponding frame of the received at least one graphics signal Gl, G2, and optionally from the received transparency information Ml, M2. Optionally, the calculator 120 may be further configured to derive the calculated values for a number of presence combinations of the at least one graphics signal Gl, G2. Herein, the term "presence combinations" refers to the various combinations of the graphics signal (s) be^¬ ing present/on (i.e. mixed-in) or absent/off (i.e. not mixed-in) in a given frame of the second video signal DF. For example, in the case of two graphics signals Gl, G2, presence combinations may include following combinations : 0 - both Gl and G2 are off (i.e. Dirty Feed DF is the same as Clean Feed CF) ;

1 - first graphics signal Gl is on and second graphics signal G2 is off;

2 - first graphics signal Gl is off and second graphics signal G2 is on;

12 - both graphics signals Gl and G2 are on, and the second one G2 is drawn after the first one Gl; etc .

The apparatus 100 further comprises a compar^¬ ator 130 that is configured to compare the calculated values with actual values of a corresponding frame of the received second video signal DF.

The apparatus 100 further comprises a selec- tor 140 that is configured to determine a presence status of the at least one graphics signal Gl, G2 in the frame of the received second video signal DF based on the performed comparison. Herein, the term "presence status" of a graphics signal refers to whether the graphics signal in question is present in a given frame of the second video signal DF or not. In an em^¬ bodiment, the selector 140 may be configured to per^¬ form the determination of the presence status by se^¬ lecting the presence combination the calculated values of which best match with the compared actual values according to a statistical criterion, as being indicative of the presence status of the at least one graphics signal in the frame of the received second video signal.

The apparatus 100 may further comprise an in^¬ formation producer 150 that is configured to produce at least one of used graphics information and associ^¬ ated used transparency information, wherein the used graphics information indicates the determined presence status.

The apparatus 100 may further comprise an in^¬ formation applying unit 160 that is configured to ap- ply the produced used graphics information and its as^¬ sociated used transparency information to one or more additional incoming video signals to produce one or more additional output video signals.

In other words, the calculator 120, and the comparator 130 may be used to try out all the differ^¬ ent combinations of Ks (i.e. 0,0; 1,0; 0,1; 1,1 in the case of two graphics signals and two mask signals) for the whole image in order to find the combination of Ks that best fits the Equation 5. Then, when the best fitting combination of Ks is found, the selector 140 may be used to determine from the best fitting combi^¬ nation of Ks which graphical elements are present.

As a result, the apparatus 100 may output a used graphics signal UG which is the effective combi^¬ nation of graphics signals Gl, G2 that is on at the given moment. The apparatus 100 may further output a used mask signal UM which is the effective combination of mask signals that is on at the given moment.

In an embodiment, the order of graphics in^¬ puts is fixed so that the order of inputs is the same order as the order in which graphics are applied in the vision mixer (and in Equations 1-5) .

Figure 2 is a flow diagram illustrating a method 200 of detecting the presence of additional graphics or graphical elements in a video signal ac^¬ cording to an embodiment of the invention.

At step 210, the following are received: a first video signal, at least one graphics signal, a second video signal comprising said first video signal mixed at least periodically with the at least one graphics signal, and optionally transparency infor^¬ mation that indicates transparency of at least a por^¬ tion of a frame of an associated graphics signal.

At step 220, calculated values of pixels are derived from a frame of the received first video sig^¬ nal, from a corresponding frame of the received at least one graphics signal, and optionally from the re^¬ ceived transparency information. Optionally, the cal^¬ culated values are derived for a number of presence combinations of the at least one graphics signal.

At step 230, the calculated values are com^¬ pared with actual values of pixels of a corresponding frame of the received second video signal.

At step 240, a presence status of the at least one graphics signal in the frame of the received second video signal is determined based on the per^¬ formed comparison. The determination of the presence status may be performed e.g. by selecting - as indica^¬ tive of the presence status of the at least one graphics signal in the frame of the received second video signal - the presence combination the calculated values of which best match with the compared actual values according to a statistical criterion.

At step 250, used graphics information and/or associated used transparency information are produced, wherein the used graphics information indicates the de^¬ termined presence status.

At step 260, the produced used graphics infor^¬ mation and its associated used transparency information are applied to one or more additional incoming video signals to produce one or more additional output video signals .

As discussed above, the vision mixer 180 can have some of the graphics channels or signals config^¬ ured as "on", and some of them as "off". In order to determine, which ones were on/off, some or all possible combinations of the on/off-setting of those graphics channels may be dealt through.

When the combinations are dealt through, it is assumed that the order of the graphics channels that are overlaid on top of the active camera signal, is al^¬ ways predetermined and fixed, and it is also assumed that this order is known (for the captured in- puts/outputs of the vision mixer 180, it is assumed that it is known which signal would be overlaid first, second, etc . ) .

For example, if the vision mixer 180 has two graphics channels, the following combinations are in^¬ spected:

0 - nothing is on (Dirty Feed is the same as Clean Feed) ;

1 - first graphics signal Gl is on;

2 - second graphics signal G2 is on;

12 - both graphics signals Gl and G2 are on, and the second one G2 is drawn on top of the first one Gl;

21 - this combination does not need to be checked, as the order of the graphics channels is al^¬ ways the same 123..., etc.

We emulate the vision mixer's 180 work of overlaying of the graphics signals by using the follow^¬ ing equation examples for every pixel of a frame (it is to be noted that these equations are provided as exam^¬ ples only and that the invention is not limited to the^¬ se equation examples. Instead, other mixing equations may be used) :

Alpha = (Mask - 16) / 220

Result = (1 - Alpha) * Background + Alph Graphics, assuming 0 ≤ Alpha ≤ 1 where :

Mask: the Y value of graphics mask pixel;

Background: Y, U, V values of the pixel of the original video frame; and

Graphics: Y, U, V values of the pixel of the graphics frame. Next, we compare the overlay results of all the combinations of the graphics, and choose the one that gives the results that are closest to the Dirty Feed signal. In order to determine how close a result- ing signal is to the Dirty Feed, we examine the differ^¬ ence between two frames (DBTF) , that is defined as a sum of all differences between corresponding pixels of the two frames.

Differences between the pixel (Yl, Ul, VI) and the pixel (Y2, U2, V2) is Euclidean metric, i.e. it is the length of the vector (Y2-Y1, U2-U1, V2-V1), i.e. it is sqrt ( sqr(Y2-Yl) + sqr(U2-Ul) + sqr(V2-Vl)).

For performance reasons we may use a simpli^¬ fied formula: abs (Y2-Y1 ) +abs (U2-U1 ) +abs (V2-V1 ) to de- termine the length.

So, first DBTF is calculated for the Clean Feed and Dirty Feed frames (DeltaO = DF - CF) . If Del- taO equals 0, then no graphics signals are on. Other^¬ wise, DBTFs are calculated for each combination of graphics signals. That results in a sequence of DeltaO, Deltal, DeltaM, where M = 2 ^AN - 1, where N is a number of graphics signals. For example, for 3 graphics signals we get this sequence of combinations: 0, 1, 2, 3, 12, 13, 23, 123.

After determining which graphics signals were

"on" in the vision mixer 180, the present invention may produce, for every frame captured from the vision mixer 180 inputs/outputs, a used graphics video frame which consists of a mix of all active graphics signals put on top of a black frame. The mixing equations for produc^¬ ing the used graphics feed are the same that are used for emulating the work of the vision mixer 180. Also, for every frame captured from the vision mixer 180 inputs/outputs, the present invention may produce a used mask video frame which represents a combination of all the masks of active graphics signals, and it can be used to overlay the used graphics frame on top of Clean Feed. In order to determine the pixel of the used masks, the present invention may use the following equation :

Mask = (DirtyFeed - CleanFeed) / (UsedGraphics - CleanFeed)

In case UsedGraphics equals CleanFeed, the pixel in question is ignored.

The equations shown above used to emulate the work of the vision mixer may differ from the actual equations used by an actual vision mixer device. That is why we need to search for a closest match of our re^¬ sults to the Dirty Feed produced by the vision mixer 180 and not the exact match.

Moreover, the inventors have discovered that sometimes a vision mixer may modify some of the regions in the image (e.g. it adds a white line on one of the borders of the image) , so the Dirty Feed is not just a product of a graphics overlay process, but also some over interference may happen inside a vision mixer. An advantage of the invention is that the algorithm de^¬ scribed above can handle that.

Furthermore, sometimes a vision mixer may use transition effects to overlay the graphics signals. That adds even more interference in the resulting Dirty Feed, in addition to simply overlaying the graphics signals. Yet, the invention allows detecting the graphics that was on/off in Vision Mixer, even in the presence of some of such transition effects (e.g., those ones that alter the pixels of auxiliary graphics mask) .

Possible Optimizations:

1. While DeltaX, that is being calculated, is already greater than the current minimal Delta, we can then stop its further calculation, and drop the combi^¬ nation from consideration.

2. Historical data can be used. If some feeds were frequently (or always) turned on (e.g. a game clock) , then it is very likely that some of them are turned on for further frames of the video, and this as^¬ sumption can be taken into consideration for choosing the order of combinations to consider.

3. If the order of the overlay of graphics is not fixed in the vision mixer 180, we can still handle that case by increasing the list of combinations we check, so, for two graphics signals, instead of check- ing 0,1,2,12, we would check 0,1,2,12,21.

4. We could determine the combination of the graphics signals even if the vision mixer 180 uses com^¬ plex transition effects that alter the pixels of the graphics signal that is to be overlaid (e.g. when graphics is sliding from the top of the screen to its target place) . In such a case, we could iterate through all possible combinations of the transition effects, in addition to all possible combinations of the graphics signals.

The exemplary embodiments can include, for ex^¬ ample, any suitable servers, workstations, PCs, laptop computers, wireless devices, other devices, and the like, capable of performing the processes of the exem^¬ plary embodiments. The devices and subsystems of the exemplary embodiments can communicate with each other using any suitable protocol and can be implemented us^¬ ing one or more programmed computer systems or devices.

One or more interface mechanisms can be used with the exemplary embodiments, including, for example, Internet access, telecommunications in any suitable form (e.g., voice, modem, and the like), wireless com^¬ munications media, and the like. For example, employed communications networks or links can include one or more wireless communications networks, cellular commu- nications networks, 3G communications networks, Public Switched Telephone Network (PSTNs) , Packet Data Net^¬ works (PDNs) , the Internet, intranets, a combination thereof, and the like.

It is to be understood that the exemplary em- bodiments are for exemplary purposes, as many varia^¬ tions of the specific hardware used to implement the exemplary embodiments are possible, as will be appre^¬ ciated by those skilled in the hardware and/or soft^¬ ware art(s) . For example, the functionality of one or more of the components of the exemplary embodiments can be implemented via one or more hardware and/or software devices.

The exemplary embodiments can store infor^¬ mation relating to various processes described herein. This information can be stored in one or more memo^¬ ries, such as a hard disk, optical disk, magneto- optical disk, RAM, and the like. One or more databases can store the information used to implement the exem^¬ plary embodiments of the present inventions. The data- bases can be organized using data structures (e.g., records, tables, arrays, fields, graphs, trees, lists, and the like) included in one or more memories or storage devices listed herein. The processes described with respect to the exemplary embodiments can include appropriate data structures for storing data collected and/or generated by the processes of the devices and subsystems of the exemplary embodiments in one or more databases .

All or a portion of the exemplary embodiments can be conveniently implemented using one or more gen^¬ eral purpose processors, microprocessors, digital sig^¬ nal processors, micro-controllers, and the like, pro- grammed according to the teachings of the exemplary embodiments of the present inventions, as will be ap^¬ preciated by those skilled in the computer and/or software art(s) . Appropriate software can be readily prepared by programmers of ordinary skill based on the teachings of the exemplary embodiments, as will be ap^¬ preciated by those skilled in the software art. In ad^¬ dition, the exemplary embodiments can be implemented by the preparation of application-specific integrated circuits or by interconnecting an appropriate network of conventional component circuits, as will be appre^¬ ciated by those skilled in the electrical art(s) . Thus, the exemplary embodiments are not limited to any specific combination of hardware and/or software.

Stored on any one or on a combination of computer readable media, the exemplary embodiments of the present inventions can include software for control^¬ ling the components of the exemplary embodiments, for driving the components of the exemplary embodiments, for enabling the components of the exemplary embodi^¬ ments to interact with a human user, and the like. Such software can include, but is not limited to, de^¬ vice drivers, firmware, operating systems, development tools, applications software, and the like. Such com- puter readable media further can include the computer program product of an embodiment of the present inven^¬ tions for performing all or a portion (if processing is distributed) of the processing performed in imple^¬ menting the inventions. Computer code devices of the exemplary embodiments of the present inventions can include any suitable interpretable or executable code mechanism, including but not limited to scripts, in^¬ terpretable programs, dynamic link libraries (DLLs) , Java classes and applets, complete executable pro- grams, Common Object Request Broker Architecture (CORBA) objects, and the like. Moreover, parts of the processing of the exemplary embodiments of the present inventions can be distributed for better performance, reliability, cost, and the like.

As stated above, the components of the exem^¬ plary embodiments can include computer readable medium or memories for holding instructions programmed ac^¬ cording to the teachings of the present inventions and for holding data structures, tables, records, and/or other data described herein. Computer readable medium can include any suitable medium that participates in providing instructions to a processor for execution. Such a medium can take many forms, including but not limited to, non-volatile media, volatile media, trans^¬ mission media, and the like. Non-volatile media can include, for example, optical or magnetic disks, mag- neto-optical disks, and the like. Volatile media can include dynamic memories, and the like. Transmission media can include coaxial cables, copper wire, fiber optics, and the like. Transmission media also can take the form of acoustic, optical, electromagnetic waves, and the like, such as those generated during radio frequency (RF) communications, infrared (IR) data com^¬ munications, and the like. Common forms of computer- readable media can include, for example, a floppy disk, a flexible disk, hard disk, magnetic tape, any other suitable magnetic medium, a CD-ROM, CD±R, CD±RW, DVD, DVD-RAM, DVD1RW, DVD±R, HD DVD, HD DVD-R, HD DVD- RW, HD DVD-RAM, Blu-ray Disc, any other suitable opti^¬ cal medium, punch cards, paper tape, optical mark sheets, any other suitable physical medium with pat- terns of holes or other optically recognizable indi^¬ cia, a RAM, a PROM, an EPROM, a FLASH-EPROM, any other suitable memory chip or cartridge, a carrier wave or any other suitable medium from which a computer can read .

While the present inventions have been de^¬ scribed in connection with a number of exemplary embodiments, and implementations, the present inventions are not so limited, but rather cover various modifica^¬ tions, and equivalent arrangements, which fall within the purview of prospective claims.

Claims

CLAIMS :

1. A method (200) of detecting the presence of additional graphics in a video signal, the method (200) comprising:

receiving (210) a first video signal, at least one graphics signal, and a second video signal comprising said first video signal mixed at least pe^¬ riodically with the at least one graphics signal;

c h a r a c t e r i z e d in that the method (200) further comprises:

deriving (220) calculated values of pixels from a frame of the received first video signal and a corresponding frame of the received at least one graphics signal;

comparing (230) the calculated values with actual values of pixels of a corresponding frame of the received second video signal; and

determining (240) a presence status of the at least one graphics signal in the frame of the received second video signal based on the performed comparison.

2. The method (200) according to claim 1, c h a r a c t e r i z e d in that said receiving (210) further comprises receiving transparency information indicating transparency of at least a portion of a frame of an associated graphics signal, and said de^¬ riving (220) further comprises deriving the calculated values from the received transparency information in addition to the frame of the received first video sig^¬ nal and the corresponding frame of the received at least one graphics signal.

3. The method (200) according to claim 1 or 2, c h a r a c t e r i z e d in that said deriving (220) further comprises deriving the calculated values for a number of presence combinations of the at least one graphics signal.

4. The method (200) according to claim 3, c h a r a c t e r i z e d in that said determining (240) said presence status is performed by selecting the presence combination the calculated values of which best match with the compared actual values according to a statistical criterion, as indicative of said presence status of the at least one graphics signal in the frame of the received second video signal.

5. The method (200) according to any of claims 1-4, further comprising:

producing (250) an effective result of at least one of used graphics information and associated resulting transparency information, based on said de^¬ termined presence status.

6. The method (200) according to claim 5, further comprising:

applying (260) the produced used graphics in^¬ formation and its associated used transparency infor^¬ mation to one or more additional incoming video sig^¬ nals to produce one or more additional output video signals .

7. An apparatus (100) for detecting the pres^¬ ence of additional graphics in a video signal, the ap^¬ paratus (100) comprising:

a receiver (110) configured to receive a first video signal, at least one graphics signal, and a second video signal comprising said first video sig^¬ nal mixed at least periodically with the at least one graphics signal;

c h a r a c t e r i z e d in that the apparatus (100) further comprises:

a calculator (120) configured to derive cal^¬ culated values of pixels from a frame of the received first video signal and a corresponding frame of the received at least one graphics signal;

a comparator (130) configured to compare the calculated values with actual values of pixels of a corresponding frame of the received second video sig^¬ nal; and a selector (140) configured to determine a presence status of the at least one graphics signal in the frame of the received second video signal based on the performed comparison.

8. The apparatus (100) according to claim 7, c h a r a c t e r i z e d in that the receiver (110) is further configured to receive transparency information indicating transparency of at least a portion of a frame of an associated graphics signal, and said cal- culator (120) is further configured to derive the cal^¬ culated values from the received transparency infor^¬ mation in addition to the frame of the received first video signal and the corresponding frame of the re^¬ ceived at least one graphics signal.

9. The apparatus (100) according to claim 7 or 8, c h a r a c t e r i z e d in that the calculator (120) is further configured to derive the calculated values for a number of presence combinations of the at least one graphics signal.

10. The apparatus (100) according to claim 9, c h a r a c t e r i z e d in that the selector (140) is configured to perform said determination of said pres^¬ ence status by selecting the presence combination the calculated values of which best match with the com- pared actual values according to a statistical crite^¬ rion, as indicative of said presence status of the at least one graphics signal in the frame of the received second video signal.

11. The apparatus (100) according to any of claims 7-10, c h a r a c t e r i z e d in further com^¬ prising an information producer (150) configured to produce an effective result of at least one of used graphics information and associated resulting trans^¬ parency information, based on said determined presence status.

12. The apparatus (100) according to claim 11, c h a r a c t e r i z e d in further comprising an information applying unit (160) configured to apply the produced used graphics information and its associ^¬ ated used transparency information to one or more ad^¬ ditional incoming video signals to produce one or more additional output video signals.

13. A computer program comprising code adapted to cause the following when executed on a da^¬ ta-processing system:

c h a r a c t e r i z e d in the computer pro^¬ gram further comprising code adapted to cause the fol- lowing when executed on a data-processing system:

14. The computer program according to claim 13, c h a r a c t e r i z e d in that said receiving (210) further comprises receiving transparency infor^¬ mation indicating transparency of at least a portion of a frame of an associated graphics signal, and said deriving (220) further comprises deriving the calcu^¬ lated values from the received transparency infor^¬ mation in addition to the frame of the received first video signal and the corresponding frame of the re- ceived at least one graphics signal.

15. The computer program according to claim 13 or 14, c h a r a c t e r i z e d in that said deriv- ing (220) further comprises deriving the calculated values for a number of presence combinations of the at least one graphics signal.

16. The computer program according to claim 15, c h a r a c t e r i z e d in that said determining

(240) said presence status is performed by selecting the presence combination the calculated values of which best match with the compared actual values ac^¬ cording to a statistical criterion, as indicative of said presence status of the at least one graphics sig^¬ nal in the frame of the received second video signal.

17. The computer program according to any of claims 13-16, c h a r a c t e r i z e d in the computer program further comprising code adapted to cause the following when executed on a data-processing system:

18. The computer program according to claim

17, c h a r a c t e r i z e d in the computer program further comprising code adapted to cause the following when executed on a data-processing system:

applying (260) the produced used graphics in- formation and its associated used transparency infor^¬ mation to one or more additional incoming video sig^¬ nals to produce one or more additional output video signals .

19. The computer program according to any of claims 13-18, wherein said computer program is stored on a computer-readable medium.