EP1611738A2 - Method for estimating logo visibility and exposure in video - Google Patents
Method for estimating logo visibility and exposure in videoInfo
- Publication number
- EP1611738A2 EP1611738A2 EP20040722778 EP04722778A EP1611738A2 EP 1611738 A2 EP1611738 A2 EP 1611738A2 EP 20040722778 EP20040722778 EP 20040722778 EP 04722778 A EP04722778 A EP 04722778A EP 1611738 A2 EP1611738 A2 EP 1611738A2
- Authority
- EP
- European Patent Office
- Prior art keywords
- tuples
- logo
- pattern
- points
- invariants
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
- G06V20/635—Overlay text, e.g. embedded captions in a TV program
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V10/757—Matching configurations of points or features
Definitions
- Keywords logo visibility, Multimedia, Advertisement, Video sequence, geometric invariants, chromatic invariants, point tuples.
- the visibility and exposure of logos in video is of important commercial concern for advertisement on television.
- a logo placed on the flat or round surface appears from different viewpoints, partially visible or occluded.
- the logo visibility time, its size (relative to the screen), position (relative to the terrain) and percent of the non-occluded part allows to directly compute its advertising impact.
- This patent discloses a method for automatic computation of the visible part of the given logo in the video sequence. For each frame of the video sequence the method computes four parameters that describe the visibility of a given logo.
- First parameter is the percentage of the visible part with respect to the whole logo.
- Second and third are the position and size of the logo with respect to the image.
- the present invention belongs to the domain of registration (or matching) methods in computer vision.
- methods of registration of two-dimensional (planar) patterns like logos under pro- jective transformation (modelling the image acquisition by a camera) are addressed.
- the registration of patterns is based on registration of features of those patterns. Therefore, registration methods are classified with respect to features that are used.
- some methods use template matching where an example of the logo original is matched to the image by applying all possible transformations to the original and evaluate the match. The best found match is the position of the logo.
- This method is very sensitive to occlusion, size changes and blur.
- Another family of methods use local features of the pattern. In particular, if the shape is composed of several curves and edges, comparison of their features can be used as registration. How- ever, the limitation of this method is the requirement for the pattern/logo to have special features th ⁇ t can not be guaranteed in practice. The algorithm is unable to locate logos that do not have such feature s .
- the advantages of the new method is its capacity to work with real conditions including high occlusion of the logo, illumination changes and blur. These latter conditions are very frequent in sport events (see Fig. 2) thus making the method very useful for this application.
- the method for registration of visible part of the logo is based on the dense and massive matching of point tuples.
- Each point tuple is matched using the comparison of its properties described by values that are invariant to geometric transformation produced by a camera and chromatic transformations produced by illumination changes.
- an example image is used to specify a user-defined zone.
- the obtained representation is used to search for learned logo in a given video sequence by analysing each individual frame , .
- the stage of computing visibility of a particular logo in 1 frame consist of registering numerous learned tuples corresponding to example image and verifying their consistency.
- the stage of registration of individual tuple is explained in section III.1.
- the stage for computing the registration of a particular logo from registered tuples is described in III.4.
- the main advantage of the invention is that a top-down approach is used to look for point tuples and that no support region is used to find points. Therefore, high resolution matching can be performed independently of any occlusion.
- Registering a tuple of several points P , in two images is done by comparing joint geometric and chromatic properties of those points. Depending on the complexity of transformation, involved in image formation a various number of points and their properties is required to perform the registration robustly.
- the geometric transformation from the plane with the logo in the scene to the sensor of the camera can be modelled by 2D affine, 2D projective or more complex transformation. Some additional transformation is required to account for lens distortion, but in the case of television cameras the distance to the object reduces the distortion effect. If 2D affine transformation is used to model camera, four points are necessary to compute two values that are independent on that transformation and thus characterizes the tuple independently of that information. If a 2D projective transformation is used, already five points are necessary to compute two different geometric invariant values.
- the known geometric invariant to projective transformation is the two cross-ratios of five points defined by:
- Another property of the tuple of points that can be used as a stable characteristic is the value based on chromatic values of individual points and that is invariant to illumination changes. If the changing illumination (sun, clouds, reflections) is modelled by the scaling transformation in rgb domain, two points are needed to compute one invariant value. The invariant to this chromatic transformation is the ratio of several rgb values. Therefore, for five points selected above four invariant values can be computed.
- the search should be done without relying on the pixel grid of the image being searched. Therefore, a sub- pixeling computations should be done during search. This is achieved by selecting instead of individual pixels the triples of neighbour pixels whose intensity (or color) values establish an interval containing the value that is searched for as shown in Fig. 7.
- the algorithms with subpixel precision is described in the Fig. 6.
- triangles of pixels (P Q t , R t as shown in Fig. 7) are selected that have values defining an interval containing the value. For example, if the searched intensity value is 128 in graylevel intensity, the triangle of pixels with values 140, 124, 127 does contain the 128 value somewhere between them and thus can be selected.
- each point S t position can vary inside the triangle P,, Q t , R, , its coordinates are expressed in barycentric form with respect to the three points defining the triangle.
- P t [x ( (w,, v f ), >>,( «,, v ( )] .
- four points will be defined with eight parameters.
- an interpolated intensity value can be computed c u t , v ( ) .
- the geometric constraint is satisfied by checking whether the corresponding comers of triangles P v P 2 , P ⁇ , P , P 5 satisfy the invariant values (withing certain tolerance).
- the chromatic value is computed by interpolation and the final expression for c 5 will depend on eight parameters c 5 ⁇ u ⁇ , v,, w 2 , v 2 , M 3 , V 3 , U 4 , V 4 ) .
- c 5 ⁇ u ⁇ , v,, w 2 , v 2 , M 3 , V 3 , U 4 , V 4 ) .
- the algorithm for learning a suitable representation of a given pattern is outlined in Fig. 4 and illustrated in Fig. 1.
- an area Learning zone (10) that contains an unoccluded pattern Logo (6) is specified by the user as in Fig. 1. Then, this area is searched for tuples of points that are representative in this whole area. In other words, tuples need to be found whose properties (geometric and chromatic invariant values) are sufficiently different from properties of all other point tuples in the learning zone. Tuples, satisfying such unicity conditions will not have similar tuples in the whole learning area and could, thus, be easily found at the search stage without mismatches.
- the algorithm of search for such unique tuples is described in Fig. 4.
- the Fig. 1 illustrates the operation of representative tuple research.
- the Candidate tuple (14) is selected so that all points have different color or intensity values to increase discrimination between points.
- a fixed neighbourhood Search zone (9) is defined around each of the points of the tuple in the Image frame (8).
- Potential neighbour tuple (13) from those search zones its geometric and chromatic invariants are computed and their values are compared with invariants of the Candidate tuple (14) that is being tested for unicity.
- the Search zone (9) is equivalent to the whole Learning zone (10) to obtain full unicity.
- Tuples that satisfy this unicity criterion are stored as part of logo representation.
- the use of those pixels in other tuples is reduced (they, however, will be used in neighbor tuples for comparison).
- the redundancy of representation is important, the participation of each pixel in a fixed number of tuples (for example ten). One avoids the use of one pixel in too many tuples to avoid extreme dependence on this point.
- the learning process will gradually use all the pixels in the Learning zone (10) (c.f. Fig. 1 ). If each pixel will be used several times in one or more Learned tuple (11) this area will be covered with several "layers" of tuples in a dense manner. With more tuples using a pixel, the whole representation will gain in reliability. At the end the whole representation will correspond to the large number of tuples characterised by their absolute intensity values, geometric invariant values and relative chromatic. When such dense representation is constructed, it can then be used for locating this logo in a new image.
- the operation of locating a logo in a new image corresponds to the search of learned tuples in this image.
- the whole image will be analysed for the presence of the learned tuples one by one.
- the density of representation with tuples allows to deal with almost random occlusion since every point is virtually related to (at least) four other points at various parts of the image.
- the Logo (6) can be occluded in any way by an Occluding object (7) as shown in Fig. 2, there will always be visible points not hidden by this object and that are related by tuples.
- the position of the logo frame is estimated. Every learned tuple contain information about its position relatively to the Learning zone (10). This information (which is in fact a transformation) could be inversed and the position of the Learning zone (10) (or reference frame) obtained from the position of the tuple. There will be false matches between tuples, therefore taking the frarne position confirmed by the majority of found tuples allows to find the right frame. This frame is also used to compute the position of the logo on the screen as well as its size relative to the screen.
- the visible part This is estimated as the number of pixels inside the frame covered by found tuples to the total number of pixels in the frar ⁇ e. This is done by computing the ratio of visually found points to the total number of points associated with particular logo. A different measure can be applied since not all points are visible all the time and something relative to pixels can be applied.
- Another operation that can be performed once all the point tuples were identified, is the modification of the visible part of the logo in order to improve its visual quality or provide a neat image of the logo. If the number of the identified points is sufficient, one can add many other points that can improve the resolution of the logo if it is viewed as a remote object.
- the last operation that can be performed is the replacement of the logo by visual information that is perceptually different from the logo appearance. This operation might be useful for hiding the logo, if its visibility in this video sequence is not desired.
- This transformation can be modelled with several approximating transformations like for example scaling of each of rgb channels, linear transformation in rgb space, etc.
- the scaling transformation corresponds to the scaling of the every chromatic channel by an independent scaling factor:
- these invariant values for every pair of pixels in the tuple. Then, these values will play a role of additional constraint for selection of tuples. This selection will thus be independent of the illumination changes, all pairs that have that value are retrieved. In the search image, these values can be computed only by combining several (N-tuples). Even if the number of those corfc. bination is high we are obliged to compute them since no other information is available.
- This st ⁇ p is an optional step in the search algorithm that is outlined in
- Fig. 1 An example image of a logo and tuple being learned
- Fig. 5 Algorithm for finding learned 5-tuple in the new image (pixel-based algorithm).
- Fig. 6 Algorithm for finding learned 5-tuple in the new image (sub-pixel algorithm).
- the algorithms of learning tuples and searching them can be implemented as an image processing software modules. This software can be run on a DSP within an embedded system or on a stardard computer that is connected to a camera.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computing Systems (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- Databases & Information Systems (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
- Illuminated Signs And Luminous Advertising (AREA)
Abstract
Description
Claims
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CH0300199 | 2003-03-27 | ||
PCT/CH2004/000182 WO2004086751A2 (en) | 2003-03-27 | 2004-03-24 | Method for estimating logo visibility and exposure in video |
Publications (1)
Publication Number | Publication Date |
---|---|
EP1611738A2 true EP1611738A2 (en) | 2006-01-04 |
Family
ID=33035093
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP20040722778 Withdrawn EP1611738A2 (en) | 2003-03-27 | 2004-03-24 | Method for estimating logo visibility and exposure in video |
Country Status (2)
Country | Link |
---|---|
EP (1) | EP1611738A2 (en) |
WO (1) | WO2004086751A2 (en) |
Families Citing this family (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7783130B2 (en) | 2005-01-24 | 2010-08-24 | The United States Of America As Represented By The Administrator Of The National Aeronautics And Space Administration | Spatial standard observer |
US20080219504A1 (en) * | 2007-03-05 | 2008-09-11 | Adams Henry W | Automatic measurement of advertising effectiveness |
CN107153809B (en) * | 2016-03-04 | 2020-10-09 | 无锡天脉聚源传媒科技有限公司 | Method and device for confirming television station icon |
CN106792153B (en) * | 2016-12-01 | 2020-07-28 | 腾讯科技(深圳)有限公司 | Video identification processing method and device and computer readable storage medium |
CN113469216B (en) * | 2021-05-31 | 2024-02-23 | 浙江中烟工业有限责任公司 | Retail terminal poster identification and integrity judgment method, system and storage medium |
Family Cites Families (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4817166A (en) | 1986-05-05 | 1989-03-28 | Perceptics Corporation | Apparatus for reading a license plate |
TW434520B (en) | 1998-06-30 | 2001-05-16 | Sony Corp | Two-dimensional code recognition processing method, device therefor and medium |
US6100941A (en) | 1998-07-28 | 2000-08-08 | U.S. Philips Corporation | Apparatus and method for locating a commercial disposed within a video data stream |
EP1172009A1 (en) | 2000-01-14 | 2002-01-16 | Koninklijke Philips Electronics N.V. | Simplified logo insertion in encoded signal |
KR100467575B1 (en) | 2001-07-23 | 2005-01-24 | 삼성전자주식회사 | Video replaying/recording system having the changeable function of background image/sound and method for changing background image/sound |
-
2004
- 2004-03-24 EP EP20040722778 patent/EP1611738A2/en not_active Withdrawn
- 2004-03-24 WO PCT/CH2004/000182 patent/WO2004086751A2/en active Application Filing
Non-Patent Citations (2)
Title |
---|
None * |
See also references of WO2004086751A2 * |
Also Published As
Publication number | Publication date |
---|---|
WO2004086751A2 (en) | 2004-10-07 |
WO2004086751A3 (en) | 2005-02-03 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
Hu et al. | Revisiting single image depth estimation: Toward higher resolution maps with accurate object boundaries | |
CN114782691B (en) | Robot target identification and motion detection method based on deep learning, storage medium and equipment | |
JP3740065B2 (en) | Object extraction device and method based on region feature value matching of region-divided video | |
CN101512601B (en) | Method for determining a depth map from images, device for determining a depth map | |
CN110210276A (en) | A kind of motion track acquisition methods and its equipment, storage medium, terminal | |
US7162102B2 (en) | Method and system for compositing images to produce a cropped image | |
US7379583B2 (en) | Color segmentation-based stereo 3D reconstruction system and process employing overlapping images of a scene captured from viewpoints forming either a line or a grid | |
US6785329B1 (en) | Automatic video object extraction | |
CN111445389A (en) | Wide-view-angle rapid splicing method for high-resolution images | |
US9982995B2 (en) | 3D scanner using structured lighting | |
CN107154017A (en) | A kind of image split-joint method based on SIFT feature Point matching | |
Stentoumis et al. | A local adaptive approach for dense stereo matching in architectural scene reconstruction | |
CN101853500A (en) | Colored multi-focus image fusing method | |
Matas | Colour-based object recognition | |
CN111402429B (en) | Scale reduction and three-dimensional reconstruction method, system, storage medium and equipment | |
CN113128433A (en) | Video monitoring image enhancement method of color migration matching characteristics | |
EP1611738A2 (en) | Method for estimating logo visibility and exposure in video | |
CN110717910B (en) | CT image target detection method based on convolutional neural network and CT scanner | |
WO2008063616A2 (en) | Apparatus for and method of robust motion estimation using line averages | |
Torres-Méndez et al. | Reconstruction of 3d models from intensity images and partial depth | |
CN112733624B (en) | People stream density detection method, system storage medium and terminal for indoor dense scene | |
JPH11510351A (en) | Apparatus and method for object tracking | |
Ross et al. | Learning static object segmentation from motion segmentation | |
Patricio et al. | A similarity-based adaptive neighborhood method for correlation-based stereo matching | |
Palma et al. | Enhanced visualization of detected 3d geometric differences |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
17P | Request for examination filed |
Effective date: 20051010 |
|
AK | Designated contracting states |
Kind code of ref document: A2 Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IT LI LU MC NL PL PT RO SE SI SK TR |
|
AX | Request for extension of the european patent |
Extension state: AL LT LV MK |
|
DAX | Request for extension of the european patent (deleted) | ||
17Q | First examination report despatched |
Effective date: 20061229 |
|
RAP1 | Party data changed (applicant data changed or rights of an application transferred) |
Owner name: COLORFOG SARL |
|
RIN1 | Information on inventor provided before grant (corrected) |
Inventor name: STARTCHIK, SERGEI |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE APPLICATION IS DEEMED TO BE WITHDRAWN |
|
18D | Application deemed to be withdrawn |
Effective date: 20171003 |