WO1999022336A1 - Specification d'objet dans une image en mode point - Google Patents

Specification d'objet dans une image en mode point Download PDF

Info

Publication number
WO1999022336A1
WO1999022336A1 PCT/US1998/022551 US9822551W WO9922336A1 WO 1999022336 A1 WO1999022336 A1 WO 1999022336A1 US 9822551 W US9822551 W US 9822551W WO 9922336 A1 WO9922336 A1 WO 9922336A1
Authority
WO
WIPO (PCT)
Prior art keywords
tag
tags
video image
test
vector quantity
Prior art date
Application number
PCT/US1998/022551
Other languages
English (en)
Inventor
Anthony J. Isadore Barreca
Original Assignee
Magic Circle Media, Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Magic Circle Media, Inc. filed Critical Magic Circle Media, Inc.
Priority to AU11199/99A priority Critical patent/AU1119999A/en
Publication of WO1999022336A1 publication Critical patent/WO1999022336A1/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence

Definitions

  • the present invention relates to the field of computer video image manipulation, and more specifically to an improved means and method for identifying objects of concern within a video image, such that identification of the objects can be maintained even as the objects move within the image.
  • the predominant current usage of the present inventive object specification system is in the identification of moving objects in a digitized movie wherein it is desirable to treat visually identifiable objects individually.
  • the inventive object specification system uses a computer to allow an operator to place a "tag" on an edge of an object in an image.
  • the computer then will identify other potential tags and the operator will accept the potential tags as desired, according to the application.
  • Potential tags are identified according to criteria which is inherent to the method and further according to criteria which can be modified as required, and tag identification is a function of the combining of such criteria according to the present inventive method.
  • An advantage of the present invention is that operator effort is minimized in the selection of tags to identify an image in a digitized picture.
  • a further advantage of the present invention is that usefulness of tags is optimized by assisting in the selection of tag locations.
  • Yet another advantage of the present invention is that objects identified according to the inventive method may be more easily tracked by a computer where the objects are in a moving video image.
  • Still another advantage of the present invention is that the inventive method is easy to implement using conventional readily available computers and peripheral devices.
  • Fig. 1 is a flow chart depicting an object specification method according to the present invention:
  • Fig. 2 is a computer system having displayed thereon a video image such as is acted upon according to the present inventive method;
  • Fig. 3 is a potential tag location diagram.
  • the best presently known mode for carrying out the invention is an object specification method instituted, primarily, through the use of a computer.
  • the predominant expected usage of the inventive object specification method is in the segregation of visually identifiable objects in a digitized video image such that the objects can be separated processed or treated by the computer.
  • the inventive object specification method is depicted in a flow diagram in Fig. 1 and is designated therein by the general reference character 10.
  • Fig. 2 is an elevational view of a computer system 12 such as is used by the originating user to practice the present invention.
  • the object specification method 10 begins with a "place tags" operation 14.
  • the place tags operation is discussed in detail in the issued United States Patent No. 5,590,262 ("the '262 patent"), discussed previously herein.
  • the originating user places a plurality of tags 16 (Fig. 2) on an outer edge 18 of an object 20 within a video image 22 displayed on a display screen 24 of the computer system 12.
  • Such operations are well known, such that one skilled in the art will understand how to cause a computer 26 of the computer system 12 to place the tags 16 when the originating user points with a mouse 28 and clicks with a mouse button 30.
  • a tag 16 is a set of pixels that is a subset of the pixels in a single frame of the video image 22 defining the object 20 that will be specified by the originating user and tracked by the computer system 12.
  • the tag 16 must contain at least one edge 18 of an object 20.
  • the minimum tag size is 9X9 pixels, but tags may be multiples of the basic Sobel mask size (3N X 3N), where N is equal to or greater than 3.
  • each tag will have a region of interest 32, which is generally that portion of the tag 16 which lies within the object 20, while the remainder of the tag 16 lies outside the region of interest. While the determination and use in tracking of the region of interest (“ROI") 32 is known in the art. due primarily to the teachings of the '262 patent, briefly, the ROI 32 is determined as follows: Using the nearest neighbor tags 16 on either side of a tag 16 for which the ROI 32 is to be calculated, compute the angle formed by these three tags 16 with the subject tag 16 placed at the vertex. Then, the pixels in the subject tag 16 that are subtended by the angle computed in the previous step form the ROI 32 for that tag 16.
  • Determination of the ROI 32 may also require the use of a tag interiority test to determine which side of the edge 18 is inside the ROI 32 and which is outside.
  • a tag interiority test is briefly described hereinafter in relation to a "functional specification of heuristics". Having established the ROI 32, it can be appreciated that the edge 18 might then be redefined as being that juncture within the tag 16 with an ROI 32 that can be found using standard computational techniques for finding differences in color, saturation, brightness, or other attributes of digitally expressed color spaces.
  • a map is an algorithm that accepts as input a single tag 16 and that outputs another tag 16 that is based on the first but that is different in some way. In addition to the new output tag, a map may also generate additional optional information.
  • Types of maps include: Filters, which are used to average or eliminate noise within a given color space or to emphasize features; Transforms, which include color space conversions, convolutions used as part of a feature extraction process such as the known Sobel process or LaPlace edge detection, and difference functions used to enhance the process of feature extraction; and, Statistics, such that statistical characteristics of tags, such as groups with ROI's are extracted.
  • map and validity tests operation 34 maps are used to "clean up" the tag so that it is more readily identified in later tracking operations. It is anticipated by the inventor that validity tests to be applied in the apply map and validity tests operation 34 will include an algorithm that asses whether an individual tag 16 is valid when it is evaluated within a set of tags 16 defining an object 20.
  • An example of a tag validity algorithm is a Tag-In-Context test, described here in relation to a functional specification of heuristics. In a "valid?" decision operation, if a tag 16 is not valid, the originating user is returned to the place tags operation 14 to try again.
  • Fig. 3 is a potential tag location diagram 42 showing a plurality of potential tag locations 16a relative to a tag 16 (the tag 16 being the location of that tag 16 in the just previous frame of the moving video image 22). As can be seen in the view of Fig.
  • n can be a distance of one pixel, one unit of pixels, or one tag size, from the previous location of the tag 16.
  • a list is created for each tag 16 providing a score for each potential tag location 16a.
  • the score possible matches and create list operation 40 is performed for each tag 16 on each object 20 of interest (since there may be more than one object 20 of interest in a frame of the video image 22) and for each frame of the moving video image 22). For the sake of clarity, this multiple looping is not shown in the simplified flow diagram of figure 1.
  • An anchor tag 16b (Fig. 2) is located at that tag 16 which has the tightest cluster of scores for the several potential tag locations 16a (Fig. 3).
  • the anchor tag 16b is the tag 16 which is used as the principal point of reference in applying certain validity tests which will discussed hereinafter.
  • the anchor tag 44 is selected by evaluating statistics associated with scoring the set of possible match tags.
  • a rigid geometry test 46 is applied to determine of the object 20 is rigid (from the point of view of an observer of the image 22) such that all of the tags 16 appear to be moving in a consistent manner - as compared to a just previous frame of the video image 22.
  • the rigid geometry test 46 tests whether the spatial relations between tags 16 delineating an object remain constant form frame to frame.
  • the rigid geometry test 46 flow continues to an articulated motion test 48 wherein it is determined whether a portion of the object 20 is. itself, rigid but is moving in relation to a remainder of the object 22 (such as. for example, a stationary dog wagging its tail). That is. the articulated motion test 48 determines when parts or sections of rigid objects
  • a rotation shift test 50 wherein it is determined if the object 20 is moving in such a manner that it appears to be both changing in dimension and moving - such as where it might be rotating on two axis at the same time.
  • the rotation shift test identifies objects 20 that exhibit perspective, rotation, or shading changes.
  • a calculate vectors and vector change frames operation 56 uses the data previously obtained in the object specification method 10 to calculate, for each tag 16, a motion vector which will continue through the running of the moving video image 22 until a new motion vector for a given frame is recorded.
  • the inventive object specification method 10 will not have to be accomplished each time the moving video image 22 is presented to the end user.
  • inventive object specification method 10 could not reasonably be accomplished each time the moving video image 22 is presented to the end user for reasons including that it requires operator intervention, and far too much computing power to accomplish in "real time”. Rather, the location of the object 20 will be known to the computer 26 (which will, typically be a different computer 26 than the one originally employed to accomplish the object specification method) will keep track of the location of the object 20 using the tag vectors established in the calculate vectors and vector change frames operation 56.
  • various tests are provided for determining the validity of a tag 16. both initially and as the tag 16 moves through time in the moving video image 22. Also, the validity of the position of tags 16 is tested in relation to the position of other tags to determine that the tags 16 adequately delineate the boundaries of the object 20.
  • Object validity tests are applied during each of the rigid geometry test 46. the articulated motion test 48 and the rotation shift test 50. The object validity tests examine a set of tags 16 to determine whether or not the specified set is sufficient to validly characterize an object 22.
  • An example of an object validity test is the Horizon test, described herein in relation to a functional specification of tests.
  • a tag interiority test is a part of the apply map and validity tests operation 14. The tag interiority test automatically determines the tag 16
  • ROI 32 by determining which side of an edge 18 is interior, that is. which side is a part of the object 20 partially defined by the tag, and computing the relevant characteristics of that ROI 32 (such as. color and/or luminance).
  • the tag interiority test uses a topological approach that looks at the overall tag set to perform its inside/outside determination.
  • a tag horizon test is to ensures that when an originating user creates a tag set bounding (delineating) a particular object 20, that no significant part of that object 20 is omitted. It may be that visual inspection by the originating user will be required to insure that this is adequately accomplished.
  • the tag horizon test is also presently a part of the apply map and validity tests operation.
  • a tag in context test identifies tags 16 that can be ignored when tracking. These tags 16 are those that can be moved without causing a change in the tag code (that is, without being able to detect a change in the tag's characterization).
  • the object specification method 10 of the present invention may be readily produced and integrated into existing systems and methods for identifying and tracking objects in a video image, and since the advantages as described herein are provided, it is expected that it will be readily accepted in the industry. For these and other reasons, it is expected that the utility and industrial applicability of the invention will be both significant in scope and long lasting in duration.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Analysis (AREA)

Abstract

L'invention concerne un procédé de spécification d'objet (10) permettant d'identifier et de suivre des étiquettes (16) délimitant un objet (20) dans une image vidéo (22). On peut appliquer le procédé de spécification d'objet (10) pour le placement d'étiquettes (1) dans des images vidéo (22) fixes et il suit ensuite les étiquettes (16) à mesure que l'image vidéo (22) passe à travers des trames pour devenir une image vidéo (22) mobile. Un utilisateur d'origine utilise un système informatique (12) pour réaliser l'opération de l'invention (10) aboutissant en une opération (56) de vecteurs de calcul et de trames de modification de vecteurs, un produit étant créé de manière à pouvoir être utilisé par un utilisateur final.
PCT/US1998/022551 1997-10-24 1998-10-23 Specification d'objet dans une image en mode point WO1999022336A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
AU11199/99A AU1119999A (en) 1997-10-24 1998-10-23 Objet specification in a bit mapped image

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US95722297A 1997-10-24 1997-10-24
US08/957,222 1997-10-24

Publications (1)

Publication Number Publication Date
WO1999022336A1 true WO1999022336A1 (fr) 1999-05-06

Family

ID=25499259

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US1998/022551 WO1999022336A1 (fr) 1997-10-24 1998-10-23 Specification d'objet dans une image en mode point

Country Status (2)

Country Link
AU (1) AU1119999A (fr)
WO (1) WO1999022336A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6862556B2 (en) 2000-07-13 2005-03-01 Belo Company System and method for associating historical information with sensory data and distribution thereof

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4739401A (en) * 1985-01-25 1988-04-19 Hughes Aircraft Company Target acquisition system and method
US5243418A (en) * 1990-11-27 1993-09-07 Kabushiki Kaisha Toshiba Display monitoring system for detecting and tracking an intruder in a monitor area
US5657251A (en) * 1995-10-02 1997-08-12 Rockwell International Corporation System and process for performing optimal target tracking
US5809161A (en) * 1992-03-20 1998-09-15 Commonwealth Scientific And Industrial Research Organisation Vehicle monitoring system

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4739401A (en) * 1985-01-25 1988-04-19 Hughes Aircraft Company Target acquisition system and method
US5243418A (en) * 1990-11-27 1993-09-07 Kabushiki Kaisha Toshiba Display monitoring system for detecting and tracking an intruder in a monitor area
US5809161A (en) * 1992-03-20 1998-09-15 Commonwealth Scientific And Industrial Research Organisation Vehicle monitoring system
US5657251A (en) * 1995-10-02 1997-08-12 Rockwell International Corporation System and process for performing optimal target tracking

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6862556B2 (en) 2000-07-13 2005-03-01 Belo Company System and method for associating historical information with sensory data and distribution thereof

Also Published As

Publication number Publication date
AU1119999A (en) 1999-05-17

Similar Documents

Publication Publication Date Title
AU714213B2 (en) Object identification in a moving video image
US6124864A (en) Adaptive modeling and segmentation of visual image streams
US6954498B1 (en) Interactive video manipulation
US9303525B2 (en) Method and arrangement for multi-camera calibration
JP4899729B2 (ja) ３次元マルチカメラビデオ監視支援システム、３次元マルチカメラビデオ監視支援方法、３次元マルチカメラビデオ監視支援プログラム
JP4226730B2 (ja) 物体領域情報生成方法及び物体領域情報生成装置並びに映像情報処理方法及び情報処理装置
US11188739B2 (en) Processing uncertain content in a computer graphics system
US7944454B2 (en) System and method for user monitoring interface of 3-D video streams from multiple cameras
US6870945B2 (en) Video object tracking by estimating and subtracting background
Zollmann et al. Image-based ghostings for single layer occlusions in augmented reality
JP5967904B2 (ja) ユーザがコンピュータシステムとやり取りを行うことができる、ビデオストリームにおける動く非剛体オブジェクトをリアルタイムで検出し及び追跡する方法及び装置
US9877010B2 (en) Camera tracker target user interface for plane detection and object creation
US20100002909A1 (en) Method and device for detecting in real time interactions between a user and an augmented reality scene
US9973744B2 (en) 3D tracked point visualization using color and perspective size
KR100998428B1 (ko) 이미지 평가 방법 및 이미지 이동 결정 방법
US20060088189A1 (en) Statistically comparing and matching plural sets of digital data
US5767857A (en) Method, apparatus, and software product for generating outlines for raster-based rendered images
US10853990B2 (en) System and method for processing a graphic object
JP2008102972A (ja) 自動３ｄモデリングシステム及び方法
Wang et al. People as scene probes
Chao et al. Augmented 3-D keyframe extraction for surveillance videos
WO1999022336A1 (fr) Specification d'objet dans une image en mode point
JP3601951B2 (ja) 移動物体の表示方法、それを用いた表示システムおよびそのためのプログラム記録媒体
JPH07121710A (ja) 画像セグメンテーション方法及び装置
CN117593421A (zh) 一种线下展示对象的电子化标示方法和装置

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): AL AM AT AU AZ BA BB BG BR BY CA CH CN CU CZ DE DK EE ES FI GB GE GH GM HR HU ID IL IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MD MG MK MN MW MX NO NZ PL PT RO RU SD SE SG SI SK SL TJ TM TR TT UA UG US UZ VN YU ZW

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): GH GM KE LS MW SD SZ UG ZW AM AZ BY KG KZ MD RU TJ TM AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE BF BJ CF CG CI CM GA GN GW ML MR NE SN TD TG

121 Ep: the epo has been informed by wipo that ep was designated in this application
DFPE Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed before 20040101)
REG Reference to national code

Ref country code: DE

Ref legal event code: 8642

NENP Non-entry into the national phase

Ref country code: KR

NENP Non-entry into the national phase

Ref country code: CA

122 Ep: pct application non-entry in european phase