EP1998567B1 - Tracking device, tracking method, tracking device control program, and computer-readable recording medium - Google Patents

Tracking device, tracking method, tracking device control program, and computer-readable recording medium Download PDF

Info

Publication number
EP1998567B1
EP1998567B1 EP07738537.5A EP07738537A EP1998567B1 EP 1998567 B1 EP1998567 B1 EP 1998567B1 EP 07738537 A EP07738537 A EP 07738537A EP 1998567 B1 EP1998567 B1 EP 1998567B1
Authority
EP
European Patent Office
Prior art keywords
information
feature information
tracking
face
detected
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
EP07738537.5A
Other languages
German (de)
French (fr)
Other versions
EP1998567A1 (en
EP1998567A4 (en
Inventor
Satoru Kuwano
Takashi Kakiuchi
Tomohiro Inoue
Masahiro Senga
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Omron Corp
Original Assignee
Omron Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority to JP2006071861 priority Critical
Application filed by Omron Corp filed Critical Omron Corp
Priority to PCT/JP2007/055066 priority patent/WO2007119355A1/en
Publication of EP1998567A1 publication Critical patent/EP1998567A1/en
Publication of EP1998567A4 publication Critical patent/EP1998567A4/en
Application granted granted Critical
Publication of EP1998567B1 publication Critical patent/EP1998567B1/en
Application status is Active legal-status Critical
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/20Analysis of motion
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06KRECOGNITION OF DATA; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
    • G06K9/00Methods or arrangements for reading or recognising printed or written characters or for recognising patterns, e.g. fingerprints
    • G06K9/00221Acquiring or recognising human faces, facial parts, facial sketches, facial expressions
    • G06K9/00228Detection; Localisation; Normalisation
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06KRECOGNITION OF DATA; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
    • G06K9/00Methods or arrangements for reading or recognising printed or written characters or for recognising patterns, e.g. fingerprints
    • G06K9/00221Acquiring or recognising human faces, facial parts, facial sketches, facial expressions
    • G06K9/00288Classification, e.g. identification
    • G06K9/00295Classification, e.g. identification of unknown faces, i.e. recognising the same non-enrolled faces, e.g. recognising the unknown faces across different face tracks
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06KRECOGNITION OF DATA; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
    • G06K9/00Methods or arrangements for reading or recognising printed or written characters or for recognising patterns, e.g. fingerprints
    • G06K9/00624Recognising scenes, i.e. recognition of a whole field of perception; recognising scene-specific objects
    • G06K9/00771Recognising scenes under surveillance, e.g. with Markovian modelling of scene activity
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00Television systems
    • H04N7/18Closed circuit television systems, i.e. systems in which the signal is not broadcast
    • H04N7/183Closed circuit television systems, i.e. systems in which the signal is not broadcast for receiving images from a single remote source
    • H04N7/185Closed circuit television systems, i.e. systems in which the signal is not broadcast for receiving images from a single remote source from a mobile camera, e.g. for remote control

Description

    TECHNICAL FIELD
  • The present invention relates to a device and a method which recognize and track a position of a specific figure inside a photographic image.
  • BACKGROUND ART
  • In a conventional device which tracks a human figure in a photographic image, a tracking-objected person requires to wear a specific device such as a pulse transmission device or a GPS (global positioning system) device in order for the device to track a specific human figure (for example, see Patent Document 1 and 2). However, the following techniques do not require the tracking-objected person to wear the specific devices. More specifically, there is a technique where (i) a position of a face of a human figure is confirmed, and (ii) panning, tilting, and zooming of a security camera are controlled so that the face is positioned in the center of a screen, so as to trace a movement of the face. Further, in the technique, features of the face are extracted from the image, in order to compare the face with a face registered in a database. This is to see whether or not the face matches the face registered in the database, in order to recognize a specific individual (for example, see Patent Document 3). Moreover, there is a technique where face information of a human figure is acquired from an image, and the face information thus acquired is tracked by use of a plurality of cameras, when the human figure is or is not registered in a database (for example, see Patent Document 4). There is also another technique where (i) automatic tracking is carried out by tracing a subject (tracking object) that is designated by pointing on a screen, and (ii) a video camera is controlled so that the subject comes in a center of a monitor screen (for example, see Patent Document 5).
  • "Tracking" in the present specification indicates an action to keep an image of a subject (tracking object) inside a photographic image (so that the subject is continuously captured in the image).
  • Patent Document 6 describes a method according to which camera information corresponding to the position at which a target to be searched for in the captured image is identified, another camera specified by the identified camera information receives a notification that the target has been detected, and when a notification is received from another camera, the posture of the own camera is controlled so that the area associated with the camera information specifying another camera can be observed.
  • Non-Patent Literature 1 relates to a real-time cooperative multi-target tracking system consisting of a group of Active Vision Agents (AVAs), which is a logical model of a network-connected computer with an active camera. The AVAs cooperatively track their target objects by dynamically exchanging object information with each other.
    • [Patent Document 1]
      Japanese Unexamined Patent Publication No. 40514/2004 (Tokukai 2004-40514) (published on February 5, 2004)
    • [Patent Document 2]
      Japanese Unexamined Patent Publication No. 20205/2005 (Tokukai 2005-20205) (published on January 20, 2005)
    • [Patent Document 3]
      Japanese Unexamined Patent Publication No. 163600/2000 (Tokukai 2000-163600) (published on June 16, 2000)
    • [Patent Document 4]
      Japanese Unexamined Patent Publication No. 324720/2003 (Tokukai 2003-324720) (published on November 14, 2003)
    • [Patent Document 5]
      Japanese Unexamined Patent Publication, No. 339056/1994 (Tokukaihei 6-339056) (published on December 6, 1994)
    • [Patent Document 6]
      Patent Application US 2005/280711 A1
    • [Non-Patent Literature 1]
      Ukita et al. : "Real-time cooperative multi-target tracking by communicating active vision agents", Information Fusion, 2003. Proceedings of the Sixth International Conference of July 8-11, 2003, Piscataway, NJ, USA, IEEE, vol. 1, 8 July 2003 (2003-07-08), pages 439-446.
    DISCLOSURE OF INVENTION
  • In a case where there are a plurality of human figures which are subject to tracking, the conventional techniques do not have means to automatically switch a tracking object when a human figure having a higher tracking priority than the human figure that is currently tracked comes into an image (frame in). Therefore, when the human figure having the higher tracking priority comes into the image while another human figure is tracked, a user, for example an observer, requires to carry out a switching operation of the tracking object, by designating the human figure which is to be tracked by, for example, pointing the human figure on a screen on which the photographic image is displayed. As such, when there is an order of priority in tracking the tracking objects, the user has to go to the trouble of switching the tracking object. Moreover, it is unknown when an object having a higher priority in the tracking order comes into the photographic image. Therefore, the user has to constantly watch the screen on which the photographic image is displayed.
  • The present invention is attained in view of the above problem, and its object is to realize a tracking device, a tracking method, a tracking device control program, and a computer-readable recording medium, each of which can autonomously track a tracking object, not having to cause trouble to a user even in a condition where a figure prioritized in tracking order is included in an image.
  • In order to attain the object, a tracking device according to the present invention tracks a tracking object by controlling a photographing section so as to continuously include the tracking object in a photographic image taken by the photographing section, the tracking device comprising:
    • feature information detection means for detecting feature information from the photographic image, wherein the feature information is face information which specifies features of the face of a person;
    • tracking object matching means for (i) comparing the feature information thus detected with a tracking object information database in which feature information of a plurality of persons is registered so that the feature information corresponds to a priority indicating tracking order of the feature information, and (ii) determining whether or not the feature information thus detected is information of a tracking object;
    • priority acquisition means for acquiring the priority of the feature information thus detected from the tracking object information, in the case where it is determined that the feature information thus detected is the information of a tracking object; and
    • control means for controlling the photographing section, based on the priority thus acquired, so as to continuously include, in the photographic image from which the feature information is detected, feature information that has a highest priority in the photographic image;
    • wherein the tracking object matching means (i) finds a similarity between the feature information detected by the feature information detection means and the feature information included in the tracking object information database, (ii) determines, in a case where the similarity thus found is greater than a predetermined authentication threshold value, that the feature information thus detected is information of a tracking object and (iii) finds authentication reliability indicative of how large the similarity thus found is as compared with the predetermined authentication threshold value; and in a case where the photographic image includes a plurality of feature information having an identical priority, the control means controls the photographing section so that one of the plurality of feature information, which has a greater authentication reliability, is continuously included in the photographic image.
  • In order to attain the object, a tracking method according to the present invention is a method which tracks a tracking object by controlling a photographing section so as to continuously include the tracking object in a photographic image taken by the photographing section, the tracking method comprising the steps of:
    • detecting feature information from the photographic image, wherein the feature information is face information which specifies features of the face of a person;
    • comparing the feature information thus detected with a tracking object information database in which feature information of a plurality of persons is registered so that the feature information corresponds to a priority indicating tracking order of a feature information;
    • determining whether or not the feature information thus detected is information of a tracking object;
    • acquiring the priority of the feature information thus detected from the tracking object information, in the case where it is determined that the feature information thus detected is the information of a tracking object; and
    • controlling the photographing section, based on the priority thus acquired, so as to continuously include, in the photographic image from which the feature information is detected, feature information that has a highest priority in the photographic image;
    • wherein the method further comprises:
      • finding a similarity between the detected feature information and the feature information included in the tracking object information database;
      • determining, in a case where the similarity thus found is greater than a predetermined authentication threshold value, that the feature information thus detected is information of a tracking object; and
      • finding authentication reliability indicative of how large the similarity thus found is as compared with the predetermined authentication threshold value; wherein, in the step of controlling the photographic section, in a case where the photographic image includes a plurality of feature information having an identical priority, the photographing section is controlled so that one of the plurality of feature information, which has a greater authentication reliability, is continuously included in the photographic image.
  • According to the arrangement and the method, when feature information included in a photographic image is detected, and the feature information thus detected is information of a tracking object, a priority indicative of a tracking order of the feature information thus detected is acquired. Acquisition of the priority is carried out for feature information which has been determined as information of the tracking object. As a result, photographing can be carried out by controlling the photographing section so that the tracking device continuously includes, in the photographic image from which the feature information is detected, the feature information having a highest priority, based on the priority thus acquired, even in the case where a plurality of feature information are included in the photographic image. This allows continuous tracking of the feature information having the highest priority. Here, the priorities are expressed by numerals for example, and may be handled as a smaller numeral expression in priority of the feature information denoting a higher priority. For example, if there are priorities that are expressed by numerals 1, 2 and 3, the priorities of these decrease in the order of 1, 2, and 3, i.e., 1 > 2 > 3. Of course, how the feature information is prioritized is not limited to this. Note that "tracking" denotes an action to keep an image of a subject (tracking object) inside a photographic image (so as to continuously capture the subject within the image).
  • Of course, if there is only one feature information that is prioritized in the tracking object information, that prioritized feature information is continuously tracked. Priority is also acquired in a case where there is just one feature information registered in the tracking object information. In such case, the one feature information is considered to have the highest priority, even if the feature information is actually not prioritized. If just a part of a plurality of the feature information is prioritized whereas the rest of the feature information are not prioritized, priority may be acquired on the basis that the rest of the feature information is considered to equally have a lowest priority.
  • Controlling of the photographing section is preferably carried out, for example, by controlling operations of a camera which is a photographing device, more specifically, controlling operations such as (i) panning/tilting and zooming that are carried out by moving a position of a camera horizontally, vertically, or forward and back, (ii) changing of exposure, or (iii) switching of cameras in the case where there are a plurality of imaging cameras. The position of the camera may be moved, for example, by placing the photographing camera on a stand positioned on a rail. Of course, the photographing section may be controlled in any way as long as the feature information having the highest priority is continuously included in the photographic image.
  • The operations of the camera are preferably controlled so as to be carried out when there is a change to the feature information having the highest priority in the photographic image, more specifically, when there is a change to the feature information having the highest priority in at least one of a position (the position change may be to one part of the feature information), size, tilt, and orientation of the feature information. Of course, the camera may be controlled so that the camera is operated based on other changes than the above.
  • Furthermore, according to the arrangement and the method, the photographing section is controlled so as to continuously include, in the photographic image from which the feature information is detected, feature information that has a highest priority in the photographic image. Therefore, even if the feature information having the highest priority is no longer tracked since the feature information having the highest priority has moved outside of a range in which the photographing section can track an image for example, it is possible to track the feature information having the highest priority in the photographic image at that time (feature information having a higher priority second to the feature information which is no longer tracked).
  • As the above, according to the arrangement and the method, the photographing section is controlled so that the tracking device continuously tracks the feature information having the highest priority, even if there is a plurality of tracking objects and a tracking order is given to the tracking objects. Therefore, the tracking device is capable of tracking the feature information having the highest priority in the photographic image, without requiring an operation from the user. Therefore, it is unnecessary for the user to specify the tracking object. As a result, the operations taken by the user are reduced. As such, the arrangement and the method allow autonomous tracking of the feature information, not having to cause trouble to the user, even if a figure that is prioritized in tracking order is included in the image.
  • In addition to the above arrangement, with the tracking device according to the present invention, the feature information is face information which specifies features of a face of a figure.
  • Face information excels in distinguishing a subject in order to specify an individual. According to the arrangement, the feature information is face information, so that authentication is easily carried out at the time of matching. The face information allows easy identification of an individual. Therefore, for example, tracking of players in sports relay broadcasting such as baseball and soccer is possible in an order of popularity, by prioritizing face information of the players.
  • Of course, the feature information is not necessarily the face information of a face of a human figure, as long as subjects are distinguishable between each other. For example, the feature information is information of an external appearance of a car. In such a case, it is possible to track cars in an order of popularity in relay broadcasting of an automobile race by prioritizing the cars, for example. If the feature information is for example information of a face of a horse, it is possible to track a horse in relay broadcasting of horse racing in order of popularity, by prioritizing the horses. Note that the above are simply exemplifications.
  • The tracking device according to the present invention is arranged such that the tracking object matching means (i) finds a similarity between the feature information detected by the feature information detection means and the feature information included in the tracking object information, (ii) determines, in a case where the similarity thus found is greater than a predetermined authentication threshold value, that the feature information thus detected is information of a tracking object, and (iii) finds authentication reliability indicative of how large the similarity thus found is as compared with the predetermined authentication threshold value; and in a case where the photographic image includes a plurality of feature information having an identical priority, the control means controls the photographing section so that one of the plurality of feature information, which has a greater authentication reliability, is continuously included in the photographic image.
  • According to the arrangement, a similarity is found between the feature information thus detected and feature information included in the tracking object information. If the similarity thus found is greater than a predetermined authentication threshold value, it is determined that the feature information thus detected is information of a tracking object. Furthermore, authentication reliability is found. This allows continuously including, in the photographic image, the feature information that has a greater authentication reliability in the case where a plurality of feature information that have an identical priority is included in the photographic image.
  • Therefore, even if a plurality of feature information that have the identical priority is included in the photographic image, it is possible to determine an object and track the feature information of the object to be always included in the photographic image. Determination of the object which is to be continuously included in the photographic image is dependent on the authentication reliability. Therefore, it is possible to further reliably track the tracking object.
  • In addition to the above arrangement, the tracking device according to an embodiment of the present invention is arranged such that the control means controls the photographing section so that at least one of a size and a position of the feature information that has the highest priority is always fixed in the photographic image.
  • According to the arrangement, at least one of a size and a position of the feature information having the highest priority is always fixed in the photographic image. The feature information that is in a stable size in the photographic image makes it easier to find the feature information having the highest priority, in the photographic image. A stable position of the feature information allows finding the feature information having the highest priority just by looking at a fixed position, for example, a center of a monitor on which the photographic image is displayed. As such, it is possible to easily carry out surveillance of the feature information having the highest priority by having at least one of the size and the position of the feature information having the highest priority fixed in the photographic image.
  • The tracking device may be realized by a computer. In such a case, a computer-readable recording medium which records a tracking device control program, which program causes the computer to operate as each of the aforementioned means so as to realize the tracking device, is also within the scope of the present invention.
  • As described above, the tracking device in accordance with the present invention includes the features defined in claim 1.
  • According to the arrangement, when feature information that is included in a photographic image is detected, and the feature information thus detected is information of a tracking object, a priority, which indicates a tracking order of the feature information thus detected, is acquired. Acquisition of the priority is carried out for feature information which has been determined as the tracking object. Therefore, it is possible to carry out photographing by controlling the photographing section so that the tracking device continuously includes, in the photographic image from which the feature information is detected, the feature information having a highest priority, based on the priority thus acquired, even if a plurality of feature information are included in the photographic image. This allows continuous tracking of the feature information having the highest priority. Of course, if there is only one feature information that is prioritized in the tracking object information, that prioritized feature information is continuously tracked.
  • Furthermore, according to the arrangement, the photographing section is controlled so as to continuously include, in the photographic image from which the feature information is detected, feature information that has a highest priority in the photographic image. Therefore, even if, for example, the feature information having the highest priority is no longer tracked since the feature information has moved outside of the range in which the photographing section can track an image, it is possible to track the feature information having a highest priority in the photographic image at that time (feature information having a higher priority second to the feature information which is no longer tracked).
  • As the above, according to the arrangement, the photographing section is controlled so that the tracking device continuously tracks the feature information having the highest priority, even if there is a plurality of tracking objects and a tracking order is provided to the tracking objects. Therefore, the tracking device is capable of tracking feature information having the highest priority in the photographic image, without requiring an operation from the user. Therefore, it is unnecessary for the user to specify the tracking object. As a result, the operations by the user are reduced. As such, the arrangement allows autonomous tracking, not having to cause trouble to the user, even if a figure that is prioritized in tracking order is included in an image.
  • BRIEF DESCRIPTION OF DRAWINGS
    • Fig. 1 illustrates an embodiment of the present invention, and is a block diagram illustrating an essential arrangement of a tracking device.
    • Fig. 2 is an explanatory view illustrating one example of a face information database, in a form of a graph.
    • Fig. 3(a) is an image view illustrating a switching of a tracking object.
    • Fig. 3(b) is an image view illustrating a switching of a tracking object.
    • Fig. 3(c) is an image view illustrating a switching of a tracking object.
    • Fig. 4(a) is an image view illustrating a switching of camera.
    • Fig. 4(b) is an image view illustrating a switching of camera.
    • Fig. 4(c) is an image view illustrating a switching of camera.
    • Fig. 5 is a flow chart illustrating a flow of a tracking process in a tracking device of the present embodiment.
    • Fig. 6 is a flow chart illustrating a flow of a camera switching process.
    • Fig. 7(a) is an image view illustrating an example where a tracking device of the present embodiment is used in relay broadcasting of horse racing.
    • Fig. 7(b) is an image view illustrating an image captured by a camera shown in Fig. 7(a).
    • Fig. 8(a) is an image view illustrating an example in which a tracking device of the present embodiment is used in relay broadcasting of a baseball game.
    • Fig. 8(b) is an image view illustrating an image captured by a camera shown in Fig. 8(a).
    • Fig. 8(c) is an image view illustrating an image captured by another camera shown in Fig. 8(a).
    • Fig. 8(d) is an image view illustrating an image captured by still another camera shown in Fig. 8(a).
    REFERENCE NUMERALS
  • 1
    Tracking device
    2
    Control section
    3
    Photographing section
    4
    Storage section
    5
    Input accepting section
    6
    Display section
    21
    Face detection section (feature information detection means)
    22
    Face matching section (tracking object matching means)
    23
    Priority acquisition section (priority acquisition means)
    24
    Movement calculation section
    25
    Photographing control section (control means)
    26
    Position recognition section (position recognition means)
    33
    Photographing device
    41
    Face information database (tracking object information)
    BEST MODE FOR CARRYING OUT THE INVENTION
  • One embodiment of the present invention is described below with reference to Figs. 1 through 8. The following description explains a tracking device of the present embodiment, which in order to track a specific object, recognizes feature information included in a photographic image as face information of a figure (human or thing), and matches the face information. However, the feature information may be any kind of information, as long as a subject that is included in the photographic image is distinguishable.
  • [Arrangement of tracking device]
  • A tracking device 1 of the present embodiment includes a control section 2, a photographing section 3, a storage section 4, an input section 5, and a display section 6.
  • The control section 2 has a microcomputer built therein. Based on a computer program, the control section 2 operates by receiving an input operation or the like from the input section 5. The control section 2 controls operations of various arrangements in the tracking device 1 as a whole. The control section 2 functions as a face detection section 21, a face matching section 22, a priority acquisition section 23, a movement calculation section 24, a photographing control section 25, and a position recognition section 26.
  • The face detection section 21 detects face information (face) that is included in a photographic image taken by the photographing section 3.
  • The face matching section 22 compares the face information detected by the face detection means with a face information database (DB) 41. In the face information DB 41, face information of a plurality of persons is registered, which face information corresponds to a priority indicating tracking order of the face information. Subsequently, the face thus detected is determined whether or not the face is a tracking object. Here, face information denotes a face image. In this case, a conventionally known method such as an eigenface method, an LFA (Local Feature Analysis) method, a graph matching method, a neural network method, a constrained mutual subspace method, a perturbation space method, or a frequency analysis method is to be used for determining sameness between the face images by comparing the face images.
  • For example, the sameness of the face images is determined as follows. Feature amount in a face image is found by use of Gabor-wavelet transform or the like. Subsequently, similarity between the feature amount in the face image and the feature amount registered on the database is found. Thus, the sameness is determined. If the similarity exceeds a threshold value set in advance, the two faces are determined as the same face. As such, the sameness of the face images can be determined by comparing the face images. Note that a determination method is not limited to this.
  • The face matching section 22 finds the similarity between the face information and face information included in the face information DB 41. If the similarity thus found is greater than a predetermined authentication threshold value, the face detection section 21 determines that the face information thus detected is a tracking object. Furthermore, the face matching section 22 finds authentication reliability, based on how greater the similarity thus found is from the predetermined authentication threshold value.
  • The priority acquisition section 23 acquires a priority of the face information thus detected from the face information DB 41.
  • The movement calculation section 24 measures a movement of the face information that has a highest priority of the face information thus detected, and calculates an amount of the movement of the face information having the highest priority. Calculation of the amount of the movement by the movement calculation section 24 may be carried out by, for example, well-known methods such as Patent Document 5.
  • The photographing control section 25 controls the photographing section 3 so that the face information having the highest priority is continuously included in the photographic image from which the face information is detected. Controlling of the photographing section 3 is carried out by following the movement of the face information having the highest priority, based on the amount of movement calculated by the movement calculation section 24. The controlling of the photographing section 3 is preferably carried out, for example, by controlling operations of a camera which is a photographing device 33, more specifically, (i) controlling operations such as panning/tilting and zooming, that are carried out by moving a position of a camera horizontally, vertically, or forward and back, (ii) changing in exposure, or (iii) switching of cameras in the case where there are a plurality of photographing cameras. The position of the camera may be moved, for example, by placing the photographing camera on a stand positioned on a rail. Of course, the photographing section 3 may be controlled in any way as long as the face information having the highest priority is continuously included in the photographic image.
  • It is preferable for the photographing section 3 to be controlled so that the operation of the camera is carried out when there is a change to the face information having the highest priority in the photographic image, more specifically, when there is a change to the face information having the highest priority, to at least one of a position (the change may be to one part contained in the face information, for example, a position of an eye), size, tilt, and a facing direction of the face information. Of course, the photographing section 3 may be controlled so that the camera is operated based on other changes than the above.
  • The photographing control section 25 may control the photographing section 3 so that at least one of the size and the position of the face having the highest priority is fixed in the photographic image. Of course, the face having the highest priority may be tracked just so as to keep the face inside the photographic image. However, the face at a stable size in the photographic image makes it easier to find the face having the highest priority in the photographic image. A stable position of the face allows finding the face information having the highest priority just by looking at a fixed position, for example, a center of a display screen on which the photographic image is displayed. Of course, the position is not limited to the center. As such, it is possible to easily carry out surveillance of a face having the highest priority by having at least one of the size and the position of the face having the highest priority fixed in the photographic image.
  • The position recognition section 26 continuously recognizes the face information (face) in the photographic image which face information (face) is detected by the face detection section 21, by continuously updating a coordinate (position of the face) of an image indicating face coordinate information. The face coordinate information is stored in a face position storage area (memory) in the tracking device (not illustrated). The face coordinate information is information indicating a position of the face information (face) thus detected by the face detection section 21. When (i) the face detection section 21 detects same face information, in a photographic image which is subsequently obtained, as the feature information which has been detected by the face detection section 21, and (ii) the position recognition section 26 recognizes the position of the face information in the photographic image, the face matching section 22 does not access the face information DB 41. That is to say, positions of face information other than the current tracking object which has the face information having the highest priority is also recognized (traced). This allows reduction of a number of times of which matching is carried out by the face matching section 22. As a result, it is possible to reduce burden on the tracking device 1.
  • The photographing section 3 includes the photographing device 33 which has a plurality of cameras, a camera driving device 31, and a camera switching device 32. The photographing section 3 photographs an image of the tracking object based on the control by the photographing control section 25. The camera driving device 31 drives the cameras in the photographing device 33. The camera switching device 32 switches the camera which is to be driven. Each of the cameras in the photographing device 33 can carry out panning/tilting, zooming, and changing of exposure.
  • The present embodiment is illustrated having a plurality of cameras, however the number of cameras may be one. In such case, the camera switching device 32 is not necessarily provided. The tracking device 1 may be arranged so that there are a plurality of tracking devices 1 which control one or more cameras. In this case, the tracking devices may track the face having the highest priority by use of a communications section provided to each of the tracking devices.
  • The storage section 4 is made of a non-volatile storage device such as a hard disk. The storage section 4 stores the face information DB 41. In the face information DB 41, face information of a plurality of persons is registered corresponding to a priority indicating a tracking order of the face information.
  • For example, with reference to Fig. 2, face information of a person A and face information of a person B that have a priority of 1 and 2, respectively, is registered in the face information DB 41. A smaller numeral expression of the priority indicates a higher priority. Fig. 2 has just two persons registered as tracking objects, however more face information may be registered. Moreover, not just registering one face information for a tracking object but registration of a plurality of face information (for example, the face information which faces front, right, left, up, and down) for a tracking object allows accurate matching by the face matching section 22. Information related to the tracing object such as a name and feature points of the tracking object (for example, in the case where the tracking object is a baseball player, a fielding position or the like) may be registered in the face information DB 41. Having such information registered in the face information DB 41 allows users to refer to the information when setting and changing the priority. Therefore, this information becomes help for the user.
  • Of course, if there is just one face information that is prioritized in the face information DB 41, the face information thus prioritized is continuously tracked. The priority is also acquired in the case where there is just one face information registered in the face information DB 41. In such case, the one face information is considered to have the highest priority, even if the face information is not prioritized. If just a part of a plurality of the face information is prioritized whereas the rest are not prioritized, the priority may be acquired on the basis that the rest of the face information is considered to equally have the lowest priority.
  • The storage section 4 may be stored with a control program which controls operations of various arrangements in the tracking device 1, an OS (operating system) program, or other various programs.
  • The input section 5 is an interface of the tracking device 1 for accepting operations from a user. The input section 5 is provided so as to enable the user to operate the tracking device 1. For example, the input section 5 is provided so that the user can input settings and changes to the priority in the face information DB 41. Therefore, the user can set and change the priority of the face information registered in the face information DB 41 in advance, via the input section 5. If the priority is not set in advance, the tracking device 1 may be arranged such that a priority is set in the case the tracking device 1 detects face information that is not prioritized. If the tracking device detects face information that is not registered in the tracking object information, the face information may be registered in the tracking object information together with a priority. For example, in sports relay broadcasting, it is possible to track a player in an order by popularity by prioritizing the players for example. This priority can be changed as necessary. In a case where a reserve player joins the game or the like, a new priority may be registered in order to carry out tracking. These are simply exemplifications. In the case where the tracking device 1 detects the face information not registered in the tracking object information, the face information is to be registered in the tracking object so as to allow the user to set the priority.
  • The input section 5 is provided so that, for example, operation keys/buttons and the like are used in order for the user to operate the input section 5. The user gives, via the input section 5, various instructions to the tracking device 1, for example start and stop of the tracking operation, rewriting of the face information DB 41, and setting and changing the priority in the face information DB 41. The input section 5 may be combined with the display screen of the display section 6. That is, for example, the input section 5 and the display screen of the display section 6 may be provided as an integrated touch panel. In such a case, items of which the user can input are displayed on the display screen so that the user can give instructions to the tracking device 1 by touching the display area of the desired item. Operation means of a stick-shape may be provided as the input section 5, in order to touch the display area.
  • The display section 6 includes a display screen (photographic screen), and outputs (displays) a photographic image photographed by the photographing device. A plurality of display sections 6 may be provided. For example, a same number of display sections 6 as the number of cameras may be provided. The display screen of the display section 6 may be partitioned so that images from a plurality of cameras are displayed on the display section 6.
  • Moreover, for example, an audio output section (not illustrated) may be provided, which audio output section outputs audio accompanying the photographic image photographed by the imaging device 33.
  • The face matching section 22 and the face information DB 41 may be provided as an external apparatus. In such a case, matching process is carried out by carrying out communications between the tracking device 1 and the external apparatus.
  • The tracking device of the present invention does not necessarily include the photographing section 3 or the face information DB 41; the control section 2 may be considered as the tracking device according to the present invention.
  • The following description explains a state where tracking is carried out by use of the tracking device 1 that has the above arrangement, with reference to Figs. 3(a) through 3(c). Assume that, as illustrated in Fig. 2, face information of a person A and a person B are registered in the face information DB 41 prioritized as 1 and 2, respectively. A smaller numerical priority indicates a higher priority.
  • First, as illustrated in Fig. 3(a), assume that (a face of) a human figure A is being tracked in a photographic image 50. Here, a human figure C is not registered in the face information DB 41. Therefore, the human figure C is not a tracking object. Accordingly, a face position of the human figure C is recognized by the position recognition section 26 as a tracing object, however tracking is not carried out.
  • Next, as illustrated in Fig. 3(b), assume that the human figure B comes into the photographic image 50 (frame in, black arrow in figure), which human figure B has a higher priority in tracking order than the human figure A. In this case, the tracking object is switched from the face of the human figure A to the face of the human figure B (white arrow in figure). Meanwhile, the face of the human figure C is not the tracking object, so just the face position is recognized. The human figure B becomes the tracking object, and while the face of the human figure B stays in the photographic image 50, just the position of the face of the human figure A is recognized as a tracking object that has a lower priority (tracing object).
  • Thereafter, as illustrated in Fig. 3(c), the face of the human figure B, as the tracking object, is made to stay in a certain range 51 of the photographic screen which displays the photographic image 50. Moreover, the position of the face of the human figure A is continuously checked (recognized) while the face of the human figure A is included in the screen, as the face that has the lower priority (tracing object). In such a condition, if the face of the human figure B disappears from the photographic image 50 due to some kind of cause, the face of the human figure A again becomes the tracking object. Meanwhile, the face of the human figure C has moved outside of the image (frame out). This ends the recognition (trace) of the face position of the human figure C.
  • The above explanation of the tracking state is made by use of one camera that carries out the tracking. However, the number of cameras may be one camera or several cameras.
  • The following description explains switching of cameras in the photographing section 3, with reference to Figs. 4(a) through 4(c). Here, assume that switching is carried out from a camera C1 to a camera C2. Each of Figs. 4(a) through 4(c) illustrate, from left to right, an image I2 photographed by the camera C2, an image I1 photographed by the camera C1, and a positional relationship between imaging ranges of the cameras C1 and C2 and a subject. In addition, the dotted areas in the images I1 and I2 illustrated in Figs. 4(a) through 4(c) are areas where the angle of view of cameras C1 and C2 overlap.
  • As illustrated in Fig. 4(a), when a face of a human figure P, which human figure P is a tracking object, is in an angle of view of the camera C1, the camera C1 tracks the face of the human figure P.
  • In Fig. 4(b), it is assumed that the human figure P, which is the tracking object, is to come off from the angle of view of the camera C1, based on a calculation by the movement calculation section 24. When the human figure P, which is the tracking object, comes into the angle of view of the camera C2, a screen changes its displayed image from the image I1 which is an image photographed by the camera C1 to the image I2 which is an image photographed by the camera C2. The photographing control section 25 controls the camera switching device 32, so as to control the switching of the cameras.
  • Subsequently, as illustrated in Fig. 4(c), when the human figure P, which is the tracking object, is in the angle of view of the camera C2, the camera C2 tracks the face of the human figure P.
  • As the above, in the tracking device 1, the photographing section 3 is controlled so that the face having the highest priority is continuously included in the photographic image, even if there is a plurality of tracking objects and a tracking order is given to the tracking objects. Therefore, the tracking device 1 can track a face having the highest priority, without requiring an operation from the user. Therefore, it is unnecessary for the user to specify the tracking object. As a result, the operations taken by the user are reduced. As such, use of the tracking device 1 allows autonomous tracking of the tracking object, not having to trouble the user, even if a human figure that is prioritized in tracking order is included in the image.
  • [Tracking process by the tracking device]
  • The following description explains a flow of tracking processes in the tracking device 1, with reference to the flow chart shown in Fig. 5. Assume that, prior to the processes explained below, prioritized face information is registered in the face information DB 41.
  • First, the face detection section 21 detects face information (hereinafter abbreviated to as "face") in a photographic image so as to determine whether or not a face is in the photographic image (step 1, hereinafter abbreviated to as like S1). Detection of the face may wait until a subject comes into the photographic image. The camera may rotate so as to include the subject in the photographic image otherwise, and subsequently start the detection when the subject comes into the photographic image.
  • When a face is detected (YES in S1), the face detection section determines whether the face thus detected is one which its position is recognized in the image (whether the face is in a previous frame) (S2).
  • An explanation dealing with the case where the face thus detected in S1 is a face in which its position is not recognized in the image (NO in S2) is explained first, and the case where the face thus detected in S1 is a face in which its position is recognized (YES in S2) is explained thereafter.
  • If the face thus detected in S1 is a face of which its position is not recognized in the image (NO in S2), the face matching section 22 checks the face information DB 41 so as to determine whether or not the face detected in S1 is registered in the face information DB 41 (S3). The determination is carried out as follows. A similarity between the face thus determined and the face information included in the face information DB 41 is found. If the similarity thus found is greater than a predetermined authentication threshold value, it is determined that the face thus determined is information of a tracking object. Furthermore, authentication reliability is found from how greater the similarity is than the authentication threshold value.
  • When the face thus detected in S1 is registered (YES in S3), the priority acquisition section 23 acquires a priority of the face thus detected in S1 (S4). Once the priority is acquired, it is determined whether or not the face currently tracked (face having the highest priority) is included in the image (S5). If the face currently tracked is included in the image (YES in S5), the priority of the face thus detected in S1 and the face currently tracked are compared (S6). In the case where the priority of the face thus detected in S1 and the priority of the face currently tracked are the same, it is determined whether or not the authentication reliability thus found at a time when determination is made in S3 is higher than the authentication reliability of the face currently tracked (S7). The authentication reliability of the face currently tracked is stored in a memory or the like not illustrated.
  • As a result of comparing the authentication reliabilities of the two faces, in the case where the authentication reliability of the face thus detected in S1 is high (YES in S7), the face that is the information of the current tracking object is taken off the tracking object. The face that is the information of the current tracking object is then registered as information of a tracking object having a low priority in the memory or the like not illustrated (S8). Also in the case where, in S6, the face thus detected in S1 has a higher priority than the face that is currently tracked, the process proceeds to S8, where the face that is the information of the current tracking object is taken off the tracking object, and is registered as the information of the tracking object having the low priority in the memory or the like not illustrated.
  • In S6, if the face thus detected in S1 is lower in priority than the face that is currently tracked, the face thus detected in S1 is registered as the information of the tracking object having the low priority in the memory or the like not illustrated (S10). Subsequently, the process proceeds to S18. In S7 also, if the face thus detected in S1 has the lower authentication reliability than the face that is currently tracked (NO in S7), the face thus detected in S1 is registered as the information of the tracking object having the low priority in the memory or the like not illustrated (S10). The process thus proceeds to S18.
  • If the face thus detected in S3 is not registered (NO in S3), the face thus detected in S1 is registered as information of a tracing object in the memory or the like not illustrated (S9). The process thus proceeds to S18.
  • If the face currently tracked is not included in the image (NO in S5), a face objected for process (here, the face thus detected in S1) is registered as the information of the tracking object in the memory or the like not illustrated (S11). The process thus proceeds to S18. Similarly, after the face which is the information of the current tracking object is registered as the information of the tracking object having the low priority in the memory or the like not illustrated (S8), the process proceeds to S11, in order to register the face objected for process (here, the face thus detected in S1) as the information of the tracking object in the memory or the like not illustrated, and proceeds to S18.
  • If the face thus detected in S1 is a face of which its position is recognized in the image (YES in S2), it is determined whether or not the face is the face that is currently tracked (S 12). If the face is the face that is currently tracked (YES in S12), it is determined whether or not the face thus detected (the face that is currently tracked) is positioned around a center of the image (S13). If the face thus detected is not positioned around the center of the image (NO in S13), the photographing control section 25 controls the photographing section 3 so that the face thus detected comes around the center of the image (S14). If the face thus detected is positioned around the center of the image (YES in S13), or, after the imaging control section 25 controls the photographing section 3 so that the face thus detected comes around the center of the image (S14), the process proceeds to S18.
  • In S12, if the face thus detected in S1 is not the face that is currently tracked (NO in S12), it is determined whether or not the face thus detected in S1 is a face having the low priority (information of a tracking object not having a high priority) (S15). If the face is of the low priority (YES in S 15), it is determined whether or not the face that is currently tracked is in the image (S16). If (i) the face is not a face having the low priority (NO in S 15) or (ii) the face currently tracked is in the image (YES in S16), the process proceeds to S18. Note that, if the face is one which is not of the low priority in S15, the face is a tracing object.
  • If the face that is currently tracked is not in the image in S16 (NO in S16), the face thus detected in S1 is registered as the information of the tracking object in the memory or the like not illustrated (S11). The process thus proceeds to S18.
  • Finally, face position information is updated by rewriting a coordinate (position of face) in an image which indicates face coordinate information that is stored in a face position storage area (memory) of the tracking device, to a detected coordinate (position where the face information was detected by the face detection section 21 in S1) (S18). Note that if a face is not detected in an image in S1 (NO in S1), information of the tracking object and the tracing object, each of which is registered in the memory or the like, is deleted from the tracking device (S17), and the process ends. Tracking process in the tracking device 1 is carried out by repetitively carrying out the above processes, as illustrated in Fig. 5.
  • The following description explains a camera switching process in a case where the photographing section 3 has a plurality of cameras, with reference to a flow chart illustrated in Fig. 6. The following description is made assuming that a camera C1 is switched to a camera C2.
  • As illustrated in Fig. 6, it is determined whether or not a face having a highest priority is included in a switching area (dotted area in Figs. 4(a) through 4(c)) of the camera C1 (S51). If there are no faces in the area (NO in S51), the process ends. If there is a face in the area (YES in S51), it is determined whether or not a position of the face having the highest priority is outside of the switching area in a previous frame of the camera C1 (S52). If the position is not outside the switching area (NO in S52), the process ends. If the position is outside the switching area (YES in S52), it is determined whether or not a position of the face having the highest priority is outside the switching area of the camera C2 (S53). If there is no such face positioned outside the area (NO in S53), the process ends. If there is the face positioned outside the area (YES in S53), it is determined whether or not the position of the face having the highest priority in the previous frame of the camera C2 is in the switching area or not (S54). If the position of the face is not in the switching area (NO in S54), the process ends. If the position of the face is in the switching area (YES in S54), cameras are switched from the camera C1 to the camera C2 (S55).
  • The switching of the camera is carried out as above. This switching of cameras between two or more cameras is carried out as similar to this. The switching is controlled so that when a camera which tracks a face that has a highest priority no longer can track the face that has the highest priority, another camera which is capable of tracking the face having the highest priority tracks the face.
  • [Example 1]
  • The following description explains the tracking device 1 of the present embodiment, assuming that the tracking device 1 is used in relay broadcasting of horse racing for example, with reference to Figs. 7(a) and 7(b). In the following description, the photographing device 33 has one camera.
  • Assume that faces of the horses are registered in the face information DB 41 as face information in advance, and a horse H2 is prioritized as 1, and a horse H5 is prioritized as 2, in an order of popularity. A camera C1 is placed on a rail 1 as illustrated in Fig. 7(a), so as to move in a progressing direction of the horses. In this case, the camera C1 is first controlled so that the horse H5 at the front comes in a center of a photographic image I1. The camera C1 continues to track the horse H5, as illustrated in Fig. 7(b). However, when the horse H2 comes into the photographic image I1, the camera C1 is controlled so that the horse H2 comes in the center of the screen in a subsequent frame.
  • As such, by setting the priority of tracking in the order of popularity, it is possible to arrange the most popular horse that is recognizable in the photographic image to be in the center of the screen.
  • Instead of the faces of the horses, faces of jockeys that ride the horses may be used. In such a case, the faces of the jockeys are to be registered in the face information DB 41. The priority may be set in the order of popularity of the jockeys, or the order of popularity of the horses on which the jockeys ride.
  • [Example 2]
  • The following description explains the tracking device 1 of the present embodiment, assuming that the tracking device 1 is used in relay broadcasting of a baseball game as illustrated in Fig. 8(a) for example, with reference to Figs. 8(a) through 8(d). In the following description, the photographing device 33 has three cameras.
  • Assume that faces of players are registered in the face information DB 41 as face information, and a respective priority is set as runner > batter > fielder.
  • As illustrated in Figs. 8(b) through 8(d), the camera C1 tracks a runner A, the camera C2 tracks a fielder B, and the camera C3 tracks a fielder C. The cameras C1 through C3 are controlled so that the respective players are positioned in the center of the screens I1 through I3. The runner A, the fielder B, and the fielder C are tracked by the camera C1, the camera C2, and the camera C3, respectively. When the runner A comes into the angle of view of the camera C2 in base running, the camera C2 brings the runner A to the center of the screen I2. That is to say, the camera C2 switches the tracking object to the runner A. When the runner A runs out from the angle of view, then the tracking object returns to the fielder B. When the runner A comes into the angle of view of the camera C3 in base running, the camera C3 also switches the tracking object to the runner A so that the runner A comes in the center of the screen I3. Once the runner A is out of the screen, the tracking object of the camera C3 returns its tracking object to the fielder C.
  • As such, it is possible to carry out tracking based on priorities.
  • Other than the above, in the tracking device 1 of the present embodiment, the priority may be set as: forward > mid fielder > defender > goal keeper, or home players > away players, for example, in the case where the device is used in relay broadcasting of a soccer game. In a case for example where a suspicious person is tracked for security, tracking may be carried out by setting a priority in an order of: a suspect on a wanted list > a person that has a criminal history > other persons. Moreover, for bicycle racing, speedboat racing, car racing, motorbike racing, a marathon, a long-distance relay road race, athletics and the like, tracking may be carried out by setting a priority as: most popular > secondly popular > ···. If it is sports relay broadcasting of an amateur baseball, the tracking may be carried out by prioritizing the players in an order of drafting. Note that these are simply exemplifications. How the tracking orders are set is not limited, and the tracking objects are also not limited.
  • The invention being thus described, it will be obvious that the same way may be varied in many ways. Such variations are not to be regarded as a departure from the spirit and scope of the invention, and all such modifications as would be obvious to one skilled in the art are intended to be included within the scope of the following claims.
  • Finally, the blocks of the tracking device 1, particularly the control section 2, may be realized by way of hardware or software as executed by a CPU as follows:
  • The tracking device 1 includes a CPU (central processing unit) and memory devices (memory media). The CPU (central processing unit) executes instructions in control programs realizing the functions. The memory devices include a ROM (read only memory) which contains programs, a RAM (random access memory) to which the programs are loaded, and a memory containing the programs and various data. The object of the present invention can also be achieved by mounting to the tracking device 1 a computer-readable recording medium containing control program code (executable program, intermediate code program, or source program) for the tracking device 1, which is software realizing the aforementioned functions, in order for the computer (or CPU, MPU) to retrieve and execute the program code contained in the recording medium.
  • The recording medium may be, for example, a tape, such as a magnetic tape or a cassette tape; a magnetic disk, such as a floppy (Registered Trademark) disk or a hard disk, or an optical disk, such as CD-ROM/MO/MD/DVD/CD-R; a card, such as an IC card (memory card) or an optical card; or a semiconductor memory, such as a mask ROM/EPROM/EEPROM/flash ROM.
  • The tracking device 1 may be arranged to be connectable to a communications network so that the program code may be delivered over the communications network. The communications network is not limited in any particular manner, and may be, for example, the Internet, an intranet, extranet, LAN, ISDN, VAN, CATV communications network, virtual dedicated network (virtual private network), telephone line network, mobile communications network, or satellite communications network. The transfer medium which makes up the communications network is not limited in any particular manner, and may be, for example, wired line, such as IEEE 1394, USB, electric power line, cable TV line, telephone line, or ADSL line; or wireless, such as infrared radiation (IrDA, remote control), Bluetooth, 802.11 wireless, HDR, mobile telephone network, satellite line, or terrestrial digital network. The present invention encompasses a carrier wave or data signal transmission in which the program code is embodied electronically.
  • INDUSTRIAL APPLICABILITY
  • The invention is applicable to systems which track a plurality of subjects, and is applicable for various purposes, for example, a security system, or for example, teleconferences, televised lectures, and sports broadcasting.

Claims (5)

  1. A tracking device (1) which tracks a tracking object by controlling a photographing section (3) so as to continuously include the tracking object in a photographic image taken by the photographing section, the tracking device (1) comprising:
    feature information detection means (21) for detecting feature information from the photographic image wherein the feature information is face information which specifies features of the face of a person ;
    tracking object matching means (22) for (i) comparing the feature information thus detected with a tracking object information database, (41) in which feature information of a plurality of persons is registered so that the feature information corresponds to a priority indicating tracking order of the feature information, and (ii) determining whether or not the feature information thus detected is information of a tracking object;
    priority acquisition means (23) for acquiring the priority of the feature information thus detected from the tracking object information, in the case where it is determined that the feature information thus detected is the information of a tracking object; and
    control means (25) for controlling the photographing section (3), based on the priority thus acquired, so as to continuously include, in the photographic image from which the feature information is detected, feature information that has a highest priority in the photographic image;
    wherein the tracking object matching means (22) (i) finds a similarity between the feature information detected by the feature information detection means (21) and the feature information included in the tracking object information database, (ii) determines, in a case where the similarity thus found is greater than a predetermined authentication threshold value, that the feature information thus detected is information of a tracking object and (iii) finds authentication reliability indicative of how large the similarity thus found is as compared with the predetermined authentication threshold value; and
    in a case where the photographic image includes a plurality of feature information having an identical priority, the control means (25) controls the photographing section (3) so that one of the plurality of feature information, which has a greater authentication reliability, is continuously included in the photographic image.
  2. The tracking device (1) as set forth in claim 1, wherein:
    the control means (25) controls the photographing section (3) so that at least one of a size and a position of the feature information that has the highest priority is always fixed in the photographic image.
  3. A tracking method for tracking a tracking object by controlling a photographing section (3) so as to continuously include the tracking object in a photographic image taken by the photographing section, the tracking method comprising the steps of:
    detecting feature information from the photographic image wherein the feature information is face information which specifies features of the face of a person ;
    comparing the feature information thus detected with a tracking object information database (41) in which feature information of a plurality of persons is registered so that the feature information corresponds to a priority indicating tracking order of the feature information;
    determining whether or not the feature information thus detected is information of a tracking object;
    acquiring the priority of the feature information thus detected from the tracking object information, in the case where it is determined that the feature information thus detected is the information of a tracking object; and
    controlling the photographing section (3), based on the priority thus acquired, so as to continuously include, in the photographic image from which the feature information is detected, feature information that has a highest priority in the photographic image;
    wherein the method further comprises:
    finding a similarity between the detected feature information and the feature information included in the tracking object information database;
    determining, in a case where the similarity thus found is greater than a predetermined authentication threshold value, that the feature information thus detected is information of a tracking object; and
    finding authentication reliability indicative of how large the similarity thus found is as compared with the predetermined authentication threshold value;
    wherein, in the step of controlling the photographic section (3), in a case where the photographic image includes a plurality of feature information having an identical priority, the photographing section (3) is controlled so that one of the plurality of feature information, which has a greater authentication reliability, is continuously included in the photographic image.
  4. A tracking device control program product for operating a tracking device (1) as set forth in claim 1 or claim 2, the control program causing a computer to function as each of the means in the tracking device (1).
  5. A computer-readable recording medium wherein a tracking device control program product as set forth in claim 4 is recorded.
EP07738537.5A 2006-03-15 2007-03-14 Tracking device, tracking method, tracking device control program, and computer-readable recording medium Active EP1998567B1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
JP2006071861 2006-03-15
PCT/JP2007/055066 WO2007119355A1 (en) 2006-03-15 2007-03-14 Tracking device, tracking method, tracking device control program, and computer-readable recording medium

Publications (3)

Publication Number Publication Date
EP1998567A1 EP1998567A1 (en) 2008-12-03
EP1998567A4 EP1998567A4 (en) 2013-05-15
EP1998567B1 true EP1998567B1 (en) 2016-04-27

Family

ID=38609152

Family Applications (1)

Application Number Title Priority Date Filing Date
EP07738537.5A Active EP1998567B1 (en) 2006-03-15 2007-03-14 Tracking device, tracking method, tracking device control program, and computer-readable recording medium

Country Status (6)

Country Link
US (1) US8599267B2 (en)
EP (1) EP1998567B1 (en)
JP (1) JP4957721B2 (en)
KR (1) KR101001060B1 (en)
CN (1) CN101401426B (en)
WO (1) WO2007119355A1 (en)

Families Citing this family (52)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101298646B1 (en) * 2007-04-30 2013-08-21 삼성전자주식회사 Method of controlling digital image processing apparatus wherein face area is efficiently searched
US8432449B2 (en) * 2007-08-13 2013-04-30 Fuji Xerox Co., Ltd. Hidden markov model for camera handoff
JP2009193424A (en) * 2008-02-15 2009-08-27 Nec Corp Traffic line capturing device and traffic line capturing method and program
JP5084550B2 (en) * 2008-02-25 2012-11-28 キヤノン株式会社 Entry monitoring system, unlocking instruction apparatus and a control method thereof, and program
JP5039607B2 (en) * 2008-02-27 2012-10-03 キヤノン株式会社 Imaging apparatus and a control method thereof
EP2104339A3 (en) * 2008-03-19 2011-05-11 Fujinon Corporation Autofocus system
US20110102570A1 (en) * 2008-04-14 2011-05-05 Saar Wilf Vision based pointing device emulation
ITMI20081628A1 (en) * 2008-09-12 2010-03-13 March Networks Corp automatic tracking an object of interest by means of a video camera
JP2010117663A (en) * 2008-11-14 2010-05-27 Fujinon Corp Autofocus system
JP5441151B2 (en) * 2008-12-22 2014-03-12 九州日本電気ソフトウェア株式会社 Face image tracking device and face image tracking method and program
JP2010200084A (en) * 2009-02-26 2010-09-09 Nikon Corp Imaging apparatus
JP5325012B2 (en) * 2009-04-27 2013-10-23 株式会社第一興商 Karaoke system with the singer's image shooting function
JP5434339B2 (en) * 2009-07-29 2014-03-05 ソニー株式会社 Imaging control apparatus, an imaging system, the imaging method, the program
JP5390322B2 (en) * 2009-09-28 2014-01-15 株式会社東芝 Image processing apparatus, and image processing method
US8675066B2 (en) * 2009-10-02 2014-03-18 Alarm.Com Incorporated Image surveillance and reporting technology
US9386281B2 (en) 2009-10-02 2016-07-05 Alarm.Com Incorporated Image surveillance and reporting technology
GB2483168B (en) 2009-10-13 2013-06-12 Pointgrab Ltd Computer vision gesture based control of a device
GB2476097A (en) * 2009-12-11 2011-06-15 Tony Jones A portable communication device for evaluating an environment
JP5495855B2 (en) * 2010-03-01 2014-05-21 キヤノン株式会社 Image processing apparatus and image processing method
JP5911201B2 (en) * 2011-04-20 2016-04-27 キヤノン株式会社 Automatic tracking camera system having automatic tracking control system and its camera device
WO2012170898A2 (en) * 2011-06-09 2012-12-13 Utah State University Research Foundation Systems and methods for sensing occupancy
KR101381439B1 (en) * 2011-09-15 2014-04-04 가부시끼가이샤 도시바 Face recognition apparatus, and face recognition method
KR101288248B1 (en) * 2011-12-09 2013-07-26 정영규 Human tracking system and method for privacy masking
US8704904B2 (en) 2011-12-23 2014-04-22 H4 Engineering, Inc. Portable system for high quality video recording
US9615015B2 (en) * 2012-01-27 2017-04-04 Disney Enterprises, Inc. Systems methods for camera control using historical or predicted event data
EP2811736A1 (en) * 2012-01-30 2014-12-10 Panasonic Corporation Optimum camera setting device and optimum camera setting method
US9124800B2 (en) * 2012-02-13 2015-09-01 Htc Corporation Auto burst image capture method applied to a mobile device, method for tracking an object applied to a mobile device, and related mobile device
WO2013131036A1 (en) 2012-03-01 2013-09-06 H4 Engineering, Inc. Apparatus and method for automatic video recording
US9723192B1 (en) 2012-03-02 2017-08-01 H4 Engineering, Inc. Application dependent video recording device architecture
AU2013225635B2 (en) 2012-03-02 2017-10-26 H4 Engineering, Inc. Waterproof Electronic Device
JP2013192154A (en) * 2012-03-15 2013-09-26 Omron Corp Monitoring device, reliability calculation program and reliability calculation method
US8938124B2 (en) * 2012-05-10 2015-01-20 Pointgrab Ltd. Computer vision based tracking of a hand
US9256781B2 (en) * 2012-05-10 2016-02-09 Pointguard Ltd. System and method for computer vision based tracking of an object
JP6089549B2 (en) * 2012-10-05 2017-03-08 富士ゼロックス株式会社 The information processing apparatus, an information processing system, and program
US9381445B2 (en) * 2013-03-15 2016-07-05 Zebra Innovations, LLC Distance detector
CN103347117B (en) * 2013-05-31 2015-08-12 东莞宇龙通信科技有限公司 Photographing method and the communication terminal communication terminal
KR101292907B1 (en) 2013-06-13 2013-08-02 정영규 Human tracking system and method for privacy masking
JP6481249B2 (en) * 2013-07-29 2019-03-13 オムロン株式会社 Programmable display device, control method, and program
JP5683663B1 (en) * 2013-09-27 2015-03-11 パナソニックIpマネジメント株式会社 The residence time measuring apparatus, the residence time measuring system and residence time measuring method
CN104573617B (en) * 2013-10-28 2018-10-02 杭州慧知连科技有限公司 An imaging control method
WO2015088795A1 (en) * 2013-12-09 2015-06-18 Todd Martin System and method for event timing and photography
JP6056774B2 (en) * 2014-01-17 2017-01-11 ソニー株式会社 An imaging apparatus and an imaging method and a program.
TWI537842B (en) * 2014-09-30 2016-06-11 Quanta Comp Inc People counting system
JP6364372B2 (en) * 2015-03-24 2018-07-25 トヨタホーム株式会社 Regional monitoring system
TWI562642B (en) * 2015-06-24 2016-12-11 Vivotek Inc Image surveillance method and image surveillance device thereof
JP2017033358A (en) * 2015-08-03 2017-02-09 富士ゼロックス株式会社 Authentication device
US20170039010A1 (en) * 2015-08-03 2017-02-09 Fuji Xerox Co., Ltd. Authentication apparatus and processing apparatus
JP2017112431A (en) * 2015-12-14 2017-06-22 キヤノン株式会社 The information processing apparatus, information processing method, and computer program
CN107623836A (en) * 2016-07-15 2018-01-23 中国电信股份有限公司 Method, device and system for processing captured face
CN107666590A (en) * 2016-07-29 2018-02-06 华为终端(东莞)有限公司 Target monitoring method, camera, controller and target monitoring system
GB2556344A (en) * 2016-10-17 2018-05-30 Nctech Ltd Camera controller
US10136049B2 (en) * 2017-01-09 2018-11-20 International Business Machines Corporation System, method and computer program product for contextual focus/zoom of event celebrities

Family Cites Families (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH04126473A (en) * 1990-09-18 1992-04-27 Toshiba Corp Automatic tracking camera device
JPH06339056A (en) 1993-05-31 1994-12-06 Oki Electric Ind Co Ltd Video camera driving device
JPH09331520A (en) 1996-06-13 1997-12-22 Nippon Telegr & Teleph Corp <Ntt> Automatic tracking system
GB2343945B (en) 1998-11-18 2001-02-28 Sintec Company Ltd Method and apparatus for photographing/recognizing a face
JP3656823B2 (en) 2000-09-04 2005-06-08 三菱電機株式会社 Automatic monitoring method and automatic monitoring equipment
KR100792283B1 (en) 2001-08-07 2008-01-07 삼성전자주식회사 Device and method for auto tracking moving object
JP3999561B2 (en) * 2002-05-07 2007-10-31 松下電器産業株式会社 Monitoring system and surveillance cameras
JP2004040514A (en) 2002-07-04 2004-02-05 Nippon Hoso Kyokai <Nhk> Automatic tracking/imaging device and method
JP4036051B2 (en) * 2002-07-30 2008-01-23 オムロン株式会社 Face matching device and the face collation method
JP2004129049A (en) 2002-10-04 2004-04-22 Matsushita Electric Ind Co Ltd Camera device, camera system, and control method of the camera system
JP4195991B2 (en) 2003-06-18 2008-12-17 パナソニック株式会社 Surveillance video monitoring system, the monitoring image generation method and surveillance video monitoring server,
JP2005020205A (en) 2003-06-24 2005-01-20 Casio Comput Co Ltd Picture photographing device
US7574016B2 (en) * 2003-06-26 2009-08-11 Fotonation Vision Limited Digital image processing using face detection information
JP2005086626A (en) * 2003-09-10 2005-03-31 Matsushita Electric Ind Co Ltd Wide area monitoring device
JP4478510B2 (en) * 2004-06-03 2010-06-09 キヤノン株式会社 Camera system, the camera, and camera control method
US7760917B2 (en) * 2005-05-09 2010-07-20 Like.Com Computer-implemented method for performing similarity searches

Also Published As

Publication number Publication date
EP1998567A1 (en) 2008-12-03
US8599267B2 (en) 2013-12-03
EP1998567A4 (en) 2013-05-15
KR101001060B1 (en) 2010-12-14
CN101401426A (en) 2009-04-01
US20090096871A1 (en) 2009-04-16
JPWO2007119355A1 (en) 2009-08-27
KR20080103586A (en) 2008-11-27
WO2007119355A1 (en) 2007-10-25
CN101401426B (en) 2010-12-01
JP4957721B2 (en) 2012-06-20

Similar Documents

Publication Publication Date Title
US9524437B2 (en) Automatic extraction of secondary video streams
US7028269B1 (en) Multi-modal video target acquisition and re-direction system and method
US7889891B2 (en) Object determining device, imaging device and monitor
US20070025722A1 (en) Image capturing apparatus and image capturing method
CN101379825B (en) Automatic tracking device and automatic tracking method
EP1427212A1 (en) Video tracking system and method
JP4541316B2 (en) Video monitoring search system
US20170295318A1 (en) Automatic generation of video from spherical content using audio/visual analysis
JP4098808B2 (en) Remote video display method, the image acquisition apparatus and its method and program
US20060227997A1 (en) Methods for defining, detecting, analyzing, indexing and retrieving events using video image processing
US9336456B2 (en) Systems, methods and computer program products for identifying objects in video data
CN101686329B (en) Operation input apparatus, operation input method
US8289392B2 (en) Automatic multiscale image acquisition from a steerable camera
US7336297B2 (en) Camera-linked surveillance system
US8350931B2 (en) Arrangement and method relating to an image recording device
CN100369487C (en) Object detection device, object detection server, and object detection method
JP5001260B2 (en) Object tracking method and an object tracking device
US20080075334A1 (en) Combined face and iris recognition system
US20070291998A1 (en) Face authentication apparatus, face authentication method, and entrance and exit management apparatus
US7227569B2 (en) Surveillance system and a surveillance camera
US7557825B2 (en) Camera system, camera, and camera control method
WO2005024698A2 (en) Method and apparatus for performing iris recognition from an image
JPH09322155A (en) Method and device for monitoring video
JP5047007B2 (en) Imaging device
JP2002374519A (en) Image monitor system, image monitor method and image monitor program

Legal Events

Date Code Title Description
AK Designated contracting states

Kind code of ref document: A1

Designated state(s): DE FR GB IT

17P Request for examination filed

Effective date: 20081007

DAX Request for extension of the european patent (to any country) (deleted)
RBV Designated contracting states (corrected)

Designated state(s): DE FR GB IT

RIC1 Information provided on ipc code assigned before grant

Ipc: G06K 9/00 20060101ALN20130411BHEP

Ipc: H04N 7/18 20060101AFI20130411BHEP

Ipc: G06T 1/00 20060101ALI20130411BHEP

Ipc: G06T 7/20 20060101ALI20130411BHEP

A4 Supplementary search report drawn up and despatched

Effective date: 20130417

17Q First examination report despatched

Effective date: 20140121

INTG Intention to grant announced

Effective date: 20151016

RIN1 Information on inventor provided before grant (corrected)

Inventor name: SENGA, MASAHIRO

Inventor name: KAKIUCHI, TAKASHI

Inventor name: KUWANO, SATORU

Inventor name: INOUE, TOMOHIRO

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): DE FR GB IT

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602007046023

Country of ref document: DE

PG25 Lapsed in a contracting state [announced from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160427

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602007046023

Country of ref document: DE

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 11

26N No opposition filed

Effective date: 20170130

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 12

PGFP Annual fee paid to national office [announced from national office to epo]

Ref country code: DE

Payment date: 20190226

Year of fee payment: 13

Ref country code: GB

Payment date: 20190313

Year of fee payment: 13

PGFP Annual fee paid to national office [announced from national office to epo]

Ref country code: DE

Payment date: 20190226

Year of fee payment: 13

Ref country code: FR

Payment date: 20190213

Year of fee payment: 13