CN101939980B - Electronic camera and image processing method - Google Patents

Electronic camera and image processing method Download PDF

Info

Publication number
CN101939980B
CN101939980B CN2009801044907A CN200980104490A CN101939980B CN 101939980 B CN101939980 B CN 101939980B CN 2009801044907 A CN2009801044907 A CN 2009801044907A CN 200980104490 A CN200980104490 A CN 200980104490A CN 101939980 B CN101939980 B CN 101939980B
Authority
CN
China
Prior art keywords
mentioned
characteristic quantity
zone
renewal
image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN2009801044907A
Other languages
Chinese (zh)
Other versions
CN101939980A (en
Inventor
石井育规
物部祐亮
小仓康伸
今川和幸
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Panasonic Intellectual Property Corp of America
Original Assignee
Matsushita Electric Industrial Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Matsushita Electric Industrial Co Ltd filed Critical Matsushita Electric Industrial Co Ltd
Publication of CN101939980A publication Critical patent/CN101939980A/en
Application granted granted Critical
Publication of CN101939980B publication Critical patent/CN101939980B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B7/00—Mountings, adjusting means, or light-tight connections, for optical elements
    • G02B7/28—Systems for automatic generation of focusing signals
    • G02B7/36—Systems for automatic generation of focusing signals using image sharpness techniques, e.g. image processing techniques for generating autofocus signals
    • G02B7/365—Systems for automatic generation of focusing signals using image sharpness techniques, e.g. image processing techniques for generating autofocus signals by analysis of the spatial frequency components of the image
    • G—PHYSICS
    • G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
    • G03B13/00—Viewfinders; Focusing aids for cameras; Means for focusing for cameras; Autofocus systems for cameras
    • G03B13/32—Means for focusing
    • G03B13/34—Power focusing
    • G03B13/36—Autofocus systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00—General purpose image data processing
    • G06T1/0007—Image acquisition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G06T7/248—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments involving reference images or patches
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/61—Control of cameras or camera modules based on recognised objects
    • H04N23/611—Control of cameras or camera modules based on recognised objects where the recognised objects include parts of the human body
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person
    • G06T2207/30201—Face
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30232—Surveillance

Abstract

It is possible to provide an electronic camera and an image processing method which can perform a highly accurate tracking process even when an object is viewed in various ways. The electronic camera includes: a tracking process unit (137), an update region detection circuit (139), an update judgment circuit (140), and a tracking result plot circuit (144). The tracking process unit (137) searches inside of a frame by using a color characteristic amount of an object to be tracked, calculates a first evaluation value indicating the matching degree between an image of a search result and the object, and identifies a target region which is estimated to contain an image of the object according to the first evaluation value. The update region detection circuit (139) searches inside of the frame by using a shape characteristic amount of the object to be tracked, calculates a seconds evaluation value indicating the matching degree between the image of the search result and the object, and identifies an update region for updating the first characteristic amount according to the second evaluation value. The update judgment circuit (140) judges whether to update the first characteristic amount depending on whether the first evaluation value or the second evaluation value satisfies a predetermined condition. If No, the tracking result plot circuit (144) plots an external frame of the target region. If Yes, the tracking result plot circuit (144) plots an external frame of the update region.

Description

Electron camera and image processing method
Technical field
The present invention relates to the object in the tracking map picture, with following the trail of electron camera and the method thereof of results suggest on display.
Background technology
In recent years, as AF (focusing automatically; Auto-Focus) AE (automatic exposure; Auto Exposure) the contraposition means of backlight debugging functions are carried the measuring ability of having the face in electron camera.Through using the face measuring ability, can make focus automatically close burnt being taken the photograph on the body, therefore, auxiliary and become effective means as user's photography.
As the image processing apparatus and the method that detect face; Following method has been proposed; The pattern of study face and the part except that face (below be called non-face), use has been imported the identifier of this parameter of learning and has been discerned face and non-face (for example, with reference to patent documentation 1).
Fig. 1 is the figure that the image processing apparatus of patent documentation 1 is shown.Fig. 2 is the figure that an example that cuts out of parts of images is shown.Parts of images cuts out portion 900 and from the image 1001 of input, cuts out parts of images 1000.Window through making a plurality of sizes is a starting point with the upper left of image, Yi Bian to the right or downside move suitable pixel (for example 1 pixel) Yi Bian scan successively, till the bottom right of image, cut out parts of images 1000 (Fig. 2).Have, said " cutting out " is meant the view data that only reads counterpart again.
Characteristic quantity evaluating part 1 (901) constituting by a plurality of identifiers.Identifier is in the specified position of parameter of utilize using the boosting method to learn, according to after rectangular characteristic (below be called) the calculated characteristics amount stated in abutting connection with difference filter.Then, if the weighted linear of the output valve of above-mentioned identifier be lower than the threshold value that calculates based on study, then characteristic quantity evaluating part 1 (901) is identified as non-face, finishes the identification of above-mentioned parts of images and handles.On the other hand,, just be identified as face, handle to next characteristic quantity evaluating part 2 (902) and shift if more than threshold value.Characteristic quantity evaluating part 2 (902) is used with the different parameter of in characteristic quantity evaluating part 1 (901), using of learning parameter and is estimated.Like this, use a plurality of characteristic quantity evaluating part to calculate evaluation of estimate, distinguish face and non-face according to the value of the evaluation of estimate that calculates.
Fig. 3 (a) and (b), (c), (d) show in abutting connection with difference filter, and Fig. 3 (e) is the example when being suitable in abutting connection with difference filter (b) to image.Represent with the white rectangle and the black rectangle of adjacency in abutting connection with difference filter, export the poor of interior pixel mean value of white rectangle and the pixel mean value in the black rectangle.The characteristic of face is, according to being determined from the bigger situation of difference in abutting connection with the pixel mean value of difference filter output, this means to resemble the high characteristic quantity of output in eyes and the mouth zone that the difference of the pixel value in zone is bigger between adjacency in this wise.For example, Fig. 3 (e) is according to the summation of the pixel that is positioned at the locational black rectangle of volume and the difference calculated characteristics amount of the summation of the pixel that is positioned at the locational white rectangle of eyebrow.Because this characteristic quantity is the interregional poor of adjacency, therefore, the local feature in image (for example line composition) is gone up kickback, the value of output characteristic on faces such as eyes, eyebrow, mouth.Have again, generally be called Haar-like characteristic (Haar-like feature) in abutting connection with difference filter.
But, only in face detects, if not just can not detect in the scope of visible face (eyes, nose, mouth), and the object except that face that can not pet tracking etc.Therefore, following method is arranged, the user reproduces the information of registration object in advance, through following the trail of this object, even the object except that face, also focusing automatically.
As the object tracking method that used in the past, the face method for tracing that only has near zone to face testing result position carry out once more face to detect, use near the template matches of searching for the former frame based on related operation, with the method that turns to purpose at a high speed based on positive search, use statistical information to add the method that dynamic prediction is searched for based on particle filter (particle filter) or polycondensation (Condensation) based on probability distribution.
These all are the initial characteristics amounts (the look histogram of color or brightness or template image itself, shape profile information etc.) of using the object that the registration of some methods wants to follow the trail of in advance.Should register characteristic quantity through using, search is carried out object tracking with the similar place of registration characteristic quantity in image.In these methods, process the initial characteristics amount in advance, get this characteristic quantity and the characteristic quantity that extracts according to each position of image between coupling.
But, can see face etc. for a long time under the object few cases of utilizing general film to photograph, in most cases all be that the appearance on the image alters a great deal.Appearance changes the problem that big such object at once will lose objects in method in the past, existing for image.
In order to address this problem, for example in patent documentation 2, used one by one the more method of new template.According to this method, even under the situation that the appearance of tracing object changes, also change more new template according to this.Therefore, can follow the trail of the object that appearance changes.
Patent documentation 1: U.S. Patent application discloses communique No. 2002/0102024
Patent documentation 2: TOHKEMY 2002-157599 communique
Invent problem to be solved
But in the method for patent documentation 2, according to each frame update template, but more new template is come in the not talkative zone that only comprises target object of always can using.Exist owing to the problem of in the zone of upgrading, sneaking into the tracking failure after non-tracing object zone makes it.
Use Fig. 5 to explain particularly.For example, suppose the correct zone of Fig. 5 (a) is decided to be 200.At this moment, in Fig. 5 of next frame (b),, therefore on 201 position, mate because the border of the color of cap is similar.In this position, because 201 left side comprises background, therefore, in the later tracking of next frame, also added the characteristic of background, head is followed the trail of given bad influence.But, if can with 202 the coupling, just can all obtain characteristic according to head, can also stably follow the trail of head later at next frame.
Summary of the invention
The present invention through using the characteristics determined renewal zone of the position of detected object thing stably, eliminates the trace error in the zone that has object in view of above-mentioned existing issue.In addition, through being modified to correct position from departing under the situation that becomes big of tram following the trail of the result, realize stable tracking process.
The means that are used to deal with problems
In order to address the above problem; Electron camera of the present invention has subject area that the object of in each frame of the image of sequence photography, confirming to become tracing object mirrors and the function that shows; Have: tracking process portion, first characteristic quantity of registration in advance of the characteristic of above-mentioned object is represented in use quantitatively, searches in the predefined scope in frame; To as the image in the resulting zone of mentioned above searching results; First evaluation of estimate of the consistent degree between the image of represents and above-mentioned object according to above-mentioned first evaluation of estimate that calculates, confirms to be estimated to be the target area of the image that has above-mentioned object; Upgrade regional calculating part; Use different with above-mentioned first characteristic quantity, represent second characteristic quantity of the characteristic of above-mentioned object quantitatively; Search in the predefined scope in above-mentioned frame, to as the image in the resulting zone of mentioned above searching results, second evaluation of estimate of the consistent degree between the image of represents and above-mentioned object; According to above-mentioned second evaluation of estimate that calculates, in above-mentioned frame, confirm to be used to upgrade the renewal zone of above-mentioned first characteristic quantity; Upgrade judging part; Whether at least one in above-mentioned first evaluation of estimate that in above-mentioned tracking process portion, calculates by inquiry and above-mentioned second evaluation of estimate that in the calculating part of above-mentioned renewal zone, calculates satisfies predetermined conditions, judges whether to upgrade above-mentioned first characteristic quantity; Registration characteristic quantity update portion is being to upgrade under the situation of above-mentioned first characteristic quantity by above-mentioned renewal judgement section judges, is used in the first new characteristic quantity that extracts in the above-mentioned renewal zone and upgrades above-mentioned first characteristic quantity; And follow the trail of drawing section as a result; By above-mentioned renewal judgement section judges for not upgrade under the situation of above-mentioned first characteristic quantity; Above-mentioned subject area is confirmed as in the above-mentioned target area that will in above-mentioned tracking process portion, confirm; And describe the information of relevant above-mentioned target area; Be to upgrade under the situation of above-mentioned first characteristic quantity by above-mentioned renewal judgement section judges, above-mentioned subject area is confirmed as in the above-mentioned renewal zone that will in the calculating part of above-mentioned renewal zone, confirm, and describes the information in relevant above-mentioned renewal zone; Above-mentioned tracking process portion uses first characteristic quantity after upgrading to confirm the fresh target zone in the new frame under the situation that above-mentioned first characteristic quantity of registration in advance is updated.
Like this, in electron camera, when tracked object, the location that can utilize other characteristics or position beyond the characteristic of following the trail of object to carry out object.Then, can be to upgrade often or the judgement of upgrading under certain condition, have and reduced the effect of using the renewal in wrong place.
In addition; Also can be; Above-mentioned tracking process portion uses the colouring information of object as above-mentioned first characteristic quantity; Confirm to be estimated to be the target area of the image that has above-mentioned object, the shape information that above-mentioned renewal zone calculating part uses object is confirmed above-mentioned renewal zone as above-mentioned second characteristic quantity.
Like this, even also can carry out stable tracking under the situation that the appearance of target object changes.
In addition; Also can be; Above-mentioned second evaluation of estimate that above-mentioned renewal judging part calculates in the calculating part of above-mentioned renewal zone is during greater than above-mentioned first evaluation of estimate that in above-mentioned tracking process portion, calculates; Perhaps, above-mentioned second evaluation of estimate that in the calculating part of above-mentioned renewal zone, calculates is judged as and upgrades above-mentioned first characteristic quantity during greater than predefined first threshold.
Like this,, therefore, do not need all to upgrade at every turn, can cut down treating capacity owing to when the evaluation of estimate of upgrading regional result of detection surpasses the evaluation of estimate of following the trail of the result, perhaps, upgrade first characteristic quantity upgrading under the situation of regional result of detection greater than first threshold.
In addition; Also can be, the average chroma of each pixel of the image of above-mentioned object be high more, and above-mentioned renewal judging part is set big more value to above-mentioned first threshold; The average chroma of above-mentioned object is low more; Above-mentioned renewal judging part is set more little value to above-mentioned first threshold, and above-mentioned second evaluation of estimate that above-mentioned renewal judging part calculates in the calculating part of above-mentioned renewal zone is during greater than above-mentioned first evaluation of estimate that in above-mentioned tracking process portion, calculates, perhaps; Above-mentioned second evaluation of estimate that in the calculating part of above-mentioned renewal zone, calculates is during greater than predefined first threshold; Be judged as and upgrade above-mentioned first characteristic quantity, the colouring information that above-mentioned tracking process portion uses object confirms to be estimated to be the target area of the image that has above-mentioned object as above-mentioned first characteristic quantity; The shape information that above-mentioned renewal zone calculating part uses object is confirmed above-mentioned renewal zone as above-mentioned second characteristic quantity.
Like this, the easness of tracking changes according to the chroma value of tracing object thing.That is, the chroma height is meant color clear, and the tracking based on first characteristic quantity is more correctly carried out in expression.Therefore,, when chroma is low, set first threshold lessly, can determine whether should upgrade first characteristic quantity continually through when chroma is high, setting first threshold significantly.Thereby, can improve the tracking precision according to object setting threshold neatly.
In addition; Also can be; The above-mentioned target area that above-mentioned renewal judging part is also confirmed in above-mentioned tracking process portion and distance between the above-mentioned renewal zone definite in the calculating part of above-mentioned renewal zone become predefined second threshold value when above, are judged as and upgrade.
Like this; Can have under the situation of very big skew in the tracking process result of tracking process portion and the position of the renewal zone result of detection that upgrades regional calculating part; Upgrade first characteristic quantity, therefore, even following the trail of under the kaput situation; Also can use first characteristic quantity after the renewal to recover tracking process, tracking performance is stable.
In addition; Also can be; Above-mentioned electron camera also has the end judging part of the tracking that judges whether to continue above-mentioned object; Above-mentioned end judging part during all less than predefined the 3rd threshold value, is judged as the tracking that can not continue above-mentioned object above-mentioned first evaluation of estimate and the above-mentioned second evaluation of estimate both sides.
Like this, can become in the tracking of object under the situation that can not continue and automatically finish tracking process, therefore, can not waste user's time ground termination.
In addition, also can be, above-mentioned tracking as a result drawing section when being judged as the tracking that can not continue above-mentioned object, target end zone with upgrade describing of zone.
Like this, owing to do not show unnecessary tracking processing result, therefore, the user is not felt well.
In addition, also can be that above-mentioned tracking drawing section is as a result described the image that can not follow the trail of user prompt when being judged as the tracking that can not continue above-mentioned object.
Like this, can let the user know that tracking is through with, therefore, the user can transfer the action of whether following the tracks of once more to.
In addition, also can be that above-mentioned tracking drawing section is as a result described the user is urged the image of setting initial position once more when being judged as the tracking that can not continue above-mentioned object.
Like this, can urge the user to carry out initial setting, therefore, even unskilled user also can easily understand and set once more.
In addition, also can be that above-mentioned tracking drawing section as a result detects through new face and to carry out initial position setting when being judged as the tracking that can not continue above-mentioned object.
Like this, under with the situation of personage, can be automatically the part of face be reset initial position as object.
In addition, also can be, above-mentioned electron camera also has: the face testing circuit, detect face in the image in each frame; With face towards identification circuit; Identification by the detected face of above-mentioned face testing circuit towards; By above-mentioned face when identification circuit is identified as face as above-mentioned object towards the side; In the calculating part of above-mentioned renewal zone, calculate conduct after the renewal reference area in the zone of the face of positive side, above-mentioned renewal zone calculating part calculates the above-mentioned renewal zone in the above-mentioned renewal reference area according to the preassigned position relation of face part and hair portion.
So just can carry out the renewal in the relation of various positions in advance, improve thereby follow the trail of precision, the renewal in this various positions relation comprises: change the place that upgrade according to upgrading regional result of detection; The result's who perhaps for example detects according to face face is towards changing the place that upgrade; The size that perhaps detects according to face uses bigger size to upgrade; Perhaps utilize and comprise that the such form of clothes is upgraded.
Like this, owing to can follow the tracks of face, therefore, can carry out and the combining of face authentication etc.
In addition, like this, even, can stably follow the tracks of tracked object for example rotating in this wise under the also immovable situation of outward appearance like toroidal.
In addition; Also can be; Above-mentioned electron camera also has the video camera control part; This video camera control part is according in target area of in above-mentioned tracking process portion, confirming and the above-mentioned renewal zone in the calculating part of above-mentioned renewal zone, confirmed certain, and change is used to adjust the camera parameters of the action of above-mentioned electron camera.
Like this, owing to can control camera parameters, therefore, can carry out setting with the corresponding camera parameters of object.
Also can be that above-mentioned video camera control part is controlled at least one side's the action of housing and the The Cloud Terrace of above-mentioned electron camera according to above-mentioned camera parameters, control, and makes all or part of the above-mentioned object that is determined cooperate assigned position and size in the frame.
Like this, through the control video camera, just can be to the suitable Position Control video camera of object.
Also can be, above-mentioned electron camera also has the target area initial setting section, and this target area initial setting section perhaps uses predefined method to decide the initial position of above-mentioned target area according to the input from the user.
So just the initial position in ability target setting zone can determine the initial position of following the tracks of.
Also can be that above-mentioned target area initial setting section is initial position with a certain side's of personage or face detection position decision.
Like this, can automatically carry out the initial setting that the personage follows the tracks of with personage or face position as initial position.
Also can be that it is initial position that above-mentioned target area initial setting section will be closed burnt place decision by the automatic focus function of AF.
Follow the tracks of as initial setting with regard to closing burnt zone like this, become easy with the interlock of AF function by the AF function.
Have again; The present invention not only can be embodied as device; Can also be embodied as with the processing means that constitute this device is the method for step; Perhaps be embodied as the program that lets computer carry out these steps, perhaps be embodied as the recording mediums such as computer-readable CD-ROM that write down this program, perhaps be embodied as information, data or signal that this program is shown.Then, these programs, information, data and signal also can communicate via communication networks such as internets.
The invention effect:
According to the present invention; Even owing under the situation that the appearance of target object has changed, also can carry out object tracing, therefore, even under the such situation in back, also can follow the trail of at object; Display box be can continue, the framing function and the automatic photography of AE, AF and video camera control utilized.
Description of drawings
Fig. 1 is that existing face detects the flow chart of handling.
Fig. 2 is that existing face according to image detects the key diagram of handling.
Fig. 3 (a)～(e) is existing key diagram in abutting connection with difference filter.
Fig. 4 is the block diagram of the image processing apparatus in the execution mode 1,2 of the present invention.
Fig. 5 (a) is the key diagram of the example of the failure tracking in the existing method (b).
Fig. 6 (a) is the key diagram of look histogram search (b).
Fig. 7 (a) is the key diagram that look histogrammic similar degree calculates (b).
Fig. 8 (a) is the key diagram in the reference picture zone of side face (b).
Fig. 9 is the flow chart of execution mode 1.
Figure 10 (a)～(c) is tracking characteristics amount and the graph of a relation that upgrades regional result of detection.
Figure 11 is the flow chart of execution mode 2.
Symbol description
100 image processing apparatus (electron camera)
101 lens
102 shutters
103 imaging apparatuss
104 AD converters
105 timing generating circuits
106 DA converters
107 memorizer control circuits
108 system, control circuits
109 image processing circuits
110 image display-memories
111 memories
112 adjustment size circuit
113 photoflash lamps
114 range finding control parts
115 convergent-divergent control parts
116 barrier control parts
117 protection portions
118 memories
119 display parts
120 nonvolatile memories
121 pattern rotating disks
123 shutter releases
124 recording portion
125 power control parts
126 power supply units
127 connectors
128 power supply units
129 interfaces
130 interfaces
131 connectors
132 connectors
133 optical finders
134 Department of Communication Forces
135 antennas
136 initial characteristics amount extraction portions
137 tracking process portions
138 follow the trail of beginning frame specified circuit
139 upgrade regional detection circuit
140 upgrade decision circuitry
141 finish decision circuitry
142 position correction circuit
143 registration characteristic quantity refresh circuits
144 follow the trail of the result describes circuit
145 video camera control circuits
146 face testing circuits
147 faces are towards identification circuit
148 image displaying parts
149 exposure control parts
Tracking results in 200 frames (a)
Tracking results in 201 frames (b)
Tram in 202 frames (b)
301 regions of search
302 follow the trail of window
303 follow the trail of the look histogram in window zone
The look histogram of 401 reference pictures
The look histogram of 402 input pictures
501 face surveyed areas
502 reference zones
S601 input picture step
The S602 face detects step
The S603 face is towards identification step
S604 initial registration characteristic extraction step
S605 tracking process step
The S606 face detects step
The S607 face detects determining step
S608 initial registration characteristic step of updating
S609 follows the tracks of frame display part video camera controlled step
S610 follow-up evaluation value comparison step
S611 follows the trail of end step
S612 initial registration characteristic quantity step of updating
S801 input picture step
The S802 face detects step
S803 initial registration characteristic extraction step
S804 tracking process step
The S805 circle detects step
S806 follows the trail of and finishes determining step
The S807 characteristic quantity upgrades determining step
S808 characteristic quantity step of updating
S809 follows the tracks of frame display part video camera controlled step
S810 follows the trail of end step
900 parts of images cut out portion
901 characteristic quantity evaluating part 1
902 characteristic quantity evaluating part 2
903 characteristic quantity evaluating part 3
1000 parts of images
1001 input pictures
Embodiment
(execution mode 1)
In this execution mode 1, to through and with having utilized face to detect the tracking with colouring information, the image processing apparatus of the tracking of the rearward-facing head that can also carry out only in the face detection, can not carrying out is narrated.That is, the image processing apparatus of execution mode 1 (electron camera) uses existing face detection and face towards identification, confirms to exist the target area of the object that becomes the follow shot object.Then, when each frame being confirmed the target area, the characteristic quantity that all extracts the color of object upgrades at every turn.Under the situation that can't carry out the face detection, use the color characteristic of the object after upgrading to confirm the target area, upgrade the registration characteristic quantity of color.
Fig. 4 is the figure of structure that the electron camera of execution mode 1 of the present invention is shown.In Fig. 4, the 100th, electron camera.The 101st, lens, the 102nd, the shutter with aperture function, the 103rd, optical image is transformed into the imaging apparatus of the signal of telecommunication, the 104th, the analog signal output transform of imaging apparatus 103 is become the AD converter (Analog Digital Converter) of digital signal.
Timing generating circuit 105 is supplied with clock signal and control signal to imaging apparatus 103, AD converter 104, DA converter (Digital Analog Converter) 106.By memorizer control circuit 107 and system, control circuit 108 control timing generating circuits 105.
109 pairs of image processing circuits are handled and the colour switching processing from the data of AD converter 104 or from the picture interpolation that the data of memorizer control circuit 107 are stipulated.
In addition, in image processing circuit 109, the calculation process that the view data that use is made a video recording is stipulated, system, control circuit 108 is controlled exposure control part 149 and range finding control part 114 according to the operation result that obtains.
Memorizer control circuit 107 is controlled AD converter 104, timing generating circuit 105, image processing circuit 109, image display-memory 110, DA converter 106, memory 111 and adjustment size circuit 112.
The data of AD converter 104 via image processing circuit 109 and memorizer control circuit 107, perhaps directly are written to AD converter 104 in image display-memory 110 or the memory 111 via memorizer control circuit 107.
The 110th, the image display-memory; The 106th, the DA converter; The 148th, thin-film transistor)-LCD (Liquid Crystal Display: the image displaying part that constitutes LCD) etc. by TFT (Thin Film Transistor:; The view data that is written to the demonstration usefulness in the image display-memory 110 via DA converter 106 after, show by image displaying part 148.
Adjustment size circuit 112 generates the adjustment sized image as low-resolution image according to the two field picture of the dynamic image of photographing.Under the situation that will use number of picture elements (size) to recording medium in the recording image data different, utilize adjustment size circuit 112 with the number of picture elements of imaging apparatus 103.
In addition, image displaying part 148 is because number of picture elements is compared quite for a short time with imaging apparatus 103, and therefore, image is used in the demonstration when also being used for being created on image displaying part 148 and showing.Constitute according to purposes, from a plurality of resolution of regulation, select the resolution of adjustment sized image.Adjustment size circuit 112 is read in the image of being preserved in the memory 111 and is adjusted size and handle, and the data of handling are written in the memory 111 with being through with.
The shutter 102 that 149 controls of exposure control part have the aperture function is through also having the photoflash lamp dimming function with photoflash lamp 113 interlocks.
The focusing of range finding control part 114 control lens 101.The varifocal of convergent-divergent control part 115 control lens 101.116 controls of barrier control part are as the action of the protection portion 117 of barrier.
Photoflash lamp 113 also has the light projector function and the photoflash lamp dimming function of AF fill-in light.
System, control circuit 108 control entire image processing devices (electron camera) 100, the constant of the action usefulness of memory 118 register system control circuits 108, variable, program etc.
Display part 119 is according to the program implementation in the system, control circuit 108, uses the display part etc. of the liquid crystal indicator, loud speaker etc. of display action state such as literal, image, sound or information etc.Display part 119 is arranged on single or in the localities a plurality of near the device of the easy visuognosis the operating portion of image processing apparatus 100, for example, and light-emitting diode), the constituting of sounding component etc. by LCD or LED (Light Emitting Diode:.
Nonvolatile memory 120 is memories of the electric property of ability ground demonstrated record, for example uses EEPROM etc.
Pattern rotating disk 121 can switch each functional mode of setting automatic photography pattern, photograph mode, panorama photographic mode, undressed pattern etc.
Shutter release 123 becomes " opening " through shutter release SW1 in the operation way of not shown shutter release button, indication AF handles, AE handles, (Auto White Balance: the action of AWB) handling etc. begins AWB.
Shutter release 123 is through shutter release SW2;, the operation of not shown shutter release button becomes " opening " when being over; Indicate a series of processing action to begin; These a series of processing are, the signal that will read from imaging apparatus 103 writes the exposure-processed of view data via AD converter 104 and memorizer control circuit 107 in memory 111; The video picture of the use computing in image processing circuit 109 and memorizer control circuit 107 is handled; Reads image data from memory 111 writes record image data and handles in recording portion 124.
Power control part 125 is made up of battery detection circuit, DC-DC converter, the switching circuit etc. that switches the piece of energising; Have or not battery, the kind of battery and the detection of the residual amount of battery are installed; According to the indication control DC-DC converter of testing result and system, control circuit 108, during necessity, supply with necessary voltage to the each several part that comprises recording medium.
Power supply unit 128 is the power supplys that are made up of the secondary cell of the primary cell of connector 127, alkaline battery or lithium battery etc. or NiCd battery or NiMH battery, Li battery etc. and AC adapter etc. 126.
Interface 129 and 130 be and recording mediums such as storage card and hard disk between interface, connector 131 and 132 carries out being connected of storage card and recording mediums such as hard disk and main body.Protection portion 117 comprises that through covering the image pickup part of the lens 101 of image processing apparatus (electron camera) 100 prevents the pollution and damaged barrier (Barrier) of image pickup part.
Optical finder 133 does not use the electronic viewfinder function of image displaying part 148, and only uses optical finder just can photograph.
USB), the various communication functions of IEEE1394, modulator-demodulator, LAN and radio communication etc. Department of Communication Force 134 has RS232C or USB (Universal Serial Bus:.
Antenna 135 is to utilize Department of Communication Force 134 is connected image processing apparatus (electron camera) 100 with other machines connector or the antenna under the radio communication situation.
Extract the initial characteristics amount in the view data that initial characteristics amount extraction portion 136 preserves and be written in the memory 111 from memory 111.The coordinate that extracts the initial characteristics amount can utilize touch-screen to specify by the user, perhaps according to carrying out automatic setting through pressing AF zone that face detection position or shutter release SW1 set etc.
Tracking process portion 137 reads in the registration characteristic quantity and carries out tracking process from memory 111, in memory 111, write and follow the trail of result's (coordinate data, evaluation of estimate).Have again, evaluation of estimate be expression registration characteristic quantity with the characteristic quantity of target area between the value of similar degree, below, more accurate more greatly with evaluation of estimate, be that the high more situation of consistent degree is that example describes.
Follow the trail of beginning frame specified circuit 138 and judge whether it is beginning frame in the view data, that in tracking, carry out, in memory 111, write judged result.
Upgrade regional detection circuit 139 and upgrade area detection, in memory 111, write and upgrade regional result of detection (coordinate data, evaluation of estimate).Have again, about upgrading the zone, evaluation of estimate also be expression registration characteristic quantity with the characteristic quantity that upgrades the zone between the value of similar degree, below, more accurate more greatly with evaluation of estimate, be that the high more situation of consistent degree is that example describes.
Upgrade decision circuitry 140 and judge whether the registration characteristic quantity of being preserved in the memory 111 is upgraded, judged result is written in the memory 111.
The judged result whether end decision circuitry 141 will finish to follow the trail of is written in the memory 111.
Position correction circuit 142 is revised the tracking result who is preserved in the memory 111 according to the renewal zone result of detection and the position relation information DB that follows the trail of the result that remain in advance in the memory 111.
Registration characteristic quantity refresh circuit 143 uses the position feature that upgrades regional result of detection to come the registration characteristic quantity that is write down in the updated stored device 111 when following the trail of the result and upgrading concerning of stating after regional result of detection is in.
Follow the trail of the result and describe circuit 144 and be written to the tracking result in the memory 111 for LCD is shown, to the view data that is written in the demonstration usefulness in the image display-memory 110 implement to be used to understand the information that tracked the position of following the trail of the result, for example follow the trail of that frame, Marseille digest, the processing of the change of literal, Show Color, obfuscation etc.
Follow the trail of the result and describe circuit 144 and be written to the tracking result in the memory 111 for LCD is shown, to the view data that is written in the demonstration usefulness in the image display-memory 110 implement to be used to understand tracked the position of following the trail of the result or upgrade the information of the position of regional result of detection, for example follow the trail of that frame, Marseille digest, the processing of the change of literal, Show Color, obfuscation etc.
Video camera control circuit 145 is controlled video camera according to the position and the size that are written to the tracking result in the memory 111, makes target object become the certain position size of image (for example, make face become central authorities and perhaps carry out convergent-divergent to show whole body etc.).
Face testing circuit 146 carries out face from image and detects, and in memory 111, writes face testing result (position, size, evaluation of estimate).
Face is which direction towards up and down towards identification circuit 147 identifications by face testing circuit 146 detected faces, and recognition result is written in the memory 111.
Under the situation that does not have these certain circuit of 136,137,138,139,140,141,142,143,144, in system, control circuit 108, also can utilize software processes, the flow process of stating after the use is followed the trail of and update processing.
Below, use Fig. 6 and Fig. 7 that the action of image processing apparatus 100 is described.
System, control circuit 108 is carried out by exposure-processed and video picture and is handled the photograph processing that constitutes; Said exposure-processed is via imaging apparatus 103, AD converter 104, image processing circuit 109, memorizer control circuit 107; In memory 111, write the view data of photographing; Said video picture is handled and is to use memorizer control circuit 107 and uses image processing circuit 109 as required, carries out various processing after reading the view data that is written in the memory 111.
If be through with photograph processing, then 108 pairs of view data that have been written in the memory 111 of system, control circuit are used adjustment size circuit 112 to generate and are used for using image in the demonstration of image displaying part 148 demonstration photographss.System, control circuit 108 is likewise adjusted size with adjustment size circuit 112 with image and is become be input to the picture size (QVGA (Quarter Video Graphics Array) etc.) in the face testing circuit, face is detected be kept in the memory 111 with image.
System, control circuit 108 detects for the face of being preserved in the memory 111 and uses view data, utilizes face testing circuit 146, uses prior art to carry out face and detects processing, and result is kept in the memory 111.
In addition, system, control circuit 108, utilizes prior art to carry out face and handles towards identification, and result is kept in the memory 111 towards identification circuit 147 through face.
Detected by face testing circuit 146 under the situation of face, initial characteristics amount extraction portion 136 is detected with reference to the face of being preserved in the memory 111 and face is registered features extraction towards the identification result, in memory 111, writes the registration characteristic quantity.For example, initial characteristics amount extraction portion 136 extracts the characteristic quantity of the look histogram of face as color, and is registered in the memory 111 according to face testing result of being preserved in the memory 111 and face orientation information.Then, tracking process portion 137 uses the characteristic quantity that is registered in the memory 111 by initial characteristics amount extraction portion 136 to carry out tracking process.
At this, use Fig. 6 to describe to using look histogrammic registration features extraction and tracking process.
Shown in Fig. 6 (a), suppose from the image detection to the frontal faces.At this moment process the look histogram in this zone (reference zone).Specifically, on histogrammic transverse axis, (Hue Saturation Value: color saturation value) value of the H of the color space (maximum 360) is dispensed on the zone that is divided into 20 HSV of each pixel that obtains from reference zone.Then, the number that is distributed in each subregion becomes the number of degrees.Set the Hi of each pixel, can calculate by Hi*18/360 and enter into which zone.Then, through the number of degrees of each subregion are equaled 1 divided by the summation that the number of picture elements of reference zone is normalized to the number of degrees.Then, shown in Fig. 6 (b), begin to become the region of search, search for while change the size positions of following the trail of window 302 with the zone 301 after the certain limit expansion from surveyed area.The look histogram 303 of following the trail of the window zone is processed while searching for by tracking process portion 137.
Then, according to the look histogram of the reference zone look histogram calculation similar value regional with following the trail of window.Use Fig. 7 that the computational methods of similar degree are described.Fig. 7 (a) is a reference picture, and the look histogram that from then on calculates is 401.In addition, Fig. 7 (b) is an input picture, the look histogram of search window shown in 402.Like this, utilizing each look histogrammic overlapping degree to calculate similar degree is evaluation of estimate.Utilizing (mathematical expression 1) to carry out similar degree calculates.
[mathematical expression 1]
S RI = Σ i = 1 dim Min ( R i , I i )
At this, Ri is the number of degrees of i subregion in the look histogram of reference zone, and at this, Ii is the number of degrees of following the trail of i subregion in the look histogram in window zone.
At this, i gets 0 to 20 value.In this example, use 20 these values to describe, but so long as the value more than 1 can be an arbitrary value.Have, this value changes according to the complexity of object again.Comprising under the situation of a plurality of colors, should be worth, seeing difference, can improve precision with tiny subregion through increasing.On the other hand,,, see difference, can use memory seldom to follow the trail of with bigger subregion through reducing this value for the few object of number of colors.
Have, the designation method in zone is made as the face detection position as a reference again, but this also can be the user use touch-screen or indicating device etc. to specify, also can use acoustic information etc. to specify.In addition, about being illustrated, but also can ask for the difference of the template between the image that cuts out according to look histogrammic coupling.In addition, through these templates being carried out the look histogram modification, can be difficult to receive the influence of brightness as preliminary treatment.As the method for asking for difference, also can be to make minimum squared distance become minimum method perhaps to utilize the relevant method of normalization etc., so long as the general method of asking distance can adopt any means.In addition, as searching method, the method for scanning area is simply narrated, but also can be the searching method based on probability distribution such as particle filter.
Below, to describing towards corresponding registration features extraction method with face.Under the situation of frontal faces, whole through the registration face, can carry out stable tracking.But, at face from the front towards under the situation of side, rearward-facing possibility is arranged in the frame afterwards, therefore, the registration location of characteristic quantity is studied.Shown in Fig. 8 (a), under the situation of side face, in face surveyed area 501, comprise colour of skin information and color development both information.At this; Shown in Fig. 8 (b),, obtain color development information if detect processing by face through the zone of the occiput in the face surveyed area about 20% is made as reference zone 502; Then object towards the back situation under, also can use the tracking of color according to color development information.Have again, stablize through waiting this ratio that changes according to easy situation and easy rearward-facing situation, just can make to follow the trail of towards the front.
Have, in this example, be illustrated with hair, but have under the situation of the few clothing of color changes such as cap and clothes at wearing, this method also works effectively.
In addition, detecting face and having under the situation of certain distance, upgrading regional detection circuit 139 settings and upgrade the sign of judgement, and be written in the memory 111 with the tracking result.Then, utilize position correction circuit 142 to revise the face detection position.
Extract the registration characteristic quantity according to this face detection position, utilization registration characteristic quantity refresh circuit 143 upgrades the registration characteristic quantities and is written in the memory 111.Registration features extraction and said method likewise carry out.
Through in this wise according to towards coming change of registration features extraction zone; Even from the side face towards the back situation under; Also can through use hair or cap come according to towards the suitable color characteristic of variation registration; Therefore, though object towards the back situation under, also can carry out stable tracking.In addition, even under the kaput situation of tracking process, through in position correction circuit 142, coming correction position according to the face testing result, and the tracking result of updated stored device 111, also can carry out the tracking of stable face.
In addition, face detects to exist and only detects the inner situation of face.So, become constant times to comprise head through making the testing result coordinate, just can carry out stable tracking.
In addition, maintaining towards the information of back, and when returning to forward backwards,, just can carry out more stable tracking through change features extraction zone.Below explanation particularly.Towards when back and, higher towards the possibility in front towards under the situation of side.So, through not being with reference zone 502, but with the face regions of face surveyed area 501 as the Characteristic Extraction zone, even also can be corresponding when then face having occurred.Thereby, can be from changing the side backwards into, utilizing the colour of skin information when positive, therefore, stable to the recovery in front from behind.
Since towards the back situation under also can follow the trail of; Therefore; Even under the situation of state backwards; Also certainly can through follow the trail of the result describe circuit follow the trail of the demonstration of frame or make the UI that follows the trail of position obfuscation as a result etc. (User Interface: user interface) control, and the control through carrying out AEAF etc. or carry out the control of video camera with video camera control circuit 145, can carry out automatic framing automatic photography.
Under the situation of the processing more than having carried out, become step shown in Figure 9 by program.The photography input picture carries out face to the input data (S601) of photographing and detects processing (S602).Have, face detects the existing method of using again.Then, the detected result of face is carried out face and handle (S603) towards identification.This face also uses existing method towards identification.Then, from the face detection position, extract the initial registration characteristic (S604) of color.In addition, in next frame, use color characteristic to confirm the tracking process (S605) of target area, carry out face afterwards and detect processing (S606).Detect (S607) if accomplish face, then according to face towards recognition result, will be replaced as the face detection position as the target area of following the trail of the result.Under the situation of not accomplishing the face detection, use the characteristic quantity of color to carry out tracking process, calculate the follow-up evaluation value relevant with the characteristic quantity of color.If following the trail of result's evaluation of estimate also is below the threshold value, then finish tracking process (S611).In addition; If the evaluation of estimate according to the tracking result of color characteristic amount is higher than threshold value; Just extract the color characteristic that uses the look histogram to represent the initial position in the target area; After having upgraded the initial registration characteristic quantity (S612), control is shifted to the demonstration of following the tracks of frame and video camera control (S609).Then, under the situation of accomplishing the face detection, towards the result who detects, select to upgrade the local of initial registration characteristic quantity and upgrade initial registration characteristic quantity (S608) according to face.Tracking after upgrading then position as a result is the LCD demonstration and the video camera control of target area.At (S609) afterwards, turning back to tracking process (S605) handles repeatedly.
(execution mode 2)
In execution mode 2, narration has utilized and has resembled the toroidal information electron camera that the tracking of the tracking of constant information and colouring information, head (not only towards the face in front, also comprising situation backwards that cannot see face) can both be carried out in rotation in this wise.That is, in execution mode 2, replace face and detect information and use the information of toroidal to decide the regional position of renewal that is used to upgrade color characteristic.
The structure of execution mode 2 is identical with the structure of Fig. 4, therefore, and to moving identical structural element omission explanation.
In execution mode 2, tracking process also likewise use the look histogram to mate to be calculated similar degree with execution mode 1 and is evaluation of estimate and follows the trail of.
In execution mode 2, also use the method same, utilize face to detect information and, register the initial characteristics amount from the initial position of user's indication (touch-screen and indicating device etc.) decision objects thing with execution mode 1.
One of information that the appearance of head is constant is " head is round ".So, utilizing shape information to upgrade the example of area detection by upgrading regional detection circuit 139 shown in this execution mode 2.In addition, so long as, can use arbitrary characteristics for the constant characteristic of the appearance of object.
Detection method about the head toroidal is narrated.Carry out rim detection near the tracking result who in memory 111, is preserved.Rim detection is used Suo Beier operator (Sobel operator) etc.Perhaps, with good grounds inter-frame difference utilizes the method for difference point.Implement Hough transformation for the point group that obtains at this (characteristic point group), obtain radius of a circle and center.Radius of the circle and the center for France for example, can take advantage of the literature "HeadFinder: フ Ritz a difference between Rousseau を bell a su ni shi ta figure tracing, Racecourse ら" (Non-Patent Document 1) discloses a method to obtain.Method about describing in the above-mentioned document is explained simply.
Can use (mathematical expression 2), utilize the constant of centre coordinate a, b and radius r to decide circle.
[mathematical expression 2]
(x-a) 2+(y-b) 2＝r 2
Therefore it is applicable to Hough transformation, and the unknown number of trying to achieve is 3, and deciding by vote space (voting space) becomes three-dimensional.Because this amount of calculation is huge, so the real-time action difficulty.So, size is limited in the scope of certain certain radius r.Afterwards, have the characteristic point group that is comprised in a plurality of radiuses of width through use and put to the vote, obtain radius parameter a, b.Thereby infer the toroidal of head.At this moment, on circle, include the evaluation of estimate that the several characteristic point group just becomes the fitting degree of expression toroidal.Position, radius information and the evaluation of estimate of this toroidal are written in the memory 111.
At this, the starting point is that the head evaluation of estimate is lower, but with respect to can stably detecting towards changing, colouring information is if evaluation of estimate reduces towards changing then.Below be described in owing under the situation that appearance changes and evaluation of estimate reduces through colouring information, use the detection position of head toroidal to carry out position correction, through revising the registration characteristic quantity, become stable method for tracing with respect to the variation of appearance.
Upgrade decision circuitry 140 and from memory 111, read in evaluation of estimate and the evaluation of estimate of upgrading regional result of detection based on the tracking result of color; When the evaluation of estimate of following the trail of the result reduces and is lower than threshold value th1; The judgement of upgrading writes judged result in memory 111.At this moment evaluation of estimate and the relation of time shaft shown in Figure 10.
Figure 10 (a) upgrades when the evaluation of estimate of upgrading regional result of detection has surpassed the evaluation of estimate of following the trail of the result.At Figure 10 (a) and (b) with (c), solid line representes to follow the trail of result's evaluation of estimate, and dotted line representes to upgrade the evaluation of estimate of regional result of detection.
[mathematical expression 3]
Follow the trail of evaluation of result value＜regional result of detection evaluation of estimate of renewal
Under this situation, be the phenomenon that produces because the appearance of target object changes and evaluation of estimate reduces and causing when following the trail of failure, thereby can upgrade recovery.
Figure 10 (b) is the figure that the evaluation of estimate of upgrading regional result of detection becomes and upgrades under the situation greater than threshold value th1.
[mathematical expression 4]
The regional result of detection evaluation of estimate of th1＜renewal
This situation is when having guaranteed to upgrade stable in the regional result of detection.Under this situation, follow the trail of result's position in the position correction of upgrading regional result of detection.Thereby, upgrade under the situation that regional result of detection follows the trail of fully can using, utilize the mode of Figure 10 (b), can therefore can construct stable system not being that successfully to select in such object be which.
In addition, describe about the timing of upgrading.The variation of color evaluation value is according to the complexity of the color of color or the object of registration and difference.For example, just high according to the evaluation of estimate of the tracking of color under the high situation of average chroma, evaluation of estimate has with appearance and changes the trend that slowly reduces.
Therefore, in the high object of average chroma, can reduce the circular frequency that detects, thereby be related to the reduction of disposed of in its entirety amount.In addition, under the low situation of average chroma, utilize look histogrammic tracking difficulty, have the trend that evaluation of estimate sharply reduces.Therefore, under the low situation of average chroma,, just can carry out stable tracking through justifying detection continually.Have again, under the low situation of average chroma, also can be set at the target area detecting determined renewal zone by circle.
In addition, in above-mentioned, narrate about carrying out method for updating according to the relation of evaluation of estimate, but also have use based on the tracking of color as a result the position be that the distance relation that position as a result that target area and circle detect is promptly upgraded between the zone carries out method for updating.For example, in 2 times of scopes of the detected radius of circle of hypothesis when be permissible range, through the tracking of color as a result position (following the trail of the center of gravity of frame) when having left this scope the position of circle detection is upgraded, can carry out stable tracking.Have again, be not limited to the multiple of circle, also can use distance arbitrarily such as Euclidean distance.This is because under this situation, even establish based on the evaluation of estimate of the tracking of color characteristic and the evaluation of estimate that detects based on circle all highly, be not object but possibility that mistake is followed the trail of is also high based on the tracking of color characteristic.
Coordinate and size that position correction circuit 142 will be followed the trail of result's target area are replaced as coordinate and the size of upgrading regional result of detection, are kept in the memory 111.At this moment, preserve round center and radius owing to upgrade regional result of detection, therefore, the value that will deduct radius from the center is as top-left coordinates, with 2 times of height as rectangle frame of radius, with top-left coordinates, height and the width that obtains rectangle frame.
In addition, shown in Figure 10 (c), in following the trail of end decision circuitry 141, when the evaluation of estimate of following the trail of the result is lower than threshold value th2 with the evaluation of estimate of upgrading regional result of detection, the judged result that finishes tracking is kept in the memory 111.
Afterwards, implement face and detect, recover automatically, perhaps finish, and specify the tracer body position once more,, just can follow the trail of again through using such method to the user notification tracking process by initial characteristics amount extraction portion 136.
According to above narration; Even towards the back situation under also can follow the trail of; Even under such state; Also can describe circuit and follow the trail of the demonstration of frame or make the UI control of following the trail of position obfuscation as a result etc. through following the trail of the result certainly, and the control through carrying out AEAF etc. or carry out the control of video camera with video camera control circuit 145, can carry out automatic framing automatic photography.
Under the situation of the processing more than having carried out, become step shown in figure 11 with program.The input data (S801) of photographing are carried out face detect (S802).The detected result of face is extracted initial registration characteristic (S803).In next frame, use the color characteristic of in step S803, registering to carry out tracking process (S804), confirm the target area.Afterwards, use and justify detection, upgrade confirm (S805) in zone as the circle of shape facility.
According to based on the evaluation of estimate of the tracking process of color characteristic and the evaluation of estimate that the circle that has used as the circle of shape facility detects, judge whether to finish tracking process (S806).In step 806, when all being lower than threshold value th2, being judged as and finishing to follow the trail of based on the evaluation of estimate of the tracking process of color characteristic amount with based on the evaluation of estimate that circle detects.Being judged as under the situation that finishes tracking process, finish tracking process, finish to follow the trail of the demonstration of frame, perhaps utilize alarm etc. to finish to the user prompt processing.Afterwards, also can in display, put down in writing and urge the user to carry out the demonstration of initial position setting once more, or detect through face and to carry out initial position setting, and recover (S810) automatically.Under situation about not finishing, shown in Figure 10 (b), use threshold value th1; That is, when the evaluation of estimate that detects based on circle has surpassed threshold value th1, upgrade color characteristic; And irrelevant with evaluation of estimate based on the tracking process of color characteristic, perhaps, shown in Figure 10 (a); When the evaluation of estimate of the renewal zone result of detection that detects based on circle has surpassed the evaluation of estimate of the tracking feature that has used color characteristic, be judged as the renewal (S807) of the characteristic quantity that carries out color characteristic.Be judged as under the more news of carrying out characteristic quantity, using the position feature amount of upgrading regional result of detection, the position feature amount of tracking results is being upgraded (S808).Then, the tracking after upgrading as a result the LCD of position show with video camera and control (S809).Afterwards, turn back to the tracking process of step S804, carry out the processing of step S804～S810 repeatedly.
About this tracking process method and similar degree computational methods, be not limited to the look histogram matching, can be based on the method for minimum squared distance or based on the relevant method of normalization etc.In addition, as searching method, can not simple scanning also, and be based on method of particle filter etc.
In addition; In this execution mode; With color characteristic as tracking characteristics, with shape facility as upgrading regional detection feature, but as the few object of non-rigid body and change color change in color is stablized and under the unsettled situation of shape; Also can be with color characteristic as upgrading regional result of detection characteristic, with shape facility as tracking characteristics.Have again, in this execution mode, utilized the face detection position as initial position, but the position that also can utilize AF is as initial position.
Have again, each functional block of block diagram (Fig. 4 etc.) typically is embodied as the LSI as integrated circuit.Also can they single-chipizations individually also can be comprised wherein a part or single-chipization fully.For example, also can be with the functional block single-chipization beyond the memory.
At this, be set at LSI, but also be called IC, system LSI, super large LSI, super LSI sometimes according to the difference of integrated level.
In addition, the method for circuit integration is not limited to LSI, also can realize with special circuit or general processor.Also can be utilized in and make (the Field Programmable Gate Array: of programmable FPGA behind the LSI field programmable gate array) with being connected or the reconfigurable processor of setting of the circuit unit of restructural LSI inside.
In addition, if the integrated circuit technology of other technologies replacement LSI of progress or the derivation of semiconductor technology occurred utilizing, can certainly use this technology to carry out the integrated of functional block.Also possibly be suitable for biotechnology etc.
In addition, also can not make in each functional block, preserve the unit single-chipization of the data of the object become coding or decoding, but become other structure.
Utilizability in the industry
Electron camera that the present invention relates to and image processing method have the tracking function of the object that the appearance of object changes, and ten minutes is useful in the framing of take pictures in the best of utilizing tracking function to realize video cameras controls such as camera function and AEAF, image.In addition, very useful in the surveillance camera of objects such as follow shot personage.

Claims (18)

1. electron camera has subject area that the object of in each frame of the image of sequence photography, confirming to become tracing object mirrors and the function that shows, has:
Tracking process portion; Use first characteristic quantity of registration in advance of the characteristic of representing above-mentioned object quantitatively; Search in the predefined scope in frame, to as the image in the resulting zone of mentioned above searching results, first evaluation of estimate of the consistent degree between the image of represents and above-mentioned object; According to above-mentioned first evaluation of estimate that calculates, confirm to be estimated to be the target area of the image that has above-mentioned object;
Upgrade regional calculating part; Use different with above-mentioned first characteristic quantity, represent second characteristic quantity of the characteristic of above-mentioned object quantitatively; Search in the predefined scope in above-mentioned frame, to as the image in the resulting zone of mentioned above searching results, second evaluation of estimate of the consistent degree between the image of represents and above-mentioned object; According to above-mentioned second evaluation of estimate that calculates, in above-mentioned frame, confirm to be used to upgrade the renewal zone of above-mentioned first characteristic quantity;
Upgrade judging part; Whether at least one in above-mentioned first evaluation of estimate that in above-mentioned tracking process portion, calculates by inquiry and above-mentioned second evaluation of estimate that in the calculating part of above-mentioned renewal zone, calculates satisfies predetermined conditions, judges whether to upgrade above-mentioned first characteristic quantity;
Registration characteristic quantity update portion is being to upgrade under the situation of above-mentioned first characteristic quantity by above-mentioned renewal judgement section judges, is used in the first new characteristic quantity that extracts in the above-mentioned renewal zone and upgrades above-mentioned first characteristic quantity; And
Follow the trail of drawing section as a result; By above-mentioned renewal judgement section judges for not upgrade under the situation of above-mentioned first characteristic quantity, above-mentioned subject area is confirmed as in the above-mentioned target area that will in above-mentioned tracking process portion, confirm, and describes the information of relevant above-mentioned target area; Be to upgrade under the situation of above-mentioned first characteristic quantity by above-mentioned renewal judgement section judges; Above-mentioned subject area is confirmed as in the above-mentioned renewal zone that will in the calculating part of above-mentioned renewal zone, confirm, and describes the information in relevant above-mentioned renewal zone
Above-mentioned tracking process portion uses first characteristic quantity after upgrading to confirm the fresh target zone in the new frame under the situation that above-mentioned first characteristic quantity of registration in advance is updated.
2. electron camera according to claim 1,
Above-mentioned tracking process portion uses the colouring information of object as above-mentioned first characteristic quantity, confirms to be estimated to be the target area of the image that has above-mentioned object,
The shape information that above-mentioned renewal zone calculating part uses object is confirmed above-mentioned renewal zone as above-mentioned second characteristic quantity.
3. electron camera according to claim 1,
Above-mentioned second evaluation of estimate that above-mentioned renewal judging part calculates in the calculating part of above-mentioned renewal zone is during greater than above-mentioned first evaluation of estimate that in above-mentioned tracking process portion, calculates; Perhaps; Above-mentioned second evaluation of estimate that in the calculating part of above-mentioned renewal zone, calculates is judged as and upgrades above-mentioned first characteristic quantity during greater than predefined first threshold.
4. electron camera according to claim 3,
The average chroma of above-mentioned object is high more, and above-mentioned renewal judging part is set big more value to above-mentioned first threshold, and the average chroma of above-mentioned object is low more, and above-mentioned renewal judging part is set more little value to above-mentioned first threshold.
5. electron camera according to claim 1,
The above-mentioned target area that above-mentioned renewal judging part is also confirmed in above-mentioned tracking process portion and distance between the above-mentioned renewal zone definite in the calculating part of above-mentioned renewal zone become predefined second threshold value when above, are judged as and upgrade.
6. electron camera according to claim 1,
Above-mentioned electron camera also has the end judging part of the tracking that judges whether to continue above-mentioned object,
Above-mentioned end judging part during all less than predefined the 3rd threshold value, is judged as the tracking that can not continue above-mentioned object above-mentioned first evaluation of estimate and the above-mentioned second evaluation of estimate both sides.
7. electron camera according to claim 6,
Above-mentioned tracking as a result drawing section when being judged as the tracking that can not continue above-mentioned object, target end zone with upgrade describing of zone.
8. electron camera according to claim 6,
Above-mentioned tracking drawing section is as a result described the image that can not follow the trail of user prompt when being judged as the tracking that can not continue above-mentioned object.
9. electron camera according to claim 6,
Above-mentioned tracking drawing section is as a result described the user is urged the image of setting initial position once more when being judged as the tracking that can not continue above-mentioned object.
10. electron camera according to claim 6,
Above-mentioned tracking drawing section as a result detects through new face and to carry out initial position setting when being judged as the tracking that can not continue above-mentioned object.
11. electron camera according to claim 1,
Above-mentioned electron camera also has:
The face testing circuit detects face in the image in each frame; And
Face is towards identification circuit, identification by the detected face of above-mentioned face testing circuit towards,
By above-mentioned face when identification circuit is identified as face as above-mentioned object towards the side; In the calculating part of above-mentioned renewal zone, calculate conduct after the renewal reference area in the zone of the face of side; Above-mentioned renewal zone calculating part calculates the above-mentioned renewal zone in the above-mentioned renewal reference area according to the preassigned position relation of face part and hair portion.
12. electron camera according to claim 1,
Above-mentioned electron camera also has the video camera control part; This video camera control part is according in target area of in above-mentioned tracking process portion, confirming and the above-mentioned renewal zone in the calculating part of above-mentioned renewal zone, confirmed certain, and change is used to adjust the camera parameters of the action of above-mentioned electron camera.
13. electron camera according to claim 12,
Above-mentioned video camera control part is controlled at least one side's the action of housing and the The Cloud Terrace of above-mentioned electron camera according to above-mentioned camera parameters, control, and makes all or part of the above-mentioned object that is determined cooperate assigned position and size in the frame.
14. electron camera according to claim 1,
Above-mentioned electron camera also has the object initial setting section, and this object initial setting section perhaps uses predefined method to decide the initial position of above-mentioned object according to the input from the user.
15. electron camera according to claim 14,
Above-mentioned object initial setting section is initial position with the detection position decision of personage or face.
16. electron camera according to claim 14,
It is initial position that above-mentioned object initial setting section will be closed burnt place decision by the automatic focus function of AF.
17. the image processing method in the electron camera, this electron camera have subject area that the object of in each frame of the image of sequence photography, confirming to become tracing object mirrors and the function that shows, above-mentioned image processing method comprises:
Tracking process portion uses first characteristic quantity of registration in advance of the characteristic of representing above-mentioned object quantitatively; Search in the predefined scope in frame; To as the image in the resulting zone of mentioned above searching results, first evaluation of estimate of the consistent degree between the image of represents and above-mentioned object is according to above-mentioned first evaluation of estimate that calculates; Confirm to be estimated to be the target area of the image that has above-mentioned object
Upgrade regional calculating part use different with above-mentioned first characteristic quantity, represent second characteristic quantity of the characteristic of above-mentioned object quantitatively; Search in the predefined scope in above-mentioned frame; To as the image in the resulting zone of mentioned above searching results, second evaluation of estimate of the consistent degree between the image of represents and above-mentioned object is according to above-mentioned second evaluation of estimate that calculates; In above-mentioned frame, confirm to be used to upgrade the renewal zone of above-mentioned first characteristic quantity
Whether at least one that upgrade in above-mentioned first evaluation of estimate that judging part calculates in above-mentioned tracking process portion by inquiry and above-mentioned second evaluation of estimate that in the calculating part of above-mentioned renewal zone, calculates satisfies predetermined conditions; Judge whether to upgrade above-mentioned first characteristic quantity
Registration characteristic quantity update portion is being to upgrade under the situation of above-mentioned first characteristic quantity by above-mentioned renewal judgement section judges, is used in the first new characteristic quantity that extracts in the above-mentioned renewal zone and upgrades above-mentioned first characteristic quantity,
Follow the trail of as a result drawing section by above-mentioned renewal judgement section judges for not upgrade under the situation of above-mentioned first characteristic quantity; Above-mentioned subject area is confirmed as in the above-mentioned target area that will in above-mentioned tracking process portion, confirm; And describe information about above-mentioned target area, and be to upgrade under the situation of above-mentioned first characteristic quantity by above-mentioned renewal judgement section judges, above-mentioned subject area will be confirmed as in the above-mentioned renewal zone that will in the calculating part of above-mentioned renewal zone, confirm; And describe about the regional information of above-mentioned renewal
Above-mentioned tracking process portion uses first characteristic quantity after upgrading to confirm the fresh target zone in the new frame under the situation that above-mentioned first characteristic quantity of registration in advance is updated.
18. an integrated circuit has subject area that the object of in each frame of the image of sequence photography, confirming to become tracing object mirrors and the function that shows, has:
Tracking process portion; Use first characteristic quantity of registration in advance of the characteristic of representing above-mentioned object quantitatively; Search in the predefined scope in frame, to as the image in the resulting zone of mentioned above searching results, first evaluation of estimate of the consistent degree between the image of represents and above-mentioned object; According to above-mentioned first evaluation of estimate that calculates, confirm to be estimated to be the target area of the image that has above-mentioned object;
Upgrade regional calculating part; Use different with above-mentioned first characteristic quantity, represent second characteristic quantity of the characteristic of above-mentioned object quantitatively; Search in the predefined scope in above-mentioned frame, to as the image in the resulting zone of mentioned above searching results, second evaluation of estimate of the consistent degree between the image of represents and above-mentioned object; According to above-mentioned second evaluation of estimate that calculates, in above-mentioned frame, confirm to be used to upgrade the renewal zone of above-mentioned first characteristic quantity;
Upgrade judging part; Whether at least one in above-mentioned first evaluation of estimate that in above-mentioned tracking process portion, calculates by inquiry and above-mentioned second evaluation of estimate that in the calculating part of above-mentioned renewal zone, calculates satisfies predetermined conditions, judges whether to upgrade above-mentioned first characteristic quantity;
Registration characteristic quantity update portion is being to upgrade under the situation of above-mentioned first characteristic quantity by above-mentioned renewal judgement section judges, is used in the first new characteristic quantity that extracts in the above-mentioned renewal zone and upgrades above-mentioned first characteristic quantity; And
Follow the trail of drawing section as a result; By above-mentioned renewal judgement section judges for not upgrade under the situation of above-mentioned first characteristic quantity, above-mentioned subject area is confirmed as in the above-mentioned target area that will in above-mentioned tracking process portion, confirm, and describes the information of relevant above-mentioned target area; Be to upgrade under the situation of above-mentioned first characteristic quantity by above-mentioned renewal judgement section judges; Above-mentioned subject area is confirmed as in the above-mentioned renewal zone that will in the calculating part of above-mentioned renewal zone, confirm, and describes the information in relevant above-mentioned renewal zone
Above-mentioned tracking process portion uses first characteristic quantity after upgrading to confirm the fresh target zone in the new frame under the situation that above-mentioned first characteristic quantity of registration in advance is updated.
CN2009801044907A 2008-02-06 2009-02-06 Electronic camera and image processing method Active CN101939980B (en)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
JP2008-026160 2008-02-06
JP2008026160 2008-02-06
PCT/JP2009/000477 WO2009098894A1 (en) 2008-02-06 2009-02-06 Electronic camera and image processing method

Publications (2)

Publication Number Publication Date
CN101939980A CN101939980A (en) 2011-01-05
CN101939980B true CN101939980B (en) 2012-08-08

Family

ID=40951973

Family Applications (1)

Application Number Title Priority Date Filing Date
CN2009801044907A Active CN101939980B (en) 2008-02-06 2009-02-06 Electronic camera and image processing method

Country Status (5)

Country Link
US (1) US8253819B2 (en)
EP (1) EP2242253B1 (en)
JP (1) JP5090474B2 (en)
CN (1) CN101939980B (en)
WO (1) WO2009098894A1 (en)

Families Citing this family (58)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4869270B2 (en) * 2008-03-10 2012-02-08 三洋電機株式会社 Imaging apparatus and image reproduction apparatus
WO2009128064A2 (en) * 2008-04-14 2009-10-22 Pointgrab Ltd. Vision based pointing device emulation
US20110199499A1 (en) * 2008-10-14 2011-08-18 Hiroto Tomita Face recognition apparatus and face recognition method
KR101051365B1 (en) * 2009-07-16 2011-07-22 주식회사 유니온커뮤니티 Access control device with camera and access control method
JP5279654B2 (en) * 2009-08-06 2013-09-04 キヤノン株式会社 Image tracking device, image tracking method, and computer program
JP5483961B2 (en) * 2009-09-02 2014-05-07 キヤノン株式会社 Image processing apparatus, subject discrimination method, program, and storage medium
GB2474536B (en) 2009-10-13 2011-11-02 Pointgrab Ltd Computer vision gesture based control of a device
JP5657235B2 (en) * 2009-11-10 2015-01-21 オリンパスイメージング株式会社 Image capturing apparatus and image capturing method
JP5427577B2 (en) * 2009-12-04 2014-02-26 パナソニック株式会社 Display control apparatus and display image forming method
KR101652261B1 (en) * 2009-12-24 2016-08-30 삼성전자주식회사 Method for detecting object using camera
US8705813B2 (en) * 2010-06-21 2014-04-22 Canon Kabushiki Kaisha Identification device, identification method, and storage medium
JP5720275B2 (en) 2011-02-03 2015-05-20 株式会社リコー Imaging apparatus and imaging method
CN102075686B (en) * 2011-02-10 2013-10-30 北京航空航天大学 Robust real-time on-line camera tracking method
US8766153B2 (en) * 2011-02-17 2014-07-01 Mitutoyo Corporation Vision measuring device and auto-focusing control method
KR20120118383A (en) * 2011-04-18 2012-10-26 삼성전자주식회사 Image compensation device, image processing apparatus and methods thereof
JP5226903B1 (en) * 2011-06-29 2013-07-03 オリンパスイメージング株式会社 TRACKING DEVICE AND TRACKING METHOD
JP5888890B2 (en) * 2011-07-21 2016-03-22 オリンパス株式会社 Focus control device
CN103098480B (en) * 2011-08-25 2018-08-28 松下电器（美国）知识产权公司 Image processing apparatus and method, three-dimensional image pickup device
JP5878924B2 (en) * 2011-11-17 2016-03-08 パナソニック インテレクチュアル プロパティ コーポレーション オブアメリカＰａｎａｓｏｎｉｃ Ｉｎｔｅｌｌｅｃｔｕａｌ Ｐｒｏｐｅｒｔｙ Ｃｏｒｐｏｒａｔｉｏｎ ｏｆ Ａｍｅｒｉｃａ Image processing apparatus, imaging apparatus, and image processing method
CN102521580A (en) * 2011-12-21 2012-06-27 华平信息技术（南昌）有限公司 Real-time target matching tracking method and system
KR101989152B1 (en) * 2012-02-10 2019-09-30 삼성전자주식회사 Apparatus and method for capturing still image during moving image photographing or reproducing
US9049364B2 (en) * 2012-02-13 2015-06-02 Htc Corporation Focus adjusting method and image capture device thereof
JP5888034B2 (en) * 2012-03-16 2016-03-16 富士通株式会社 User detection device, method and program
US9256781B2 (en) * 2012-05-10 2016-02-09 Pointguard Ltd. System and method for computer vision based tracking of an object
US8938124B2 (en) * 2012-05-10 2015-01-20 Pointgrab Ltd. Computer vision based tracking of a hand
US8441548B1 (en) * 2012-06-15 2013-05-14 Google Inc. Facial image quality assessment
CN103516972B (en) * 2012-06-19 2017-03-29 联想(北京)有限公司 Camera control method and electronic equipment
JP2014116716A (en) * 2012-12-07 2014-06-26 Samsung Display Co Ltd Tracking device
JP2014139734A (en) * 2013-01-21 2014-07-31 Sony Corp Information processing device and method, and program
JP2014207659A (en) * 2013-03-19 2014-10-30 パナソニック株式会社 Image processing device and image processing method and imaging apparatus
KR101316805B1 (en) 2013-05-22 2013-10-11 주식회사 파이브지티 Automatic face tracing and recognition method and system
US10474921B2 (en) 2013-06-14 2019-11-12 Qualcomm Incorporated Tracker assisted image capture
US20160140727A1 (en) * 2013-06-17 2016-05-19 Aselsan Elektronik Sanayi Ve Ticaret Anonim Sirket I A method for object tracking
CN104424634B (en) * 2013-08-23 2017-05-03 株式会社理光 Object tracking method and device
JP6268960B2 (en) * 2013-11-15 2018-01-31 オムロン株式会社 Image recognition apparatus and data registration method for image recognition apparatus
US9437000B2 (en) * 2014-02-20 2016-09-06 Google Inc. Odometry feature matching
JP6415196B2 (en) * 2014-09-08 2018-10-31 キヤノン株式会社 IMAGING DEVICE AND IMAGING DEVICE CONTROL METHOD
JP6310093B2 (en) 2014-11-12 2018-04-11 エスゼット ディージェイアイ テクノロジー カンパニー リミテッドＳｚ Ｄｊｉ Ｔｅｃｈｎｏｌｏｇｙ Ｃｏ．，Ｌｔｄ Target object detection method, detection apparatus, and robot
JP6521626B2 (en) * 2014-12-25 2019-05-29 キヤノン株式会社 Object tracking device, method and program
US9934443B2 (en) * 2015-03-31 2018-04-03 Daon Holdings Limited Methods and systems for detecting head motion during an authentication transaction
US10740907B2 (en) * 2015-11-13 2020-08-11 Panasonic Intellectual Property Management Co., Ltd. Moving body tracking method, moving body tracking device, and program
TWI585394B (en) * 2015-12-09 2017-06-01 由田新技股份有限公司 Automatic focusing system
JP6656987B2 (en) * 2016-03-30 2020-03-04 株式会社エクォス・リサーチ Image recognition device, mobile device, and image recognition program
CN106934395B (en) * 2017-02-15 2020-06-30 国家计算机网络与信息安全管理中心 Rigid body target tracking method adopting combination of SURF (speeded Up robust features) and color features
CN107181976B (en) * 2017-04-28 2021-01-29 华为技术有限公司 Bullet screen display method and electronic equipment
CN110059521B (en) * 2018-01-18 2022-05-13 浙江宇视科技有限公司 Target tracking method and device
US20190244030A1 (en) * 2018-02-07 2019-08-08 Hitachi, Ltd. Object tracking in video using better object area
US11164015B2 (en) * 2018-05-08 2021-11-02 Ford Global Technologies, Llc Simultaneous diagnosis and shape estimation from a perceptual system derived from range sensors
JP6779474B2 (en) * 2019-01-16 2020-11-04 みこらった株式会社 Levitation mobile stand
CN110517614B (en) * 2019-08-28 2022-11-22 苏州精速智能科技有限公司 Method for detecting bad conductive particles of liquid crystal module
US11308923B2 (en) * 2019-10-09 2022-04-19 Motorola Mobility Llc Electronic device with face orienting user interface
US11076111B1 (en) * 2019-11-13 2021-07-27 Twitch Interactive, Inc. Smart color-based background replacement
CN111062965B (en) * 2019-12-26 2023-01-17 西华大学 Low-complexity double-threshold multi-resolution mouth detection method based on assembly line
US11004212B1 (en) * 2020-01-02 2021-05-11 Hong Kong Applied Science and Technology Research Institute Company Limited Object tracking method and system using iterative template matching
US11553136B2 (en) * 2020-02-19 2023-01-10 Canon Kabushiki Kaisha Subject tracking device, subject tracking method, and storage medium
JP7214681B2 (en) * 2020-02-19 2023-01-30 キヤノン株式会社 SUBJECT TRACKING DEVICE, SUBJECT TRACKING METHOD, COMPUTER PROGRAM AND STORAGE MEDIUM
JP6785481B1 (en) * 2020-05-22 2020-11-18 パナソニックＩｐマネジメント株式会社 Image tracking device
CN115222817B (en) * 2022-08-31 2022-11-29 成都千嘉科技股份有限公司 Pointer positioning method of pointer type pressure gauge

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1722826A (en) * 2004-07-14 2006-01-18 松下电器产业株式会社 System, device and method for object tracing
CN1892702A (en) * 2005-07-05 2007-01-10 欧姆龙株式会社 Tracking apparatus

Family Cites Families (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH056434A (en) * 1991-06-27 1993-01-14 Matsushita Electric Ind Co Ltd Automatic tracking device
JP3552456B2 (en) 1997-05-01 2004-08-11 セイコーエプソン株式会社 Moving object tracking method
JPH11167455A (en) * 1997-12-05 1999-06-22 Fujitsu Ltd Hand form recognition device and monochromatic object form recognition device
US6687386B1 (en) * 1999-06-15 2004-02-03 Hitachi Denshi Kabushiki Kaisha Object tracking method and object tracking apparatus
US6859552B2 (en) * 2000-11-07 2005-02-22 Minolta Co., Ltd. Image retrieving apparatus
JP2002157599A (en) 2000-11-17 2002-05-31 Nippon Telegr & Teleph Corp <Ntt> Method for detecting and recognizing object, recording medium recording its program and object monitoring and tracking device
US7099510B2 (en) * 2000-11-29 2006-08-29 Hewlett-Packard Development Company, L.P. Method and system for object detection in digital images
US7298412B2 (en) * 2001-09-18 2007-11-20 Ricoh Company, Limited Image pickup device, automatic focusing method, automatic exposure method, electronic flash control method and computer program
KR100459893B1 (en) * 2002-01-08 2004-12-04 삼성전자주식회사 Method and apparatus for color-based object tracking in video sequences
JP2004362210A (en) * 2003-06-04 2004-12-24 Nippon Telegr & Teleph Corp <Ntt> Device and method for tracing object and its program and recording medium with its program recorded thereon
US7315631B1 (en) * 2006-08-11 2008-01-01 Fotonation Vision Limited Real-time face tracking in a digital image acquisition device
JP4824411B2 (en) * 2005-01-20 2011-11-30 パナソニック株式会社 Face extraction device, semiconductor integrated circuit
JP2007072520A (en) * 2005-09-02 2007-03-22 Sony Corp Video processor
JP4657934B2 (en) * 2006-01-23 2011-03-23 富士フイルム株式会社 Face detection method, apparatus and program
GB2452512B (en) * 2007-09-05 2012-02-29 Sony Corp Apparatus and method of object tracking
TW200929005A (en) * 2007-12-26 2009-07-01 Altek Corp Human face detection and tracking method
JP4730431B2 (en) * 2008-12-16 2011-07-20 日本ビクター株式会社 Target tracking device
JP5383366B2 (en) * 2009-07-28 2014-01-08 キヤノン株式会社 Image processing apparatus, image processing method, and program thereof
JP5662670B2 (en) * 2009-10-27 2015-02-04 キヤノン株式会社 Image processing apparatus, image processing method, and program

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1722826A (en) * 2004-07-14 2006-01-18 松下电器产业株式会社 System, device and method for object tracing
CN1892702A (en) * 2005-07-05 2007-01-10 欧姆龙株式会社 Tracking apparatus

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
JP特开2004-362210A 2004.12.24
JP特开平10-304348A 1998.11.13
JP特开平11-167455A 1999.06.22

Also Published As

Publication number Publication date
JPWO2009098894A1 (en) 2011-05-26
EP2242253A4 (en) 2014-05-14
US8253819B2 (en) 2012-08-28
JP5090474B2 (en) 2012-12-05
EP2242253A1 (en) 2010-10-20
EP2242253B1 (en) 2019-04-03
CN101939980A (en) 2011-01-05
WO2009098894A1 (en) 2009-08-13
US20110001840A1 (en) 2011-01-06

Similar Documents

Publication Publication Date Title
CN101939980B (en) Electronic camera and image processing method
US10346464B2 (en) Cross-modiality image matching method
US10013764B2 (en) Local adaptive histogram equalization
JP7026225B2 (en) Biological detection methods, devices and systems, electronic devices and storage media
CN107707871B (en) Image processing apparatus, image capturing apparatus, image processing method, and storage medium
JP5054063B2 (en) Electronic camera, image processing apparatus, and image processing method
US8306262B2 (en) Face tracking method for electronic camera device
CN106295638A (en) Certificate image sloped correcting method and device
CN107087107A (en) Image processing apparatus and method based on dual camera
CN103733607A (en) Device and method for detecting moving objects
CN105594194A (en) Image display device and image display method
CN104243787A (en) Photographing method and equipment, and photo management method
CN103004179B (en) Tracking means and tracking
CN102103754A (en) Subject tracking apparatus, subject region extraction apparatus, and control methods therefor
CN103179338A (en) Tracking device and tracking method
US9106829B2 (en) Apparatus and method for providing guide information about photographing subject in photographing device
JP6924064B2 (en) Image processing device and its control method, and image pickup device
CN102857690A (en) Tracking apparatus, tracking method, shooting device and shooting method
CN111126108A (en) Training method and device of image detection model and image detection method and device
CN105335959A (en) Quick focusing method and device for imaging apparatus
CN102087401A (en) Auto focusing method, recording medium for recording the method, and auto focusing apparatus
CN109740442A (en) Localization method, device, storage medium and electronic equipment
CN111967525A (en) Data processing method and device, server and storage medium
CN116543014A (en) Panorama-integrated automatic teacher tracking method and system
CN104038689A (en) Image processing device and image processing method

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
C14 Grant of patent or utility model
GR01 Patent grant
ASS Succession or assignment of patent right

Owner name: MATSUSHITA ELECTRIC (AMERICA) INTELLECTUAL PROPERT

Free format text: FORMER OWNER: MATSUSHITA ELECTRIC INDUSTRIAL CO, LTD.

Effective date: 20140930

C41 Transfer of patent application or patent right or utility model
TR01 Transfer of patent right

Effective date of registration: 20140930

Address after: Seaman Avenue Torrance in the United States of California No. 2000 room 200

Patentee after: PANASONIC INTELLECTUAL PROPERTY CORPORATION OF AMERICA

Address before: Osaka Japan

Patentee before: Matsushita Electric Industrial Co.,Ltd.