CN101939980B - Electronic camera and image processing method - Google Patents

Electronic camera and image processing method Download PDF

Info

Publication number
CN101939980B
CN101939980B CN2009801044907A CN200980104490A CN101939980B CN 101939980 B CN101939980 B CN 101939980B CN 2009801044907 A CN2009801044907 A CN 2009801044907A CN 200980104490 A CN200980104490 A CN 200980104490A CN 101939980 B CN101939980 B CN 101939980B
Authority
CN
China
Prior art keywords
update
object
tracking
image
evaluation value
Prior art date
Application number
CN2009801044907A
Other languages
Chinese (zh)
Other versions
CN101939980A (en
Inventor
今川和幸
小仓康伸
物部祐亮
石井育规
Original Assignee
松下电器产业株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority to JP2008026160 priority Critical
Priority to JP2008-026160 priority
Application filed by 松下电器产业株式会社 filed Critical 松下电器产业株式会社
Priority to PCT/JP2009/000477 priority patent/WO2009098894A1/en
Publication of CN101939980A publication Critical patent/CN101939980A/en
Application granted granted Critical
Publication of CN101939980B publication Critical patent/CN101939980B/en

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00Details of television systems
    • H04N5/222Studio circuitry; Studio devices; Studio equipment ; Cameras comprising an electronic image sensor, e.g. digital cameras, video cameras, TV cameras, video cameras, camcorders, webcams, camera modules for embedding in other devices, e.g. mobile phones, computers or vehicles
    • H04N5/225Television cameras ; Cameras comprising an electronic image sensor, e.g. digital cameras, video cameras, camcorders, webcams, camera modules specially adapted for being embedded in other devices, e.g. mobile phones, computers or vehicles
    • H04N5/232Devices for controlling television cameras, e.g. remote control ; Control of cameras comprising an electronic image sensor
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS, OR APPARATUS
    • G02B7/00Mountings, adjusting means, or light-tight connections, for optical elements
    • G02B7/28Systems for automatic generation of focusing signals
    • G02B7/36Systems for automatic generation of focusing signals using image sharpness techniques, e.g. image processing techniques for generating autofocus signals
    • G02B7/365Systems for automatic generation of focusing signals using image sharpness techniques, e.g. image processing techniques for generating autofocus signals by analysis of the spatial frequency components of the image
    • GPHYSICS
    • G03PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03BAPPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
    • G03B13/00Viewfinders; Focusing aids for cameras; Means for focusing for cameras; Autofocus systems for cameras
    • G03B13/32Means for focusing
    • G03B13/34Power focusing
    • G03B13/36Autofocus systems
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00General purpose image data processing
    • G06T1/0007Image acquisition
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/20Analysis of motion
    • G06T7/246Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G06T7/248Analysis of motion using feature-based methods, e.g. the tracking of corners or segments involving reference images or patches
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00Details of television systems
    • H04N5/222Studio circuitry; Studio devices; Studio equipment ; Cameras comprising an electronic image sensor, e.g. digital cameras, video cameras, TV cameras, video cameras, camcorders, webcams, camera modules for embedding in other devices, e.g. mobile phones, computers or vehicles
    • H04N5/225Television cameras ; Cameras comprising an electronic image sensor, e.g. digital cameras, video cameras, camcorders, webcams, camera modules specially adapted for being embedded in other devices, e.g. mobile phones, computers or vehicles
    • H04N5/232Devices for controlling television cameras, e.g. remote control ; Control of cameras comprising an electronic image sensor
    • H04N5/23218Control of camera operation based on recognized objects
    • H04N5/23219Control of camera operation based on recognized objects where the recognized objects include parts of the human body, e.g. human faces, facial parts or facial expressions
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10016Video; Image sequence
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30196Human being; Person
    • G06T2207/30201Face
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30232Surveillance

Abstract

It is possible to provide an electronic camera and an image processing method which can perform a highly accurate tracking process even when an object is viewed in various ways. The electronic camera includes: a tracking process unit (137), an update region detection circuit (139), an update judgment circuit (140), and a tracking result plot circuit (144). The tracking process unit (137) searchesinside of a frame by using a color characteristic amount of an object to be tracked, calculates a first evaluation value indicating the matching degree between an image of a search result and the object, and identifies a target region which is estimated to contain an image of the object according to the first evaluation value. The update region detection circuit (139) searches inside of the frameby using a shape characteristic amount of the object to be tracked, calculates a seconds evaluation value indicating the matching degree between the image of the search result and the object, and identifies an update region for updating the first characteristic amount according to the second evaluation value. The update judgment circuit (140) judges whether to update the first characteristic amount depending on whether the first evaluation value or the second evaluation value satisfies a predetermined condition. If No, the tracking result plot circuit (144) plots an external frame of the target region. If Yes, the tracking result plot circuit (144) plots an external frame of the update region.

Description

电子摄像机和图像处理方法 Electronic cameras and image processing method

技术领域 FIELD

[0001] 本发明涉及追踪图像中的物体,将追踪结果提示在显示器上的电子摄像机及其方法。 [0001] The present invention relates to the tracking of the object image, the tracking results suggest that the electronic camera and a method on the display.

背景技术 Background technique

[0002] 近年来,作为AF(自动对焦;Auto_Focus) • AE (自动曝光;Auto Exposure) •逆光修正功能的对位手段,在电子摄像机中搭载有脸检测功能。 [0002] In recent years, as the AF (Auto Focus; Auto_Focus) • AE (Auto Exposure; Auto Exposure) • alignment means backlight correction function, equipped with face detection function in an electronic camera. 通过使用脸检测功能,能够使焦点自动地合焦在被摄体上,因此,作为用户的摄影辅助而成为有效的手段。 By using the face detection function, the focus can be automatically focus on the subject, therefore, as the user's photographic auxiliary means becomes effective.

[0003] 作为检测脸的图像处理装置和方法,已提出了下述方法,学习脸和除脸以外的部分(以下称作非脸)的图案,使用已输入了该学习到的參数的识别器来识别脸与非脸(例如,參照专利文献I)。 [0003] As an image processing apparatus and method for detecting a face, a method has been proposed, and a portion other than the face learning face (hereinafter, referred to as non-face) pattern identification has been entered using the learned parameters is to identify the face and non-face (e.g., refer to Patent Document I).

[0004] 图I是示出专利文献I的图像处理装置的图。 [0004] Figure I is a diagram illustrating an image processing apparatus of Patent Document I of FIG. 图2是示出部分图像的切出的一例的图。 FIG 2 is a diagram illustrating an example of the partial image cut out. 部分图像切出部900从输入的图像1001中切出部分图像1000。 Cutting out partial image 1001 from the image input unit 900 in the image cut-out section 1000. 通过使多个尺寸的窗以图像的左上为起点,一边向右侧或下侧挪动适当的象素(例如I个象素)ー边依次进行扫描,直到图像的右下为止,来切出部分图像1000(图2)。 A plurality of windows by the size of the image to the upper left as a starting point, while moving the appropriate pixels (e.g., pixels I) ー sequentially scanned to the right side or the lower side until the lower right of the image, cut out portion image 1000 (FIG. 2). 再有,所述“切出”是指仅读取对应部分的图像数据。 Furthermore, the "cut" refers to read only the portions corresponding to the image data.

[0005] 特征量评价部I (901)由多个识别器的组合构成。 [0005] The feature amount evaluation unit I (901) is constituted by a combination of a plurality of identifiers. 识别器在利用使用boosting方法学习到的參数所指定的位置,根据后述的矩形特征(以下称作邻接差分滤波器)计算特征量。 Boosting learning method using the identifier to a parameter in the location specified by using, according to the rectangular feature to be described later (hereinafter referred to as adjacent difference filter) is calculated feature amount. 然后,若上述识别器的输出值的加权线性和低于基于学习计算出的阈值,则特征量评价部1(901)识别为非脸,结束上述部分图像的识别处理。 Then, when the output value of the weighted linear classifier learning and lower than the calculated evaluation value threshold, the feature amount based on a unit (901) identifying a non-face ends of the partial image recognition processing. 另ー方面,若在阈值以上,就识别为脸,处理向下一个特征量评价部2(902)转移。ー other hand, if above the threshold, it is recognized as a face, a feature amount evaluation process down portion 2 (902) transfers. 特征量评价部2(902)使用与在特征量评价部1(901)中使用的学习參数不同的參数进行评价。 Feature amount evaluation unit 2 (902) uses the learning parameter used in the feature amount evaluation unit 1 (901) different parameters were evaluated. 这样,使用多个特征量评价部计算评价值,根据计算出的评价值的值辨别脸与非脸。 Thus, the calculation unit using a plurality of evaluation feature amount evaluation value, the evaluation value based on the value calculated with the non-face face discrimination.

[0006] 图3(a)、(b)、(c)、(d)示出了邻接差分滤波器,图3(e)是对图像适用了邻接差分滤波器(b)时的例子。 [0006] FIG. 3 (a), (b), (c), (d) shows the adjacent difference filter, FIG. 3 (e) is an example when the adjacent difference filter (b) applied to the image. 邻接差分滤波器用邻接的白色矩形和黒色矩形表示,输出白色矩形内的象素平均值与黑色矩形内的象素平均值的差。 Adjacent difference filter is represented by a white rectangle and the adjacent rectangle black color, the average pixel difference within the black rectangle with the average of the pixels within the output white rectangle. 脸器官的特征是,根据从邻接差分滤波器输出的象素平均值的差较大的情况而被确定,这意味着象眼睛和嘴这样地在邻接间区域的象素值的差较大的区域中输出高的特征量。 Characterized in that the face organ, is determined based on a great difference between the average pixel output from the adjacent difference filter, which means that the eyes and mouth in such a way as a large pixel value between adjacent regions in the difference high output region feature amount. 例如,图3(e)根据位于额位置上的黒色矩形的象素的总和与位于眉位置上的白色矩形的象素的总和之差计算特征量。 For example, FIG. 3 (e) is calculated according to the difference between the sum of the feature amounts of the pixel of the white rectangle on the sum of the black color pixel is located on a rectangular frontal position located eyebrow position. 由于该特征量是邻接的区域间的差,因此,在图像内的局部特征(例如线成分)上強烈反应,在眼睛、眉、嘴等脸器官上输出特征的值。 Since the feature quantity is a difference between adjacent regions, therefore, a strong reaction on local features (e.g. line component) in the image, the value of the output characteristic in the eyes, eyebrows, mouth and other facial organs. 再有,邻接差分滤波器一般称作Haar-Iike特征(Haar-likeieature)。 Further, the adjacent difference filter is generally referred to as a Haar-Iike wherein (Haar-likeieature).

[0007] 但是,仅在脸检测中,若不是看得见脸(眼睛、鼻子、嘴)的范围内就不能检测,并且不能追踪宠物等的除脸以外的物体。 [0007] However, only in the face detection, if it is visible can not be detected within the range of the face (eyes, nose, mouth), and the object can not be tracked beyond the other face of pets. 因此,有下述方法,用户预先再现登记物体的信息,通过追踪该物体,即使是除脸以外的物体,也自动地对焦。 Thus, there is a method, user information registered in advance reproduction object by tracking the object, even objects other than the face, it is automatically focus.

[0008] 作为以往使用的物体追踪方法,有仅对脸检测结果位置的附近区域进行再次脸检测的脸追踪方法、使用基于相关运算的前ー帧附近捜索的模板匹配、以高速化为目的的基于积极捜索的方法、使用基于概率分布的统计信息加入动态预测进行捜索的基于粒子过滤器(particle filter)或缩聚(Condensation)的方法。 [0008] As a conventional object tracking method of use, only the area near the face detection result of the position is performed again face face detection tracking method, using template matching vicinity of the front frame ー Dissatisfied cable operation based on the correlation of object at a high speed into Dissatisfied method based positive cable, cable Dissatisfied performed using statistical information based on probability distribution based on the dynamic prediction added particle filter (particle filter) or polycondensation (condensation) method.

[0009] 这些都是预先使用某些方法登记想进行追踪的物体的初始特征量(顔色或亮度的色直方图或者模板图像本身、形状•轮廓信息等)。 [0009] These are some of the previously used method for registering an initial feature amount want to track the object (the color histogram of color or brightness, or a template image itself, shape, contour information, etc. •). 通过使用该登记特征量,在图像中搜索与登记特征量类似的地方来进行物体追踪。 By using this feature amount registered in the image registration searches similar to the local feature amount of object tracking. 在这些方法中,预先制成初始特征量,取该特征量与根据图像的各位置所提取的特征量之间的匹配。 In these methods, an initial feature amount previously prepared, taking matching between the feature amount extracted at each position of the image in accordance with the feature quantity.

[0010] 但是,利用一般的影片进行摄影的对象少数情况下能长时间看见脸等,多数情况下都是图像上的样子变化很大。 [0010] However, in general the use of film for photographing objects a long time to see a few cases such as the face, in most cases all the way on the image vary widely. 在以往的方法中存在对于图像上样子变化大的这样的对象马上就会丢失目标的问题。 Problems for such an object on the image of the target will be lost immediately change the way in the conventional method.

[0011] 为了解决该问题,例如在专利文献2中使用了逐次更新模板的方法。 [0011] In order to solve this problem, for example, a method of sequentially updating a template in Patent Document 2. 根据该方法,即使在追踪对象的样子变化的情况下,也按照该变化更新模板。 According to this method, even in the case of tracking objects like changes, but also changes in accordance with the updated template. 因此,能追踪样子变化的物体。 Therefore, to track changes in the way objects.

[0012] 专利文献I :美国专利申请公开第2002/0102024号公报 [0012] Patent Document I: U.S. Patent Application Publication No. 2002/0102024 Patent Publication

[0013] 专利文献2 :日本特开2002-157599号公报 [0013] Patent Document 2: Japanese Laid-Open Patent Publication No. 2002-157599

[0014] 发明所要解决的问题 [0014] The invention solves the problem

[0015] 但是,在专利文献2的方法中,按照每个帧更新模板,但并不能说总是能够使用仅包含对象物体的区域来更新模板。 [0015] However, in the method of Patent Document 2, the template is updated for each frame, but can not always be used, said area comprising only the update target object to the template. 存在由于在进行更新的区域中混入非追踪对象区域而使之后的追踪失败的问题。 Tracking problems after the failure due to the mixed area of ​​the non-performing update the tracking target area.

[0016] 使用图5具体地说明。 [0016] specifically described with reference to FIG 5. 例如,假设将图5(a)的正确区域定为200。 For example, it is assumed in FIG. 5 (a) is set to the correct region 200. 这时,在下一帧的图5(b)中,由于帽子的顔色的边界类似,因此在201的位置上匹配。 In this case, in FIG. 5 (b) of the next frame, since similar colors hat boundary, the position 201 on the match. 在该位置中,由于201的左侧包含背景,因此,在下一帧以后的追踪中,也加入了背景的特征,对头部追踪给予坏影响。 In this position, since the left contains background 201, Therefore, after the next frame in the track, but also the added feature background, to give a bad influence on the head tracking. 但是,若能够与202匹配,就能够根据头部全体而得到特征,能够在下一帧以后也稳定地追踪头部。 However, if 202 is capable of matching characteristic can be obtained according to all the head, the head can be stably tracked after the next frame.

发明内容 SUMMARY

[0017] 本发明鉴于上述现有问题,通过使用能稳定地检测对象物的位置的特征决定更新区域,来消除存在对象物的区域的追踪错误。 [0017] The present invention in view of the above conventional problems, can be stably detected by using the features of the object position determined update region, the tracking error exists to eliminate the area of ​​the object. 此外,通过在追踪结果从正确位置的偏离变大的情况下修正成正确的位置,来实现稳定的追踪处理。 Further, by correcting the position to the right in the case of deviation from the correct position of the tracking result becomes large, to achieve a stable tracking process.

[0018] 用于解决问题的手段 [0018] Solution to Problem

[0019] 为了解决上述问题,本发明的电子摄像机具有在连续摄影的图像的各帧内确定成为跟踪对象的对象物所映入的对象区域并进行显示的功能,具有:追踪处理部,使用定量地表示上述对象物的特征的预先登记的第一特征量,在帧内的预先设定的范围内进行捜索,对作为上述捜索结果所得到的区域内的图像,计算表示与上述对象物的图像之间的一致度的第一评价值,根据计算出的上述第一评价值,确定被推定为存在上述对象物的图像的目标区域;更新区域计算部,使用与上述第一特征量不同的、定量地表示上述对象物的特征的第二特征量,在上述帧内的预先设定的范围内进行捜索,对作为上述捜索结果所得到的区域内的图像,计算表示与上述对象物的图像之间的一致度的第二评价值,根据计算出的上述第二评价值,在上述帧内确定用于更新上述第 [0019] In order to solve the above problems, the present invention has an electronic camera in the continuous shooting image frame is determined to become the object of the tracking object in the target region and reflection display function, comprising: a tracking processing unit, quantitative wherein said object represents a first feature amount previously registered, performed in the frame Dissatisfied cable predetermined range, the image in the area as the search result obtained Dissatisfied, calculates an image and the object a first evaluation value of the degree of matching between, based on the first evaluation value calculated, is estimated to determine the presence of the object image of a target area; update area calculation unit, using the first feature different, the second feature amount quantitatively indicating characteristics of the object performs Dissatisfied cable in said frame preset range, in the area of ​​the image as the search result obtained Dissatisfied, calculates an image of the target object and second evaluation value of the degree of coincidence between, based on the second evaluation value calculated, is determined within the frame for updating the first 一特征量的更新区域;更新判断部,通过调查在上述追踪处理部中计算出的上述第一评价值和在上述更新区域计算部中计算出的上述第二评价值中的至少ー个是否满足预先设定的条件,来判断是否更新上述第一特征量;登记特征量更新部,在由上述更新判断部判断为更新上述第一特征量的情况下,用在上述更新区域内提取的新的第一特征量更新上述第一特征量;以及追踪结果描绘部,在由上述更新判断部判断为不更新上述第一特征量的情况下,将在上述追踪处理部中确定的上述目标区域确定为上述对象区域,并描绘有关上述目标区域的信息,在由上述更新判断部判断为更新上述第一特征量的情况下,将在上述更新区域计算部中确定的上述更新区域确定为上述对象区域,并描绘有关上述更新区域的信息,上述追踪处理部在预先登记的上述第一特征量 A feature amount of the update region; update determination unit, by investigating at least ー calculated in the above-described tracking processing unit in the first evaluation value and the second evaluation value calculated in the update area calculation unit is satisfied in a predetermined condition, determines whether to update the first feature amount; registered feature amount updating unit, is determined by the determination unit is a case where updating to update the first feature amount, using the extracted within the update area of ​​the new a first feature amount updating the first feature amount; and a tracking result drawing unit, in a case where the above-described update determination unit determines not to update the first feature, the above-mentioned target region determined in said tracking processing unit is determined as the target region and drawing target region on the above information, at by the update determination unit determines to update the first feature of the case, the update area determined in said update area calculation unit is determined as the target area, and drawing information about the update area, said tracking processing unit in the first feature amounts registered in advance 更新的情况下,使用更新后的第一特征量确定新的帧内的新目标区域。 The case of updating the first feature amount is determined using the updated new frame a new target area.

[0020] 这样,在电子摄像机中,在跟踪物体时,能利用追踪物体的特征以外的其他特征或位置进行物体的定位。 [0020] Thus, in the electronic camera, the tracking object, the object can be positioned using other features or characteristics of the object location tracking outside. 然后,能进行是时常进行更新还是在一定条件下进行更新的判断,具有減少了使用错误地方的更新的效果。 Then, it is able to be determined or updated frequently updated under certain conditions, have reduced local update the wrong effect.

[0021] 此外,也可以是,上述追踪处理部使用对象物的顔色信息作为上述第一特征量,来确定被推定为存在上述对象物的图像的目标区域,上述更新区域计算部使用对象物的形状信息作为上述第二特征量,来确定上述更新区域。 [0021] Further, it may be that the above-described tracking processing unit using color information of the object as the first feature quantity to determine the target area estimated to include the image of the target object, the update area calculation unit using the object shape information as the second feature quantity to determine the update area.

[0022] 这样,即使是对象物体的样子产生变化的情况下也能进行稳定的跟踪。 [0022] Thus, even when the target object appearance changes can produce stable tracking.

[0023] 另外,也可以是,上述更新判断部在上述更新区域计算部中计算出的上述第二评价值大于在上述追踪处理部中计算出的上述第一评价值吋,或者,在上述更新区域计算部中计算出的上述第二评价值大于预先设定的第一阈值时,判断为更新上述第一特征量。 [0023] Further, it may be that the above-described update determination section calculated in the update area calculation unit and the second evaluation value is greater than the first evaluation value calculated in the above-inch tracking processing unit, or, in the update when the area calculation unit calculates the second evaluation value is larger than a first predetermined threshold value, it is determined to update the first feature.

[0024] 这样,由于在更新区域探测结果的评价值超过追踪结果的评价值时,或者在更新区域探测结果大于第一阈值的情况下,更新第一特征量,因此,不需要毎次都进行更新,能够削减处理量。 [0024] Thus, due to the update area detection result of the evaluation value exceeds the value of the evaluation result of the tracking, or in the case of the update area detection result is greater than a first threshold value, update the first feature amount, it is not necessary for all times every update, can reduce the amount of processing.

[0025] 此外,也可以是,上述对象物的图像的每个象素的平均彩度越高,上述更新判断部对上述第一阈值设定越大的值,上述对象的平均彩度越低,上述更新判断部对上述第一阈值设定越小的值,上述更新判断部在上述更新区域计算部中计算出的上述第二评价值大于在上述追踪处理部中计算出的上述第一评价值时,或者,在上述更新区域计算部中计算出的上述第二评价值大于预先设定的第一阈值吋,判断为更新上述第一特征量,上述追踪处理部使用对象物的顔色信息作为上述第一特征量,来确定被推定为存在上述对象物的图像的目标区域,上述更新区域计算部使用对象物的形状信息作为上述第二特征量,来确定上述更新区域。 [0025] It is also possible that the higher the average saturation of each pixel of the image of the object, and the update determination unit sets a larger value to the first threshold value, the lower the average saturation of the target the first assessment calculated in the above-described tracking processing unit and the update determination unit to set a smaller value of the first threshold value, and the update determination unit calculated in the update area calculation unit is greater than the second evaluation value when the value, or calculated in the update area calculation unit and the second evaluation value is larger than a first predetermined threshold value inch, it is determined to update the first feature, the above-described tracking processing using color information of the object unit as the first feature quantity to determine the target area estimated image of the object exists, and the update area calculation unit using the shape information of the object as the second feature quantity to determine the update area.

[0026] 这样,追踪的容易度根据追踪对象物的彩度值而改变。 [0026] Thus, the ease of tracking varies according to the saturation value of the object to be tracked. 即,彩度高是指颜色鮮明,表示更正确地进行基于第一特征量的追踪。 That is, high color refers to the color bright, represent more accurately track based on the first feature amount. 因此,通过在彩度高时较大地设定第一阈值,在彩度低时较小地设定第一阈值,能够决定是否应该频繁地更新第一特征量。 Thus, by setting a larger first threshold value when the color is high, the first threshold value is set smaller at low saturation can be determined whether to update the first feature amount frequently. 从而,能根据对象物灵活地设定阈值,能够提闻追踪精度。 Thereby, a threshold value can be set flexibly according to the object, the tracking accuracy can be enhanced smell.

[0027] 此外,也可以是,上述更新判断部还在上述追踪处理部中确定的上述目标区域与在上述更新区域计算部中确定的上述更新区域之间的距离成为预先设定的第二阈值以上时,判断为进行更新。 [0027] It is also possible that the distance between the target region and the update determination unit also determines the above-described tracking processing unit and the update area determined in said update area calculation unit becomes a second threshold value set in advance or more, it is determined to update.

[0028] 这样,能够在追踪处理部的追踪处理结果和更新区域计算部的更新区域探测结果的位置有很大偏移的情况下,更新第一特征量,因此,即使在追踪失败了的情况下,也能使用更新后的第一特征量恢复追踪处理,追踪性能稳定。 Case [0028] In this manner, a large shift in position of the tracking result of the update area detection processing result and the update area calculation unit tracking processing unit case, to update the first feature amount, and therefore, even if the tracking fails under, the first feature amount can be used to update the tracking process recovery, stable tracking performance.

[0029] 此外,也可以是,上述电子摄像机还具有判断是否不能继续上述对象物的追踪的结束判断部,上述结束判断部在上述第一评价值和上述第二评价值双方都小于预先设定的第三阈值时,判断为不能继续上述对象的追踪。 [0029] Further, it may be that the electronic camera determines whether or not to continue further with termination determination unit tracking of the object, said determination end portion of the first evaluation value and the second evaluation value is both less than the preset when the third threshold value, it is determined that the tracking of the target can not be continued.

[0030] 这样,能够在对象物的追踪成为不能继续的情况下自动地结束追踪处理,因此,能够不浪费用户的时间地終止处理。 [0030] Thus, it is possible to automatically be ended tracking processing can not be continued in a case where the object to be tracked, it is possible without wasting the user's time to terminate the processing.

[0031] 此外,也可以是,上述追踪结果描绘部在判断为不能继续上述对象的追踪时,结束目标区域和更新区域的描绘。 [0031] In addition, it can be said tracking result drawing unit when it is determined not to continue tracking the object, drawing the end of the target area and the update area.

[0032] 这样,由于不显示多余的跟踪处理结果,因此,不使用户感到不适。 [0032] Thus, since no redundant tracking processing result display, therefore, it does not cause discomfort to the user.

[0033] 另外,也可以是,上述追踪结果描绘部在判断为不能继续上述对象的追踪时,描绘对用户提示不能追踪的图像。 [0033] Further, it may be that said tracking result drawing unit when it is determined not to continue tracking the object can not be tracked prompts the user drawing an image.

[0034] 这样,能够让用户知道跟踪已经结束了,因此,用户能够转为是否再次进行跟踪的行动。 [0034] In this way, allowing users to know the track has ended, so users can keep track of whether to action again.

[0035] 此外,也可以是,上述追踪结果描绘部在判断为不能继续上述对象的追踪时,描绘对用户催促再次设定初始位置的图像。 [0035] In addition, it can be said tracking result drawing unit when it is determined not to continue tracking the object, prompting the user to set an image depicting the initial position again.

[0036] 这样,能够催促用户进行初始设定,因此,即使是不熟练的使用者,也能够容易地理解到应该再次设定。 [0036] Thus, the user can be prompted initially set, so even an inexperienced user can be easily understood to be set again.

[0037] 另外,也可以是,上述追踪结果描绘部在判断为不能继续上述对象的追踪时,通过新的脸检测进行初始位置设定。 [0037] Further, it may be that said tracking result drawing unit is not to continue tracking the object, the initial position setting by the new face detection is determined.

[0038] 这样,在将人物作为对象的情况下,能够自动地对脸的部分重新设定初始位置。 [0038] Thus, as in the case where the character object, on the part of the face can automatically reset the initial position.

[0039] 此外,也可以是,上述电子摄像机还具有:脸检测电路,在各帧内的图像内检测脸;和脸朝向识别电路,识别由上述脸检测电路检测到的脸的朝向,在由上述脸朝向识别电路识别为作为上述对象物的脸朝向侧面时,在上述更新区域计算部中计算出作为朝向正侧面的脸的区域的更新基准区域之后,上述更新区域计算部根据脸部分与头发部分的预先指定的位置关系,计算上述更新基准区域内的上述更新区域。 [0039] Further, it may be that the electronic camera further comprising: a face detection circuit which detects a face in each frame of the image; and a facing direction recognition circuit recognizes a face detected by the face direction detecting circuit, by the above-described face orientation recognition circuit recognizes as the face of the object toward the side surface, after calculating a toward the positive side of the face of updating the reference region area in the update area calculation unit, the update area calculation unit based on the face portion and the hair the positional relationship specified in advance portions, calculates the updated area in the update reference area.

[0040] 这样就能预先进行各种各样的位置关系中的更新,从而追踪精度提高,该各种各样的位置关系中的更新包括:按照更新区域探測结果改变应该更新的地方;或者例如按照脸检测的结果的脸朝向改变应该更新的地方;或者按照脸检测的尺寸,使用更大的尺寸进行更新;或者利用包括衣服这样的形式进行更新等。 [0040] Such advance can be updated in a variety of positional relationship, thereby to improve tracking accuracy, the positional relationship of the various updating comprises: changing the place should be updated with the update area detection result; or e.g. the results of face detection in accordance with the face orientation changing the place to be updated; or according to the size of the face detection, using a larger size of the update; or by including such forms garments updates.

[0041] 这样,由于能跟踪脸,因此,能进行与脸认证等的结合。 [0041] Thus, since the track face can, therefore, face authentication can be performed in conjunction with the like.

[0042] 此外,这样,在例如如圆形状这样地即使旋转外观也不改变的情况下,能够稳定地跟踪跟踪物体。 [0042] Furthermore, such, for example, as in the case of circular shape in this way does not change appearance even when rotating, it is possible to stably track the tracking object.

[0043] 此外,也可以是,上述电子摄像机还具有摄像机控制部,该摄像机控制部根据在上述追踪处理部中确定的目标区域和在上述更新区域计算部中确定的上述更新区域中的某个,变更用于调整上述电子摄像机的动作的摄像机參数。 [0043] Further, it may be that the electronic camera further comprises a camera control unit, the camera control unit determines a target area in said tracking processing unit and the update area determined in said update area calculation unit in accordance with , change the operation of the electronic camera is used to adjust the camera parameters.

[0044] 这样,由于能控制摄像机參数,因此,能够进行与对象相应的摄像机參数设定。 [0044] Thus, since the camera parameters can be controlled, and therefore, it can be performed with the object corresponding camera setting parameters.

[0045] 也可以是,上述摄像机控制部根据上述摄像机參数控制上述电子摄像机的壳体和云台的至少一方的动作,进行控制,使被确定的上述对象物的全体或一部分配合帧内的规定位置和尺寸。 [0045] may be the camera control unit controls the operation based on the camera parameter of at least one of the housing of the electronic camera and PTZ control, so that all or part of the object is determined to fit within the frame specified position and size. [0046] 这样,通过控制摄像机,就能向对象物的适合位置控制摄像机。 [0046] Thus, by controlling the camera, the camera can be controlled to a suitable position of the object.

[0047] 也可以是,上述电子摄像机还具有目标区域初始设定部,该目标区域初始设定部根据来自用户的输入,或者使用预先设定的方法来决定上述目标区域的初始位置。 [0047] may be, the electronic camera further comprising a target area initial setting unit, the target area initial setting unit determines the initial position of the target region based on input from a user, or a method of using the previously set.

[0048] 这样就能设定目标区域的初始位置,能够决定跟踪的初始位置。 [0048] This allows setting an initial position of the target area, it is possible to determine the initial position of the track.

[0049] 也可以是,上述目标区域初始设定部将人物或脸的某一方的检测位置决定为初始位置。 [0049] may be the target area initial setting unit detects the position of either the person or the face is determined as an initial position.

[0050] 这样,能够将人物或脸位置作为初始位置,能够自动地进行人物跟踪的初始设定。 [0050] In this manner, the person or the face position as the initial position, the initial setting can be performed automatically tracking people.

[0051] 也可以是,上述目标区域初始设定部将由AF自动对焦功能合焦的地方决定为初始位置。 [0051] may be the target area initial setting unit by the local autofocus AF focus is determined as an initial position.

[0052] 这样就能将由AF功能合焦的区域作为初始设定来进行跟踪,与AF功能的联动变得容易。 [0052] This will focus by the area of ​​the AF function as the initial setting to track, in conjunction with the AF function is facilitated.

[0053] 再有,本发明不仅能够实现为装置,还能够实现为以构成该装置的处理手段为步骤的方法,或者实现为让计算机执行这些步骤的程序,或者实现为记录了该程序的计算机可读CD-ROM等记录介质,或者实现为示出该程序的信息、数据或信号。 [0053] Further, the present invention not only as an apparatus, but also can be implemented as to constitute a processing means of the apparatus for the steps of the method, or as a computer to execute the steps of the program, or as a record of the program computer readable recording medium like CD-ROM, or as illustrating the program information, or data signals. 然后,这些程序、信息、数据和信号也可以经由因特网等通信网络进行通信。 Then, these program, information, data and signals may also communicate via a communication network such as the Internet.

[0054] 发明效果: [0054] Effect of the Invention:

[0055] 根据本发明,由于即使在对象物体的样子变化了的情况下也能够进行对象追踪,因此,即使在物体朝向后面这样的情况下也能够追踪,能继续显示框,进行利用AE、AF和摄像机控制的成帧功能和自动摄影。 [0055] According to the present invention, even when the appearance of the target object is changed it is possible to perform object tracking, and therefore, even if the object toward the lower case of such a back can be tracked, can continue to display frame, for use AE, AF framing and camera control and automatic photography.

附图说明 BRIEF DESCRIPTION

[0056] 图I是现有的脸检测处理的流程图。 [0056] Figure I is a flowchart of a conventional face detection process.

[0057] 图2是现有的根据图像的脸检测处理的说明图。 [0057] FIG. 2 is an explanatory view of a conventional face detection process image.

[0058] 图3(a)〜(e)是现有的邻接差分滤波器的说明图。 [0058] FIG. 3 (a) ~ (e) is an explanatory view of a conventional adjacent difference filter.

[0059] 图4是本发明的实施方式1、2中的图像处理装置的方框图。 [0059] FIG. 4 is a block diagram of the image processing apparatus 1 in the embodiment of the present invention.

[0060] 图5(a) (b)是现有方法中的失败跟踪的例子的说明图。 [0060] FIG. 5 (a) (b) is an example of a conventional method of tracking failure described in FIG.

[0061] 图6(a) (b)是色直方图搜索的说明图。 [0061] FIG. 6 (a) (b) is an explanatory view of a color histogram search.

[0062] 图7(a) (b)是色直方图的类似度计算的说明图。 [0062] FIG. 7 (a) (b) is an explanatory view of a color histogram calculated degree of similarity.

[0063] 图8(a) (b)是侧脸的參考图像区域的说明图。 [0063] FIG. 8 (a) (b) is an explanatory view showing a side face of the reference image region.

[0064] 图9是实施方式I的流程图。 [0064] FIG. 9 is a flowchart of embodiments of formula I.

[0065] 图10(a)〜(C)是跟踪特征量与更新区域探测结果的关系图。 [0065] FIG. 10 (a) ~ (C) is a graph tracking the feature amount of the update area detection result.

[0066] 图11是实施方式2的流程图。 [0066] FIG. 11 is a flowchart of the second embodiment.

[0067] 符号说明 [0067] Description of Symbols

[0068] 100图像处理装置(电子摄像机) [0068] 100 image processing apparatus (electronic camera)

[0069] 101 透镜 [0069] 101 lens

[0070] 102 快门 [0070] The shutter 102

[0071] 103 摄像元件 [0071] 103 imaging device

[0072] 104 AD 变换器 [0072] 104 AD converter

[0073] 105 定时发生电路[0074] 106 DA 变换器 [0073] The timing generating circuit 105 [0074] 106 DA converter

[0075] 107 存储器控制电路 [0075] The memory control circuit 107

[0076] 108 系统控制电路 [0076] The system control circuit 108

[0077] 109 图像处理电路 [0077] The image processing circuit 109

[0078] 110 图像显示存储器[0079] 111 存储器 [0078] The image display memory 110 [0079] 111 memory

[0080] 112 调整尺寸电路 [0080] The resizing circuit 112

[0081] 113 闪光灯 [0081] 113 flash

[0082] 114 测距控制部 [0082] The ranging control section 114

[0083] 115 缩放控制部 [0083] 115 Zoom control unit

[0084] 116 屏障控制部 [0084] The barrier control unit 116

[0085] 117 保护部 [0085] The protecting portion 117

[0086] 118 存储器 [0086] The memory 118

[0087] 119 显示部 [0087] The display unit 119

[0088] 120 非易失性存储器 [0088] The nonvolatile memory 120

[0089] 121 模式转盘 [0089] The mode dial 121

[0090] 123 快门开关 [0090] The shutter switch 123

[0091] 124 记录部 [0091] 124 recording unit

[0092] 125 电源控制部 [0092] Power control section 125

[0093] 126 电源部 [0093] The power supply unit 126

[0094] 127 连接器 [0094] Connector 127

[0095] 128 电源部 [0095] The power supply unit 128

[0096] 129 接ロ [0096] 129 connected ro

[0097] 130 接ロ [0097] 130 connected ro

[0098] 131 连接器 [0098] Connector 131

[0099] 132 连接器 [0099] Connector 132

[0100] 133 光学取景器 [0100] The optical viewfinder 133

[0101] 134 通信部 [0101] communication unit 134

[0102] 135 天线 [0102] Antenna 135

[0103] 136 初始特征量提取部 [0103] The initial feature amount extraction unit 136

[0104] 137 追踪处理部 [0104] Tracking processing unit 137

[0105] 138 追踪开头帧指定电路 [0105] Tracking the beginning of the frame designation circuit 138

[0106] 139 更新区域探测电路 [0106] 139 Update area detection circuit

[0107] 140 更新判断电路 [0107] 140 Update determination circuit

[0108] 141 结束判断电路 [0108] End Analyzing circuit 141

[0109] 142 位置修正电路 [0109] 142 position correction circuit

[0110] 143 登记特征量更新电路 [0110] registered feature amount updating circuit 143

[0111] 144 追踪结果描绘电路 [0111] tracking result drawing circuit 144

[0112] 145 摄像机控制电路[0113] 146 脸检测电路 [0112] The camera control circuit 145 [0113] Face detection circuit 146

[0114] 147 脸朝向识别电路 [0114] facing direction recognition circuit 147

[0115] 148 图像显示部 [0115] The image display unit 148

[0116] 149 曝光控制部 [0116] The exposure control unit 149

[0117] 200 帧(a)中的跟踪结果 In [0117] 200 (a) tracking result

[0118] 201 帧(b)中的跟踪结果 In [0118] 201 (b) tracking result

[0119] 202 帧(b)中的正确位置 In [0119] 202 (b) correct position

[0120] 301 搜索区域 [0120] 301 search area

[0121] 302 追踪窗 [0121] 302 track window

[0122] 303 追踪窗区域的色直方图 [0122] 303 tracking window area color histogram

[0123] 401 參考图像的色直方图 [0123] 401 color histogram of the reference image

[0124] 402 输入图像的色直方图 [0124] 402 input color image histogram

[0125] 501 脸检测区域 [0125] Face detection area 501

[0126] 502 參考区域 [0126] 502 reference region

[0127] S601 输入图像步骤 [0127] S601 step input image

[0128] S602 脸检测步骤 [0128] S602 Face detection step

[0129] S603 脸朝向识别步骤 [0129] S603 step face orientation identification

[0130] S604 初始登记特征提取步骤 [0130] S604 Initial registered feature extraction step

[0131] S605 追踪处理步骤 [0131] S605 Tracking processing step

[0132] S606 脸检测步骤 [0132] S606 Face detection step

[0133] S607 脸检测判断步骤 [0133] S607 Face detection determination step

[0134] S608 初始登记特征更新步骤 [0134] S608 step of updating the initial registered feature

[0135] S609 跟踪框显示部•摄像机控制步骤 [0135] S609 tracking box displays the camera control unit • step

[0136] S610 追踪评价值比较步骤 [0136] S610 Tracking evaluation value comparison step

[0137] S611 追踪结束步骤 [0137] S611 track completion of step

[0138] S612 初始登记特征量更新步骤 [0138] S612 Initial registered feature amount updating step

[0139] S801 输入图像步骤 [0139] S801 step input image

[0140] S802 脸检测步骤 [0140] S802 Face detection step

[0141] S803 初始登记特征提取步骤 [0141] S803 Initial registered feature extraction step

[0142] S804 追踪处理步骤 [0142] S804 Tracking processing step

[0143] S805 圆检测步骤 [0143] S805 circle detection step

[0144] S806 追踪结束判断步骤 [0144] S806 Tracking termination determination step

[0145] S807 特征量更新判断步骤 [0145] S807 feature amount update determination step

[0146] S808 特征量更新步骤 [0146] S808 feature amount updating step

[0147] S809 跟踪框显示部•摄像机控制步骤 [0147] S809 tracking box displays the camera control unit • step

[0148] S810 追踪结束步骤 [0148] S810 track completion of step

[0149] 900 部分图像切出部 [0149] The image cut-out unit section 900

[0150] 901 特征量评价部I [0150] feature amount evaluation unit 901 I

[0151] 902 特征量评价部2[0152] 903 特征量评价部3 [0151] feature amount evaluation unit 2 902 [0152] 903 feature amount evaluation unit 3

[0153] 1000 部分图像 [0153] 1000 parts of the image

[0154] 1001 输入图像 [0154] 1001 of the input image

具体实施方式 Detailed ways

[0155](实施方式I) [0155] (Embodiment I)

[0156] 在本实施方式I中,对通过并用利用了脸检测和顔色信息的追踪,还能够进行仅在脸检测中不能进行的朝向后面的头部的追踪的图像处理装置进行叙述。 [0156] In the present embodiment, I, and by using for the face detection and tracking using the color information, the image processing apparatus is also capable of tracking only the head toward the back can not be performed in the face detection will be described. 即,实施方式I的图像处理装置(电子摄像机)使用现有的脸检测和脸朝向识别,确定会存在成为跟踪摄影对象的对象物的目标区域。 That is, embodiments of formula I of the image processing apparatus (electronic camera) using an existing face detection and face orientation identification, there will be determined photographic subject tracking target region of the object. 然后,毎次对每个帧确定目标区域吋,都提取对象物的顔色的特征量进行更新。 Then, every time a target area is determined inch for every frame, extracts the color feature amount of the object to be updated. 在无法进行脸检测的情况下,使用更新后的对象物的顔色特征确定目标区域,更新顔色的登记特征量。 In a case where face detection is not possible, using the updated color feature of the object to determine a target area, updating the color feature amount registration.

[0157] 图4是示出本发明的实施方式I的电子摄像机的结构的图。 [0157] FIG. 4 is a diagram showing a configuration of the electronic camera according to an embodiment of the present invention I. 在图4中,100是电子摄像机。 In FIG. 4, 100 is an electronic camera. 101是透镜,102是具有光圈功能的快门,103是将光学像变换成电信号的摄像元件,104是将摄像元件103的模拟信号输出变换成数字信号的AD变换器(Analog Digital 101 is a lens, a shutter 102 having a diaphragm function, 103 is an optical image pickup element is converted into an electric signal, 104 is an analog signal output from the imaging element 103 into a digital signal in an AD converter (Analog Digital

しonverterノ。 Shi onverter Techno.

[0158] 定时发生电路105向摄像元件103、AD变换器104、DA变换器(Digital AnalogConverter) 106供给时钟信号和控制信号。 [0158] Circuit 105 103, AD converter 104, DA converter (Digital AnalogConverter) 106 is supplied a clock signal and a control signal to the timing of the imaging device occurs. 由存储器控制电路107和系统控制电路108控制定时发生电路105。 Control circuit 107 and the system control circuit 108 controls the timing generation circuit 105 by the memory.

[0159] 图像处理电路109对来自AD变换器104的数据或来自存储器控制电路107的数据进行规定的图像插补处理和顔色变换处理。 Image interpolation processing and color conversion processing [0159] The image processing circuit 109 to the data from the AD converter 104 or data from the memory control circuit 107 performs predetermined.

[0160] 此外,在图像处理电路109中,使用摄像到的图像数据进行规定的运算处理,系统控制电路108根据得到的运算结果,对曝光控制部149和测距控制部114进行控制。 [0160] Further, in the image processing circuit 109, a pickup image data to predetermined arithmetic processing, the system control circuit 108 based on the calculation result, the exposure control unit 149 and the ranging control unit 114 controls.

[0161] 存储器控制电路107控制AD变换器104、定时发生电路105、图像处理电路109、图像显示存储器110、DA变换器106、存储器111和调整尺寸电路112。 [0161] The memory control circuit 107 controls the AD converter 104, the timing generating circuit 105, the image processing circuit 109, image display memory 110, DA converter 106, a memory 111, and resize circuit 112.

[0162] 将AD变换器104的数据经由图像处理电路109和存储器控制电路107,或者将AD变换器104直接经由存储器控制电路107写入到图像显示存储器110或存储器111中。 [0162] The data of the AD converter 104 via the image processing control circuit 107 and memory circuit 109, the AD converter 104 or directly via the memory control circuit 107 is written to the image display memory 110 or the memory 111.

[0163] 110是图像显示存储器,106是DA变换器,148是由TFT (Thin Film Transistor :薄膜晶体管)-!XD(Liquid Crystal Display :液晶显示器)等构成的图像显示部,被写入到图像显示存储器110中的显示用的图像数据经由DA变换器106后,由图像显示部148进行显示。 [0163] 110 is an image display memory, 106 is the DA converter, 148 is a TFT (Thin Film Transistor: TFT) - XD!: An image display unit configured (Liquid Crystal Display The liquid crystal display) or the like, is written in the image displaying the image data in the display memory 110 via the DA converter 106, the image displayed by the display unit 148.

[0164] 调整尺寸电路112根据摄影到的动态图像的帧图像,生成作为低分辨率图像的调整尺寸图像。 [0164] The resizing circuit 112 to the frame image of moving image photography, image resizing generates low-resolution image. 在要使用与摄像元件103的象素数不同的象素数(尺寸)向记录介质中记录图像数据的情况下利用调整尺寸电路112。 Using the resizing circuit 112 to be used in the case where the image pickup element 103 is different from the number of number of pixels (size) of the pixel image data is recorded to the recording medium.

[0165] 此外,图像显示部148因为象素数与摄像元件103相比相当小,因此,也用于生成在图像显示部148中进行显示时的显示用图像。 [0165] Further, since the image display unit 148 with the number of pixels of the image pickup element 103 is quite small compared, and for generating a display is performed on the image display unit 148 in the image. 构成为根据用途,从规定的多个分辨率中选择调整尺寸图像的分辨率。 Use according to configuration, the image size selected from a plurality of adjustment resolution as the predetermined resolution. 调整尺寸电路112读入存储器111中所保存的图像进行调整尺寸处理,将结束了处理的数据写入到存储器111中。 Resizing circuit 112 reads the memory 111 stored the image resizing process, the end of the writing process to the data memory 111.

[0166] 曝光控制部149控制具有光圈功能的快门102,通过与闪光灯113联动还具有闪光灯调光功能。 [0166] The exposure control unit 149 controls the shutter 102 having a diaphragm function, through the linkage 113 further includes a flash flash dimming function.

[0167] 测距控制部114控制透镜101的聚焦。 [0167] ranging control section 114 controls the focusing lens 101. 缩放控制部115控制透镜101的变焦距。 Zoom control unit 115 controls the zoom lens 101. 屏障控制部116控制作为屏障的保护部117的动作。 Barrier control unit 116 controls the operation of the protection as a barrier portion 117.

[0168] 闪光灯113还具有AF辅助光的投光功能和闪光灯调光功能。 [0168] The flash 113 also has a floodlight function of AF auxiliary light and the flash light control function.

[0169] 系统控制电路108控制整个图像处理装置(电子摄像机)100,存储器118记录系统控制电路108的动作用的常量、变量、程序等。 [0169] The system control circuit 108 controls the entire image processing device (electronic camera) 100, a system memory 118 recording operation control circuit 108 with constants, variables, programs and the like.

[0170] 显示部119是按照系统控制电路108中的程序的执行,使用文字、图像、声音等显示动作状态或信息等的液晶显示装置、扬声器等的显示部等。 [0170] the display unit 119 is executing a program in accordance with the system control circuit 108, using text, images, sounds and other display devices, such as a speaker and the like such as liquid crystal display unit for displaying information or the operation state. 显示部119在图像处理装置100的操作部附近的容易视觉辨认的装置上设置在单个或多个地方上,例如,由LCD或LED (Light Emitting Diode :发光二极管)、发声元件等的组合构成。 A display unit 119 visually recognized easily on a device in the vicinity of the operation unit of the image processing apparatus 100 is provided on a single or a plurality of places, e.g., by an LCD or LED (Light Emitting Diode: LED), a combination of sound and the like constituting the element.

[0171] 非易失性存储器120是能电气性地证明•记录的存储器,例如使用EEPROM等。 [0171] The nonvolatile memory 120 is a memory capable of electrically demonstrated • recording, for example, an EEPROM.

[0172] 模式转盘121能够切换设定自动摄影模式、摄影模式、全景摄影模式、未加工模式等的各功能模式。 [0172] mode switching dial 121 can set the automatic shooting mode, shooting mode, panoramic shooting mode, the processing mode is not the respective function modes.

[0173] 快门开关123通过快门开关SW1,在未图示的快门按钮的操作途中变为“开”,指示AF处理、AE处理、AWB (Auto White Balance :自动白平衡)处理等的动作开始。 [0173] The shutter switch 123 by the shutter switch SW1, the midst of operation of a shutter button (not shown) becomes "open", indicating the AF processing, AE processing, AWB (Auto White Balance: Auto White Balance) processing operation starts.

[0174] 快门开关123通过快门开关SW2,在未图示的快门按钮的操作完了时变为“开”,指示一系列处理的动作开始,该一系列处理是,将从摄像元件103读取的信号经由AD变换器104和存储器控制电路107,向存储器111中写入图像数据的曝光处理;在图像处理电路109和存储器控制电路107中的使用运算的显像处理;从存储器111中读取图像数据,在记录部124中写入图像数据的记录处理。 [0174] The shutter switch 123 by the shutter switch SW2, when the shutter button (not shown) over an operation to "open", indicating the start of a series of operation processing, the series of processing is read from the image pickup element 103 a control signal via the AD converter 104 and memory circuit 107, the image exposure processing to write data to the memory 111; developing processing circuit 107 uses the image processing operation and a memory control circuit 109; an image read from the memory 111 data, image data written in the recording process in the recording unit 124.

[0175] 电源控制部125由电池检测电路、DC-DC变换器、切换通电的块的开关电路等构成,进行有无安装电池、电池的种类及电池残量的检测,根据检测结果和系统控制电路108的指示控制DC-DC变换器,在必要的期间向包括记录介质在内的各部分供给必要的电压。 The switching circuit block [0175] The power supply control unit 125 by the battery detection circuit, DC-DC converter, the switching power configuration, presence or absence of installing the battery, the battery type and the remaining amount of the battery detected according to the detection result and control system 108 indicates a control circuit DC-DC converter, to provide the necessary voltage to each section includes a recording medium including the necessary period of time.

[0176] 电源部128是由连接器127、碱性电池或锂电池等的一次电池或NiCd电池或NiMH电池、Li电池等的二次电池及AC适配器等126构成的电源。 [0176] 126 power supply unit 128 is constituted by a power connector 127 and the like, alkaline batteries or lithium primary battery or NiCd or NiMH batteries, Li batteries such as secondary batteries and an AC adapter.

[0177] 接ロ129和130是与存储卡和硬盘等记录介质之间的接ロ,连接器131和132进行存储卡和硬盘等记录介质与主体的连接。 [0177] 129 and 130 are connected ro ro contact between the recording medium and the memory card and hard disk, connectors 131 and 132 connected to the main body of a recording medium and a hard disk memory card. 保护部117是通过覆盖包括图像处理装置(电子摄像机)100的透镜101在内的摄像部来防止摄像部的污染和破损的屏障(Barrier)。 Protection unit 117 is used to prevent contamination and damage to the imaging portion of a barrier (Barrier) by covering the image pickup portion includes a lens 101 of the image processing apparatus (electronic camera) 100 of the inner.

[0178] 光学取景器133没有使用图像显示部148的电子取景器功能,而仅使用光学取景器就能进行摄影。 [0178] The optical viewfinder 133 without using the electronic viewfinder function of the image display unit 148, only using the optical viewfinder can be photographed.

[0179]通信部 134 具有RS232C 或USB (Universal Serial Bus :通用串行总线)、IEEE1394、调制解调器、LAN和无线通信等的各种通信功能。 [0179] The communication unit 134 has a RS232C or USB (Universal Serial Bus: Universal Serial Bus), IEEE1394, modem, LAN, and wireless communication various communication functions.

[0180] 天线135是利用通信部134将图像处理装置(电子摄像机)100与其他机器连接的连接器或无线通信情况下的天线。 [0180] The antenna 135 is an antenna 100 connected to the other machines connected to the image processing apparatus (electronic camera) or a case where a wireless communication unit 134 of the communication. [0181] 初始特征量提取部136从存储器111中保存的图像数据中提取初始特征量并写入到存储器111中。 [0181] Initial feature amount extraction unit 136 extracts an initial feature amount from the image data stored in the memory 111 and written into the memory 111. 提取初始特征量的坐标可以由用户利用触摸屏进行指定,或者根据通过按下脸检测位置或快门开关SWl而设定的AF区域等进行自动设定。 An initial feature amount coordinates can be specified using the touch screen by the user, or automatically set according to the AF area and the like, or by pressing the face detecting position and the shutter switch SWl is set.

[0182] 追踪处理部137从存储器111中读入登记特征量进行追踪处理,在存储器111中写入追踪结果(坐标数据、评价值)。 [0182] Tracking processing unit 137 reads the registered feature amount from the memory 111 in the tracking process, writing tracking result (coordinate data and evaluation value) in the memory 111. 再有,评价值是表示登记特征量与目标区域的特征量之间的类似度的值,以下,以评价值越大越准确、即一致度越高的情况为例进行说明。 Further, the evaluation value is a value of the similarity between the registered feature amount and the feature amount of the target area, the following, in order to evaluate more accurately the larger the value, the higher the degree of matching i.e., a case will be described as an example.

[0183] 追踪开头帧指定电路138判断是否是图像数据中的、在追踪中进行的开头帧,在存储器111中写入判断結果。 [0183] Tracking the beginning of the frame designation circuit 138 determines whether the image data is the beginning of the frame in the track, the result of the determination is written in the memory 111.

[0184] 更新区域探测电路139进行更新区域的探測,在存储器111中写入更新区域探測结果(坐标数据、评价值)。 [0184] update area detection circuit 139 Update area detection, writing the update area detection result (coordinate data and evaluation value) in the memory 111. 再有,关于更新区域,评价值也是表示登记特征量与更新区域的特征量之间的类似度的值,以下,以评价值越大越准确、即一致度越高的情况为例进行说明。 Further, with respect to the update area, the evaluation value is represented by the degree of similarity between a feature quantity value of the feature amount registered with the update area, the following, in order to evaluate more accurately the larger the value, the higher the degree of matching i.e., a case will be described as an example.

[0185] 更新判断电路140判断是否对存储器111中所保存的登记特征量进行更新,将判断结果写入到存储器111中。 If [0185] Update determination circuit 140 determines the feature quantity stored in registration memory 111 is updated, the determination result is written to the memory 111.

[0186] 结束判断电路141将是否结束追踪的判断结果写入到存储器111中。 [0186] The circuit 141 determines whether the end of the tracking end determination result is written in the memory 111.

[0187] 位置修正电路142根据预先保持在存储器111中的更新区域探测结果与追踪结果的位置关系信息DB,对存储器111中所保存的追踪结果进行修正。 [0187] The position correction circuit 142 in the memory 111 beforehand held in the update area detection result and the positional relationship of the tracking result information DB, on the tracking results stored in the memory 111 is corrected.

[0188] 登记特征量更新电路143在追踪结果和更新区域探測结果处于后述的关系时,使用更新区域探测结果的位置特征来更新存储器111中所记录的登记特征量。 [0188] registered feature amount updating circuit 143 Tracking result in the update area detection result and the relationship is described, the use position wherein the update area detection result to update the registered feature amount recorded in the memory 111.

[0189] 追踪结果描绘电路144为了使IXD显示已写入到存储器111中的追踪结果,对已写入在图像显示存储器110中的显示用的图像数据实施用于了解已追踪到追踪结果的位置的信息、例如追踪框、马赛克化、文字、显示颜色的变更、模糊化等的处理。 [0189] tracking result drawing circuit 144 in order to make the display IXD tracking result written into the memory 111, the display position of the image data written in the display memory 110 for understanding the embodiments have been traced to the trace result in an image information, such as tracking frame, tiling, text, display color changes, blurring and other processing.

[0190] 追踪结果描绘电路144为了使IXD显示已写入到存储器111中的追踪结果,对已写入在图像显示存储器110中的显示用的图像数据实施用于了解已追踪到追踪结果的位置或者更新区域探测结果的位置的信息、例如追踪框、马赛克化、文字、显示颜色的变更、模糊化等的处理。 [0190] tracking result drawing circuit 144 in order to make the display IXD tracking result written into the memory 111, the display position of the image data written in the display memory 110 for understanding the embodiments have been traced to the trace result in an image or update area detection result of the position information, such as tracking frame, mosaic oriented, text, display color changes, blurring processing and the like.

[0191] 摄像机控制电路145根据已写入到存储器111中的追踪结果的位置和尺寸对摄像机进行控制,使得对象物体成为图像的一定位置•尺寸(例如,使脸成为中央或者进行缩放以显示全身等)。 [0191] The camera control circuit 145 controls the camera according to the tracking result written into the memory 111 of the position and size of the target object becomes a predetermined position such that • the size of the image (e.g., making a face or be scaled to show the central body Wait).

[0192] 脸检测电路146从图像进行脸检测,在存储器111中写入脸检测结果(位置、尺寸、评价值)。 [0192] Face detection circuit 146 for detecting a face from an image, writing face detection result (a position, size, evaluation value) in the memory 111.

[0193] 脸朝向识别电路147识别由脸检测电路146检测到的脸是朝向上下左右的哪个方向,将识别结果写入到存储器111中。 [0193] facing direction recognition circuit 147 detected by the face recognition face detection circuit 146 which is towards the vertical and horizontal directions, the recognition result is written into the memory 111.

[0194] 在不具有这些136、137、138、139、140、141、142、143、144的某个电路的情况下,在系统控制电路108中,也可以利用软件处理,使用后述的流程进行追踪和更新处理。 [0194] In the case of such a circuit does not have a 136,137,138,139,140,141,142,143,144, the system control circuit 108 may be implemented by software processing, the process described below is used tracking and updating process.

[0195] 以下,使用图6和图7说明图像处理装置100的动作。 [0195] Here, using FIGS. 6 and 7 illustrate operation of the image processing apparatus 100.

[0196] 系统控制电路108执行由曝光处理和显像处理构成的摄影处理,所述曝光处理是经由摄像元件103、AD变换器104、图像处理电路109、存储器控制电路107,向存储器111中写入摄影到的图像数据,所述显像处理是使用存储器控制电路107以及根据需要使用图像处理电路109,读取已写入在存储器111中的图像数据后进行各种处理。 [0196] The system control circuit 108 performs shooting processing composed of exposure processing and developing processing, the exposure process via the image pickup element 103, AD converter 104, the image processing circuit 109, the memory control circuit 107, a write to memory 111 into photographic image data, the developing process is performed after the various processing using the memory control circuit 107 and, if necessary using the image processing circuit 109, the image data read in the memory 111 written.

[0197] 若结束了摄影处理,则系统控制电路108对已写入到存储器111中的图像数据,使用调整尺寸电路112生成用于在图像显示部148中显示摄影图像的显示用图像。 [0197] When the end of photographic processing, the system control circuit 108 has been written into image data memory 111, using the resizing circuit 112 generates a captured image displayed in the image display unit 148 display image. 系统控制电路108同样地用调整尺寸电路112将图像调整尺寸成应该输入到脸检测电路中的图像尺寸(QVGA(Quarter Video Graphics Array)等),将脸检测用图像保存在存储器111中。 The system control circuit 108 in the same manner with the resizing circuit 112 to resize the image to be inputted to the image size (QVGA (Quarter Video Graphics Array), etc.) the face detection circuit, the detected face image stored in the memory 111. [0198] 系统控制电路108对于存储器111中所保存的脸检测用图像数据,利用脸检测电路146,使用现有技术执行脸检测处理,并将处理结果保存在存储器111中。 [0198] The system control circuit 108 detects the image data stored in the memory 111 to the face by the face detection circuit 146 performs face detection using the prior art process, and the results stored in the memory 111.

[0199] 此外,系统控制电路108通过脸朝向识别电路147,利用现有技术执行脸朝向识别处理,并将处理结果保存在存储器111中。 [0199] Further, the system control circuit 108 through the facing direction recognition circuit 147, the implementation of the prior art face orientation recognition processing, and the processing result stored in the memory 111.

[0200] 在由脸检测电路146检测到了脸的情况下,初始特征量提取部136參照存储器111中所保存的脸检测和脸朝向识别处理结果进行登记特征量的提取,向存储器111中写入登记特征量。 [0200] In the case where the detection by the face detection circuit 146 to the face, the initial feature amount extraction unit 136 refers to the memory 111 saved in face detection and face orientation recognition processing result registered feature amount extraction, writing into the memory 111 registered feature amount. 例如,初始特征量提取部136根据存储器111中所保存的脸检测结果和脸朝向信息,提取脸的色直方图作为顔色的特征量,并登记在存储器111中。 For example, the initial feature amount extraction unit 111. The memory 136 toward the stored face information and face detection results, extracting a face feature quantity as a color histogram of color, and is registered in the memory 111. 然后,追踪处理部137使用由初始特征量提取部136登记在存储器111中的特征量进行追踪处理。 Then, tracking processing using the registration unit 137 by the initial feature amount extraction unit 136 in the memory 111 for the feature quantity tracking process.

[0201] 在此,使用图6对使用色直方图的登记特征量的提取和追踪处理进行说明。 [0201] Here, FIG. 6 using the color histogram feature amount extracting and tracking the registration process will be described.

[0202] 如图6(a)所示,假设从图像检测到正面脸。 [0202] FIG. 6 (a), assume that the image is detected from the front face. 这时制成该区域(參考区域)的色直方图。 In this case formed in the region (reference region) of a color histogram. 具体地说,在直方图的横轴上,从參考区域得到的各象素的HSV(Hue SaturationValue :色饱和度值)色空间的H的值(最大360)被分配在划分成20个的区域中。 More specifically, on the horizontal axis of the histogram of each pixel obtained from the reference region of HSV: the value of H (Hue SaturationValue color saturation value) color space (maximum 360) is allocated 20 is divided into regions in. 然后,分配在各个分区中的个数成为度数。 Then, the number allocated to each partition becomes degrees. 设定各象素的Hi,由Hi*18/360能够计算进入到了哪个区域。 Each pixel is set Hi, the Hi * 18/360 which can be calculated into the region. 然后,通过将各分区的度数除以參考区域的象素数来归ー化为度数的总和等于I。 Then, normalized by dividing the number of degrees of each partition of pixels of the reference region is equal to the sum of the degree ー into I. 然后,如图6(b)所示,从检测区域开始以一定范围扩展后的区域301成为搜索区域,ー边改变追踪窗302的尺寸•位置ー边进行搜索。 Then, as shown in FIG 6 (b), the region from the start to the detection region after a certain search area range extension 301 become, ー while changing the size of the tracking window 302 • ー edge position search. 追踪处理部137 —边搜索一边制成追踪窗区域的色直方图303。 Tracking processing unit 137 - Edge search color made while tracking window area of ​​the histogram 303.

[0203] 接着,根据參考区域的色直方图与追踪窗区域的色直方图计算类似值。 [0203] Next, the color histogram of the reference region and calculating the tracking window area color histogram similarity value. 使用图7说明类似度的计算方法。 A method for calculating similarity using FIG. 图7 (a)是參考图像,从此计算出的色直方图是401。 FIG. 7 (a) is the reference image from a color histogram is computed 401. 此外,图7 (b)是输入图像,402中示出搜索窗的色直方图。 Further, FIG. 7 (b) is an input image, a search window 402 shown in the color histogram. 这样,利用各个色直方图的重叠程度来计算出类似度即评价值。 Thus, to calculate the degree of similarity that is an evaluation value using the degree of overlapping of respective color histograms. 利用(数学式I)进行类似度计算。 Using (Formula I) A similar calculation.

[0204][数学式I] [0204] [Formula I]

dim dim

[0205] SJtl =〉' Min (Ri, Ii) [0205] SJtl => 'Min (Ri, Ii)

j=i j = i

[0206] 在此,Ri是參考区域的色直方图中的第i个分区的度数,在此,Ii是追踪窗区域的色直方图中的第i个分区的度数。 [0206] Here, the reference region Ri of a color histogram is the i-th degree partitions, here, is the tracking of Ii window area color histogram partition i-th degree.

[0207] 在此,i取0至20的值。 [0207] Here, i takes a value from 0 to 20. 在本例中,使用20这个值进行说明,但只要是I以上的值,可以是任意值。 In the present embodiment, a value 20 will be described, but as long as the value of I above, it may be any value. 再有,该值根据对象的复杂度而变更。 Further, the value is changed according to the complexity of the object. 在包含多个颜色的情况下,通过増大该值,用细小的分区看差別,能够提高精度。 In the case of containing a plurality of colors, by enlargement of the big value, to see the difference with a fine partition, the accuracy can be improved. 另ー方面,对于颜色数量少的物体,通过减小该值,以较大的分区看差别,能够使用很少的存储器进行追踪。ー another aspect, the object for a small number of colors, by reducing the value to see the difference the larger partition, it is possible to use less memory track.

[0208] 再有,作为參考区域的指定方法,设为脸检测位置,但这也可以是用户使用触摸屏或指示器等进行指定,也可以使用声音信息等进行指定。 [0208] Further, as a reference region specifying method, to detect the position of the face, but may be a touch screen or a user specified pointer, audio information may be used for specified. 此外,关于根据色直方图的匹配进行了说明,但也可以求取切出的图像间的模板的差异。 Further, with regard it has been described based on the color histogram matching, but can also strike the difference between the template cut out image. 此外,通过作为预处理而对这些模板进行色直方图修正,能够难以受到亮度的影响。 Further, as a pretreatment is performed by correcting the color histogram of these templates, it can be hardly affected by luminance. 作为求取差异的方法,也可以是使最小平方距离成为最小的方法或者利用归ー化相关的方法等,只要是一般的求距离的方法,可以采用任意方法。 As a method of obtaining the difference, it may be minimized to minimize the square of the distance method, or a method using normalized correlation ー the like, as long as the seek distance general method, any method may be employed. 此外,作为搜索方法,对简单地扫描区域的方法进行了叙述,但也可以是粒子过滤器等基于概率分布的捜索方法。 Further, as a search method, a method of simply scanning the area has been described, but may be a particle filter Dissatisfied search method based on probability distribution and the like. [0209] 下面,对与脸朝向相应的登记特征量的提取方法进行说明。 [0209] Next, a method for extracting the face direction corresponding registration feature amount will be described. 在正面脸的情况下,通过登记脸整体,能进行稳定的追踪。 In the case of the front face, by registering as a whole face, enabling stable track. 但是,在脸从正面朝向了侧面的情况下,在以后的帧中有朝向后面的可能性,因此,对特征量的登记位置进行研究。 However, when the face orientation of a side from the front, towards the rear there is a possibility at a later frame, and therefore, the position of the feature amount registration study. 如图8(a)所示,在侧脸的情况下,在脸检测区域501内包含肤色信息和发色信息两者。 As shown in FIG 8 (a), in the case where the side faces, comprising both the color information and hair color information in the face detection area 501. 在此,如图8(b)所示,通过将脸检测区域内的后头部区域20%左右设为參考区域502,若由脸检测处理取得发色信息,则在对象朝向了后面的情况下,也能够根据发色信息进行使用顔色的追踪。 Here, FIG. 8 (b), the reference area 502 is set by about 20% in the occipital region of the face detection area, if the information acquired by the color-face detection processing, in the case where the object toward the back of the , the track can be performed using the color information based hair color. 再有,通过按照容易朝向正面的情况和容易朝向后面的情况等来变更该比例,就能够使追踪稳定。 Further, according to the front toward the back of the case easily and readily toward a case like changed by the ratio, it is possible to stably track. [0210] 再有,在本例中,以头发进行了说明,但在穿戴有帽子和衣服等顔色变动少的衣物的情况下,本方法也有效地起作用。 [0210] Further, in this embodiment, to the hair it has been described, but in the case of hat worn clothes and less color variation of the laundry, the present methods effectively.

[0211] 此外,在检测到脸并且与追踪结果具有一定距离的情况下,更新区域探测电路139设置进行更新判断的标志,并写入到存储器111中。 [0211] Further, in a case where the face is detected and a distance from the tracking result, the update area detection circuit 139 Update determination flag settings, and written to the memory 111. 然后,利用位置修正电路142修正脸检测位置。 Then, the face detection position corrected by the position correction circuit 142.

[0212] 根据该脸检测位置提取登记特征量,利用登记特征量更新电路143更新登记特征量并写入到存储器111中。 [0212] According to the registered feature amount extraction face detection position, using the registered feature amount updating circuit 143 updates the registered feature amount into the memory 111 and written. 登记特征量的提取与上述方法同样地进行。 Registered feature amount extraction method in the same manner as described above.

[0213] 通过这样地按照朝向来变更登记特征量的提取区域,即使在从侧脸朝向了后面的情况下,也能够通过使用头发或帽子来按照朝向的变化登记适当的顔色特征,因此,即使在对象朝向了后面的情况下,也能进行稳定的追踪。 [0213] Thus in accordance with registered feature amount is changed toward the extraction region by, even when the side faces toward the back from, it is possible to register the proper color feature according to change in orientation of the hair or by using a cap, and therefore, even if in the case where the object toward the back, the tracking can be performed stably. 此外,即使在追踪处理失败了的情况下,通过在位置修正电路142中根据脸检测结果来修正位置,并更新存储器111的追踪结果,也能进行稳定的脸的追踪。 Further, even in a case where the tracking process has failed, by the position correction circuit 142 corrects the position based on the face detection result and updating the tracking result in the memory 111, can be performed stably tracked face.

[0214] 此外,脸检测存在仅检测脸内部的情況。 [0214] Further, there is a case of detecting only the inside face detection face. 于是,通过使检测结果坐标成为常数倍以包含头部,就能进行稳定的追踪。 Accordingly, the detection result by the coordinate to be constant multiple head comprising, tracking can be carried out stably.

[0215] 此外,在保持有朝向了后面的信息,而从朝后恢复到朝前时,通过变更特征量的提取区域,就能进行更稳定的追踪。 [0215] Further, in the holding information toward the back, while the recovery from the rearward to forward, by changing the feature amount extraction region, a more stable tracking can be carried out. 以下具体地说明。 The following specifically. 在朝向了后面时接着又朝向了侧面的情况下,朝向正面的可能性较高。 When the case toward a side toward the rear and then, the higher the possibility of facing the front. 于是,通过不是将參考区域502,而是将脸检测区域501的脸侧区域作为特征量提取区域,即使接着出现了脸时也能够对应。 Thus, not by reference to the area 502, but the side of the face region 501 as the face detection region feature amount extraction area, then even if there has been a corresponding face is possible. 从而,能够在从朝后转变为侧面、正面时利用肤色信息,因此,从后面向正面的恢复稳定。 Thus, it is possible to transition from the side toward the front side when using the color information, and therefore, from the front face to restore stability.

[0216] 由于在朝向了后面的情况下也能够进行追踪,因此,即使在朝后状态的情况下,也当然能够通过追踪结果描绘电路进行追踪框的显示或使追踪结果位置模糊化等的瓜(UserInterface :用户界面)控制,而且通过进行AE -AF等的控制或用摄像机控制电路145进行摄像机的控制,能进行自动成帧•自动摄影。 [0216] Since the direction of the latter situation can be tracked, and therefore, even when the state after moving, but also of course be the tracking result drawing circuit by displaying the tracking frame or to the position of the tracking result blurring like melons (UserInterface: user interface) control, and by controlling the AE -AF or the like by the camera control circuit 145 controls the camera can automatically • automatic photographing framing.

[0217] 在由程序执行了以上的处理的情况下,成为图9所示的步骤。 [0217] In the case where the program executed by the above processing, the step shown in FIG. 9 become. 摄影输入图像,对摄影到的输入数据(S601)进行脸检测处理(S602)。 Input image photography, photography input data (S601) for the face detection process (S602). 再有,脸检测使用现有的方法。 Further, the face detection using the conventional method. 接着,对脸检测出的结果进行脸朝向识别处理(S603)。 Next, the result of the face detected by the face orientation recognition processing performed (S603). 该脸朝向识别也使用现有的方法。 This facing direction recognition also uses a conventional method. 然后,从脸检测位置中提取颜色的初始登记特征(S604)。 Then, color feature extracting initial registration (S604) from the face detection position. 另外,在下ー帧中,使用顔色特征进行确定目标区域的追踪处理(S605),之后进行脸检测处理(S606)。 Further, ー in the next frame, tracking processing using the color feature (S605) to determine the target area, followed by the face detection process (S606). 如果完成脸检测(S607),则根据脸朝向识别結果,将作为追踪结果的目标区域置換成脸检测位置。 If face detection is completed (S607), the face orientation according to the recognition result, the face detection position is replaced with a target region tracking results. 在没完成脸检测的情况下,使用顔色的特征量进行追踪处理,计算出与顔色的特征量有关的追踪评价值。 In the case where not completed the face detection using the color feature amount of the tracking process, the tracking calculated evaluation value related to the color feature amount. 若追踪结果的评价值也是阈值以下,则结束追踪处理(S611)。 If the track is also a result of the evaluation value threshold value, the end of the tracking process (S611). 此外,若根据颜色特征量的追踪结果的评价值高于阈值,就从目标区域内的初始位置中提取使用色直方图表示的顔色特征,在更新了初始登记特征量之后(S612),将控制转移向跟踪框的显示和摄像机控制(S609)。 Also, if higher than the evaluation value threshold based on the tracking result of the color feature amount, using the color are extracted from an initial position within the target area represented by the color feature histogram, after the initial registered feature amount is updated (S612), the control transfer to the tracking box display and camera control (S609). 然后,在完成脸检测的情况下,根据脸朝向检测的结果,选择更新初始登记特征量的地方并更新初始登记特征量(S608)。 Then, in a case where the completion of the face detection, face orientation according to a result of detection, to select the initial registered feature amount updating place and update the initial registered feature amount (S608). 然后进行更新后的追踪结果位置即目标区域的LCD显示和摄像机控制。 Then the position of the tracking result is updated target area i.e., LCD display and camera control. 在(S609)之后,返回到追踪处理(S605)反复进行处理。 After (S609), returns to the tracking process (S605) repeats the process.

[0218](实施方式2) [0218] (Embodiment 2)

[0219] 在实施方式2中,叙述利用了象圆形状信息这样地在旋转中不变的信息和顔色信息的追踪、头部(不仅朝向正面的脸,也包含看不见脸的朝后的情況)的追踪都能进行的电子摄像机。 Situation after [0219] In the second embodiment, the use is described as a circular shape information in this manner a constant rotation and color information of the tracking head (not facing the front face, also contains invisible face towards ) can track the electronic camera. 即,在实施方式2中,取代脸检测信息而使用圆形状的信息来决定用于更新顔色特征的更新区域的位置。 That is, in the second embodiment, instead of the face detection information and the information used to determine the circular location update for updating a color feature regions.

[0220] 实施方式2的结构与图4的结构相同,因此,对动作相同的结构要素省略说明。 [0220] FIG configuration similar to Embodiment 4, therefore, the operation of the same components will be omitted.

[0221] 在实施方式2中,追踪处理也与实施方式I同样地使用色直方图匹配来计算类似度即评价值并进行追踪。 [0221] In the second embodiment, the tracking process is also used in the same manner as in I embodiment calculates the color histogram matching similarity degree evaluation value i.e. track.

[0222] 在实施方式2中也使用与实施方式I同样的方法,利用脸检测信息和来自用户的指示(触摸屏和指示器等)决定对象物的初始位置,登记初始特征量。 [0222] Also using the same method as in Embodiment I In Embodiment 2, using the face detection information and an instruction from the user (pointer and the touch screen, etc.) determines the initial position of the object, the initial registration of the feature amount.

[0223] 头部的样子不变的信息之ー是“头部是圆”。 [0223] ー same way the head of the information is "head is round." 于是,在本实施方式2中示出由更新区域探测电路139利用形状信息进行更新区域的探測的例子。 Thus, the illustrated circuit is detected by the update area detection information 139 is updated by the shape of the region in the present example of embodiment 2 of the embodiment. 除此以外,只要是对于物体的样子不变的特征,可以使用任意特征。 In addition, as long as the object look for invariant feature may be any feature.

[0224] 关于头部圆形状的检测方法进行叙述。 [0224] For circular head detecting method will be described. 在存储器111中所保存的追踪结果的附近进行边缘检測。 Edge detection in the vicinity of the tracking result stored in the memory 111. 边缘检测使用索贝尔算子(Sobel operator)等。 Sobel edge detection operator (Sobel operator) and the like. 或者,有根据帧间差分利用差分点的方法。 Alternatively, there is a method using inter-frame difference according to the difference point. 对于在此得到的点群(特征点群)实施霍夫变换,求出圆的半径和中心。 For point group (feature point group) obtained in this Hough transform, and obtains radius of the circle center. 圆的半径和中心的求法例如可以利用文献“ HeadFinder :フレ一ム間差分をべ一スにした人物追跡、馬場ら”(非专利文献I)中记载的方法来求得。 And the center of the radius of the circle can be used, for example, Document Seeking "HeadFinder: fu Ritz difference between a Rousseau wo shi ta ni su べ a trace figures, Baba ra" (Non-Patent Document I) described be determined. 关于上述文献中描述的方法简单地说明。 The method described in the above documents will be briefly described.

[0225] 可以使用(数学式2),利用中心坐标a、b和半径r的常量来决定圆。 [0225] can be used (Equation 2), using the central coordinates of the constants a, b, and the radius r of the circle is determined.

[0226][数学式2] [0226] [Mathematical Formula 2]

[0227] (x~a)2+(yb)2 = r2 [0227] (x ~ a) 2+ (yb) 2 = r2

[0228] 将它适用于霍夫变换,求得的未知数为3个,因此表决空间(voting space)成为三维的。 [0228] The Hough transform is applied to it, for the three unknowns is obtained, so the voting space (voting space) becomes three-dimensional. 由于该计算量庞大,因此实时动作困难。 Due to the huge amount of calculation, and therefore difficult to real-time action. 于是,将尺寸限定在某个一定的半径r的范围内。 Thus, the size is defined within a range of a radius r. 之后,通过使用具有宽度的多个半径内所包含的特征点群进行表决,来求出半径參数a、b。 Thereafter, vote by using the feature point group having a plurality of radii of a width included, obtains radius parameters a, b. 从而推定头部的圆形状。 Thereby estimating the circular shape of the head. 这时,在圆上包含有几个特征点群就成为表示圆形状的拟合程度的评价值。 In this case, it contains several circle becomes the feature point group represented by the evaluation value of the degree of circular shape fitting. 将该圆形状的位置、半径信息和评价值写入到存储器111中。 The position of the circular shape, radius information, and evaluation values ​​are written into the memory 111.

[0229] 在此,着眼点在干,头部评价值较低,但相对于朝向变化能够稳定地检测,而颜色信息若朝向改变则评价值降低。 [0229] Here, the focus in a dry, low head evaluation value, but changes in direction can be stably detected, and if the color information with respect to the orientation changing the evaluation value decreases. 以下叙述在由于样子变化而评价值通过颜色信息降低的情况下,使用头部圆形状的检测位置进行位置修正,通过修正登记特征量,相对于样子的变化变得稳定的追踪方法。 In the following description since the evaluation value change like the case where the color information is reduced by using a circular detected position of the head position correction, by correcting the registered feature amount with respect to the state of change becomes stable tracking method.

[0230] 更新判断电路140从存储器111中读入基于颜色的追踪结果的评价值和更新区域探测结果的评价值,在追踪结果的评价值降低并低于阈值thl时,进行更新的判断,在存储器111中写入判断結果。 [0230] The update determination circuit 140 reads the evaluation value of the update area detection result of the evaluation value based on the tracking result of the color from the memory 111, reduced and lower than the threshold thl the evaluation value of the tracking result, the determination updating, in the memory 111 writes the determination result. 图10中示出这时的评价值与时间轴的关系。 FIG 10 shows the relationship at this time and the evaluation value of the timeline.

[0231] 图10(a)在更新区域探测结果的评价值超过了追踪结果的评价值时进行更新。 When the update [0231] FIG. 10 (a) the evaluation value exceeds the update area detection result of the evaluation value of the tracking result. 在图10(a)、(b)和(c)中,实线表示追踪结果的评价值,虚线表示更新区域探测结果的评价值。 In FIG 10 (a), (b) and (c), a solid line indicates the evaluation value of the tracking result, the broken line indicates that the update area detection result of the evaluation value.

[0232][数学式3] [0232] [Formula 3]

[0233] 追踪结果评价值< 更新区域探测结果评价值 [0233] Tracking result evaluation value <update area detection result evaluation value

[0234] 该情况下,在是因为对象物体的样子变化且评价值降低而产生的现象并且引起追踪失败吋,能够进行更新从而恢复。 [0234] In this case, because the target object changes appearance, and the evaluation value decreases, and a phenomenon is generated due to tracking failure inch, it can be updated to recover.

[0235] 图10(b)是更新区域探测结果的评价值变得大于阈值thl的情况下进行更新的图。 [0235] FIG. 10 (b) is the case where the result of the update area detection evaluation value becomes larger than the threshold thl updated FIG.

[0236][数学式4] [0237] thl <更新区域探测结果评价值 [0236] [Mathematical Formula 4] [0237] thl <update area detection result of the evaluation value

[0238] 该情况是保证了更新区域探测结果中的稳定性吋。 [0238] This case is ensuring the stability of the update area detection inch results. 该情况下,在更新区域探测结果的位置修正追踪结果的位置。 In this case, the correction of the position of the tracking result in the location update area detection result. 从而,在能够使用更新区域探测结果充分地进行追踪的情况下,利用图10(b)的方式,能够在不是这样的物体中顺利地选择出是哪个,因此能构筑稳定的系统。 Thus, in the case of the update area detection result can be used sufficiently for tracking, using FIG. 10 (b) can be smoothly selected in which the object not the case, it is possible to build a stable system.

[0239] 另外,关于更新的定时进行说明。 [0239] In addition, regarding the timing of updating is described. 顔色评价值的变化根据登记的顔色或对象物的顔色的复杂度而不同。 The change in color evaluation value of the color complex color or object registration differs. 例如,在平均彩度高的情况下,根据颜色的追踪的评价值就高,评价值具有与样子变化一起慢慢降低的趋势。 For example, in the case of high average color based on the evaluation value of the tracking color is high, having the evaluation value with changes in appearance tends to decrease gradually.

[0240] 因此,在平均彩度高的物体中,能降低圆形检测的频率,从而关系到整体处理量的削減。 [0240] Thus, the average color of the object is high, to reduce the frequency of circle detection, thus related to the overall reduction of processing amount. 此外,在平均彩度低的情况下,利用色直方图的追踪困难,具有评价值急剧降低的趋势。 In addition, at low average saturation, use the color histogram of tracking difficulties, the evaluation value has sharply decreased. 因此,在平均彩度低的情况下,通过频繁地进行圆检测,就能进行稳定的追踪。 Thus, at low average saturation, frequently by circle detection, tracking can be carried out stably. 再有,在平均彩度低的情况下,也可以将由圆检测所确定的更新区域设定为目标区域。 Further, at a low average saturation, it may be determined by the setting of the circle detection area update target area.

[0241] 此外,在上述中,关于根据评价值的关系进行更新的方法进行了叙述,但也有使用基于颜色的追踪结果位置即目标区域与圆检测的结果位置即更新区域之间的距离关系进行更新的方法。 [0241] Further, in the above-described, regarding the method of updating based on the relationship between evaluation values ​​has been described, but also the use of color-based tracking result of the position i.e. results position of the target region and the circle detection is updated from the relationship between the area newer methods. 例如,在假设检测到的圆半径的2倍范围内为容许范围时,通过在颜色的追踪结果位置(追踪框的重心)离开了该范围时对圆检测的位置进行更新,能进行稳定的追踪。 For example, carried out in the range of 2 times the assumed radius detected is the allowable range by the tracking result of the color position (tracking frame center of gravity) leaves the range of the position of the circle detection updates, enabling stable tracking . 再有,不限于圆的倍数,也可以使用欧几里德距离等任意的距离。 Further, the multiple is not limited to a circle, you may be used an arbitrary distance Euclidean distance. 这是因为,该情况下,即使设基于颜色特征的追踪的评价值和基于圆检测的评价值都高,基于颜色特征的追踪不是对象物而是误追踪的可能性也高。 This is because, in this case, even if the evaluation value set based on the color feature and the evaluation value of the tracking circle detection are high, the possibility is not based on the color characteristics of the object, but the tracking of a tracking error is also high.

[0242] 位置修正电路142将追踪结果的目标区域的坐标和尺寸置換成更新区域探测结果的坐标和尺寸,保存在存储器111中。 [0242] The position correction circuit 142 the coordinates and size of the tracking result is replaced with the target area update area detection result of the coordinates and size, stored in the memory 111. 这时,由于更新区域探测结果保存有圆的中心和半径,因此,将从中心减去半径的值作为左上坐标,将半径的2倍作为矩形框的高度,以得到矩形框的左上坐标、高度和宽度。 In this case, since the update area detection result is stored in the center and radius of the circle, and therefore, the value obtained by subtracting the radius from the center as the upper-left coordinate, the height of two times the radius of a rectangular frame, to obtain upper left coordinates of the rectangular frame, the height and width.

[0243] 此外,如图10(c)所示,在追踪结束判断电路141中,在追踪结果的评价值和更新区域探测结果的评价值低于阈值th2时,将结束追踪的判断结果保存在存储器111中。 [0243] Further, FIG. 10 (c), in the tracking end determination circuit 141, the evaluation value and the update area detection result is below the evaluation value of the tracking result when the threshold th2, the tracking end determination results stored in 111 memory.

[0244] 之后,实施脸检测,由初始特征量提取部136进行自动恢复,或者向用户通知追踪处理已结束,并再次指定追踪物体位置,通过使用这样的方法,就能进行再追踪。 After [0244], face detection embodiment, the amount of the initial feature extraction unit 136 and automatic recovery, or to notify the user tracking process has been completed, and track objects specified position again, by using such a method, tracking can be carried out again.

[0245] 根据以上叙述,即使在朝向了后面的情况下也能够追踪,即使在这样的状态下,也当然能够通过追踪结果描绘电路进行追踪框的显示或使追踪结果位置模糊化等的n控制,而且通过进行AE • AF等的控制或用摄像机控制电路145进行摄像机的控制,能进行自动成巾贞•自动摄影。 Display [0245] According to the above description, even toward the latter case it is possible to track, even in such a state, of course, can be tracked block drawing circuit by the tracking result or cause the tracking result of the position blurring like n control , and by controlling the like AE • AF or camera control by the camera control circuit 145, can automatically to the automatic photographing • Chen towel.

[0246] 在用程序执行了以上的处理的情况下,成为如图11所示的步骤。 [0246] In the case of performing the above processing procedure, the step shown in FIG. 11 becomes. 对摄影到的输入数据(S801)进行脸检测(S802)。 Photography data to the input (S801) for face detection (S802). 对脸检测出的结果提取初始登记特征(S803)。 The initial registered feature extraction (S803) to the result of the face detected. 在下ー帧中,使用在步骤S803中登记的颜色特征进行追踪处理(S804),确定目标区域。ー next frame using the color feature registered in step S803 the tracking process (S804), determine the target area. 之后,使用作为形状特征的圆进行圆检测,进行更新区域的确定(S805)。 Thereafter, a circular shape as is characteristic circle detection, determination (S805) the update region.

[0247] 根据基于顔色特征的追踪处理的评价值和使用了作为形状特征的圆的圆检测的评价值,判断是否结束追踪处理(S806)。 [0247] The evaluation value of the circle with the detection and evaluation value was used as shape feature tracking process based on color characteristics of the determined tracking process (S806) is finished. 在步骤806中,在基于颜色特征量的追踪处理的评价值和基于圆检测的评价值都低于阈值th2时,判断为结束追踪。 In step 806, when the tracking process based on the evaluation value of the color feature amount and the evaluation value based on the detected circle is below a threshold value Th2, it is determined that the end of the track. 在判断为结束追踪处理的情况下,结束追踪处理,结束追踪框的显示,或者利用警报等向用户提示处理结束。 In the case where it is determined that the end of the tracking process, the tracking process ends, the end of the display of the tracking frame or by using other prompts alert the user ends the processing. 之后,也可以在显示器中记载催促用户再次进行初始位置设定的显示,或通过脸检测进行初始位置设定,并进行自动恢复(S810)。 Thereafter, the user may be urged described again in the display shows the initial set position, or initial position set by the face detection and automatic recovery is performed (S810). 在不结束的情况下,如图10(b)所示,使用阈值thl,即,而在基于圆检测的评价值超过了阈值thl时,更新顔色特征,而与基于颜色特征的追踪处理的评价值无关,或者,如图10(a)所示,在基于圆检测的更新区域探测结果的评价值超过了使用了顔色特征的追踪特征的评价值时,判断为进行顔色特征的特征量的更新(S807)。 In the case of not ending, when FIG. 10 (b), the use of the threshold thl, i.e., while exceeding the threshold value thl based on the evaluation value of the circle detection, update the color characteristics, and the tracking process based on color characteristics of the commentary when independent value, or, as shown in FIG 10 (a), in the evaluation value based on a result of the update area detection by circle detection exceeds the tracking feature using the color feature evaluation value, it is determined that color feature amount updating feature (S807). 在判断为进行特征量的更新的情况下,使用更新区域探测结果的位置•特征量,对跟踪结果的位置•特征量进行更新(S808)。 In a case where the feature amount is determined to perform updating using the update area detection result • position feature quantity, the position of the tracking result is updated • feature quantity (S808). 然后,进行更新后的追踪结果位置的LCD显示和摄像机控制(S809)。 Then, the position of the tracking result is updated after the LCD display and camera control (S809). 之后,返回到步骤S804的追踪处理,反复进行步骤S804〜S810的处理。 Thereafter, the processing returns to step S804 of the tracking, the process repeats steps S804~S810.

[0248] 关于该追踪处理方法和类似度计算方法,不限于色直方图匹配方法,可以是基于最小平方距离的方法或者基于归ー化相关的方法等。 [0248] For tracking processing method and the similarity calculating method is not limited to the color histogram matching method, a method may be based on a least square distance or a method based on normalized correlation ー like. 此外,作为搜索方法,也可以不是简单的扫描,而是基于粒子过滤器的方法等。 Further, as a search method may not simply scan, but on the particulate filter and the like.

[0249] 此外,在本实施方式中,将颜色特征作为跟踪特征,将形状特征作为更新区域探測特征,但在如非刚体且顔色变化少的物体这样的对颜色的变化稳定而形状不稳定的情况下,也可以将颜色特征作为更新区域探测结果特征,将形状特征作为跟踪特征。 [0249] In the present embodiment, as the color feature tracking feature, feature shape as the update area detection feature, but in non-rigid small and stable change in color change in color of such objects and dimensionally stable case, the color may be characterized as a result of the update area detection feature, feature shape as the tracking feature. 再有,在本实施方式中,利用了脸检测位置作为初始位置,但也可以利用AF的位置作为初始位置。 Further, in the present embodiment, by using the face detection position as the initial position, but can also be utilized as the initial position of the AF position.

[0250] 再有,将方框图(图4等)的各功能块典型地实现为作为集成电路的LSI。 [0250] Further, the block diagram (FIG. 4, etc.) of each function block are typically realized as an LSI. 也可以将它们个别地单芯片化,也可以包含其中一部分或全部地单芯片化。 They may also be a single chip individually, or part or all of which may comprise one chip. 例如,也可以将存储器以外的功能块单芯片化。 For example, the functional blocks may be other than the one chip memory.

[0251 ] 在此,设定为LSI,但有时也根据集成度的不同而称作1C、系统LSI、超大LSI、超级LSI。 [0251] Here, LSI is set, but sometimes also be referred to as 1C, system LSI, super LSI, super LSI depending on the integration degree.

[0252] 此外,电路集成化的方法不限于LSI,也可以用专用电路或通用处理器来实现。 [0252] Further, the method of circuit integration is not limited to LSI, and a dedicated circuit or a general-purpose processor. 也可以利用在制造LSI后可编程的FPGA(Field Programmable Gate Array :现场可编程门阵列)和可重构LSI内部的电路单元的连接或设定的可重构处理器。 May also be utilized After LSI manufacture, a programmable FPGA (Field Programmable Gate Array: a field programmable gate array) and the connection or setting inside the LSI can be reconfigurable circuit cells are reconfigurable processor.

[0253]另外,若出现了利用半导体技术的进步或派生的其他技术替换LSI的集成电路化技木,当然也可以使用该技术进行功能块的集成化。 [0253] Further, when replacing the LSI appears by a semiconductor technology or other technological advances in integrated circuit technology of wood-derived, of course, this technique may be used for integration of the functional blocks. 也可能适用生物技术等。 Application of biotechnology is also possible.

[0254] 此外,也可以不使各功能块中的、保存成为编码或解码的对象的数据的单元单芯片化,而是成为另外的结构。 [0254] It is also possible without the respective functional blocks, one chip unit stored data becomes an object of encoding or decoding, but becomes another structure.

[0255] エ业上的可利用性 [0255] Applicability on Industry Ester

[0256] 本发明涉及的电子摄像机和图像处理方法具有物体的样子变化的对象的追踪功能,在利用追踪功能实现的最佳拍照摄影功能和AE • AF等摄像机控制、图像的成帧中十分有用。 [0256] electronic camera and an image processing method of the present invention having a change in appearance of the object tracking object tracking function is useful to achieve optimum use of camera photographing function like a camera and AE • AF control, framing image . 此外,在追踪摄影人物等对象物的监视摄像机中十分有用。 In addition, very useful in tracking an object such person photography surveillance cameras. · ·

Claims (18)

1. 一种电子摄像机,具有在连续摄影的图像的各帧内确定成为跟踪对象的对象物所映入的对象区域并进行显示的功能,具有: 追踪处理部,使用定量地表示上述对象物的特征的预先登记的第一特征量,在帧内的预先设定的范围内进行搜索,对作为上述搜索结果所得到的区域内的图像,计算表示与上述对象物的图像之间的一致度的第一评价值,根据计算出的上述第一评价值,确定被推定为存在上述对象物的图像的目标区域; 更新区域计算部,使用与上述第一特征量不同的、定量地表示上述对象物的特征的第二特征量,在上述帧内的预先设定的范围内进行搜索,对作为上述搜索结果所得到的区域内的图像,计算表示与上述对象物的图像之间的一致度的第二评价值,根据计算出的上述第二评价值,在上述帧内确定用于更新上述第一特征量的更新区域; An electronic camera, each frame having a continuous shooting image is determined to become the object of the tracking object in the target region and reflection display function, comprising: a tracking processing unit, using a quantitative showing of the object of the a first feature amount previously registered feature, frame search within a predetermined range, the image in the area as the search result is obtained, calculates a degree of coincidence between the image and the object to be a first evaluation value based on the first evaluation value calculated, is estimated to determine the presence of the object image of a target area; update area calculation unit, using the first feature different, indicating that the object is quantitatively second feature quantity of the feature, the search frame within said predetermined range, the image in the area as the search result is obtained, calculates a degree of matching between the first image and the object two evaluation value based on the second evaluation value calculated, in determining the intra-frame update area for updating the first feature amount; 新判断部,通过调查在上述追踪处理部中计算出的上述第一评价值和在上述更新区域计算部中计算出的上述第二评价值中的至少一个是否满足预先设定的条件,来判断是否更新上述第一特征量; 登记特征量更新部,在由上述更新判断部判断为更新上述第一特征量的情况下,用在上述更新区域内提取的新的第一特征量更新上述第一特征量;以及追踪结果描绘部,在由上述更新判断部判断为不更新上述第一特征量的情况下,将在上述追踪处理部中确定的上述目标区域确定为上述对象区域,并描绘有关上述目标区域的信息,在由上述更新判断部判断为更新上述第一特征量的情况下,将在上述更新区域计算部中确定的上述更新区域确定为上述对象区域,并描绘有关上述更新区域的信息, 上述追踪处理部在预先登记的上述第一特征量被更新的情况下,使用 New determination unit, by investigating whether at least one predetermined condition is satisfied calculated in the above-described tracking processing unit in the first evaluation value and the second evaluation value calculated in the update area calculation unit is to determine whether to update the first feature amount; registered feature amount updating unit, when updating by the determination unit determines to update the first feature of the case, the update area extracted in said new first feature amount updating the first feature amount; and a tracking result drawing unit, in a case where the above-described update determination unit determines not to update the first feature, the above-mentioned target region determined in said tracking processing unit is determined as the target area, and drawing on the above information of the target region in the determination by the updating unit determines to update the first feature of the case, the update area determined in said update area calculation unit is determined as the target area, and drawing information about the update area of the tracking process section in the case of the first previously registered feature amount is updated using 更新后的第一特征量确定新的帧内的新目标区域。 The updated first feature quantity to determine a new frame a new target area.
2.根据权利要求I所述的电子摄像机, 上述追踪处理部使用对象物的颜色信息作为上述第一特征量,来确定被推定为存在上述对象物的图像的目标区域, 上述更新区域计算部使用对象物的形状信息作为上述第二特征量,来确定上述更新区域。 The electronic camera according to claim I according to the tracking processing unit using color information of the object as the first feature quantity to be estimated is determined as the image of the object exists in the target area, the update area calculation unit using the shape information of the object as the second feature quantity to determine the update area.
3.根据权利要求I所述的电子摄像机, 上述更新判断部在上述更新区域计算部中计算出的上述第二评价值大于在上述追踪处理部中计算出的上述第一评价值时,或者,在上述更新区域计算部中计算出的上述第二评价值大于预先设定的第一阈值时,判断为更新上述第一特征量。 3. The electronic camera according to claim I, said second evaluation unit and the update determination calculated in the update area calculation unit is greater than the first evaluation value calculated in the above-described tracking processing unit, or, when calculated in the update area calculation unit and the second evaluation value is larger than a first predetermined threshold value, it is determined to update the first feature.
4.根据权利要求3所述的电子摄像机, 上述对象物的平均彩度越高,上述更新判断部对上述第一阈值设定越大的值,上述对象物的平均彩度越低,上述更新判断部对上述第一阈值设定越小的值。 4. The electronic camera according to claim 3, the higher the average saturation of the object, and the update determination unit sets a larger value to the first threshold value, the lower the average saturation of the object, said update Analyzing unit setting a smaller value of the first threshold.
5.根据权利要求I所述的电子摄像机, 上述更新判断部还在上述追踪处理部中确定的上述目标区域与在上述更新区域计算部中确定的上述更新区域之间的距离成为预先设定的第二阈值以上时,判断为进行更新。 The electronic camera according to claim I, the distance between the target area and the update determination unit also determines the above-described tracking processing unit and the update area determined in said update area calculation unit becomes a preset when the second threshold value or more, it is determined to update.
6.根据权利要求I所述的电子摄像机, 上述电子摄像机还具有判断是否不能继续上述对象物的追踪的结束判断部, 上述结束判断部在上述第一评价值和上述第二评价值双方都小于预先设定的第三阈值时,判断为不能继续上述对象物的追踪。 The electronic camera according to claim I, further comprising the electronic camera determines whether or not to continue the tracking end determination unit object of the ending portion of the first determination evaluation value and the second evaluation value is less than both the when a third predetermined threshold value, it is determined not to continue to track the target object.
7.根据权利要求6所述的电子摄像机, 上述追踪结果描绘部在判断为不能继续上述对象物的追踪时,结束目标区域和更新区域的描绘。 7. The electronic camera according to claim 6, said tracking result drawing unit determines the tracking can not be continued when the above-described object, the end of the drawing area and the update target area.
8.根据权利要求6所述的电子摄像机, 上述追踪结果描绘部在判断为不能继续上述对象物的追踪时,描绘对用户提示不能追踪的图像。 8. The electronic camera according to claim 6, said tracking result drawing unit when it is determined not to continue the tracking of the object, the depiction of images can not be tracked user prompts.
9.根据权利要求6所述的电子摄像机, 上述追踪结果描绘部在判断为不能继续上述对象物的追踪时,描绘对用户催促再次设定初始位置的图像。 9. The electronic camera according to claim 6, said tracking result drawing unit when it is determined not to continue tracking the object, prompting the user to set an image depicting the initial position again.
10.根据权利要求6所述的电子摄像机, 上述追踪结果描绘部在判断为不能继续上述对象物的追踪时,通过新的脸检测进行初始位置设定。 10. The electronic camera according to claim 6, said tracking result drawing unit is unable to continue the tracking of the object, the initial position setting by the new face detection is determined.
11.根据权利要求I所述的电子摄像机, 上述电子摄像机还具有: 脸检测电路,在各帧内的图像内检测脸;以及脸朝向识别电路,识别由上述脸检测电路检测到的脸的朝向, 在由上述脸朝向识别电路识别为作为上述对象物的脸朝向侧面时,在上述更新区域计算部中计算出作为朝向侧面的脸的区域的更新基准区域之后,上述更新区域计算部根据脸部分与头发部分的预先指定的位置关系,计算上述更新基准区域内的上述更新区域。 11. The electronic camera according to claim I, the electronic camera further comprising: a face detection circuit for detecting a face in the image in each frame; and a facing direction recognition circuit recognizes a face detected by the face direction detecting circuit Thereafter, when the identification circuit identification by the face orientation of a face of the object toward the side, calculated as toward the side face of updating the reference region area in the update area calculation unit, the update area calculation unit based on the face portion a pre-designated portion of hair positional relationship with the update area calculation within the update reference area.
12.根据权利要求I所述的电子摄像机, 上述电子摄像机还具有摄像机控制部,该摄像机控制部根据在上述追踪处理部中确定的目标区域和在上述更新区域计算部中确定的上述更新区域中的某个,变更用于调整上述电子摄像机的动作的摄像机参数。 12. The electronic camera according to claim I, the electronic camera further comprises a camera control unit, the camera control unit determines a target area in said tracking processing unit and the update area determined in said update area calculation unit according to the certain, change camera parameters to adjust the operation of the electronic camera.
13.根据权利要求12所述的电子摄像机, 上述摄像机控制部根据上述摄像机参数控制上述电子摄像机的壳体和云台的至少一方的动作,进行控制,使被确定的上述对象物的全体或一部分配合帧内的规定位置和尺寸。 13. The electronic camera as claimed in claim 12, wherein said camera control unit according to at least one of the parameters controlling the camera housing and head of the electronic camera operation, controls so that the object is determined to be a part of or the entire fitting a predetermined location and size within the frame.
14.根据权利要求I所述的电子摄像机, 上述电子摄像机还具有对象物初始设定部,该对象物初始设定部根据来自用户的输入,或者使用预先设定的方法来决定上述对象物的初始位置。 14. The electronic camera according to claim I, the electronic camera further includes an initial setting unit of the object, the object setting section determines the initial object based on input from a user, or a method of using a preset initial position.
15.根据权利要求14所述的电子摄像机, 上述对象物初始设定部将人物或脸的检测位置决定为初始位置。 15. The electronic camera according to claim 14, the initial setting of the object detection unit the position of the person or the face is determined as an initial position.
16.根据权利要求14所述的电子摄像机, 上述对象物初始设定部将由AF自动对焦功能合焦的地方决定为初始位置。 16. The electronic camera according to claim 14, of the object portion by an initial setting where autofocus AF focus is determined as an initial position.
17. 一种电子摄像机中的图像处理方法,该电子摄像机具有在连续摄影的图像的各帧内确定成为跟踪对象的对象物所映入的对象区域并进行显示的功能,上述图像处理方法包括: 追踪处理部使用定量地表示上述对象物的特征的预先登记的第一特征量,在帧内的预先设定的范围内进行搜索,对作为上述搜索结果所得到的区域内的图像,计算表示与上述对象物的图像之间的一致度的第一评价值,根据计算出的上述第一评价值,确定被推定为存在上述对象物的图像的目标区域, 更新区域计算部使用与上述第一特征量不同的、定量地表示上述对象物的特征的第二特征量,在上述帧内的预先设定的范围内进行搜索,对作为上述搜索结果所得到的区域内的图像,计算表示与上述对象物的图像之间的一致度的第二评价值,根据计算出的上述第二评价值,在上述帧 17. A method of image processing in the electronic camera, the camera having an electronic image of each frame in the continuous shooting determination target object to be tracked in the target region and the reflection display function, the image processing method comprising: wherein the tracking processing unit represented by the above-described object is quantitatively using a first feature amount previously registered, to search within a predetermined range of the frame, in the region of the image as the search result is obtained, and calculate a a first evaluation value of a degree of matching between an image of the object, based on the first evaluation value calculated, is estimated to determine the presence of a target area image of the object, the update area calculation unit using the above-mentioned first feature different amounts of second feature quantity, quantitatively represents the characteristic of the object, in said search frame preset range, in the area of ​​the image as the search result is obtained, the target is calculated and represents second evaluation value of the degree of matching between the image of the object, based on the second evaluation value calculated in the frame 确定用于更新上述第一特征量的更新区域, 更新判断部通过调查在上述追踪处理部中计算出的上述第一评价值和在上述更新区域计算部中计算出的上述第二评价值中的至少一个是否满足预先设定的条件,来判断是否更新上述第一特征量, 登记特征量更新部在由上述更新判断部判断为更新上述第一特征量的情况下,用在上述更新区域内提取的新的第一特征量更新上述第一特征量, 追踪结果描绘部在由上述更新判断部判断为不更新上述第一特征量的情况下,将在上述追踪处理部中确定的上述目标区域确定为上述对象区域,并描绘有关上述目标区域的信息,在由上述更新判断部判断为更新上述第一特征量的情况下,将在上述更新区域计算部中确定的上述更新区域确定为上述对象区域,并描绘有关上述更新区域的信息, 上述追踪处理部在预先登记的上 Determining the update area for updating the first feature, the update determination unit investigation calculated in the above-described tracking processing unit in the first evaluation value and the second evaluation value calculated in the update area calculation unit is the at least one predetermined condition is satisfied, it determines whether to update the first feature amount, the feature amount registration section in the updated by the update determination unit determines to update the first feature of the case, the extracted within the update area new first feature amount updating the first feature amount, tracking result drawing unit in a case where the above-described update determination unit determines not to update the first feature, the above-mentioned target region determined in said tracking processing unit determines the to the target region, and information on these drawing target region in the determination by the updating unit determines to update the first feature of the case, the update area determined in said update area calculation unit is determined as the target area and depicting information about the update area, and said tracking processing unit registered in advance 第一特征量被更新的情况下,使用更新后的第一特征量确定新的帧内的新目标区域。 When the first feature amount is updated, a first feature amount using the updated new intra determine a new target area.
18. 一种集成电路,具有在连续摄影的图像的各帧内确定成为跟踪对象的对象物所映入的对象区域并进行显示的功能,具有: 追踪处理部,使用定量地表示上述对象物的特征的预先登记的第一特征量,在帧内的预先设定的范围内进行搜索,对作为上述搜索结果所得到的区域内的图像,计算表示与上述对象物的图像之间的一致度的第一评价值,根据计算出的上述第一评价值,确定被推定为存在上述对象物的图像的目标区域; 更新区域计算部,使用与上述第一特征量不同的、定量地表示上述对象物的特征的第二特征量,在上述帧内的预先设定的范围内进行搜索,对作为上述搜索结果所得到的区域内的图像,计算表示与上述对象物的图像之间的一致度的第二评价值,根据计算出的上述第二评价值,在上述帧内确定用于更新上述第一特征量的更新区域; 更 18. An integrated circuit, each frame having a continuous shooting image is determined to become the object of the tracking object in the target region and reflection display function, comprising: a tracking processing unit, using a quantitative showing of the object of the a first feature amount previously registered feature, frame search within a predetermined range, the image in the area as the search result is obtained, calculates a degree of coincidence between the image and the object to be a first evaluation value based on the first evaluation value calculated, is estimated to determine the presence of the object image of a target area; update area calculation unit, using the first feature different, indicating that the object is quantitatively second feature quantity of the feature, the search frame within said predetermined range, the image in the area as the search result is obtained, calculates a degree of matching between the first image and the object two evaluation value based on the second evaluation value calculated, in determining the intra-frame update area for updating the first feature amount; more 新判断部,通过调查在上述追踪处理部中计算出的上述第一评价值和在上述更新区域计算部中计算出的上述第二评价值中的至少一个是否满足预先设定的条件,来判断是否更新上述第一特征量; 登记特征量更新部,在由上述更新判断部判断为更新上述第一特征量的情况下,用在上述更新区域内提取的新的第一特征量更新上述第一特征量;以及追踪结果描绘部,在由上述更新判断部判断为不更新上述第一特征量的情况下,将在上述追踪处理部中确定的上述目标区域确定为上述对象区域,并描绘有关上述目标区域的信息,在由上述更新判断部判断为更新上述第一特征量的情况下,将在上述更新区域计算部中确定的上述更新区域确定为上述对象区域,并描绘有关上述更新区域的信息, 上述追踪处理部在预先登记的上述第一特征量被更新的情况下,使用 New determination unit, by investigating whether at least one predetermined condition is satisfied calculated in the above-described tracking processing unit in the first evaluation value and the second evaluation value calculated in the update area calculation unit is to determine whether to update the first feature amount; registered feature amount updating unit, when updating by the determination unit determines to update the first feature of the case, the update area extracted in said new first feature amount updating the first feature amount; and a tracking result drawing unit, in a case where the above-described update determination unit determines not to update the first feature, the above-mentioned target region determined in said tracking processing unit is determined as the target area, and drawing on the above information of the target region in the determination by the updating unit determines to update the first feature of the case, the update area determined in said update area calculation unit is determined as the target area, and drawing information about the update area of the tracking process section in the case of the first previously registered feature amount is updated using 新后的第一特征量确定新的帧内的新目标区域。 The first new feature amount to determine a new frame a new target area.
CN2009801044907A 2008-02-06 2009-02-06 Electronic camera and image processing method CN101939980B (en)

Priority Applications (3)

Application Number Priority Date Filing Date Title
JP2008026160 2008-02-06
JP2008-026160 2008-02-06
PCT/JP2009/000477 WO2009098894A1 (en) 2008-02-06 2009-02-06 Electronic camera and image processing method

Publications (2)

Publication Number Publication Date
CN101939980A CN101939980A (en) 2011-01-05
CN101939980B true CN101939980B (en) 2012-08-08

Family

ID=40951973

Family Applications (1)

Application Number Title Priority Date Filing Date
CN2009801044907A CN101939980B (en) 2008-02-06 2009-02-06 Electronic camera and image processing method

Country Status (5)

Country Link
US (1) US8253819B2 (en)
EP (1) EP2242253B1 (en)
JP (1) JP5090474B2 (en)
CN (1) CN101939980B (en)
WO (1) WO2009098894A1 (en)

Families Citing this family (41)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4869270B2 (en) * 2008-03-10 2012-02-08 三洋電機株式会社 Imaging apparatus and image reproduction apparatus
US20110102570A1 (en) * 2008-04-14 2011-05-05 Saar Wilf Vision based pointing device emulation
WO2010044214A1 (en) * 2008-10-14 2010-04-22 パナソニック株式会社 Face recognition device and face recognition method
KR101051365B1 (en) * 2009-07-16 2011-07-22 주식회사 유니온커뮤니티 Access control device with camera and access control method
JP5279654B2 (en) * 2009-08-06 2013-09-04 キヤノン株式会社 Image tracking device, image tracking method, and computer program
JP5483961B2 (en) * 2009-09-02 2014-05-07 キヤノン株式会社 Image processing apparatus, subject discrimination method, program, and storage medium
GB2474536B (en) 2009-10-13 2011-11-02 Pointgrab Ltd Computer vision gesture based control of a device
JP5657235B2 (en) * 2009-11-10 2015-01-21 オリンパスイメージング株式会社 Image capturing apparatus and image capturing method
JP5427577B2 (en) * 2009-12-04 2014-02-26 パナソニック株式会社 Display control apparatus and display image forming method
KR101652261B1 (en) * 2009-12-24 2016-08-30 삼성전자주식회사 Method for detecting object using camera
US8705813B2 (en) * 2010-06-21 2014-04-22 Canon Kabushiki Kaisha Identification device, identification method, and storage medium
JP5720275B2 (en) * 2011-02-03 2015-05-20 株式会社リコー Imaging apparatus and imaging method
CN102075686B (en) * 2011-02-10 2013-10-30 北京航空航天大学 Robust real-time on-line camera tracking method
US8766153B2 (en) * 2011-02-17 2014-07-01 Mitutoyo Corporation Vision measuring device and auto-focusing control method
KR20120118383A (en) * 2011-04-18 2012-10-26 삼성전자주식회사 Image compensation device, image processing apparatus and methods thereof
WO2013001940A1 (en) * 2011-06-29 2013-01-03 オリンパスイメージング株式会社 Tracking device, and tracking method
JP5888890B2 (en) * 2011-07-21 2016-03-22 オリンパス株式会社 Focus control device
WO2013027320A1 (en) * 2011-08-25 2013-02-28 パナソニック株式会社 Image processing device, three-dimensional image capture device, image processing method, and image processing program
US9171222B2 (en) * 2011-11-17 2015-10-27 Panasonic Intellectual Property Corporation Of America Image processing device, image capturing device, and image processing method for tracking a subject in images
CN102521580A (en) * 2011-12-21 2012-06-27 华平信息技术(南昌)有限公司 Real-time target matching tracking method and system
KR101989152B1 (en) * 2012-02-10 2019-09-30 삼성전자주식회사 Apparatus and method for capturing still image during moving image photographing or reproducing
US9049364B2 (en) * 2012-02-13 2015-06-02 Htc Corporation Focus adjusting method and image capture device thereof
JP5888034B2 (en) * 2012-03-16 2016-03-16 富士通株式会社 User detection device, method and program
US9256781B2 (en) * 2012-05-10 2016-02-09 Pointguard Ltd. System and method for computer vision based tracking of an object
US8938124B2 (en) * 2012-05-10 2015-01-20 Pointgrab Ltd. Computer vision based tracking of a hand
US8441548B1 (en) * 2012-06-15 2013-05-14 Google Inc. Facial image quality assessment
CN103516972B (en) * 2012-06-19 2017-03-29 联想(北京)有限公司 Camera control method and electronic equipment
JP2014116716A (en) * 2012-12-07 2014-06-26 Samsung Display Co Ltd Tracking device
JP2014139734A (en) * 2013-01-21 2014-07-31 Sony Corp Information processing device and method, and program
JP2014207659A (en) * 2013-03-19 2014-10-30 パナソニック株式会社 Image processing device and image processing method and imaging apparatus
KR101316805B1 (en) 2013-05-22 2013-10-11 주식회사 파이브지티 Automatic face tracing and recognition method and system
WO2014203026A1 (en) * 2013-06-17 2014-12-24 Aselsan Elektronik Sanayi Ve Ticaret Anonim Sirketi A method for object tracking
CN104424634B (en) * 2013-08-23 2017-05-03 株式会社理光 Object tracking method and device
JP6268960B2 (en) * 2013-11-15 2018-01-31 オムロン株式会社 Image recognition apparatus and data registration method for image recognition apparatus
US9437000B2 (en) * 2014-02-20 2016-09-06 Google Inc. Odometry feature matching
JP6415196B2 (en) * 2014-09-08 2018-10-31 キヤノン株式会社 Imaging device and imaging device control method
CN105518702B (en) * 2014-11-12 2018-06-26 深圳市大疆创新科技有限公司 A kind of detection method, detection device and robot to target object
JP6521626B2 (en) * 2014-12-25 2019-05-29 キヤノン株式会社 Object tracking device, method and program
US9934443B2 (en) * 2015-03-31 2018-04-03 Daon Holdings Limited Methods and systems for detecting head motion during an authentication transaction
TWI585394B (en) * 2015-12-09 2017-06-01 由田新技股份有限公司 Automatic focusing system
CN106934395A (en) * 2017-02-15 2017-07-07 国家计算机网络与信息安全管理中心 The rigid-object tracking that a kind of use SURF features and color characteristic are blended

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1722826A (en) 2004-07-14 2006-01-18 松下电器产业株式会社 System, device and method for object tracing
CN1892702A (en) 2005-07-05 2007-01-10 欧姆龙株式会社 Tracking apparatus

Family Cites Families (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH056434A (en) * 1991-06-27 1993-01-14 Matsushita Electric Ind Co Ltd Automatic tracking device
JP3552456B2 (en) 1997-05-01 2004-08-11 セイコーエプソン株式会社 Moving object tracking method
JPH11167455A (en) * 1997-12-05 1999-06-22 Fujitsu Ltd Hand form recognition device and monochromatic object form recognition device
US6687386B1 (en) * 1999-06-15 2004-02-03 Hitachi Denshi Kabushiki Kaisha Object tracking method and object tracking apparatus
US6859552B2 (en) * 2000-11-07 2005-02-22 Minolta Co., Ltd. Image retrieving apparatus
JP2002157599A (en) 2000-11-17 2002-05-31 Nippon Telegr & Teleph Corp <Ntt> Method for detecting and recognizing object, recording medium recording its program and object monitoring and tracking device
US7099510B2 (en) 2000-11-29 2006-08-29 Hewlett-Packard Development Company, L.P. Method and system for object detection in digital images
US7298412B2 (en) * 2001-09-18 2007-11-20 Ricoh Company, Limited Image pickup device, automatic focusing method, automatic exposure method, electronic flash control method and computer program
KR100459893B1 (en) * 2002-01-08 2004-12-04 삼성전자주식회사 Method and apparatus for color-based object tracking in video sequences
JP2004362210A (en) 2003-06-04 2004-12-24 Nippon Telegr & Teleph Corp <Ntt> Device and method for tracing object and its program and recording medium with its program recorded thereon
JP4824411B2 (en) * 2005-01-20 2011-11-30 パナソニック株式会社 Face extraction device, semiconductor integrated circuit
JP2007072520A (en) * 2005-09-02 2007-03-22 Sony Corp Video processor
JP4657934B2 (en) * 2006-01-23 2011-03-23 富士フイルム株式会社 Face detection method, apparatus and program
US7315631B1 (en) * 2006-08-11 2008-01-01 Fotonation Vision Limited Real-time face tracking in a digital image acquisition device
GB2452512B (en) * 2007-09-05 2012-02-29 Sony Corp Apparatus and method of object tracking
TWI358674B (en) * 2007-12-26 2012-02-21 Altek Corp
JP4730431B2 (en) * 2008-12-16 2011-07-20 日本ビクター株式会社 Target tracking device
JP5383366B2 (en) * 2009-07-28 2014-01-08 キヤノン株式会社 Image processing apparatus, image processing method, and program thereof
JP5662670B2 (en) * 2009-10-27 2015-02-04 キヤノン株式会社 Image processing apparatus, image processing method, and program

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1722826A (en) 2004-07-14 2006-01-18 松下电器产业株式会社 System, device and method for object tracing
CN1892702A (en) 2005-07-05 2007-01-10 欧姆龙株式会社 Tracking apparatus

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
JP特开2004-362210A 2004.12.24
JP特开平10-304348A 1998.11.13
JP特开平11-167455A 1999.06.22

Also Published As

Publication number Publication date
CN101939980A (en) 2011-01-05
EP2242253B1 (en) 2019-04-03
US20110001840A1 (en) 2011-01-06
WO2009098894A1 (en) 2009-08-13
JP5090474B2 (en) 2012-12-05
US8253819B2 (en) 2012-08-28
EP2242253A4 (en) 2014-05-14
EP2242253A1 (en) 2010-10-20
JPWO2009098894A1 (en) 2011-05-26

Similar Documents

Publication Publication Date Title
KR101401165B1 (en) Image-capturing apparatus and method, expression evaluation apparatus, and recording medium
CN100565320C (en) Imaging apparatus, control method of imaging apparatus, and computer program
CN101193208B (en) Image shooting device, image processing device, image processing method and image processing program
CN101246541B (en) Image-processing device and image-processing method, image-pickup device
US8836777B2 (en) Automatic detection of vertical gaze using an embedded imaging device
CN1909603B (en) Image processing method and imaging apparatus
US20080316327A1 (en) Image capture device with contemporaneous reference image capture mechanism
JP4262082B2 (en) Image processing apparatus, control method therefor, program, and storage medium
US20090290796A1 (en) Image processing apparatus and image processing method
US20090135269A1 (en) Electronic Camera and Image Processing Device
US20130329955A1 (en) Real-Time Face Tracking with Reference Images
US8494286B2 (en) Face detection in mid-shot digital images
US7623733B2 (en) Image combination device, image combination method, image combination program, and recording medium for combining images having at least partially same background
US20090087099A1 (en) Image processing apparatus, image capturing apparatus, image processing method and recording medium
JP5239126B2 (en) Electronic camera
JP4264663B2 (en) Imaging apparatus, image processing apparatus, image processing method therefor, and program causing computer to execute the method
JP2005215750A (en) Face detecting device and face detecting method
JP5098259B2 (en) Camera
US8503818B2 (en) Eye defect detection in international standards organization images
JP2007286255A (en) Camera
JP2004320287A (en) Digital camera
JP4898532B2 (en) Image processing apparatus, photographing system, blink state detection method, blink state detection program, and recording medium on which the program is recorded
US8520091B2 (en) Auto white balance correction value calculation device, method, program, and image pickup device
JP4904243B2 (en) Imaging apparatus and imaging control method
US8629915B2 (en) Digital photographing apparatus, method of controlling the same, and computer readable storage medium

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
C14 Grant of patent or utility model
GR01 Patent grant
ASS Succession or assignment of patent right

Owner name: MATSUSHITA ELECTRIC (AMERICA) INTELLECTUAL PROPERT

Free format text: FORMER OWNER: MATSUSHITA ELECTRIC INDUSTRIAL CO, LTD.

Effective date: 20140930

C41 Transfer of patent application or patent right or utility model
TR01 Transfer of patent right

Effective date of registration: 20140930

Address after: Seaman Avenue Torrance in the United States of California No. 2000 room 200

Patentee after: Intellectual property company of Matsushita Electric Industries (U.S.)

Address before: Osaka Japan

Patentee before: Matsushita Electric Industrial Co., Ltd.