EP2219366B1 - Image capturing device, image capturing method, and image capturing program - Google Patents

Image capturing device, image capturing method, and image capturing program Download PDF

Info

Publication number
EP2219366B1
EP2219366B1 EP10151652.4A EP10151652A EP2219366B1 EP 2219366 B1 EP2219366 B1 EP 2219366B1 EP 10151652 A EP10151652 A EP 10151652A EP 2219366 B1 EP2219366 B1 EP 2219366B1
Authority
EP
European Patent Office
Prior art keywords
image
images
group
blur
image capturing
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
EP10151652.4A
Other languages
German (de)
English (en)
French (fr)
Other versions
EP2219366A1 (en
Inventor
Kazuhisa Matsunaga
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Casio Computer Co Ltd
Original Assignee
Casio Computer Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Casio Computer Co Ltd filed Critical Casio Computer Co Ltd
Publication of EP2219366A1 publication Critical patent/EP2219366A1/en
Application granted granted Critical
Publication of EP2219366B1 publication Critical patent/EP2219366B1/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/95—Computational photography systems, e.g. light-field imaging systems
    • H04N23/951—Computational photography systems, e.g. light-field imaging systems by using two or more images to influence resolution, frame rate or aspect ratio
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/10—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from different wavelengths
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/68—Control of cameras or camera modules for stable pick-up of the scene, e.g. compensating for camera body vibrations
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/68—Control of cameras or camera modules for stable pick-up of the scene, e.g. compensating for camera body vibrations
    • H04N23/681—Motion detection
    • H04N23/6811—Motion detection based on the image signal
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/68—Control of cameras or camera modules for stable pick-up of the scene, e.g. compensating for camera body vibrations
    • H04N23/682—Vibration or motion blur correction
    • H04N23/684—Vibration or motion blur correction performed by controlling the image sensor readout, e.g. by controlling the integration time
    • H04N23/6845—Vibration or motion blur correction performed by controlling the image sensor readout, e.g. by controlling the integration time by combination of a plurality of images sequentially taken
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/80—Camera processing pipelines; Components thereof
    • H04N23/84—Camera processing pipelines; Components thereof for processing colour signals
    • H04N23/843—Demosaicing, e.g. interpolating colour pixel values
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/10—Circuitry of solid-state image sensors [SSIS]; Control thereof for transforming different wavelengths into image signals
    • H04N25/11—Arrangement of colour filter arrays [CFA]; Filter mosaics
    • H04N25/13—Arrangement of colour filter arrays [CFA]; Filter mosaics characterised by the spectral characteristics of the filter elements
    • H04N25/134—Arrangement of colour filter arrays [CFA]; Filter mosaics characterised by the spectral characteristics of the filter elements based on three different wavelength filter elements

Definitions

  • the present invention relates to an image capturing device, image capturing method, and an image capturing program.
  • an electronic hand-blur correction method is known as a method of correcting blur caused by hand movement for a digital camera.
  • a plurality of images is consecutively captured and pixels corresponding to each of the images are added together.
  • S/N signal to noise
  • whether to perform multiplane addition processing is determined by the digital camera before photography (such as when the shutter button is being half-pressed) based on whether the environment is one in which blur caused by hand movement is likely, for example, whether the environment in which photography is to be carried out is slightly dark or not. Therefore, conditions at the time of decision and at the time of actual photography (for example, when the shutter button is full-pressed) do not always match, resulting in an expected result of hand-blur correction not always being achieved.
  • Patent document US2008/0170126 relates to a method and system for image stabilization when taking pictures with a long exposure time. More specifically, it teaches to cut the long exposure time in several shorter periods and to select a reference image from the resulting plurality of frames based on image sharpness or the one that exhibits the least blur. It further teaches to use a shorter exposure time for one of the frames, so as to reduce the risk of having it affected by hand-blur, and using this frame as reference image in order to compute an output image as a weighted average of the corresponding values in the multiple frames.
  • FIG. 1 is a block diagram showing an electrical configuration of a digital camera which is an embodiment of an image capturing device according to the present invention.
  • This digital camera has a recording mode for carrying out photography in a base operational mode, and a playback mode for playing back a photographed image.
  • the digital camera of the present embodiment includes a photographic lens 1, a CMOS imaging device 3 which receives photographic light gathered by way of the photographic lens 1 through a shutter 2 and also applies photoelectric conversion to an optical image of a subject formed on the light-receiving surface and outputs as an image signal, an A/D converter 4 which converts an output signal of the CMOS imaging device 3 to digital image data, and DRAM 5 which sequentially stores image data after conversion. It should be added that data of a plurality of images is stored in the DRAM 4.
  • Operation of the shutter 2 is controlled by way of the shutter control unit 7b according to a command of the CPU core 7a, and operation of the CMOS imaging device 3 and the A/D converter 4 is controlled by way of the light-receiving controller 7c according to a command of the CPU core 7a.
  • One image of image data stored in the DRAM 5, i.e. RAW data is interpolated with color information for each pixel by way of the demosaic unit 7d and converted to YUV data, and then displayed on the liquid crystal display screen 8 through the liquid crystal display controller 6. It should be noted that the converted YUV data is also temporarily stored in the DRAM 5.
  • the external storage medium 11 can be configured of a memory card that is removable from the camera main unit, a flash memory built into the camera main unit, or the like.
  • a characteristic amount calculation unit 7e, SRAM 7f, a block matching unit 7g, an image transformation synthesis addition unit 7h, as well as a shutter button 9, and a mode button 10 are connected to the CPU core 7a.
  • various programs for operation of the CPU core 7a and various data used during execution of the programs are stored in the memory of the CPU core 7a, the programs and data may be stored in memory that is separate from the CPU core 7a.
  • the CPU core 7a follows the program and controls operation of each unit of the digital camera according to a pressing operation of the shutter button 9, functioning as an image capturing means, a hand-blur determination means, an image-blur determination means, and a selection means according to the present invention.
  • the block matching unit 7g functions as a calculation means according to the present invention, and uses the SRAM 7f as working memory, carries out matching among a plurality of images captured by way of the CMOS imaging device 3, acquires a vector indicating a relative displacement amount and displacement direction of the subject portion, and performs outputs thereof to the CPU core 7a.
  • the image deformation and synthesis addition unit 7h functions as an image processing means according to the present invention that aligns a plurality of images captured by way of the CMOS imaging device 3 and carries out synthesizing thereof, in accordance with an instruction of the CPU core 7a.
  • FIG. 2 is a flowchart showing a processing procedure of the CPU core 7a at a time in which the recording mode has been set.
  • digital hand-blur correction mode is set by way of a user operating a mode button 10 of the digital camera (Step A1). As a result, the digital camera automatically carries out hand-blur correction.
  • the CPU core 7a upon detection thereof, carries out consecutive photography of 9 images consisting of images of a first group, images of a second group, and a third image (Step A2). More specifically, 9 images consecutive in time are acquired by way of the CMOS imaging device 3. Of the 9 images, the initial one image, which corresponds to the above-mentioned third image, is acquired by carrying out subject-blur suppression photography for a third exposure time at a high sensitivity capable of dealing with photography in which the subject is blurred or at a dark location, for example, a photographic sensitivity equivalent to ISO 1600.
  • the next four images, which correspond to the images of the first group, are acquired by carrying out photography at an ordinary exposure time (a "first exposure time”).
  • the last four images, which correspond to the images of the second group, are acquired by carrying out photography at a shutter time (a "second exposure time") two levels shorter than an ordinary exposure time.
  • optical hand-blur correction in which the CMOS imaging device 3 operates by shifting (changing position) is also carried out.
  • the CPU core 7a acquires RAW data for each of the 9 images consecutive in time by way of the CMOS imaging device 3, and stores in the DRAM 5 (Step A3). Furthermore, RAW data of each are converted into YUV data and luminance image data consisting of only a luminance component by way of a demosaic unit 7d, and image data after conversion are stored in the DRAM 5 (Step A3).
  • the YUV data are image data to be used for synthesis, and the luminance image data are compressed image data of a sufficient size for positional alignment work during synthesis of a plurality of images (YUV data).
  • the first image of the luminance image data is set as a reference image, and in that reference image, a window of predetermined size is set in a center portion in the frame.
  • Block matching in which only a subject portion is set as a tracking target is carried out by the block matching unit 7g for the blocks 103 acquired in this manner.
  • the CPU core 7a generates a projective matrix representing a positional relationship between adjacent frame images using a specific motion vector showing a displacement amount of or below a predetermined distance (Step A5). That is to say, a projective matrix is obtained based only on a correct motion vector, which exclude motion vectors that should not have been acquired (were clearly acquired erroneously).
  • the projective matrix in the present embodiment is generated by way of the RANSAC method.
  • the RANSAC method follows a model estimation algorithm which repeatedly examines all of hypothetical sets of data generated based on a randomly selected sample.
  • a set of positions is randomly selected, a candidate of a projective transformation matrix is calculated corresponding to the set of selected positions, and the results of projective transformation when the coordinates of all positions are subject to the projective transformation based on the calculated formula are evaluated for a degree of matching. The calculation and evaluation are repeated, and the projective transformation matrix used when the best evaluated values are acquired is adopted.
  • a projective transformation matrix 2 representing a positional relationship between frame 3, with frame 2 as a reference image, is obtained.
  • Step A6 a projective transformation matrix is generated for images from the second to the fifth consecutive in time. More specifically, a projective transformation matrix 3 representing a positional relationship between frame 4, with frame 3 as a reference image, is obtained; then a projective transformation matrix 5 representing a positional relationship between frame 5, with frame 4 as a reference image, is obtained; and then a projective transformation matrix 6 representing a positional relationship between frame 6, with frame 5 as a reference image, is obtained.
  • Step A7 After the previously described projective transformation matrices for the images (luminance image data) up to frame 5 are generated, that is to say, after the positional relationships of the subject between each of the successive images from the second to the fifth images are calculated ("YES" in Step A6), then a parallel displacement component between each of the adjacent frames is calculated from the projective transformation matrix acquired in Step A5 (Step A7).
  • the calculation of the parallel displacement component can be obtained according to (the formula) (s 2 + t 2 ) 1/2 , as shown in FIG. 5 .
  • Step A8 For the image for which the size of the parallel displacement component calculated in Step A7 is a minimum, it is determined whether image blur of the subject has occurred (Step A8). For the image blur of the subject, for all the blocks 103, the differences of the sum of squares are taken for the blocks of identical position in adjacent frames, and if those values are greater than a predetermined value for even one block, it is determined that image blur has occurred.
  • Step A9 the image for subject-blur suppression, which is the first image (Frame 1), is selected (Step A9), and the image is stored in the external storage medium 11.
  • Step A8 if it is determined that image blur of the subject has not occurred, for an image having a size of the parallel displacement component that is a minimum, it is determined whether the size of the parallel displacement component is less than or equal to a predetermined value (for example, 2 pixels), and it is determined whether blur caused by hand movement has occurred (Step A10).
  • a predetermined value for example, 2 pixels
  • Step A10 If the size of the parallel displacement component in Step A10 is less than or equal to 2 pixels, then it is determined that blur caused by hand movement is sufficiently small, and as shown in FIG. 6 , an image such that the parallel displacement component is a minimum is selected (Step A11), and the image is stored in the external storage medium 11.
  • Step A10 the size of the parallel displacement component in Step A10 is not less than or equal to 2 pixels, as shown in FIG. 6 , the image deformation and synthesis addition unit 7h synthesizes, by way of multiplane addition, images 6 through 9 photographed using the second exposure time to generate a new image (Step A12), and stores the new image in the external storage medium 11.
  • the image deformation and synthesis addition unit 7h can carry out high sensitivity processing having an increased sensitizing effect by way of carrying out multiplane addition synthesis processing using Bayer array data for the sixth to ninth images. More specifically, as shown in FIG. 7 , of the Bayer array data for each pixel, a degree of correlation of G (green) images, which accurately reflect a luminance component and a resolution, is obtained and positional adjustment is carried out, and also interpolation processing is carried out for vertical, horizontal, and diagonal pixels that are missing in the Bayer array data, and the full G pixel data are generated. Positional adjustment and color interpolation processing are carried out for the R (red) and B (blue) data, in accordance with positional adjustment results of G (green) data.
  • the R (red), B (blue), and G (green) data are converted to full-size RGB data.
  • it is also suitable to perform interpolation processing according to a rotational position by also detecting a rotational direction, and carrying out coordinate rotation conversion by way of affine transformation for positional adjustment.
  • it is also suitable to extract an area image of a main subject of a center or a foreground, and to perform addition after moving to a position such that an outline form or a characteristic amount of a subject or a background overlaps, and then carry out positional adjustment.
  • a plurality of images photographed with an ordinary exposure time for photography of a single image (a first exposure time)
  • a plurality of images photographed with a second exposure time for multiplane addition in which a shutter time is shortened are consecutively photographed.
  • a projective transformation matrix representing a positional relationship between adjacent images in images consecutive in time is obtained for all pairs of adjacent images, and an image with minimum blur caused by hand movement in the images of the first group photographed using the ordinary exposure time is selected. If image blur has not occurred in the subject of the image and blur caused by hand movement is sufficiently small, then the image is selected.
  • an optimal image having less blur depending on the photographing situation and to perform output thereof Furthermore, in the first group of images that are photographed using the ordinary exposure time, an image having the minimum blur caused by hand movement is selected, and if image blur of the subject has occurred in the selected image, the image for subject-blur suppression is used. Therefore, it is possible to select an optimal image having less blur depending on the photographic situation and to perform output thereof. Furthermore, if blur caused by hand movement has occurred in the images selected above, multiplane addition is applied to the images of the second group photographed for multiplane addition, and a new image is generated. Therefore, it is possible to output an optimal image having less blur depending on the photographic situation.
  • a size of the gyro sensor value at a time of image photography does not always reflect a size of blur caused by hand movement in the image, causing a hand-blur determination by way of the gyro sensor to be inaccurate.
  • a positional relationship is obtained based on a captured image after the aforementioned optical hand-blur correction, and therefore, it is possible to detect the occurrence of a blur accurately.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computing Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Spectroscopy & Molecular Physics (AREA)
  • Studio Devices (AREA)
EP10151652.4A 2009-02-17 2010-01-26 Image capturing device, image capturing method, and image capturing program Active EP2219366B1 (en)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
JP2009033913A JP4748230B2 (ja) 2009-02-17 2009-02-17 撮像装置、撮像方法及び撮像プログラム

Publications (2)

Publication Number Publication Date
EP2219366A1 EP2219366A1 (en) 2010-08-18
EP2219366B1 true EP2219366B1 (en) 2020-05-27

Family

ID=42102652

Family Applications (1)

Application Number Title Priority Date Filing Date
EP10151652.4A Active EP2219366B1 (en) 2009-02-17 2010-01-26 Image capturing device, image capturing method, and image capturing program

Country Status (7)

Country Link
US (1) US8310553B2 (zh)
EP (1) EP2219366B1 (zh)
JP (1) JP4748230B2 (zh)
KR (1) KR101109532B1 (zh)
CN (1) CN101895679B (zh)
HK (1) HK1146436A1 (zh)
TW (1) TWI420901B (zh)

Families Citing this family (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP5868053B2 (ja) * 2011-07-23 2016-02-24 キヤノン株式会社 画像処理方法、画像処理装置、およびプログラム
KR101804215B1 (ko) * 2012-03-06 2017-12-05 삼성전자주식회사 강건하게 비균일 모션 블러를 추정하는 방법 및 장치
US8928765B2 (en) * 2012-06-08 2015-01-06 Apple Inc. Noise reduction based on motion sensors
KR101904203B1 (ko) * 2012-06-20 2018-10-05 삼성전자주식회사 시프트 알고리즘을 이용하여 대용량 소스 이미지의 특징점 정보를 추출하는 장치 및 방법
US9449374B2 (en) 2014-03-17 2016-09-20 Qualcomm Incoporated System and method for multi-frame temporal de-noising using image alignment
JP6604831B2 (ja) * 2015-11-30 2019-11-13 キヤノン株式会社 画像処理装置、画像処理装置の制御方法及びプログラム
WO2017173585A1 (zh) * 2016-04-05 2017-10-12 华为技术有限公司 一种拍照方法及终端
TWI693577B (zh) 2018-09-19 2020-05-11 聚晶半導體股份有限公司 影像產生方法與電子裝置
CN113228102A (zh) * 2019-01-09 2021-08-06 奥林巴斯株式会社 图像处理装置、图像处理方法和图像处理程序
JP2020184669A (ja) * 2019-05-07 2020-11-12 シャープ株式会社 画像処理装置、撮像装置、画像処理方法、プログラム
WO2022097102A1 (en) * 2020-11-06 2022-05-12 Buyaladdin.com, Inc. Vertex interpolation in one-shot learning for object classification

Family Cites Families (23)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4285889B2 (ja) * 2000-06-30 2009-06-24 株式会社リコー 撮像装置
US7061524B2 (en) * 2001-11-13 2006-06-13 The Board Of Trustees Of The Leland Stanford Junior University Motion/saturation detection system and method for synthesizing high dynamic range motion blur free images from multiple captures
US6879731B2 (en) * 2003-04-29 2005-04-12 Microsoft Corporation System and process for generating high dynamic range video
JP2005136760A (ja) * 2003-10-31 2005-05-26 Nikon Corp デジタルスチルカメラ
US8570389B2 (en) * 2004-07-22 2013-10-29 Broadcom Corporation Enhancing digital photography
JP4378272B2 (ja) * 2004-12-15 2009-12-02 キヤノン株式会社 撮影装置
US7773115B2 (en) * 2004-12-15 2010-08-10 Texas Instruments Incorporated Method and system for deblurring digital camera images using reference image and motion estimation
US7538794B2 (en) * 2005-01-31 2009-05-26 Hewlett-Packard Development Company, L.P. Method and apparatus for motion estimation in a digital imaging device
DE602005003917T2 (de) * 2005-02-03 2008-12-04 Sony Ericsson Mobile Communications Ab Verfahren und Vorrichtung zur Erzeugung von Bildern mit hoher Dynamik aus mehreren Belichtungen
US8654201B2 (en) * 2005-02-23 2014-02-18 Hewlett-Packard Development Company, L.P. Method for deblurring an image
JP4086166B2 (ja) * 2005-03-02 2008-05-14 船井電機株式会社 電子写真装置
WO2007032082A1 (ja) * 2005-09-16 2007-03-22 Fujitsu Limited 画像処理方法及び画像処理装置
JP2007243774A (ja) * 2006-03-10 2007-09-20 Olympus Imaging Corp 電子的ぶれ補正装置
US20080170126A1 (en) 2006-05-12 2008-07-17 Nokia Corporation Method and system for image stabilization
JP4867554B2 (ja) 2006-09-29 2012-02-01 カシオ計算機株式会社 電子カメラ、撮像制御プログラム及び撮像制御方法
JP4315971B2 (ja) 2006-11-09 2009-08-19 三洋電機株式会社 撮像装置
US7825969B2 (en) * 2006-12-15 2010-11-02 Nokia Corporation Image stabilization using multi-exposure pattern
US7796872B2 (en) * 2007-01-05 2010-09-14 Invensense, Inc. Method and apparatus for producing a sharp image from a handheld device containing a gyroscope
JP4860568B2 (ja) * 2007-03-30 2012-01-25 富士フイルム株式会社 撮影装置
JP4936222B2 (ja) * 2007-06-14 2012-05-23 カシオ計算機株式会社 動きベクトル照合装置、画像合成装置及びプログラム
KR101371775B1 (ko) * 2007-07-09 2014-03-24 삼성전자주식회사 카메라의 손떨림 보정 방법 및 장치
TWI351219B (en) * 2007-09-28 2011-10-21 Altek Corp Image capturing apparatus with image-compensating function and method for compensating image thereof
KR101442610B1 (ko) * 2008-02-18 2014-09-19 삼성전자주식회사 디지털 촬영장치, 그 제어방법 및 제어방법을 실행시키기위한 프로그램을 저장한 기록매체

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
None *

Also Published As

Publication number Publication date
US8310553B2 (en) 2012-11-13
TW201101813A (en) 2011-01-01
JP2010193058A (ja) 2010-09-02
CN101895679B (zh) 2012-09-26
US20100209009A1 (en) 2010-08-19
EP2219366A1 (en) 2010-08-18
CN101895679A (zh) 2010-11-24
JP4748230B2 (ja) 2011-08-17
KR101109532B1 (ko) 2012-01-31
TWI420901B (zh) 2013-12-21
HK1146436A1 (en) 2011-06-03
KR20100094397A (ko) 2010-08-26

Similar Documents

Publication Publication Date Title
EP2219366B1 (en) Image capturing device, image capturing method, and image capturing program
US7825969B2 (en) Image stabilization using multi-exposure pattern
JP5445363B2 (ja) 画像処理装置、画像処理方法および画像処理プログラム
US8581992B2 (en) Image capturing apparatus and camera shake correction method, and computer-readable medium
JP6157242B2 (ja) 画像処理装置及び画像処理方法
TWI387330B (zh) 具有用於拍攝平移圖像之平移模式的攝像裝置
JPWO2004062270A1 (ja) 画像処理装置
US20080101710A1 (en) Image processing device and imaging device
KR102592745B1 (ko) 자세 추정 장치, 자세 추정 방법 및 기록 매체에 저장된 컴퓨터 프로그램
JP2007274504A (ja) デジタルカメラ
JP2016111568A (ja) 像ぶれ補正制御装置、撮像装置およびそれらの制御方法、プログラム
JP4942572B2 (ja) 画像加算装置および方法並びにプログラム
JP5387341B2 (ja) 撮像装置
JP4936222B2 (ja) 動きベクトル照合装置、画像合成装置及びプログラム
JP4052348B2 (ja) 画像処理装置、及び画像処理方法
JP4687619B2 (ja) 画像処理装置、画像処理方法及びプログラム
JP6218520B2 (ja) 画像処理装置、画像処理方法、及びプログラム
JP2017038281A (ja) 撮像装置およびその制御方法
JP4919165B2 (ja) 画像合成装置及びプログラム
JP5301690B2 (ja) 撮影装置および方法並びにプログラム
JP7458769B2 (ja) 画像処理装置、撮像装置、画像処理方法、プログラムおよび記録媒体
JP4374726B2 (ja) 撮影装置、及び記憶媒体
JP6478481B2 (ja) 画像処理装置、画像処理方法及びプログラム
JP2013118471A (ja) 映像処理装置およびその制御方法
JP2005223384A (ja) 撮像装置

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20100126

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO SE SI SK SM TR

AX Request for extension of the european patent

Extension state: AL BA RS

17Q First examination report despatched

Effective date: 20111215

APBK Appeal reference recorded

Free format text: ORIGINAL CODE: EPIDOSNREFNE

APBN Date of receipt of notice of appeal recorded

Free format text: ORIGINAL CODE: EPIDOSNNOA2E

APBR Date of receipt of statement of grounds of appeal recorded

Free format text: ORIGINAL CODE: EPIDOSNNOA3E

APAF Appeal reference modified

Free format text: ORIGINAL CODE: EPIDOSCREFNE

APBT Appeal procedure closed

Free format text: ORIGINAL CODE: EPIDOSNNOA9E

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

INTG Intention to grant announced

Effective date: 20191220

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE PATENT HAS BEEN GRANTED

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 1275787

Country of ref document: AT

Kind code of ref document: T

Effective date: 20200615

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602010064438

Country of ref document: DE

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200527

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200828

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200527

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200827

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200527

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200928

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200927

REG Reference to a national code

Ref country code: NL

Ref legal event code: MP

Effective date: 20200527

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200527

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200527

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200827

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 1275787

Country of ref document: AT

Kind code of ref document: T

Effective date: 20200527

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200527

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200527

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200527

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200527

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200527

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200527

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200527

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200527

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200527

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200527

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200527

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602010064438

Country of ref document: DE

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

26N No opposition filed

Effective date: 20210302

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200527

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200527

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20210126

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20210126

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20210131

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20210131

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20210131

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20210131

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20210126

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20210126

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20210131

REG Reference to a national code

Ref country code: DE

Ref legal event code: R079

Ref document number: 602010064438

Country of ref document: DE

Free format text: PREVIOUS MAIN CLASS: H04N0005232000

Ipc: H04N0023600000

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20100126

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200527

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200527

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20231128

Year of fee payment: 15