EP2059046A1 - Verfahren und System zum Kombinieren von Videos zur Anzeige in Echtzeit - Google Patents

Verfahren und System zum Kombinieren von Videos zur Anzeige in Echtzeit Download PDF

Info

Publication number
EP2059046A1
EP2059046A1 EP08017405A EP08017405A EP2059046A1 EP 2059046 A1 EP2059046 A1 EP 2059046A1 EP 08017405 A EP08017405 A EP 08017405A EP 08017405 A EP08017405 A EP 08017405A EP 2059046 A1 EP2059046 A1 EP 2059046A1
Authority
EP
European Patent Office
Prior art keywords
angle
narrow
videos
video
wide
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
EP08017405A
Other languages
English (en)
French (fr)
Other versions
EP2059046B1 (de
Inventor
Jeroen Van Baar
Wojciech Matusik
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Mitsubishi Electric Corp
Original Assignee
Mitsubishi Electric Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Mitsubishi Electric Corp filed Critical Mitsubishi Electric Corp
Publication of EP2059046A1 publication Critical patent/EP2059046A1/de
Application granted granted Critical
Publication of EP2059046B1 publication Critical patent/EP2059046B1/de
Not-in-force legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/222—Studio circuitry; Studio devices; Studio equipment
    • H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
    • H04N5/2624—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects for obtaining an image which is composed of whole input images, e.g. splitscreen
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/90—Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/222—Studio circuitry; Studio devices; Studio equipment
    • H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
    • H04N5/2625—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects for obtaining an image which is composed of images from a temporal image sequence, e.g. for a stroboscopic effect
    • H04N5/2627—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects for obtaining an image which is composed of images from a temporal image sequence, e.g. for a stroboscopic effect for providing spin image effect, 3D stop motion effect or temporal freeze effect
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/222—Studio circuitry; Studio devices; Studio equipment
    • H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
    • H04N5/2628—Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • H04N7/181—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources

Definitions

  • This invention relates generally to image processing, and more particularly to combining multiple input image sequences to generate a single output image sequence.
  • an output image can be generated from multiple input images.
  • Compositing combines visual elements (objects) from separate input images to create the illusion that all of the elements are parts of the same scene.
  • Mosaics and panoramas combine entire input images into a single output image.
  • a mosaic consists of non-overlapping images arranged in some tessellation.
  • a panorama usually refers to a wide-angle representation of a view.
  • parallax analysis motion parallax is used to estimate a 3D structure of a scene, which allows the images to be combined.
  • Layer decomposition is generally restricted to scenes that can be decomposed into multiple depth layers.
  • Pixel correspondences require stereo techniques and depth estimation.
  • the output image often includes annoying artifacts, such as streaks and halos at depth edges.
  • the prior art methods are complex and not suitable for real-time applications.
  • a set of input videos is acquired of a scene by multiple narrow-angle cameras. Each camera has a different field of view of the scene. That is, the fields of view are substantially abutting with minimal overlap.
  • a wide-angle camera acquires a wide-angle input video of the entire scene.
  • a field of view of the wide-angle camera substantially overlaps the fields of view of the set of narrow-angle cameras.
  • a resolution of the output video is approximately the sum of the resolutions of the input videos.
  • the invention uses the wide-angle videos for correcting and combing the narrow-angle videos.
  • Correction is not limited to geometrical correction, as in the prior art, but also includes colorimetric correction. Colorimetric correction ensures that the output video can be displayed with uniform color and gain as if the output video was acquired by a single camera.
  • the invention also has as an objective the simultaneous acquisition and display of the videos with real-time performance.
  • the invention does not require manual alignment and camera calibration. The amount of overlap, if any, between the views of the cameras can be minimized.
  • Figure 1A is a schematic of a system for combining input videos to generate an output video according to an embodiment of the invention
  • Figure 1B is a schematic of a set of narrow-angle input images and a wide angle input image
  • Figure 2 is a flow diagram of a method for combining input videos to generate an output video according to an embodiment of the invention
  • Figure 3 is a front view of a display device according to an embodiment of the invention.
  • Figure 4 shows an offset parameter according to an embodiment of the invention.
  • Figure 1 shows a system for combining a set of narrow-angle input videos 111 acquired of a scene by a set of narrow-angle cameras 101 to generate an output video 110 in real-time for a display device 108 according to an embodiment of our invention.
  • the input videos 111 are combined using a wide-angle input video 112 acquired by a wide-angle camera 102.
  • the output video 110 can be presented on a display device 108.
  • the display device includes a set of projection display devices.
  • the projectors can be front or rear.
  • Figure 1B shows a set of narrow angle images 111.
  • Image 111' is a reference image described below.
  • the wide-angle image 112 is indicated by dashes.
  • the input images do not need to be rectangular.
  • the dotted line 301 is for one display screen, and the solid line 302 indicates a largest inscribed rectangle.
  • wide-angle and narrow-angle as used herein are simply relative. That is, the field of view of the wide-angle camera 102 substantially overlaps the fields of view of the narrow-angle cameras 101.
  • the narrow-angle cameras basically have a normal angle, and the wide-angle camera simply has a zoom factor of 2x.
  • Our wide-angle camera should not be confused with a conventional fish-eye lens camera, which takes an extremely wide, hemispherical image. Our wide-angle camera does not have any noticeable distortion. If we use a conventional fish-eye lens, then we can correct the distortion of image 112 according to the lens distortion parameters.
  • the field of view of the wide-angle camera 102 should encompass the combined field of views of the set of narrow-angle cameras 101.
  • the field of view of the wide-angle camera 102 is slightly larger than the combined views of the four narrow-angle cameras 101. Therefore, the resolution of the output video is approximately the sum of the resolutions of the set of input videos 111.
  • the cameras 101-102 are connected to a cluster of computers 103 via a network 104.
  • the computers are conventional and include processors, memories and input/output interfaces by buses.
  • the computers implement the method according to our invention.
  • the use of a wide-angle camera in our invention has several advantages. First, the overlap, if any, between the set of input videos 111 can be minimal. Second, misalignment errors are negligible. Third, the invention can be applied to complex scenes. Fourth, the output video can be corrected for both geometry and color.
  • the wide-angle resolution video 112 provides both geometry and color correction information.
  • the narrow-angle cameras 101 are arranged in a 2 x 2 array, and the single wide-angle camera 102 is arranged above or between the narrow-angle cameras as shown in Figure 1A .
  • the field of view of the wide-angle camera combines the fields of view of the narrow-angle cameras 101.
  • Each camera is connected to one of the computers 103 via the network 104.
  • Each computer is equipped with graphics hardware comprising a graphics processing unit (GPU) 105.
  • GPU graphics processing unit
  • the frame rates of the cameras are synchronized. However, this is not necessary if the number of moving elements (pixels) in the scene is small.
  • a modem GPU such as used for high-speed computer graphic applications, can process images extremely fast, i.e., in real-time. Therefore, we load the GPU with transformation and geometry parameters to combine and transform the input videos in real-time as described below.
  • Each computer and GPU is connected to the display device 108 on which the output video is displayed.
  • the display device 108 on which the output video is displayed.
  • Each display is connected to one of the computers.
  • the invention can also be worked with different combinations of computers, GPUs and display devices.
  • the invention can be worked with a single computer, GPU and display device, and multiple cameras.
  • FIG. 2 shows details of the method according to the invention.
  • the selected images are acquired at about the same time. For example, the first image in each video. Exact correspondence in timing can be achieved by synchronizing the cameras. It should be noted, that set 200 of temporally corresponding images could be selected periodically to update GPU parameters as described below as needed.
  • the homography enables us to determine 250 the geometries 251 for a single largest inscribed rectangular image 302 that encompasses the transformed image.
  • the geometry also takes into consideration a geometry of the display device 108, e.g., the arrangement and size of the one (or more) display screens.
  • the display geometry defines an appearance of the output video.
  • the size can be specified in terms of pixels, e.g., the width and height, or the width and aspect ratio.
  • the homographies 231 between the narrow-angle videos and the geometry of the output video are stored in the GPUs 105 of the various processors 103.
  • subsequent images in the set of narrow-angle input videos 111 can be streamed 260 through the GPUs to produce the output video 110 in real-time according to the homographies and the geometry of the display screen.
  • the GPU parameters can be updated dynamically as needed to adapt to a changing environment while streaming.
  • the homographies, geometries and color correction can be periodically updated in the GPUs, e.g., once a minute or some other interval, to accommodate a changing scene and varying lighting conditions. This is particularly appropriate for outdoor scenes, where large objects can periodically enter and leave the scene.
  • the updating can also be sensitive to moving objects or shadows in the scene.
  • a scale invariant feature detector e.g., a scale invariant feature transformation (SIFT), Lowe, "Distinctive image features from scale invariant keypoints," International Journal of Computer Vision, 60(2):91-110, 2004 , incorporated herein by reference.
  • SIFT scale invariant feature transformation
  • Other feature detectors such as a corner and line (edge) detectors can either be used instead, or to increase the number of features. It should be noted, that the feature detection can be accelerated by using the GPUs.
  • the perspective transformation 240 during the combining can be approximated by 3x3 projective transformation matrices, or homographies 231.
  • the homographies are determined from the correspondences 221 of the features 211. Given that some of the correspondence candidates could be falsely matched, we use a modified RANSAC approach to determine the homographies, Fischler et al., "Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography," Commun. ACM, 24(6):381-395, 1981 , incorporated herein by reference.
  • the lens distortion is corrected by estimating parameters of the first two terms of a power series. If the lens distortion parameters are known, than the correction can be implemented on the GPU as per pixel look-up operations.
  • Figure 3 is a front view of four display devices.
  • the dashed lines 301 indicate the seams between four display screens.
  • the first step locates the largest rectangle 302 inside the transformed and combined image 241.
  • the largest rectangle can also conform to the aspect ratio of the display device.
  • the parameters that are stored in the GPUs include the 3x3 homographies used to transform the narrow-angle images to the coordinate system of the selected reference image 111', the x and y offset 401 for each transformed image, see Figure 4 , and the size (width and height) of each transformed input image.
  • the offsets and size are determined from the combined image 241 and the configuration of the display device 108.
  • each image is transformed using the homographies 231.
  • the transformation with the homography is a projective transformation. This operation is supported by the GPU 105.
  • the GPU can perform the resizing to match the display geometry by interpolations within its texture function.
  • the images can be blended into the output video using a multiband blending technique, U.S. 6,755,537 , "Method for globally aligning multiple projected images,” issued to Raskar et al., June 29, 2004, incorporated herein by reference.
  • the blending maintains a uniform intensity across the output image.
  • Our color correction method includes the following steps. We determine a cluster of pixels in a local neighborhood near each feature in each input image 111. We match the cluster of pixels with adj acent or nearby clusters of pixels. Then, we determine an offset and 3x3 color transform between the images.
  • the above color transform is based on the content of the input images. To avoid some colors being overrepresented, we can track the peaks of the 3D histogram that are included. Peak locations that are already represented are skipped in favor of locations that have not yet been included.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Processing (AREA)
  • Studio Devices (AREA)
EP08017405A 2007-11-09 2008-10-02 Verfahren und System zum Kombinieren von Videos zur Anzeige in Echtzeit Not-in-force EP2059046B1 (de)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
US11/937,659 US20090122195A1 (en) 2007-11-09 2007-11-09 System and Method for Combining Image Sequences

Publications (2)

Publication Number Publication Date
EP2059046A1 true EP2059046A1 (de) 2009-05-13
EP2059046B1 EP2059046B1 (de) 2010-08-25

Family

ID=40344687

Family Applications (1)

Application Number Title Priority Date Filing Date
EP08017405A Not-in-force EP2059046B1 (de) 2007-11-09 2008-10-02 Verfahren und System zum Kombinieren von Videos zur Anzeige in Echtzeit

Country Status (5)

Country Link
US (1) US20090122195A1 (de)
EP (1) EP2059046B1 (de)
JP (1) JP2009124685A (de)
CN (1) CN101431617B (de)
DE (1) DE602008002303D1 (de)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB2471708A (en) * 2009-07-09 2011-01-12 Thales Holdings Uk Plc Image combining with light point enhancements and geometric transforms
US10996460B2 (en) 2017-04-13 2021-05-04 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Multi-aperture imaging device, imaging system and method of providing a multi-aperture imaging device
US11070731B2 (en) 2017-03-10 2021-07-20 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Multi-aperture imaging device, imaging system and method for making available a multi-aperture imaging device
CN114727009A (zh) * 2017-04-13 2022-07-08 弗劳恩霍夫应用研究促进协会 用于对局部视场成像的设备、多孔径成像设备及提供这些设备的方法

Families Citing this family (66)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TW200910937A (en) * 2007-08-29 2009-03-01 Imagia Technologies Co Ltd The generation device of continuous crisscross scenes
CN101562706B (zh) * 2009-05-22 2012-04-18 杭州华三通信技术有限公司 一种图像拼接方法和设备
US8817071B2 (en) 2009-11-17 2014-08-26 Seiko Epson Corporation Context constrained novel view interpolation
US9594960B2 (en) * 2010-09-14 2017-03-14 Microsoft Technology Licensing, Llc Visualizing video within existing still images
US20130083196A1 (en) * 2011-10-01 2013-04-04 Sun Management, Llc Vehicle monitoring systems
US8666159B1 (en) 2012-06-04 2014-03-04 Google Inc. Real time feature extraction
WO2014083489A1 (en) 2012-11-28 2014-06-05 Corephotonics Ltd. High-resolution thin multi-aperture imaging systems
KR101417527B1 (ko) * 2012-12-28 2014-07-10 한국항공우주연구원 항공기에서 촬영되는 항공 영상을 이용하는 지형 변화 탐지 장치 및 방법
CN109040553B (zh) 2013-06-13 2021-04-13 核心光电有限公司 双孔径变焦数字摄影机
CN105359006B (zh) 2013-07-04 2018-06-22 核心光电有限公司 小型长焦透镜套件
CN109120823B (zh) 2013-08-01 2020-07-14 核心光电有限公司 具有自动聚焦的纤薄多孔径成像系统及其使用方法
KR101583888B1 (ko) 2013-12-19 2016-01-08 현대자동차주식회사 스파클 감이 향상된 도료 조성물과 그 도장방법
CN103679665A (zh) * 2014-01-03 2014-03-26 北京华力创通科技股份有限公司 几何校正方法及装置
US9392188B2 (en) 2014-08-10 2016-07-12 Corephotonics Ltd. Zoom dual-aperture camera with folded lens
CN107209404B (zh) 2015-01-03 2021-01-15 核心光电有限公司 微型长焦镜头模块和使用该镜头模块的相机
EP3492958B1 (de) 2015-04-02 2022-03-30 Corephotonics Ltd. Doppelschwingspulenmotorstruktur in einer dualoptischen modulkamera
CN111175926B (zh) 2015-04-16 2021-08-20 核心光电有限公司 紧凑型折叠式相机中的自动对焦和光学图像稳定
KR102007379B1 (ko) 2015-05-28 2019-08-05 코어포토닉스 리미티드 이중-조리개 디지털 카메라의 광학식 손떨림 방지 및 자동-초점을 위한 양-방향성 강성
CN112672022B (zh) 2015-08-13 2022-08-02 核心光电有限公司 视频支持和切换/无切换动态控制的双孔径变焦摄影机
CN105141920B (zh) * 2015-09-01 2018-06-19 电子科技大学 一种360度全景视频拼接系统
KR102225727B1 (ko) 2015-09-06 2021-03-10 코어포토닉스 리미티드 소형의 접이식 카메라의 롤 보정에 의한 자동 초점 및 광학식 손떨림 방지
US20170118475A1 (en) * 2015-10-22 2017-04-27 Mediatek Inc. Method and Apparatus of Video Compression for Non-stitched Panoramic Contents
US10578948B2 (en) 2015-12-29 2020-03-03 Corephotonics Ltd. Dual-aperture zoom digital camera with automatic adjustable tele field of view
EP3518520B1 (de) 2016-05-30 2020-08-26 Corephotonics Ltd. Kugellagergeführter schwingspulenmotor
EP3381181B1 (de) 2016-06-19 2022-04-06 Corephotonics Ltd. Rahmensynchronisation in einem kamerasystem mit dualer blende
WO2018007951A1 (en) 2016-07-07 2018-01-11 Corephotonics Ltd. Dual-camera system with improved video smooth transition by image blending
EP4224233A1 (de) 2016-07-07 2023-08-09 Corephotonics Ltd. Linearkugelgeführter schwingspulenmotor für gefaltete optik
CN107644394B (zh) * 2016-07-21 2021-03-30 完美幻境(北京)科技有限公司 一种3d图像的处理方法及装置
US20180101813A1 (en) * 2016-10-12 2018-04-12 Bossa Nova Robotics Ip, Inc. Method and System for Product Data Review
US10762653B2 (en) * 2016-12-27 2020-09-01 Canon Kabushiki Kaisha Generation apparatus of virtual viewpoint image, generation method, and storage medium
EP3563193B1 (de) 2016-12-28 2021-03-31 Corephotonics Ltd. Gefaltete kamerastruktur mit einem erweiterten lichtfaltelement-scanbereich
US10970915B2 (en) * 2017-01-06 2021-04-06 Canon Kabushiki Kaisha Virtual viewpoint setting apparatus that sets a virtual viewpoint according to a determined common image capturing area of a plurality of image capturing apparatuses, and related setting method and storage medium
CN113805405B (zh) 2017-01-12 2023-05-23 核心光电有限公司 紧凑型折叠式摄影机及其组装方法
IL302577A (en) 2017-02-23 2023-07-01 Corephotonics Ltd Lens designs for a folded camera
CN114137790A (zh) 2017-03-15 2022-03-04 核心光电有限公司 具有全景扫瞄范围的系统、移动电子装置及其方法
US10904512B2 (en) 2017-09-06 2021-01-26 Corephotonics Ltd. Combined stereoscopic and phase detection depth mapping in a dual aperture camera
US10951834B2 (en) 2017-10-03 2021-03-16 Corephotonics Ltd. Synthetically enlarged camera aperture
CN110140076B (zh) 2017-11-23 2021-05-21 核心光电有限公司 紧凑型折叠式摄影机结构
CN114609746A (zh) 2018-02-05 2022-06-10 核心光电有限公司 折叠摄像装置
KR102494003B1 (ko) 2018-02-12 2023-01-30 코어포토닉스 리미티드 광학 이미지 안정화 기능을 갖는 폴디드 카메라
US10694168B2 (en) 2018-04-22 2020-06-23 Corephotonics Ltd. System and method for mitigating or preventing eye damage from structured light IR/NIR projector systems
EP4109174A1 (de) 2018-04-23 2022-12-28 Corephotonics Ltd. Strahlengangfaltelement mit erweitertem drehbereich mit zwei freiheitsgraden
CN108564557B (zh) * 2018-05-31 2020-08-25 京东方科技集团股份有限公司 图像校正方法及装置
KR102289149B1 (ko) 2018-08-04 2021-08-12 코어포토닉스 리미티드 카메라 위의 전환 가능한 연속 디스플레이 정보 시스템
WO2020039302A1 (en) 2018-08-22 2020-02-27 Corephotonics Ltd. Two-state zoom folded camera
US11287081B2 (en) 2019-01-07 2022-03-29 Corephotonics Ltd. Rotation mechanism with sliding joint
WO2020183312A1 (en) 2019-03-09 2020-09-17 Corephotonics Ltd. System and method for dynamic stereoscopic calibration
CN110180151A (zh) * 2019-05-06 2019-08-30 南昌嘉研科技有限公司 一种游泳训练辅助系统
CN110223250B (zh) * 2019-06-02 2021-11-30 西安电子科技大学 基于单应变换的sar几何校正方法
KR102365748B1 (ko) 2019-07-31 2022-02-23 코어포토닉스 리미티드 카메라 패닝 또는 모션에서 배경 블러링을 생성하는 시스템 및 방법
US11659135B2 (en) 2019-10-30 2023-05-23 Corephotonics Ltd. Slow or fast motion video using depth information
IT201900021399A1 (it) * 2019-11-18 2021-05-18 Telecom Italia Spa Metodo e sistema per lo stitching video
US11949976B2 (en) 2019-12-09 2024-04-02 Corephotonics Ltd. Systems and methods for obtaining a smart panoramic image
CN114641983A (zh) 2019-12-09 2022-06-17 核心光电有限公司 用于获得智能全景图像的系统及方法
EP4052176A4 (de) 2020-02-22 2023-01-04 Corephotonics Ltd. Geteilte bildschirmfunktion für makrofotografie
CN111277764B (zh) * 2020-03-10 2021-06-01 西安卓越视讯科技有限公司 基于gpu加速的4k实时视频全景拼接方法
KR20230159624A (ko) 2020-04-26 2023-11-21 코어포토닉스 리미티드 홀 바 센서 보정을 위한 온도 제어
CN117372250A (zh) 2020-05-17 2024-01-09 核心光电有限公司 全视场参考图像的图像拼接
CN113747002B (zh) * 2020-05-29 2023-04-07 青岛海信移动通信技术股份有限公司 终端及图像拍摄方法
KR20240001277A (ko) 2020-05-30 2024-01-03 코어포토닉스 리미티드 슈퍼 매크로 이미지를 얻기 위한 시스템 및 방법
US11637977B2 (en) 2020-07-15 2023-04-25 Corephotonics Ltd. Image sensors and sensing methods to obtain time-of-flight and phase detection information
CN114730064A (zh) 2020-07-15 2022-07-08 核心光电有限公司 扫描折叠相机的视点像差校正
KR102640229B1 (ko) 2020-07-31 2024-02-22 코어포토닉스 리미티드 큰 스트로크 선형 위치 감지를 위한 홀 센서-자석 구조
CN116679420A (zh) 2020-08-12 2023-09-01 核心光电有限公司 用于光学防抖的方法
TW202422200A (zh) 2021-03-11 2024-06-01 以色列商核心光電有限公司 彈出式照相機系統
KR102638173B1 (ko) 2021-06-08 2024-02-19 코어포토닉스 리미티드 슈퍼-매크로 이미지의 초점면을 틸팅하기 위한 시스템 및 카메라

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0946058A1 (de) * 1996-09-10 1999-09-29 Sergei Ivanovich Miroshnichenko Hochauflösendes fernsehsystem
WO2002093916A2 (en) 2001-05-14 2002-11-21 Elder James H Attentive panoramic visual sensor
US20030034982A1 (en) * 2001-08-02 2003-02-20 Jacky Talayssat Device for graphically processing video objects
US20040017386A1 (en) * 2002-07-26 2004-01-29 Qiong Liu Capturing and producing shared multi-resolution video
US20060139233A1 (en) 2003-06-27 2006-06-29 Neale Adam R Image display apparatus for displaying composite images

Family Cites Families (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH06225192A (ja) * 1993-01-25 1994-08-12 Sony Corp パノラマ効果用カメラ
US6078701A (en) * 1997-08-01 2000-06-20 Sarnoff Corporation Method and apparatus for performing local to global multiframe alignment to construct mosaic images
JP2002158922A (ja) * 2000-11-20 2002-05-31 Fuji Photo Film Co Ltd 画像表示制御装置および方法
JP2003333425A (ja) * 2002-05-15 2003-11-21 Fuji Photo Film Co Ltd カメラ
US7321386B2 (en) * 2002-08-01 2008-01-22 Siemens Corporate Research, Inc. Robust stereo-driven video-based surveillance
JP2004135209A (ja) * 2002-10-15 2004-04-30 Hitachi Ltd 広視野高解像度映像の生成装置及び方法
SE0302065D0 (sv) * 2003-07-14 2003-07-14 Stefan Carlsson Video - method and apparatus
KR101042638B1 (ko) * 2004-07-27 2011-06-20 삼성전자주식회사 파노라마 영상 생성을 위한 디지털 촬상장치 및 그 생성방법
CA2587644C (en) * 2004-11-12 2015-01-13 Mok3, Inc. Method for inter-scene transitions
JP2006171939A (ja) * 2004-12-14 2006-06-29 Canon Inc 画像処理装置及び方法
US7646400B2 (en) * 2005-02-11 2010-01-12 Creative Technology Ltd Method and apparatus for forming a panoramic image
JP2007043466A (ja) * 2005-08-03 2007-02-15 Mitsubishi Electric Corp 画像合成装置並びに多カメラ監視システム
CN1958266B (zh) * 2005-11-04 2011-06-22 鸿富锦精密工业（深圳）有限公司 模具结构
US20070250898A1 (en) * 2006-03-28 2007-10-25 Object Video, Inc. Automatic extraction of secondary video streams
US7719568B2 (en) * 2006-12-16 2010-05-18 National Chiao Tung University Image processing system for integrating multi-resolution images

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0946058A1 (de) * 1996-09-10 1999-09-29 Sergei Ivanovich Miroshnichenko Hochauflösendes fernsehsystem
WO2002093916A2 (en) 2001-05-14 2002-11-21 Elder James H Attentive panoramic visual sensor
US20030034982A1 (en) * 2001-08-02 2003-02-20 Jacky Talayssat Device for graphically processing video objects
US20040017386A1 (en) * 2002-07-26 2004-01-29 Qiong Liu Capturing and producing shared multi-resolution video
US20060139233A1 (en) 2003-06-27 2006-06-29 Neale Adam R Image display apparatus for displaying composite images

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB2471708A (en) * 2009-07-09 2011-01-12 Thales Holdings Uk Plc Image combining with light point enhancements and geometric transforms
US11070731B2 (en) 2017-03-10 2021-07-20 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Multi-aperture imaging device, imaging system and method for making available a multi-aperture imaging device
US10996460B2 (en) 2017-04-13 2021-05-04 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Multi-aperture imaging device, imaging system and method of providing a multi-aperture imaging device
CN114727009A (zh) * 2017-04-13 2022-07-08 弗劳恩霍夫应用研究促进协会 用于对局部视场成像的设备、多孔径成像设备及提供这些设备的方法
US11457152B2 (en) 2017-04-13 2022-09-27 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Device for imaging partial fields of view, multi-aperture imaging device and method of providing same
EP3610637B1 (de) * 2017-04-13 2024-05-01 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Vorrichtungen zur abbildung von teilgesichtsfeldern, multiaperturabbildungsvorrichtungen und verfahren zum bereitstellen derselben
EP4376404A3 (de) * 2017-04-13 2024-07-24 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Vorrichtung zur abbildung von teilgesichtsfeldern, multiaperturabbildungsvorrichtung und verfahren zum bereitstellen derselben

Also Published As

Publication number Publication date
DE602008002303D1 (de) 2010-10-07
US20090122195A1 (en) 2009-05-14
CN101431617A (zh) 2009-05-13
CN101431617B (zh) 2011-07-27
JP2009124685A (ja) 2009-06-04
EP2059046B1 (de) 2010-08-25

Similar Documents

Publication Publication Date Title
EP2059046B1 (de) Verfahren und System zum Kombinieren von Videos zur Anzeige in Echtzeit
US7864215B2 (en) Method and device for generating wide image sequences
EP3163535B1 (de) Weitbereichsbilderfassungsverfahren und -vorrichtung
EP1421795B1 (de) Mehrprojektorsmosaic mit automatischer positionierung
EP3111647B1 (de) Aneinanderheften von bildern und automatische farbkorrektur
US9398215B2 (en) Stereoscopic panoramas
US7855752B2 (en) Method and system for producing seamless composite images having non-uniform resolution from a multi-imager system
EP1813105B1 (de) Verfahren und systeme zum erzeugen von nahtlos zusammengestzten bildern ohne notwendigkeit von überlappung der quellbilder
US9307165B2 (en) In-camera panorama image stitching assistance
US7006709B2 (en) System and method deghosting mosaics using multiperspective plane sweep
US9195121B2 (en) Markerless geometric registration of multiple projectors on extruded surfaces using an uncalibrated camera
TWI383680B (zh) 整合式影像監視系統及其方法
US20040257540A1 (en) Single or multi-projector for arbitrary surfaces without calibration nor reconstruction
US20080253685A1 (en) Image and video stitching and viewing method and system
WO2007149323A2 (en) Mesh for rendering an image frame
US10805534B2 (en) Image processing apparatus and method using video signal of planar coordinate system and spherical coordinate system
US7813578B2 (en) Method and apparatus for unobtrusively correcting projected image
US8019180B2 (en) Constructing arbitrary-plane and multi-arbitrary-plane mosaic composite images from a multi-imager
Liu et al. Head-size equalization for better visual perception of video conferencing
US8149260B2 (en) Methods and systems for producing seamless composite images without requiring overlap of source images
Patidar et al. Automatic image mosaicing: an approach based on FFT
Mehta et al. Image stitching techniques
JP2009141508A (ja) テレビ会議装置、テレビ会議方法、プログラムおよび記録媒体
Raskar et al. P‐153: Low‐Cost Multi‐Projector Curved Screen Displays
JPH1032755A (ja) 大画面画像合成処理装置

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MT NL NO PL PT RO SE SI SK TR

AX Request for extension of the european patent

Extension state: AL BA MK RS

17P Request for examination filed

Effective date: 20090428

17Q First examination report despatched

Effective date: 20090618

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

AKX Designation fees paid

Designated state(s): DE FR GB

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): DE FR GB

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REF Corresponds to:

Ref document number: 602008002303

Country of ref document: DE

Date of ref document: 20101007

Kind code of ref document: P

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

26N No opposition filed

Effective date: 20110526

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602008002303

Country of ref document: DE

Effective date: 20110526

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 8

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20150930

Year of fee payment: 8

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20150908

Year of fee payment: 8

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20150929

Year of fee payment: 8

REG Reference to a national code

Ref country code: DE

Ref legal event code: R119

Ref document number: 602008002303

Country of ref document: DE

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20161002

REG Reference to a national code

Ref country code: FR

Ref legal event code: ST

Effective date: 20170630

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20161002

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20161102

Ref country code: DE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170503