EP1142332A1 - Camera a vision peripherique - Google Patents

Camera a vision peripherique

Info

Publication number
EP1142332A1
EP1142332A1 EP00969431A EP00969431A EP1142332A1 EP 1142332 A1 EP1142332 A1 EP 1142332A1 EP 00969431 A EP00969431 A EP 00969431A EP 00969431 A EP00969431 A EP 00969431A EP 1142332 A1 EP1142332 A1 EP 1142332A1
Authority
EP
European Patent Office
Prior art keywords
camera
view
field
hemispherical
assembly
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Withdrawn
Application number
EP00969431A
Other languages
German (de)
English (en)
Inventor
Hugo Strubbe
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Koninklijke Philips NV
Original Assignee
Koninklijke Philips Electronics NV
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Koninklijke Philips Electronics NV filed Critical Koninklijke Philips Electronics NV
Publication of EP1142332A1 publication Critical patent/EP1142332A1/fr
Withdrawn legal-status Critical Current

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/14—Systems for two-way working
    • H04N7/141—Systems for two-way working between two video terminals, e.g. videophone
    • H04N7/142—Constructional details of the terminal equipment, e.g. arrangements of the camera and the display
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/50—Constructional details
    • H04N23/55—Optical parts specially adapted for electronic image sensors; Mounting thereof
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • H04N7/181—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources

Definitions

  • This invention relates to a camera viewing system with peripheral vision.
  • Camera viewing systems are now used for a variety of purposes including surveillance, security, video conferencing, etc., and, ideally, allow for simultaneous remote viewing, close viewing, and wide-angle and include systems in which a camera converts the optical image into an electrical signal, the image signals are digitized electronically, the digitized images are transformed by image processors controlled by a microcomputer into two or three dimensional images which are stored in an output image buffer or system, and the stored images are in turn scanned out by display electronics to a video display device for viewing.
  • This limited field of view may be expanded by tilting and panning as is well known in the art but such a system suffers from drawbacks including the significant amount of time required to make a full rotation to view the expanded view making the system unsuitable for real-time applications.
  • Omni-directional cameras can capture a hemispherical field of view but the captured images have limited resolution and a blind spot on top where the camera is mounted.
  • Systems which employ multiple cameras with a corresponding reflector or with multiple reflectors pointing in different directions, may also be employed to achieve a large field of view. In practice, however, it is difficult to design such a configuration in which the centers of projection of all the imaging systems coincide so that a true omnidirectional system may be achieved.
  • a single planar mirror does not enhance the field of view of a traditional imaging system.
  • Multiple planar mirrors (each pointing in a different direction) can be used with a single imaging system. However, for any 2 given imaging system, each planar mirror produces a different center of projection. Multiple planar mirrors therefore require the use of multiple imaging systems (multiple lens and multiple image detectors).
  • a fish-eye lens can also be used to obtain a hemispherical field of view. Such fish-eye lens cameras have a short focal length and can expand the field of view to be as large as a hemisphere but are larger and more complex than conventional lenses and often distort the center field of view (as well as the periphery).
  • An object of the invention is to provide camera viewing systems with a panoramic or hemispherical field of view with peripheral vision that efficiently capture and display visual information within a substantially hemispheric or entire field of view containing particularly peripheral content, and that permit electronic manipulation and display of the image while minimizing distortion effects.
  • a computer vision system for producing a substantially hemispheric or panoramic image comprising a camera viewing system adapted to provide a substantially panoramic or hemispherical image wherein the camera viewing system comprises at least one dual opposing camera assembly in which at least two cameras or camera assemblies are mounted in back-to-back relationship, i.e.
  • At least one said camera or camera assembly providing at least a hemispherical field of view and at least one said camera or camera assembly providing a direct forward field of view that is less than hemispherical to collectively display a substantially hemispherical or entire field of view, and provide at least one image signal representative of said image; and further comprising an image signal processing apparatus coupled to receive said at least one image signal to convert said image signal into image signal data; means to convert said image signal data to form a video image, either analog or digital; and a display device to permit the viewing of the video data.
  • a camera viewing system which comprises: at least one dual opposing camera assembly having a first camera effective to provide a direct forward field of view that is less than hemispherical, and a second camera effective to provide a hemispherical field of view that includes areas of a field of view that is peripheral to the field of view of said first camera, said first and second cameras being mounted in back- to-back relationship one to another.
  • a computer vision system which comprises a camera viewing system in which a first camera or camera and lens assembly is effective to provide a direct forward field of view that is less than hemispherical, a second camera or camera and lens assembly which has the same optical axis as the first camera or camera and lens assembly is arranged in back-to-back relationship therewith; a mirror is arranged to reflect light from a hemispherical field of view, said second camera or camera and lens assembly and said mirror being effective to provide a reflected hemispherical field of view that includes areas of a field of view that is peripheral to the field of view of said first camera or camera and lens assembly.
  • the second camera or camera and lens assembly is arranged to receive light that is reflected from the mirror, i.e. the second camera or camera and lens assembly "looks off the mirror.
  • the mirror is oriented to a line which is the common optical axis of the back-to-back mounted first and second camera or camera and lens assemblies.
  • Cameras and camera and lens assemblies (hereafter collectively referred to as camera assembly or camera assemblies) suitable for use in this invention may be any optical device that is capable of receiving a focussed image from any source (preferably from a lens) and transforming that image into an electronic signal or into hard copy storage such as photographic film.
  • Any video source and image capturing device that captures an image and converts it into memory, electronic or otherwise, can serve as the first camera assembly herein.
  • the image can be stored onto an assortment of media, e.g., photographic film, electronic storage, or other conventional storage means. Electronic storage is preferred because of the ease of electronic manipulation thereof and the invention is further described herein in terms of electronic media.
  • the image processing apparatus will include an image scanner or other capture-and-conversion method to change the image into electronic format so that electronic manipulation may be performed.
  • the system may also include means for digitizing an incoming video image signal, transforming a portion of the video image, if necessary, and producing one or more output images for viewing in a teleconferencing environment.
  • Input and output buffers can be constructed using commercially available video random access memory chips.
  • a control interface can be accomplished using microcontrollers and transform processors, image filtering and display by means well known in the art. Transmission may be via telephone lines, with the system including signal compression and decompression units, or wireless transmission with appropriate broadcasting apparatus may be utilized if desired.
  • the display device may be a monitor, television, virtual reality helmet or other virtual reality display system or a projection display.
  • the display device is a monitor, which may be conventional or a specially designed modular LCD monitor; it may be a single-view or multi-view monitor; and most preferably is a split screen monitor.
  • the monitor may also comprise a picture-in-picture feature, if desired.
  • the split-screen monitor is particularly preferred when used with the imaging systems of the invention since such a monitor enables the user to zoom in on the speaker ("X" in the figures) while retaining the peripheral scene ("y” or “yy” and “z” or “zz” in the figures) and while maintaining a high resolution of both areas of the substantially panoraic or hemispheric scene in real-time.
  • Each camera assembly is schematically illustrated as comprising a lens and an imaging device such as a CCD imager.
  • a conventional camera with zoom feature has been used with particular success as the first camera assembly.
  • various types of cameras for wide-angle viewing such as NTSC, and HDTV-type cameras may be used.
  • the camera assembly can be mounted and supported in conventional manner.
  • the second camera assembly will be any device that captures or senses an image or receives light within a hemispherical field of view and may be a conventional camcorder or an omnidirectional video camera which provides a digital video signal output.
  • each sensing unit will be an assembly commonly referred to as an omnicam and comprised of an off-the-shelf video camera, an off-the-shelf lens, and a curved mirror which provides a single effective center of projection for the complete sensing unit. This allows the construction of distortion-free images for any user selected portion of the acquired omnidirectional image.
  • the mirror may be any suitable reflector of any suitable configuration that is capable of receiving light from a hemispherical field of view and reflecting the same to be received by the second camera assembly.
  • Suitable examples of mirrors and of suitable shapes therefor are preferably a substantially paraboloid-shaped reflector as described in WO 97/43854 positioned to orthographically reflect an image of a substantially hemispheric scene.
  • Other mirrors or reflectors suitable for use include a conical reflecting surface and/or hyperboloidal mirrors.
  • the omnicam sensing unit is capable of imaging a complete hemispherical field of view.
  • the curved mirror used to accomplish this may be hyperbolic when the imaging projection used is perspective, and parabolic when the imaging projection used is orthographic.
  • the omnicam will include a telecentric lens to achieve orthographic projection and a parabolic mirror to achieve a hemispherical field of view. This configuration offers calibration advantages over one that uses a hyperbolic mirror.
  • the second camera assembly is an omni-directional camera and lens assembly commonly designated an omnicam, which covers nearly a half sphere as the peripheral viewing device
  • said first camera assembly is a camera with pan, tilt and zoom features or a CCD camera and lens assembly mounted in such a manner that it picks up the area that the omnicam cannot see, and arranged in back-to-back relationship on the same axis whereby the first camera assembly looks forward, and the second camera assembly looks at the hemisphere.
  • a standard camera is mounted in back- to-back relationship with an omnicam camera and related components to produce a computer vision system that has an undistorted center view and a peripheral area view.
  • Such a system may focus on one object or subject, while providing image data on peripheral objects or subjects.
  • the undistorted center view and the peripheral area view are aligned such that parallax error between the two views is minimized and preferably is eliminated.
  • the camera assemblies are preferably mounted on a pan, and tilt "PT" drive mechanism.
  • the camera viewing system is eminently suitable for use in video conferencing and security systems and is substantially free of the disadvantage of prior systems discussed above.
  • Figure 1 is a side view of an exemplary embodiment of an omni-directional imaging apparatus of this invention
  • Figure 2 is a perspective view of a split-screen monitor according to one embodiment of the invention
  • Figure 3 is a perspective view of a split-screen monitor according to a second embodiment of the invention.
  • Figure 4 is a block diagram of a computer vision system according to an embodiment of the invention.
  • a computer vision system (100) in which a camera viewing system (110) has a first camera assembly comprising a camera (111) with a lens (108), the camera assembly being effective to provide a direct forward field of view that is less than hemispherical and that provides an image that at least encompasses object x, and a second camera assembly comprising a camera (112) with a lens (109) and mirror (113), the camera assembly and mirror being effective to provide a hemispherical field of view that includes areas of a field of view that is hemispherical and provides an image that includes at least objects y or yy and z or zz in a field of view that is peripheral to the field of view of said first camera assembly (111,108), said first and second camera assemblies (108,109,111,112) being mounted in back-to-back relationship one to another.
  • the second camera assembly (112,109) receives the reflected image of the hemispherical scene from the mirror (113), which is mounted to reflect a substantially hemispherical scene.
  • the cameras (111) and (112) generate a signal which may be sent through a cable or telephone line (120) to a computer (125), all as is well known in the art.
  • the stored electronic image data may then be accessed by a transform processor and can be electronically manipulated by panning, up/down, zooming, etc.
  • the second camera assembly and mirror are preferably mounted on a suitable support (114) which may be a thin wire or transparent column or tubing effective to support the camera without interfering with the image. Additionally, if desired, the entire assembly may be mounted on a PT drive mechanism.
  • a first camera (111), and lens (108), preferably with a zoom feature for magnification, is mounted on a pan- tilt base mechanism (116) and is effective to provide a direct forward and close-up field of view that looks forward at object X, and an omnicam second camera (112) and lens (109) looks at the hemisphere and is effective to receive from a mirror (113) a reflected substantially panoramic or hemispherical field of view that includes objects x, yy, and zz.
  • the cameras (111,112), lenses (108,109), and mirror (113) are mounted, on a suitable camera support base (114), if desired, in back-to-back relationship along a common optical axis (115).
  • the cameras (111,112) generate a signal which is sent through a cable or telephone line (120) to computer (125), to provide a close-up image of the selected participant or participants and a field of view that includes the peripheral environment that exists at the time of imaging.
  • the computer 125 is programmed to allow the user to view any desired portion of the hemispherical scene, to zoom in on a selected portion, or to pan the scene in any desired manner.
  • it is necessary and customary to zoom in to the person speaking (x) while excluding the neighboring participant (y or z) or participants (yy,zz) adjacent to the speaker from the field of vision. As a result, the context of the other persons participating in the conference is lost.
  • a split-screen monitor 130 may display images of central and peripheral areas of the panoramic or hemispherical scene of high resolution.
  • the view of x (the speaker), which is data passed from the first camera assembly with zoom lens is shown in the upper % to l / ⁇ of the screen, and the peripheral data content of the scene, yy and zz (other members of the teleconference or background) is shown in the lower V ⁇ to V-i of the screen.
  • FIG 4 illustrates a block diagram of one embodiment of a system that incorporates one embodiment of an image signal processing apparatus for controlling data produced by the cameras of a computer viewing system described in Figs. 1 to 3 and is similar to a control system illustrated in EPO 740,177 A2.
  • cameras 111, 112 obtain views of an area as described herein above.
  • the image signal or output signal of the cameras is passed through analog to digital converters (131, 132), if necessary.
  • the output of the cameras can be thought of as a stream of pixels and the output from the converters can be thought of as data representative of the pixels from the cameras.
  • the output of the converters is passed through MUX (134) which allows the pixel data from each converter to reach the memory (135).
  • Controller (136) cycles the lines of MUX so that all outputs of the A D converters are stored in the memory (135). It is also possible to eliminate MUX and store the output of each A/D converter in a separate memory. It is also possible to eliminate the A/D converters when using digital cameras or sensors. Controller (136) is implemented using a microprocessor which provides control signals to counters that control the switching of MUX and counters used to provide addressing to memory (135). As the pixel information is read from memory (135), all of the information is passed over telecommunications network (137) to telecommunications bridge (138).
  • the information from memory is provided to bridge via modem (139); however, the data may be passed to bridge (138) without use of modem (139) if a digital connection is made between the memory and the bridge.
  • the bridge then distributes all of the data received from memory to each user in communication with the bridge. If the bridge provides analog link to users, a modem should be used at each user port. If the bridge provides digital link with each user a modem is not required.
  • the pixel data is passed to video memory (140) which then passes the data to display (126-131) under control of user input device (125), which is preferably a computer keyboard although a mouse or joystick may also be employed.

Abstract

L'invention concerne un système de visualisation par caméra permettant une vision directe et périphérique, comprenant au moins un ensemble double de caméras opposées dos à dos, placées de préférence sur le même axe optique. Au moins une des caméras assure au moins un champ de vision hémisphérique et au moins une caméra assure une champ de vision directe vers l'avant, inférieur à un champ hémisphérique.
EP00969431A 1999-10-19 2000-10-04 Camera a vision peripherique Withdrawn EP1142332A1 (fr)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US421349 1995-04-12
US42134999A 1999-10-19 1999-10-19
PCT/EP2000/009770 WO2001030079A1 (fr) 1999-10-19 2000-10-04 Camera a vision peripherique

Publications (1)

Publication Number Publication Date
EP1142332A1 true EP1142332A1 (fr) 2001-10-10

Family

ID=23670145

Family Applications (1)

Application Number Title Priority Date Filing Date
EP00969431A Withdrawn EP1142332A1 (fr) 1999-10-19 2000-10-04 Camera a vision peripherique

Country Status (3)

Country Link
EP (1) EP1142332A1 (fr)
JP (1) JP2003512783A (fr)
WO (1) WO2001030079A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
RU2600308C1 (ru) * 2015-11-03 2016-10-20 Вячеслав Михайлович Смелков Устройство компьютерной системы панорамного телевизионного наблюдения

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2006017402A2 (fr) * 2004-08-06 2006-02-16 Ipix Corporation Système et méthode de surveillance
JP2011530258A (ja) * 2008-08-04 2011-12-15 コーニンクレッカ フィリップス エレクトロニクス エヌ ヴィ 周辺観測手段を備えた通信装置
US9677840B2 (en) 2014-03-14 2017-06-13 Lineweight Llc Augmented reality simulator
US10324290B2 (en) 2015-12-17 2019-06-18 New Skully, Inc. Situational awareness systems and methods

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPS59115677A (ja) * 1982-12-22 1984-07-04 Hitachi Ltd 画像処理装置
DE3928244C1 (fr) * 1989-08-26 1990-10-25 Messerschmitt-Boelkow-Blohm Gmbh, 8012 Ottobrunn, De
US5883663A (en) * 1996-12-02 1999-03-16 Siwko; Robert P. Multiple image camera for measuring the alignment of objects in different planes
AU6515798A (en) * 1997-04-16 1998-11-11 Isight Ltd. Video teleconferencing
US6215519B1 (en) * 1998-03-04 2001-04-10 The Trustees Of Columbia University In The City Of New York Combined wide angle and narrow angle imaging system and method for surveillance and monitoring

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See references of WO0130079A1 *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
RU2600308C1 (ru) * 2015-11-03 2016-10-20 Вячеслав Михайлович Смелков Устройство компьютерной системы панорамного телевизионного наблюдения

Also Published As

Publication number Publication date
WO2001030079A1 (fr) 2001-04-26
JP2003512783A (ja) 2003-04-02

Similar Documents

Publication Publication Date Title
US5686957A (en) Teleconferencing imaging system with automatic camera steering
Nalwa A true omnidirectional viewer
US6118474A (en) Omnidirectional imaging apparatus
EP0897636B1 (fr) Appareil de prise d'image omnidirectionnel
US6977676B1 (en) Camera control system
US6426774B1 (en) Panoramic camera
US7667730B2 (en) Composite surveillance camera system
JP3862895B2 (ja) パノラマ画像形成器
US7224382B2 (en) Immersive imaging system
US7593041B2 (en) System and method for a software steerable web camera with multiple image subset capture
US20100045773A1 (en) Panoramic adapter system and method with spherical field-of-view coverage
US6628897B2 (en) Camera system
KR102087450B1 (ko) 초광각 영상 처리를 위한 시스템 및 방법
US20040263611A1 (en) Omni-directional camera design for video conferencing
EP1652004A2 (fr) Systeme video panoramique a imagerie en temps reel sans distorsion
JP2000089395A (ja) コンパクト形高解像度パノラマ画面表示システム
WO1998047291A2 (fr) Videoconference
WO2005002201A2 (fr) Appareil pour videoteleconference
EP1142332A1 (fr) Camera a vision peripherique
JP4448001B2 (ja) 撮像装置
JP2006025340A (ja) 広角撮像装置、撮像システムおよびその制御方法
JP4199940B2 (ja) 撮像装置及び撮像システム
JP2859417B2 (ja) 表示・撮像装置
CA2251879C (fr) Appareil de projection omnidirectionnel

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AT BE CH CY DE DK ES FI FR GB GR IE IT LI LU MC NL PT SE

AX Request for extension of the european patent

Free format text: AL;LT;LV;MK;RO;SI

17P Request for examination filed

Effective date: 20011026

RBV Designated contracting states (corrected)

Designated state(s): DE FR GB

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE APPLICATION HAS BEEN WITHDRAWN

18W Application withdrawn

Effective date: 20041224