CN1985266B - Panoramic vision system and method - Google Patents
Panoramic vision system and method Download PDFInfo
- Publication number
- CN1985266B CN1985266B CN2004800431488A CN200480043148A CN1985266B CN 1985266 B CN1985266 B CN 1985266B CN 2004800431488 A CN2004800431488 A CN 2004800431488A CN 200480043148 A CN200480043148 A CN 200480043148A CN 1985266 B CN1985266 B CN 1985266B
- Authority
- CN
- China
- Prior art keywords
- overall view
- distortion
- display
- image
- parameter
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Lifetime
Links
- 230000004438 eyesight Effects 0.000 title claims abstract description 55
- 238000000034 method Methods 0.000 title claims abstract description 34
- 230000000007 visual effect Effects 0.000 claims description 82
- 238000012937 correction Methods 0.000 claims description 73
- 230000003287 optical effect Effects 0.000 claims description 36
- 238000006243 chemical reaction Methods 0.000 claims description 28
- 238000010586 diagram Methods 0.000 claims description 21
- 230000006870 function Effects 0.000 claims description 17
- 238000007323 disproportionation reaction Methods 0.000 claims description 10
- 230000008569 process Effects 0.000 claims description 10
- 239000000203 mixture Substances 0.000 claims description 7
- 238000001514 detection method Methods 0.000 claims description 6
- 230000008676 import Effects 0.000 claims description 6
- 230000005236 sound signal Effects 0.000 claims description 6
- 230000004075 alteration Effects 0.000 claims description 5
- 230000005540 biological transmission Effects 0.000 claims description 5
- 230000008859 change Effects 0.000 claims description 5
- 230000006835 compression Effects 0.000 claims description 5
- 238000007906 compression Methods 0.000 claims description 5
- 238000012545 processing Methods 0.000 claims description 5
- 208000013021 vision distortion Diseases 0.000 claims description 4
- 238000013480 data collection Methods 0.000 claims description 3
- 238000012544 monitoring process Methods 0.000 claims description 3
- 238000007689 inspection Methods 0.000 claims description 2
- 230000006698 induction Effects 0.000 claims 2
- 239000002131 composite material Substances 0.000 abstract 1
- 238000005516 engineering process Methods 0.000 description 4
- 230000002708 enhancing effect Effects 0.000 description 4
- 238000001914 filtration Methods 0.000 description 4
- 238000005259 measurement Methods 0.000 description 4
- 230000019771 cognition Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 239000012634 fragment Substances 0.000 description 3
- 238000013507 mapping Methods 0.000 description 3
- 239000003550 marker Substances 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 230000015572 biosynthetic process Effects 0.000 description 2
- 230000010354 integration Effects 0.000 description 2
- 230000011664 signaling Effects 0.000 description 2
- 230000002123 temporal effect Effects 0.000 description 2
- 238000007514 turning Methods 0.000 description 2
- 241001269238 Data Species 0.000 description 1
- 241000482967 Diloba caeruleocephala Species 0.000 description 1
- 208000003164 Diplopia Diseases 0.000 description 1
- 206010039203 Road traffic accident Diseases 0.000 description 1
- XUIMIQQOPSSXEZ-UHFFFAOYSA-N Silicon Chemical compound [Si] XUIMIQQOPSSXEZ-UHFFFAOYSA-N 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000004888 barrier function Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000001149 cognitive effect Effects 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000009792 diffusion process Methods 0.000 description 1
- 239000006185 dispersion Substances 0.000 description 1
- 210000003128 head Anatomy 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000003909 pattern recognition Methods 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 210000004180 plasmocyte Anatomy 0.000 description 1
- 238000005498 polishing Methods 0.000 description 1
- 239000004926 polymethyl methacrylate Substances 0.000 description 1
- 230000001915 proofreading effect Effects 0.000 description 1
- 230000001902 propagating effect Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 238000010008 shearing Methods 0.000 description 1
- 229910052710 silicon Inorganic materials 0.000 description 1
- 239000010703 silicon Substances 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 238000002604 ultrasonography Methods 0.000 description 1
- 230000004304 visual acuity Effects 0.000 description 1
- 238000012800 visualization Methods 0.000 description 1
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
- B60R1/20—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
- B60R1/22—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle
- B60R1/23—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle with a predetermined field of view
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
- B60R1/20—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
- B60R1/22—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle
- B60R1/28—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle with an adjustable field of view
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4038—Image mosaicing, e.g. composing plane images from plane sub-images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/40—Image enhancement or restoration using histogram techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/80—Geometric correction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/90—Dynamic range modification of images or parts thereof
- G06T5/92—Dynamic range modification of images or parts thereof based on global image properties
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/63—Control of cameras or camera modules by using electronic viewfinders
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/698—Control of cameras or camera modules for achieving an enlarged field of view, e.g. panoramic image capture
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/95—Computational photography systems, e.g. light-field imaging systems
- H04N23/951—Computational photography systems, e.g. light-field imaging systems by using two or more images to influence resolution, frame rate or aspect ratio
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
- H04N25/60—Noise processing, e.g. detecting, correcting, reducing or removing noise
- H04N25/61—Noise processing, e.g. detecting, correcting, reducing or removing noise the noise originating only from the lens unit, e.g. flare, shading, vignetting or "cos4"
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
- B60R2300/10—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of camera system used
- B60R2300/102—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of camera system used using 360 degree surveillance camera system
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
- B60R2300/10—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of camera system used
- B60R2300/105—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of camera system used using multiple cameras
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
- B60R2300/30—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing
- B60R2300/302—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing combining image information with GPS information or vehicle data, e.g. vehicle speed, gyro, steering angle data
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
- B60R2300/30—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing
- B60R2300/303—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing using joined images, e.g. multiple camera images
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
- B60R2300/30—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing
- B60R2300/306—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing using a re-scaling of images
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
- B60R2300/40—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the details of the power supply or the coupling to vehicle components
- B60R2300/402—Image calibration
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
- B60R2300/60—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by monitoring and displaying vehicle exterior scenes from a transformed perspective
- B60R2300/607—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by monitoring and displaying vehicle exterior scenes from a transformed perspective from a bird's eye viewpoint
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
- B60R2300/70—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by an event-triggered choice to display a specific image among a selection of captured images
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
- B60R2300/80—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement
- B60R2300/802—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement for monitoring and displaying vehicle exterior blind spot views
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
- B60R2300/80—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement
- B60R2300/8053—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement for bad weather conditions or night vision
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
- B60R2300/80—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement
- B60R2300/8066—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement for monitoring rearward traffic
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Mechanical Engineering (AREA)
- Computing Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Processing (AREA)
- Studio Devices (AREA)
Abstract
A distortion corrected panoramic vision system and method provides a visually correct composite image acquired through wide angle optics and projected onto a viewing surface. The system uses image acquisition devices to capture a scene up to 360 DEG or 4p steradians broad. An image processor calibrates unevenness of color or brightness, and makes space converse for each image.
Description
Technical field
The present invention relates to vision system, more particularly, relate to a kind of overall view visual system with image distortion correction.
Background technology
Improved environment sensing technology is more and more required by many practical applications, for example, and surveillance, video conference visible system and vehicle vision system.The essence of these application is to need to monitor a wide operating area, and constitutes combination picture with convenient different user models understanding.In order to minimize number of cameras and cost, the video camera with full shot can photograph wide zone, but because its intrinsic geometric configuration and optical nonlinearity can produce distortion.In surveillance was used, a plurality of panoramic shooting heads can cover the whole outside and the interior zone of building, and described system can provide dynamic observing of this zone, and can manually or automatically follow the trail of through this regional object.
In vehicle vision system, a plurality of panoramic cameras can provide the observations of 360 degree on every side of whole zone and barrier.Described system can go to display view angle specific operation person's pattern, for example turns to, the conversion of reversing and track to be to improve environment perception.Other advantage of vehicle vision system is to reduce windage and the noise that rearview mirror causes, and by removing the width that this outstanding rearview mirror reduces vehicle.These systems also can detect the object in moving, and the warning near object is provided, and follows the trail of this object by a plurality of vision areas.Vision system also can strengthen Infravision significantly by various technology, for example, and infrared ray, radar and photosensitive device.
Vision system comprises one or more image acquisition equipments, is connected to one or more visual display units.Image pickup apparatus can be in conjunction with the different camera lens with different focal lengths and depth of field, for example, and plane camera lens, full shot, flake (fish-eye) camera lens and annular lens.Fish eye lens and annular lens have the wide visual field scope and the bigger depth of field.They can grasp wide and dark field range.Yet they may make image fault, and particularly the edge part makes image seem disproportionate.In the camera lens of any kind, the optical distortion that also has near the light dispersion and radially imperfection, camera lens biasing, lens focus and the camera lens outside tangential to cause because of camera lens.In vision system, also has the image fault of other type that causes because of brightness variation and chromatic aberation.The quality of these distortion effect images and sharpness.
Have overall view visual system now when correctly mixing a plurality of images, not removal of images distortion.United States Patent (USP) 6,498,620B2 just disclose a kind of like this panorama system, and name is called " vehicle rearview system ".This system is made up of image capture apparatus, image synthesizer and display system.No matter be in any document of quoting of this patent or the application, all be not provided for proofreading and correct the electronic image disposal system of how much, optics, lossing normal color and luminance picture distortion.Publication number is that the U.S. Patent application of 2003/0103141A1 has been described a kind of vehicle vision system, only includes the gamma correction based on pre-calibration, does not comprise other optics and geometry correction.
Uncontinuity between the image of distortion and a plurality of image has reduced operator's visual cognition speed, and therefore, his/her environment cognition may produce mistake.This is one of important obstruction that starts effective overall view visual system.Therefore need provide the overall view visual system that can accurately present surrounding environment by eliminating how much, optics, lossing normal color, brightness and other distortion and the composograph at a plurality of visual angles being provided.This correction can help visualization and identification, and improves visual image quality.
Summary of the invention
An aspect of of the present present invention provides a kind of overall view visual system with relevant video camera, display optics and geometrical property, and described system comprises:
(a) a plurality of image acquisition equipments, grab image frames data from scene, and produce the image inductor input, and described image frame data jointly covers the visual field of the highest 360 ° of scopes, and described image acquisition equipment has how much and the optical distortion parameter;
(b) digital quantizer is connected to described a plurality of image acquisition equipment with to described image frame data and image inductor input sample and be converted to Digital Image Data;
(c) image processor is connected to described digital quantizer, and described image processor comprises:
(i) image measuring apparatus is in order to receive described Digital Image Data and to measure picture contrast and the brightness histogram relevant with described Digital Image Data;
(ii) gamma correction module, control the inductor exposure and optimize the brightness range of selecting the zone in the described Digital Image Data by analyzing, and proofread and correct the brightness disproportionation that the gamma correction figure that generates from video camera, display optics and geometrical property data calculates from the picture contrast of described image measuring apparatus and brightness histogram and from the ambient light rank of environment light sensor;
(iii) convolution level, receive the optics of imperfection of expression optics and geometric distortion and video camera and display and geometric distortion parameter and display and observe parameter, and these parameter convolution are generated brightness with free-air correction figure and offer gamma correction module and the distortion correction module that is connected;
(iv) distortion correction module, be connected to described convolution level, reception is observed free-air correction figure that the parameter convolution obtains described Digital Image Data is used the distortion correction conversion, to generate calibrated Digital Image Data by described geometry and optical distortion parameter and display;
(v) display controller is connected to described distortion correction module to form composograph according to described calibrated Digital Image Data; And
(d) display system is connected to described image processor to observe the surface described composograph of demonstration for observation, and described composograph does not have the vision distortion.
On the other hand, the invention provides a kind of use have relevant video camera, display optics and geometrical property and how much and the overall view visual system of optical distortion parameter the method for composograph to provide panorama to observe that covers the highest 360 ° or 4 π sterad scopes is provided, described method comprises:
(a) obtain image frame data from scene, described image frame data is common to cover the field range of the highest 360 ° or 4 π sterads, and produces set of diagrams and import as inductor;
(b) described image frame data of conversion and image inductor are input as Digital Image Data, and described Digital Image Data is relevant with the ambient light rank with brightness of image;
(c) measure picture contrast and the brightness histogram relevant with described Digital Image Data, by analyzing described picture contrast and brightness histogram and control the inductor exposure and optimize in the described Digital Image Data brightness range of selecting the zone, and proofread and correct the brightness disproportionation that the gamma correction figure that generates from video camera, display optics and geometrical property data calculates from the ambient light rank of environment light sensor;
(d) optics of the imperfection of reception expression optics and geometric distortion and video camera and display and geometric distortion parameter and display are observed parameter, and these parameter convolution are generated brightness and free-air correction figure;
(e) receive by described geometry and optical distortion parameter and display and observe free-air correction figure that the parameter convolution obtains described Digital Image Data is used the distortion correction conversion, to generate calibrated Digital Image Data;
(f) form composograph according to described calibrated Digital Image Data; And
(g) observing the surface described composograph of demonstration for observation, described composograph does not have the vision distortion.
On the other hand, the invention provides a kind of image processor, be used to have the overall view visual system of relevant video camera, display optics and geometrical property and geometry and optical distortion parameter, described overall view visual system uses a plurality of image acquisition equipments with grab image frames from scene and generate Digital Image Data and image inductor input, and use digital quantizer to be input as Digital Image Data to change described Digital Image Data and image inductor, described image processor comprises:
(a) image measuring apparatus is in order to receive described Digital Image Data and to measure picture contrast and the brightness histogram relevant with described Digital Image Data;
(b) gamma correction module, control the inductor exposure and optimize the brightness range of selecting the zone in the described Digital Image Data by analyzing, and proofread and correct the brightness disproportionation that the gamma correction figure that generates from video camera, display optics and geometrical property data calculates from the picture contrast of described image measuring apparatus and brightness histogram and from the ambient light rank of environment light sensor;
(c) convolution level, receive the optics of imperfection of expression optics and geometric distortion and video camera and display and geometric distortion parameter and display and observe parameter, and these parameter convolution are generated brightness with free-air correction figure and offer gamma correction module and the distortion correction module that is connected;
(d) distortion correction module, be connected to described convolution level, reception is observed free-air correction figure that the parameter convolution obtains described Digital Image Data is used the distortion correction conversion, to generate calibrated Digital Image Data by described geometry and optical distortion parameter and display; And
(e) display controller is connected to described distortion correction module to form composograph according to described calibrated Digital Image Data.
In the first embodiment of the present invention, a kind of vehicle vision system is provided, cover horizontal field of view scope and vertical visual field scope up to 180 ° up to 360 °, emphasis is in the observation in rear portion, both sides and the corner of car.Described environment display image can for example vehicle-state or navigation data combine with other information of vehicles data.The image first-selection is presented on the preceding control panel, near driver's field range.Also can show additional observation image to improve spreadability and dirigibility, for example, substitute inside instrument panel (dashboard) the corner display of rearview mirror, can eliminate the projection and the windage of external component.When a view was more important than other view, described system was applicable to and disposes display again and switch between each view based on control input.
For example, when reversing, display is reset to show the image in the tailstock wider range; Perhaps, the right-hand rotation lamp open and vehicle when fair speed is travelled, show the image on right side; Perhaps open at the right-hand rotation lamp, vehicle stops or during low speed driving, shows the image of right hand corner.In a preferred embodiment, described system provides dependent help, is used as the parking assistant, or provides track modified line warning by the identification to kerb line, track and object, and definite distance.
In another embodiment, the invention provides a kind of monitoring system, cover a certain structure (for example buildings, bridge etc.) inner and/or outside horizontal extent and vertical visual field scope up to 180 ° up to 360 °.Described system can provide a plurality of images after distortion correction to produce sequential chart image-tape or scanning panoramic view.Described system can carry out the object tracking in detection of dynamic and the monitoring range, and object identification is to identify people, vehicle etc.
In another embodiment, the invention provides a kind of video conference vision system, produce wide reach 180 ° or 360 ° broad horizon of annular.In this embodiment, the present invention helps to observe the participant of meeting or party, and the function of using a video camera to provide camera lens to further the spokesman.
The various details of the embodiment of the invention and advantage of the present invention will be showed by following description and accompanying drawing.
Description of drawings
In the accompanying drawing:
Fig. 1 is the overview of vision system of the present invention and built-in parts thereof;
Fig. 2 A is as the structural representation of the image processor of a vision system part among Fig. 1;
Fig. 2 B is the logical flow chart of image processor among Fig. 1;
Fig. 3 A has the synoptic diagram that rearview mirror is used for the conventional truck of both sides and afterbody observation;
Fig. 3 B is the setting of video camera among the vehicle vision system embodiment and the synoptic diagram of range of observation thereof;
Fig. 3 C is the synoptic diagram of the video camera range of observation selected among the vehicle vision system embodiment;
Fig. 4 A is the synoptic diagram on the observation surface of vision system display among the vehicle vision system embodiment;
Fig. 4 B is the synoptic diagram on identical observation surface shown in Fig. 4 A when the right-hand rotation pilot lamp is opened among the vehicle vision system embodiment;
Fig. 5 A is the synoptic diagram of selectable big field range on the observation surface of vision system display among the vehicle vision system embodiment;
The synoptic diagram of observation surface after resetting when Fig. 5 B is vehicle backing among Fig. 5 A;
The synoptic diagram of display through resetting when Fig. 6 A is the vehicle modified line;
Fig. 6 B is that vehicle is turned right and the synoptic diagram of driver's seat display through resetting when not knowing;
Fig. 7 is the vertical view that shows vehicle and approaching object;
Fig. 8 is the synoptic diagram of an example of the control input in the vehicle vision system;
Fig. 9 A is the synoptic diagram of the video camera in the semisphere that is arranged on conference table central authorities among the video conference vision system embodiment, 180 ° of (radians of 2 π) visuals field;
Fig. 9 B is the synoptic diagram of the display of video conference vision system embodiment;
Figure 10 is equipped with the synoptic diagram of two video cameras as the part of surveillance embodiment on corridor or the building wall;
Figure 11 A is the synoptic diagram of the display of surveillance embodiment of the present invention;
Figure 11 B be among Fig. 9 A same display at the synoptic diagram of time after a while.
Embodiment
Fig. 1 has showed the one-piece construction of the vision system of setting up according to the present invention 100, it comprises that the like video camera of a plurality of image acquisition equipments 110 is in order to grab image frames, digital quantizer 118 is a Digital Image Data in order to the converted image frame data, image processor 200 is in order to adjust brightness and correcting image distorsion and to constitute composograph according to Digital Image Data and controlled variable, to image processor 200, and display device 120 is in order to show on the surface 160 that synthetic image is for observation observing in order to the parameter of transmitting user 150 and inductor 194 for controller 101.The finally synthetic image of the present invention covers up to 360 °, comprises a plurality of interested specific regions, and undistorted.This system has strengthened the environment awareness to a great extent.
In the present invention, camera lens does not need perfection or produces perfect image.All image fault comprises the image fault that lens geometry structure or imperfection cause, all by mathematical simulation and electronic calibration.The importance of this feature is that but it can allow the user to use the wide-angle lens of cheap extended depth-of-field.The shortcoming of this class camera lens is to produce image fault and brightness disproportionation; This class camera lens object that can stretch is particularly near the edge of viewfinder range.This effect is well-known, and for example, flake or annular lens are like this.They make object seem disproportionate.There is a large amount of optical solutions to can be used for compensating these distortions.Yet additional a large amount of optical device will make described equipment bigger, heavier and more expensive.The cost of optical device is determined by cost of labor in a large number, for example, and polishing, aligning etc.But electronic component becomes more cheaply and is more effective.As everyone knows, optical device can not be eliminated some distortion fully.
Another shortcoming of optical device solution is the functional fault that causes after susceptibility of aiming at and the collision.Therefore, be starved of electronic image and proofread and correct so that vision system is represented real setting, when making its function admirable, price is also low relatively.This is confirmed in vision system, and electronic component is that most probable is required to finish other task.These tasks comprise panorama conversion and display control.These basic functions are provided with the image processor of describing 200 after a while.
Image inductor array 114 is arranged in the video camera 110.This inductor array is applicable to the utmost point drastic changeization of the light intensity that relaxes the viewfinder range that passes through camera chain.For the vision system that effectively is operated under the different light situation, need very big dynamic range.This dynamic range requires to come from from the variation of ambient lighting intensity in daytime to evening, and the variation that causes of the light source of chance at night.The video camera inductor is a plurality of fragments of inlaying basically, and each fragment part in the viewfinder range receives light, and the recording light line strength is exported as voltage.Image-capture controller 116 sends the picture frame strength information, and based on these data, receives order to be used for synthetic optimization, lens aperture control and white balance control from image processor 200.
Digital quantizer 118 receives data from image-capture controller 116, and to change these data be Digital Image Data.In another embodiment, the function of digital quantizer 118 can be integrated in image inductor 114 or the image processor 200.Under the situation that exists voice data to obtain, digital quantizer 118 receives voice data from audio frequency apparatus array 117.Producing digital audio-frequency data subsequently also merges it and Digital Image Data.Then Digital Image Data is sent to image processor 200.
Fig. 2 A shows the details of image processor 200.The function of this image processor be receive Digital Image Data, measurement image statistical value, strengthen picture quality, compensate for brightness is uneven and proofread and correct various distortions in these data, with the one or more undistorted composographs of final generation.This image processor comprises optics and geometric data interface 236, it comprises camera optics data, projection optics data and perspective geometry data, and control data interface 202, image measurement module 210, gamma correction module 220, distortion convolution level 250, distortion correction module 260 and display controller 280.In order to understand the function of image processor 200, characteristic and reason that these distortions simply are discussed are important.
Surrounding environment light and pointolite can cause the utmost point drastic changeization of gray scale, thereby produce relatively poor picture quality and make pictorial element be difficult to identification.In the present invention, image processor 200 is measured whole and the image-region of selecting, and analyze they with the control exposure to obtain optimum picture quality in interesting areas.The higher source luminance for example intensity of headlight and spotlight is reduced fully, and the details of low-light level object is enhanced and is beneficial to the identification of element.
Image inductor can be the inductor of any kind and any resolution, and in the embodiments of the invention, solid-state CCD of high resolving power or cmos image inductor all are being appropriate aspect size, cost, integrated level and the dirigibility.Typical application comprises by aperture control and is adjusted to the wide ambient light rank of variation range integral time (integration time).Except aperture control and control integral time, 220 pairs of interesting areas of gamma correction module are carried out histogram analysis, and the contrast that enlarges the contrast in low irradiance zone and reduce the high illuminance zone is discerned with the pictorial element that enhancing is provided.
Brightness disproportionation is that undesired brightness changes in the image.The Main physical reason that causes brightness disproportionation is to pass the light of optical system different piece through different distance and areal concentration.The decay of the intensity of light and the distance of its process square proportional.This phenomenon had both occurred in the camera optics device, also occurred in the display optics device.Except these pure physical causes, the imperfection of optical system also causes brightness disproportionation.The example of described imperfection is that the diplopia and the side of projection lens rises and falls, and the inconsistency of the projection ray that produces.If brightness changes for 3 different color component differences, just it is inconsistent to be called colourity.
The optical distortion of another kind of type is a chromatic aberation.It come from optical element for example camera lens have different refractive indexes for different wavelength.The light of propagating by optical element reflects with different angles according to different wavelength.This makes color generation lateral excursion in the image.Lateral chromatic aberration causes the different colours component separating and the diffusion of point target.A point target seems to have burr on the surface observing.The chromatic aberation of another kind of type comes down to axial, produces because of camera lens has different focuses to different wavelength of light.Such chromatic aberation can not be proofreaied and correct by the electronic calibration method.
Various other distortions may appear in the vision system.Camera lens tangentially or the biasing of imperfection radially, camera lens, projector imperfection and the keystone distortion that causes because of the off-axis projection be some common distortions.
Except above-mentioned distortion, image processor 200 is mapped in the observation surface with particular characteristics with the scene that captures, for example, and shape, size and aspect ratio.For example, image can be made of the different piece of the scene of catching, and projects on the part of windshield of vehicle, in this case, because observe the molded non-planar on surface and specific size, distortion can occur.Image processor among the present invention is proofreaied and correct all these distortions, followingly will provide description.
With reference to the details of the image processor among Fig. 2 A 200, Digital Image Data is received by image measurement module 210, will record the contrast and the brightness histogram of the image of interesting areas at this.These histograms are analyzed with control inductor exposure by gamma correction module 220, and adjust Digital Image Data to improve picture material, are used for observing and detecting.Some adjustment comprises high brightness compression, expanded contrast, enhancing details and reduces noise.
Together with these metrical informations, gamma correction module 220 receives video camera and projection optics data from optics and geometric data interface 236.The accurate light ray propagation of these data based optical elements calculates and calibration is determined.These data are very important for gamma correction, because they provide the optical path length by the different light rays of video camera and optical projection system.The video camera and the figure of projection correction that separate or make up are generated to calculate the correction of each pixel.
But this correction chart off-line obtains, and perhaps can dynamically calculate according to environment.Therefore the function of gamma correction module 220 is to receive Digital Image Data and produce the adjusted Digital Image Data of brightness.Under the inconsistent situation of colourity, gamma correction module 220 first-selections are used gamma correction to three different color components respectively.The physics realization of gamma correction module can be software program or special-purpose treatment circuit, for example computational logic in digital signal processor or the integrated circuit.
The adjusted view data of brightness should be carried out the correction of how much, optics and other spatial distortion.These distortions are called " distortion (warp) " proofreaies and correct, and this alignment technique is known as " scalloping " in this manual." digital picture distortion " (George Wolberg that the relevant introduction of scalloping can be shown at George Wolberg, Digital Image Warping, IEEE Computer Society Press, 1998) find in, the application quotes its related content at this.
In one embodiment of the present of invention, distortion figure is according to increasing progressively one of polynomial expression and position polynomial expression parametrization.Scalloping is that coordinate conversion, mapping output pixel to the actual parameter of importing pixel are represented basically.Ideally, each output pixel of gridden data set representations is to the mapping of importing pixel.Yet because the shearing size of look-up table, gridden data represents to realize it being quite irremissible with respect to hardware.Among the present invention, the scalloping technology provides a kind of effective method to represent pixel grid point data collection by some parameters.In one embodiment, these parametric representations are finished by polynomial of degree n, and n is by the complexity decision of combination distortion.
In another embodiment of the present invention, the zones of different of output region is divided into a plurality of small pieces with inherent geometric attribute, to reduce polynomial number of times.On the principle, the quantity of these small pieces is big more and to be suitable for the degree of polynomial of each small pieces high more, and the parametrization of gridden data collection is just accurate more.Yet in using in real time, this need balance each other with processing power.Described distortion figure therefore expression output pixel represents camera optics, display optics and display geometry characteristic to the mapping of importing pixel, comprises the specification of finally synthetic image and observes surperficial style characteristic.In addition, any controlled variable comprises user's input parameter, also with the above-mentioned parameter combination, and is illustrated in the once conversion.
Except coordinate conversion, also need sampling or filter function usually.In case output image data is mapped on the input pixel, input regional designated around the pixel is used for filtering.This zone is called filter footprint territory (filter footprint).Filtering is a weighted mean function basically, obtains the intensity of the color component of output pixel based on pixels all in the overlay area.In specific example, the oval overlay area of anisotropy is used to obtain optimal image quality.As everyone knows, the overlay area area is big more, and the quality of output image is high more.Carries out image filtering when image processor 200 is carried out coordinate conversion in the present invention.
For correcting image distorsion, above-mentioned all geometry and optical distortion parameter are combined in together in distortion convolution level 250, are used for different color components.These parameters comprise camera optics data, projection optics data, the perspective geometry data that obtain by optics and geometric data interface 236, and import by the control that control interface 202 obtains.Optics that these are combined and geometric distortion parameter are obtained by distortion correction module 260 subsequently.The function of this module is that each positions of elements, shape and color intensity in the scene are transformed on the display element.The shape of observing surface 160 is also included within the perspective geometry data.This surface needs not be the plane, can be the shape of any routine, if obtain exterior view (surface map) and with other distortion parameter combination.Described display surface figure (display surface map) carries out process of convolution with remaining distortion data.
In one embodiment of the invention, distortion correction module 260 obtains to cover the distortion figure (warp map) in whole distortion parameter space.This processing has provided detailed explanation in U.S. Patent application No.2003/0020732-A1 and No.2003/0043303-A1, quote its related content at this.For every group of distortion parameter, will calculate conversion, the distortion that is produced on the observation surface 160 that is transmitted to given shape by camera optics device, display optics device the time with compensating images.But the formation of distortion parameter group and conversion Calculation off-line are finished, and are stored in the storer and pass through interface accessing for image processor 200.In the situation that parameter changes, also finish to partial dynamic at least.
Display image can be made up of form, and this form can be independently image or the image of cascade each other.For each form, distortion correction module 260 is added from the skewed surface equation and computer memory conversion and filtering parameter, and the carries out image conversion is so that show.
For each picture frame, distortion correction module 260 is at first found out immediate grid number strong point in the distortion parameter space.Distortion correction module 260 is added the existing conversion that should organize parameter to mate actual distortion parameter subsequently.Distortion correction module 260 is used this conversion to compensate all distortions to Digital Image Data subsequently.Be combined from the Digital Image Data of each frame of each video camera and meet the composograph of observing surface and minor structure thereof with formation.Corresponding numerical data when image is formed on the observation surface, does not have the lip-deep optimum viewing area of observable distortion and adaptive observation after overcorrect.The physics realization of distortion correction module 260 can be the software program in the nextport universal digital signal processor NextPort, or special-purpose treatment circuit, for example special IC.The physics example of image processor 200 is combined in the sxW1 and REON chip of Silicon Optix company.And, image processor 200 applicable to the red, green and blue composition of individual processing Digital Image Data to proofread and correct lateral chromatic aberration and distortion.
Fig. 2 B is the logical flow chart of image processor 200 in example of the present invention. the digital data stream in this process flow diagram is represented with thick line, and the data stream of calculating is represented with fine rule. as shown in FIG., brightness and contrast's histogram records from numerical data in step 10. camera optics, display optics and display geometry parameter obtain in step 14 and 16. and these data obtain in step 20 with brightness and contrast's histogram subsequently, inequality in this brightness of image is adjusted. from the optics and the geometric data of step 14 and 16, and the controlled variable of step 26 acquisition, collection in step 30 subsequently. in this step, all distortion parameters are merged. and the conversion that transforms distortion effect is calculated in step 40. the Digital Image Data that this compensation conversion is applied to obtain in the step 20 subsequently through the brightness adjustment.
As previously mentioned, for higher picture quality, pixel data is filtered in step 50.Therefore, carry out coordinate conversion and Flame Image Process in the step 50 simultaneously.In step 60, the Digital Image Data behind the distortion compensation is used to generate composograph and is used for showing.
After the Digital Image Data of a frame was processed and synthetic, display controller 280 produced image according to these data.In one embodiment of the invention, described image is formed on the display device 120, and display device 120 can be direct visual display unit or projection display apparatus.In one embodiment, the image from projection display apparatus 120 is projected on the observation surface 160 by display optics 122.Display optics 122 is a large amount of optical elements, observes on the surface 160 in order to projection ray is guided to from display device 120.Any specific display system all has extra optics and geometric distortion, need proofread and correct.In this example, image processor 200 merges these distortions and proofreaies and correct distortion after merging.
It should be noted that the selection that the invention is not restricted to any particular display system; This display device can be liquid crystal, light emitting diode, cathode-ray tube (CRT), el cell, plasma display or any other visual display device.But this display device can be direct viewing, perhaps can be projected on the integrated plate by display optics, as view screen.First-selected ground, the brightness of display system is adjusted by environment light sensor, independent signal and user's dimmer switch.The size on this observation surface preferably can be by the user according to for example adjusting from the distance of screen from operator's eyes.
For some application of the present invention, may need record data stream or send it to different clients by network.In both cases, at first need and must compressed video data stream to reach storage and bandwidth limit.In an example, compression stage 132 flows from image processor 200 receiver, videos, and compresses this digital of digital video data.Data storage after the compression in recording unit 142 for using in the future.In another embodiment, data are encoded in code level 134, and are sent on the network by network interface 144.
After having explained general feature of the present invention in detail, below will introduce a plurality of embodiment of vision system 100.In one embodiment, the invention provides a kind of vehicle vision system.Fig. 3 A has showed a kind of existing automobile 900, has two side rearview mirrors and a room mirror, covers the visual field, right side 902, the survey visual field 904, a left side and the visual field, rear portion 906.As everyone knows, traditional position of rear view mirror and field range cause occurring blind spot, and may have distortion because of the visual angle of broad.The image that provides from these three mirrors is difficult to obtain complete surrounding environment cognition.
In an example of the present invention, shown in Fig. 3 B, automobile 900 ' is equipped with video camera 110 ' and the video camera 111 ' that is arranged on the operating seat front.These positions by with the overlapping coverage that increases of driver's direct vision scope.Video camera 113 ' in this example is positioned at the centre of automobile tail or roof rear portion.Can provide nonoverlapping consecutive image by the specific region of selecting different field ranges.This can prevent that obscuring from appearring in the driver.
In another example that Fig. 3 C shows, automobile 900 " be provided with video camera 110 at two turnings of headstock " and video camera 111 ", and be provided with video camera 113 in the centre position of automobile tail ".In this example, emphasis is according to user's input or controlled variable scalable field range.For example, when side marker light is opened, turn to side view 903 " available.The coverage that it should be noted that this visual field is the function that the user imports, and on principle, covers the whole zone in the dotted line.Equally, drive side view 905 " be used for common driving model, it also can expand to the whole zone that covers in the dotted line.The visual field, rear portion in this example has two kinds of patterns according to controlled variable.When reversing, show the reversing visual field, rear portion 907 ".This visual field can cover automobile rear fully, comprises the object on the road surface.This guarantees to move backward safely and greatly facilitate double parking.Yet automobile uses the narrower visual field, driving rear portion 906 when driving model ".These positions and angle guarantee that the driver can obtain near the view of automobile external.This example greatly facilitates double parking and lane change.When if driver's direct field of vision is covered by approaching object, this vision system will strengthen at least one visual field of vehicle front and rear corner.
Fig. 4 A is the synoptic diagram of an example on the observation surface 160 seen of driver.Rear portion field-of-view image 166 is in the bottom of observing the surface, and right side field-of-view image 162 and left visual field image 164 lay respectively at upper right quarter and the upper left quarter of observing surface 160.Fig. 5 A has showed an example that uses than the observation surface 160 of wide display, and wherein side view shows the both sides that are positioned at rear portion field-of-view image 166 '.Image processor 200 has panorama conversion and image stitching function to synthesize this special demonstration.Fig. 4 B and Fig. 5 B have showed respectively when automobile is turned right or moveed backward, the example of Fig. 4 A after resetting and the display of Fig. 5 A.
Fig. 6 A is the synoptic diagram on the observation surface 160 of automobile modified line, the display reset when changing to right lane.In this example, automobile right front and rear are visual fully, thereby can know all objects on automobile right side.Fig. 6 B is that the right-hand rotation pilot lamp is reset the synoptic diagram with the display that is used to turn right when opening.Turn to side view 903 among Fig. 3 C " be presented at the middle and upper part of display now, promptly turn to side view 163 ".The visual field, rear portion 166 " and the visual field, right side 162 " be presented at the bottom of display.
Fig. 7 has showed another example of observing surface 160.The specific setting of this of display is by collecting visual information and determining that from the ultrasound wave, laser and the radar inductor that are integrated in the vision system 100 distance realizes from image acquisition equipment.In the example of this displaying, deceive thick line and have the automobile of gray shade to be provided with vision system 100, and the setting around showing, i.e. other vehicle.Image processor uses the distance of this inductor inspection vehicle outside surface and a certain target basis part.When the distance between car outside surface and target falls into the defined limit of car speed, can provide the Voice ﹠ Video warning.This embodiment of the present invention improves cognitive also help of environment to a great extent and helps to drive a car.The bidimensional image of each object is directly grasped by video camera.The shape of these vehicles and object and size infer that by size and shape according to track and roadside the image of video camera rebulids.Can rebulid the more accurate image of Driving Scene by the look-up table of certain objects in pattern-recognition and the database.
It should be noted that the embodiment that the invention is not restricted to these displayings; The variation of number of cameras and position and resetting of observation surface all fall into scope of the present invention.All these optional embodiments in perspective geometry data, projection optics data and the camera optics data overlay image processor 200.
Preferably, display brightness can be adjusted by environment light sensor, adjusts by the signal from headlight, or adjusts by manual dimmer switch.In addition, preferably, the size of display surface also can be adjusted by the driver.In addition, the focal length of display image is preferably in driver's focal range, and as United States Patent (USP) 5,949,331 is described, quotes the related content of this patent at this.Yet the display system focal length preferably can be according to the speed adjustment of vehicle to guarantee that image is forever in driver's field depth.In higher speed, the driver pays close attention to larger distance naturally.This adjustment realizes by velometer signal 156 or transmission signals 154.
Fig. 8 has showed that the user imports an example of 150 in the vehicle vision system.Signaling interface 151 is from different parts received signals, and is connected with controller 101 (referring to Fig. 1).For example, side marker light is opened when turning right, and transmits a turn signal 152 and gives signaling interface 151, and be sent on the controller 101, is sent to image processor 200 at last.In case after image processor 200 received this signal, image processor configuration display was to emphasize right side display image 162 on observation surface 160.Fig. 4 B has showed view screen 160 in this case.The right side field-of-view image has occupied half of observing surface 160 now, and second half is used for rear portion field-of-view image 166.Fig. 5 B has showed the same case of the embodiment that uses wide observation surface.Other signal causes other function.
For example, when stopping, the first-selected signal that produces of transmission is to emphasize the visual field, rear portion.Other signal comprises the turn signal 152 that sends when bearing circle when a side is rotated above a pre-set limit, brake signal 153 when brake pedal is operated, transmit the transmission of Information signal 154 of relevant travel speed and method, and velometer signal 156, under the measurement friction speed, import unlike signal.In the present invention, described display system is adjusted to different situations according to any of these controlled variable, and resets automatically so that crucial image can be used.Image processor 200 can be used in combination various control signals, and we have only mentioned several examples at this.
Various Useful Informations can be presented to be observed on the surface 160, comprises highway map, roadside information, GPS information and Local Weather Report.Vision system 100 is visited these data by downloaded files or by the radio communication of driver's request.Vision system 100 is added in these stacked datas the image that shows with the combining display system on the image frame data subsequently.Important warning system is also first-selected to be received by this vision system, and shows and broadcast by audio signal interface by display system.Vision system 100 also can pass through the exchange image data information and the turnpike driving system integration.Vision system 100 also can be integrated with distance determiner and target marker, and United States Patent (USP) 6,498 has relevant the introduction among the 620B2, quoted by the application at this.
In one embodiment of the invention, vehicle vision system 100 includes compression stage 132 and recording unit 142 continuous recording with the incident setting up video camera and audio frequency apparatus and detect.The function of these feature combinations can be preserved the incident of wanting to write down except that the driver before bump and the specific fragment behind the bump, in for example traffic accident of accident, vehicle is abducted with traffic resets after breaking rules.These information can be used by law enforcement agency, juridical authorities and insurance company.
In a different embodiment, the invention provides a kind of video conference vision system, can cover up to 180 ° or 360 ° field range according to given setting.In this embodiment, sound signal and vision signal all need.In the vision system 100 that Fig. 1 shows, except the video input from video camera, audio frequency apparatus array 117 provides the audio frequency input.Audio signal data is converted to numerical data by digital quantizer 118, and is superimposed upon on the digital picture frame data.This system provides the image shift function, sweeps each spokesman and quizmaster based on the triangulation of sound signal.This pan and zoom function are provided by image processor 200.Under the situation that lacks optical zoom, provide digital zooming.
Fig. 9 A relates to an example of video conferencing system.Video camera 110 shown in the figure is positioned at the central authorities of conference table.In this specific example, video camera 110 has the semisphere lens system and grasps desktop and all things of top.Original image be put upside down, be stretched and disproportionate.Yet observing the image that shows on the surface 160 is undistorted and the conversion of process panorama.
Fig. 9 B has showed observation surface 160, and wherein spokesman's image is exaggerated, and other people of participant shows with linear fashion on same observation surface.
In another embodiment, the invention provides a kind of surveillance with motion detection and Target Recognition. Fig. 2 A has showed an example of image processor 200, being used for surveillance. motion detector and tracker 270 assessments are the picture frame of input continuously, and, sending moving region coordinate to the picture frame of distortion convolution level 250. inputs and be used to monitor zone current window outside based on default detection levels of motion and signal alarm. the coordinate of tracking is used to calculate form to show the image of detected target after calibrated by distortion convolution level 250. and the target image of correcting distortion can be by motion compensation and temporal interpolation (temporal interpolation) the enhancing resolution to target.This system also can carry out Target Recognition and classification.
Figure 10 has showed an example of surveillance.Video camera 110 and 111 is arranged in the corridor, is installed on the wall in the corridor.Traffic in these two camera supervised corridors.The moving when image processor 200 among this embodiment uses motion detector and tracker 270 to follow the trail of certain target through the corridor.The observation surface 160 of the special time in this example is shown in Figure 11 A, and the observation surface 160 of time is shown in Figure 11 B after a while.When target in the visual field when a video camera moves to another video camera, what this target was passed through is omnidistance crawled.This vision system also can be carried out the enhancing to resolution, improves the details and the identification of target by decimation in time (temporalextraction), and shows it in the image apex of Figure 11 A and Figure 11 B.Head portion in the accompanying drawing provides the full resolution image of moving target or moving target has been carried out digital zooming.Identification of targets and follow the trail of the detected target that connects in the comparing motion of a frame by a frame and realize.The outstanding tracked target of outline line is so that identification.
Claims (55)
1. the overall view visual system with relevant video camera, display optics and geometrical property is characterized in that, described system comprises:
(a) a plurality of image acquisition equipments, grab image frames data from scene, and produce the image inductor input, and described image frame data jointly covers the visual field of the highest 360 ° of scopes, and described image acquisition equipment has how much and the optical distortion parameter;
(b) digital quantizer is connected to described a plurality of image acquisition equipment with to described image frame data and image inductor input sample and be converted to Digital Image Data;
(c) image processor is connected to described digital quantizer, and described image processor comprises:
(i) image measuring apparatus is in order to receive described Digital Image Data and to measure picture contrast and the brightness histogram relevant with described Digital Image Data;
(ii) gamma correction module, control the inductor exposure and optimize the brightness range of selecting the zone in the described Digital Image Data by analyzing, and proofread and correct the brightness disproportionation that the gamma correction figure that generates from video camera, display optics and geometrical property data calculates from the picture contrast of described image measuring apparatus and brightness histogram and from the ambient light rank of environment light sensor;
(iii) convolution level, receive the optics of imperfection of expression optics and geometric distortion and video camera and display and geometric distortion parameter and display and observe parameter, and these parameter convolution are generated brightness with free-air correction figure and offer gamma correction module and the distortion correction module that is connected;
(iv) distortion correction module, be connected to described convolution level, reception is observed free-air correction figure that the parameter convolution obtains described Digital Image Data is used the distortion correction conversion, to generate calibrated Digital Image Data by described geometry and optical distortion parameter and display;
(v) display controller is connected to described distortion correction module to form composograph according to described calibrated Digital Image Data; And
(d) display system is connected to described image processor to observe the surface described composograph of demonstration for observation, and described composograph does not have the vision distortion.
2. system according to claim 1 is characterized in that, described image processor is applicable to that the red, green and blue composition of individual processing Digital Image Data is to proofread and correct lateral chromatic aberration and distortion.
3. overall view visual system according to claim 2 is characterized in that, described gamma correction module individual processing red, green and blue composition is inconsistent to proofread and correct colourity.
4. overall view visual system according to claim 1 is characterized in that, described a plurality of image acquisition equipments and described digital quantizer are realized by a plurality of digital cameras.
5. overall view visual system according to claim 3 is characterized in that described display system is a light projection system, and wherein said geometry is merged with precorrection relevant geometry and optical distortion with the optical distortion parameter.
6. overall view visual system according to claim 3, it is characterized in that, further comprise at least one environment light sensor, signal and at least one user's dimmer switch that at least one is independent, described environment light sensor, independent signal and user's dimmer switch are individually or combine the brightness of adjusting described display system.
7. overall view visual system according to claim 3, it is characterized in that, further comprise controller, wherein said controller be used to transmit one group of customer parameter to described image processor with based on one group of overview function selecting and parameter with described customer parameter and distortion parameter merging.
8. overall view visual system according to claim 3 is characterized in that, further comprises the control interface that is connected to described image processor, is used for receiving and transmission user input and controlled variable.
9. overall view visual system according to claim 3 is characterized in that, wherein said image processor is further used for sending exposure control command to described image acquisition equipment and controls to be used for aperture and time shutter.
10. overall view visual system according to claim 3 is characterized in that, further comprises observing the surface, and the position of the lip-deep composograph of wherein said observation and big I are adjusted according to one group of user's input.
11. overall view visual system according to claim 3 is characterized in that, described system is a vehicle vision system.
12. overall view visual system according to claim 11 is characterized in that, described display system is used for the composograph that forms vision field of driver the place ahead in the vehicle at least one at following group: instrument panel, windshield and roof.
13. overall view visual system according to claim 12 is characterized in that, described display system is used to form two rearview mirrors of observing lip-deep composograph with the simulating vehicle both sides of windshield and instrument panel interior angle.
14. overall view visual system according to claim 11 is characterized in that, the focal length of described composograph is positioned at vehicle driver's focal range.
15. overall view visual system according to claim 11 is characterized in that, the focal length of described display system is adjusted to guarantee that image is forever in driver's field depth according to the speed of vehicle.
16. overall view visual system according to claim 11 is characterized in that, described vehicle vision system is reset according to the input in following group in a plurality of the setting: turn signal, transmission system, brake system, steering wheel system and velometer.
17. overall view visual system according to claim 16 is characterized in that, when the Vehicular turn signal was set to turn right, described display system strengthened the visual field, right side of vehicle; When the Vehicular turn signal is set to, strengthen the left visual field of vehicle.
18. overall view visual system according to claim 16, it is characterized in that turn right when surpassing default angle when direction of traffic, described display system strengthens the visual field, right side of vehicle, turn left when surpassing default angle when steering wheel for vehicle, strengthen the left visual field of vehicle.
19. overall view visual system according to claim 16 is characterized in that, when vehicle backing, described display system strengthens the vehicle rear visual field.
20. overall view visual system according to claim 16 is characterized in that, if when driver's direct field of vision is covered by approaching object, described vision system strengthens at least one visual field of vehicle front and rear corner.
21. overall view visual system according to claim 20, it is characterized in that, further comprise at least one laser inductor and radar inductor, and wherein said image processor further uses the distance between described laser inductor and/or radar inductor inspection vehicle outside surface and a certain target, and the distance between car outside surface and described target provides the Voice ﹠ Video warning when falling into the defined limit of car speed.
22. overall view visual system according to claim 21, it is characterized in that, described display system is caught two dimensional image through the configuration back by video camera and is searched database according to track and roadside pattern and object and infer 3-D view, shows the vertical view of vehicle and setting on every side thereof.
23. overall view visual system according to claim 20, it is characterized in that, described image processor is further used for detecting at least one in road surface, track and the roadside mark, and provides audio frequency and visual alert when the mark of the approaching described roadside of vehicle outside surface.
24. overall view visual system according to claim 11, it is characterized in that, further comprise in following group at least one: environment light sensor, come headlight inductor and driver's dimmer switch, and the brightness and contrast of wherein said display system according to described environment light sensor, come in headlight inductor and the driver's dimmer switch reading of at least one to adjust.
25. overall view visual system according to claim 11 is characterized in that, further comprises compression stage and the recording unit continuous recording with the incident setting up video camera and audio frequency apparatus and detect.
26. overall view visual system according to claim 11 is characterized in that, further comprises integrated GPS navigation unit, it is presented at highway map and roadside information on the described display system.
27. overall view visual system according to claim 3 is characterized in that, further comprises a plurality of motion detectors and tracker, described overall view visual system is as monitoring vision system.
28. overall view visual system according to claim 27 is characterized in that, described image processor further uses described motion detector and tracker to carry out motion detection and target tracking.
29. overall view visual system according to claim 27 is characterized in that, after described motion detector and tracker detected motion, described image processor provided one group of video and audio signal.
30. overall view visual system according to claim 27; it is characterized in that; described image processor is the protection zone of one group of appointment of definition further; follow the trail of a plurality of targets; and the generation Voice ﹠ Video is reported to the police when having target to enter described appointment protected location; and when having target to enter described appointment protection zone, described display system is reset to highlight described target.
31. overall view visual system according to claim 3, it is characterized in that, further comprise one group of frequency induction device, and described digital quantizer further changes the frequency induction device and be input as digital audio-frequency data, described overall view visual system is as the video conference vision system.
32. overall view visual system according to claim 31, it is characterized in that, the intensity and the direction of the sound signal that described image processor and described display system further produce according to the video conference user produce and show the video conference user images of convergent-divergent.
33. having the overall view visual system of relevant video camera, display optics and geometrical property and how much and optical distortion parameter, a use generates the method for composograph that covers the highest 360 ° or 4 π sterad scopes to provide panorama to observe, it is characterized in that described method comprises:
(a) obtain image frame data from scene, described image frame data is common to cover the field range of the highest 360 ° or 4 π sterads, and produces set of diagrams and import as inductor;
(b) described image frame data of conversion and image inductor are input as Digital Image Data, and described Digital Image Data is relevant with the ambient light rank with brightness of image;
(c) measure picture contrast and the brightness histogram relevant with described Digital Image Data, by analyzing described picture contrast and brightness histogram and control the inductor exposure and optimize in the described Digital Image Data brightness range of selecting the zone, and proofread and correct the brightness disproportionation that the gamma correction figure that generates from video camera, display optics and geometrical property data calculates from the ambient light rank of environment light sensor;
(d) optics of the imperfection of reception expression optics and geometric distortion and video camera and display and geometric distortion parameter and display are observed parameter, and these parameter convolution are generated brightness and free-air correction figure;
(e) receive by described geometry and optical distortion parameter and display and observe free-air correction figure that the parameter convolution obtains described Digital Image Data is used the distortion correction conversion, to generate calibrated Digital Image Data;
(f) form composograph according to described calibrated Digital Image Data; And
(g) observing the surface described composograph of demonstration for observation, described composograph does not have the vision distortion.
34. method according to claim 33 is characterized in that, the red, green and blue composition of further handling described Digital Image Data individually is to proofread and correct the lateral chromatic aberration distortion.
35. method according to claim 34 is characterized in that, wherein the brightness adjustment is carried out to proofread and correct colourity inconsistent at described red, green and blue composition individually.
36. method according to claim 35 is characterized in that, wherein calibration shift realizes by following steps:
(i) obtain the grid data collection in the whole space that covers described geometry and optical distortion parameter;
(ii) calculate the calibration shift of each data set on the grid of corresponding step (i); And
(iii) insert contiguous grid conversion with realistic picture frame conversion parameter.
37. method according to claim 36 is characterized in that, wherein step (i) and (ii) off-line execution, and at each picture frame correction visit switch grid.
38. method according to claim 36 is characterized in that, wherein step (i) and (ii) dynamically finishing under situation of change.
39. method according to claim 36 is characterized in that, wherein relevant with described conversion pixel pattern is by distortion figure expression.
40., it is characterized in that wherein said distortion figure is according to increasing progressively one of polynomial expression and position polynomial expression parametrization according to the described method of claim 39.
41. method according to claim 35 is characterized in that, further generates and transmit the exposure control command to a plurality of image acquisition equipments.
42. method according to claim 35 is characterized in that, comprises that further reception control and customer parameter are to carry out process of convolution with distortion parameter.
43. method according to claim 35 is characterized in that, further comprises motion detection and tracking.
44. according to the described method of claim 42, it is characterized in that, further comprise brightness, contrast and the size of importing composograph shown in the adjustment according to inductor and user.
45. according to the described method of claim 42, it is characterized in that, further reset described display system according to one group of control input.
46. according to the described method of claim 42, it is characterized in that, further adjust the focal length of described display system according to one group of controlled variable.
47. image processor, be used to have the overall view visual system of relevant video camera, display optics and geometrical property and geometry and optical distortion parameter, described overall view visual system uses a plurality of image acquisition equipments with grab image frames from scene and generate Digital Image Data and image inductor input, and use digital quantizer to be input as Digital Image Data to change described Digital Image Data and image inductor, described image processor comprises:
(a) image measuring apparatus is in order to receive described Digital Image Data and to measure picture contrast and the brightness histogram relevant with described Digital Image Data;
(b) gamma correction module, control the inductor exposure and optimize the brightness range of selecting the zone in the described Digital Image Data by analyzing, and proofread and correct the brightness disproportionation that the gamma correction figure that generates from video camera, display optics and geometrical property data calculates from the picture contrast of described image measuring apparatus and brightness histogram and from the ambient light rank of environment light sensor;
(c) convolution level, receive the optics of imperfection of expression optics and geometric distortion and video camera and display and geometric distortion parameter and display and observe parameter, and these parameter convolution are generated brightness with free-air correction figure and offer gamma correction module and the distortion correction module that is connected;
(d) distortion correction module, be connected to described convolution level, reception is observed free-air correction figure that the parameter convolution obtains described Digital Image Data is used the distortion correction conversion, to generate calibrated Digital Image Data by described geometry and optical distortion parameter and display; And
(e) display controller is connected to described distortion correction module to form composograph according to described calibrated Digital Image Data.
48., it is characterized in that the red, green and blue composition that is further used for processes digital image data individually is to proofread and correct the lateral chromatic aberration distortion according to the described image processor of claim 47.
49., it is characterized in that the described gamma correction module red, green and blue composition of processes digital image data individually is inconsistent to proofread and correct colourity according to the described image processor of claim 48.
50. according to the described image processor of claim 49, it is characterized in that, be further used for generating and transmit the exposure control command to image acquisition equipment.
51. according to the described image processor of claim 49, it is characterized in that, be further used for receiving control and customer parameter to carry out process of convolution with distortion parameter.
52. according to the described image processor of claim 49, it is characterized in that, be further used for motion detection and follow the trail of the objective.
53. according to the described image processor of claim 49, it is characterized in that, be further used for importing brightness, contrast and the size of adjusting described composograph according to a plurality of inductors and one group of user.
54. according to the described image processor of claim 51, it is characterized in that, be further used for resetting described display system according to one group of control input.
55. according to the described image processor of claim 51, it is characterized in that, be further used for adjusting the focal length of described display system according to one group of controlled variable.
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| PCT/US2004/023849 WO2006022630A1 (en) | 2004-07-26 | 2004-07-26 | Panoramic vision system and method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN1985266A CN1985266A (en) | 2007-06-20 |
| CN1985266B true CN1985266B (en) | 2010-05-05 |
Family
ID=35967743
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2004800431488A Expired - Lifetime CN1985266B (en) | 2004-07-26 | 2004-07-26 | Panoramic vision system and method |
Country Status (4)
| Country | Link |
|---|---|
| EP (1) | EP1771811A4 (en) |
| JP (1) | JP4543147B2 (en) |
| CN (1) | CN1985266B (en) |
| WO (1) | WO2006022630A1 (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105898338A (en) * | 2015-12-18 | 2016-08-24 | 乐视致新电子科技（天津）有限公司 | Panorama video play method and device |
| WO2018094697A1 (en) * | 2016-11-25 | 2018-05-31 | 深圳市窝窝头科技有限公司 | Fast three-dimensional space projection and photographing visual identification system |
| CN111149356A (en) * | 2017-06-27 | 2020-05-12 | 布鲁姆斯技术有限公司 | Method for projecting immersive audiovisual content |
Families Citing this family (56)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| FR2926908A1 (en) * | 2008-01-30 | 2009-07-31 | Renault Sas | Motor vehicle i.e. car, driving assisting method, involves allocating quality score to each of pixels in each of corrected images, and selecting pixel having high quality in order to elaborate top view of environment of motor vehicle |
| JP5078148B2 (en) * | 2008-03-10 | 2012-11-21 | 株式会社リコー | Image processing apparatus and image pickup apparatus |
| CN101252687B (en) * | 2008-03-20 | 2010-06-02 | 上海交通大学 | Method for implementing multichannel combined interested area video coding and transmission |
| DE102008045007A1 (en) * | 2008-08-29 | 2010-03-04 | Man Nutzfahrzeuge Aktiengesellschaft | A method for image-based monitoring of a vehicle environment, in particular an environment of a commercial vehicle |
| GB2471708A (en) * | 2009-07-09 | 2011-01-12 | Thales Holdings Uk Plc | Image combining with light point enhancements and geometric transforms |
| JP4970516B2 (en) | 2009-09-30 | 2012-07-11 | 日立オートモティブシステムズ株式会社 | Surrounding confirmation support device |
| CN102052916B (en) * | 2009-11-04 | 2013-05-01 | 沈阳迅景科技有限公司 | Method for three-dimensional measurement of panoramic real scenes |
| BR112012019871A8 (en) * | 2010-02-08 | 2018-06-19 | Obschestvo S Ogranichennoy Otveststvennostiyu Sistemy Peredovykh Tekh | method and device for determining the speed of travel and vehicle coordinates and subsequently identifying and automatically recording road traffic violations |
| CN102233850A (en) * | 2010-05-04 | 2011-11-09 | 全福精密科技有限公司 | Driving auxiliary device with panoramic and driving recording functions |
| CN101945287B (en) * | 2010-10-14 | 2012-11-21 | 浙江宇视科技有限公司 | ROI encoding method and system thereof |
| JP2012144162A (en) * | 2011-01-12 | 2012-08-02 | Toyota Motor Corp | Travel support apparatus |
| KR101037547B1 (en) * | 2011-03-20 | 2011-05-26 | 황승발 | A high efficient control system for lighting of cctv camera |
| JP5932018B2 (en) * | 2011-03-28 | 2016-06-08 | エイヴィエル・テスト・システムズ・インコーポレーテッド | Deconvolution method for exhaust measurement |
| JP5741321B2 (en) * | 2011-08-23 | 2015-07-01 | 株式会社富士通ゼネラル | Driving assistance device |
| US10284776B2 (en) | 2011-11-07 | 2019-05-07 | Sony Interactive Entertainment Inc. | Image generation apparatus and image generation method |
| EP2779621B1 (en) | 2011-11-07 | 2021-12-22 | Sony Interactive Entertainment Inc. | Image generation device, image generation method and program |
| US9894272B2 (en) | 2011-11-07 | 2018-02-13 | Sony Interactive Entertainment Inc. | Image generation apparatus and image generation method |
| EP2779620B8 (en) * | 2011-11-07 | 2016-09-28 | Sony Interactive Entertainment Inc. | Image generation device, and image generation method |
| JP2013214947A (en) * | 2012-03-09 | 2013-10-17 | Ricoh Co Ltd | Image capturing apparatus, image capturing system, image processing method, information processing apparatus, and program |
| US9387813B1 (en) * | 2012-03-21 | 2016-07-12 | Road-Iq, Llc | Device, system and method for aggregating networks and serving data from those networks to computers |
| US20130293683A1 (en) * | 2012-05-03 | 2013-11-07 | Harman International (Shanghai) Management Co., Ltd. | System and method of interactively controlling a virtual camera |
| KR101847825B1 (en) * | 2012-05-24 | 2018-04-12 | 현대모비스 주식회사 | Around View System and Method for Camera Brightness Correction thereof |
| KR101937272B1 (en) * | 2012-09-25 | 2019-04-09 | 에스케이 텔레콤주식회사 | Method and Apparatus for Detecting Event from Multiple Image |
| CN104798128A (en) * | 2012-10-05 | 2015-07-22 | 维迪诺蒂有限公司 | Annotation method and apparatus |
| DE102012024661A1 (en) * | 2012-12-17 | 2014-06-18 | Connaught Electronics Ltd. | Method for white balance of an image representation taking into account color values excluding a subset of pixels, camera system and motor vehicle with a camera system |
| JP2014150476A (en) * | 2013-02-04 | 2014-08-21 | Olympus Imaging Corp | Photographing apparatus, image processing method, and image processing program |
| CN103568955A (en) * | 2013-09-30 | 2014-02-12 | 深圳市领华数据信息有限公司 | Car interior glass projection method and system |
| JPWO2015064095A1 (en) * | 2013-10-29 | 2017-03-09 | 京セラ株式会社 | Image correction parameter output device, camera system, and correction parameter output method |
| DE102014205511A1 (en) * | 2014-03-25 | 2015-10-01 | Conti Temic Microelectronic Gmbh | METHOD AND DEVICE FOR DISPLAYING OBJECTS ON A VEHICLE INDICATOR |
| JP2015186170A (en) | 2014-03-26 | 2015-10-22 | ソニー株式会社 | Image processing apparatus and image processing method |
| JP2015207802A (en) | 2014-04-17 | 2015-11-19 | ソニー株式会社 | Image processor and image processing method |
| JP6685293B2 (en) * | 2014-06-19 | 2020-04-22 | イヴェコ・エッセ・ピ・ア | Back vision system to help drive the vehicle |
| SE538494C2 (en) * | 2014-11-07 | 2016-08-02 | BAE Systems Hägglunds AB | External perception system and procedure for external perception in combat vehicles |
| KR101629577B1 (en) * | 2014-12-10 | 2016-06-13 | 현대오트론 주식회사 | Monitoring method and apparatus using a camera |
| TWI536313B (en) | 2015-06-30 | 2016-06-01 | 財團法人工業技術研究院 | Method for adjusting vehicle panorama system |
| CN107534757B (en) * | 2015-08-10 | 2020-04-10 | Jvc 建伍株式会社 | Vehicle display device and vehicle display method |
| JP6582874B2 (en) * | 2015-10-28 | 2019-10-02 | 株式会社リコー | COMMUNICATION SYSTEM, COMMUNICATION DEVICE, COMMUNICATION METHOD, AND PROGRAM |
| CN108886562B (en) * | 2016-03-22 | 2021-07-13 | 株式会社理光 | Image processing system, image processing method, and computer-readable storage medium |
| KR101765556B1 (en) * | 2016-05-11 | 2017-08-23 | (주)캠시스 | Apparatus and method for processing the image according to the velocity of automobile |
| JP6674630B2 (en) | 2016-05-30 | 2020-04-01 | カシオ計算機株式会社 | Image processing apparatus, image processing method, and program |
| KR102522565B1 (en) | 2016-08-31 | 2023-04-17 | 삼성전자주식회사 | Image display apparatus and operating method for the same |
| CN108001357A (en) * | 2016-10-31 | 2018-05-08 | 中交北斗技术有限责任公司 | A kind of high accuracy Big Dipper positioning intelligent rearview mirror |
| CN106775309A (en) * | 2016-12-06 | 2017-05-31 | 北京尊豪网络科技有限公司 | A kind of method and device for showing information of real estate |
| JP7059522B2 (en) | 2017-06-09 | 2022-04-26 | ソニーグループ株式会社 | Control device, control method, program and imaging system |
| CN109204326B (en) * | 2017-06-29 | 2020-06-12 | 深圳市掌网科技股份有限公司 | Driving reminding method and system based on augmented reality |
| JP6988206B2 (en) * | 2017-07-07 | 2022-01-05 | 株式会社タダノ | Crane car |
| JP7004410B2 (en) * | 2017-11-27 | 2022-01-21 | 株式会社東海理化電機製作所 | Vehicle visibility device and display control method |
| JP6989212B2 (en) * | 2017-11-27 | 2022-01-05 | 株式会社東海理化電機製作所 | Vehicle visibility device |
| CN108973860A (en) * | 2018-06-27 | 2018-12-11 | 北斗星通（重庆）汽车电子有限公司 | Vehicle-mounted blind area detection system |
| JP6857695B2 (en) * | 2018-09-14 | 2021-04-14 | シャープ株式会社 | Rear display device, rear display method, and program |
| CN109263652B (en) * | 2018-11-14 | 2020-06-09 | 江铃汽车股份有限公司 | Method for measuring and checking front visual field of driver |
| KR102559686B1 (en) * | 2018-12-17 | 2023-07-27 | 현대자동차주식회사 | Vehicle and vehicle image controlling method |
| CN110796597B (en) * | 2019-10-10 | 2024-02-02 | 武汉理工大学 | Vehicle-mounted all-round image splicing device based on space-time compensation |
| CN111917985B (en) * | 2020-08-14 | 2021-11-16 | 广东申义实业投资有限公司 | Vehicle, method and device for three-dimensional panoramic visual display and storage medium |
| CN111968184B (en) * | 2020-08-24 | 2024-04-02 | 北京茵沃汽车科技有限公司 | Method, device and medium for realizing view follow-up in panoramic looking-around system |
| CN116980540A (en) * | 2023-07-27 | 2023-10-31 | 湖北空间智能技术有限公司 | Low-illumination image processing method and device for pod and panoramic pod system |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5978521A (en) * | 1997-09-25 | 1999-11-02 | Cognex Corporation | Machine vision methods using feedback to determine calibration locations of multiple cameras that image a common object |
| WO2003107274A2 (en) * | 2002-06-12 | 2003-12-24 | Silicon Optix Inc. | System and method for electronic correction of optical anomalies |
| CN1252978C (en) * | 1998-01-23 | 2006-04-19 | 夏普公司 | Image processing device and image processing method |
Family Cites Families (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5465163A (en) * | 1991-03-18 | 1995-11-07 | Canon Kabushiki Kaisha | Image processing method and apparatus for processing oversized original images and for synthesizing multiple images |
| US5289321A (en) * | 1993-02-12 | 1994-02-22 | Secor James O | Consolidated rear view camera and display system for motor vehicle |
| US6498620B2 (en) | 1993-02-26 | 2002-12-24 | Donnelly Corporation | Vision system for a vehicle including an image capture device and a display system having a long focal length |
| US5699444A (en) * | 1995-03-31 | 1997-12-16 | Synthonics Incorporated | Methods and apparatus for using image data to determine camera location and orientation |
| JP3974964B2 (en) * | 1996-11-08 | 2007-09-12 | オリンパス株式会社 | Image processing device |
| JP3232408B2 (en) * | 1997-12-01 | 2001-11-26 | 日本エルエスアイカード株式会社 | Image generation device, image presentation device, and image generation method |
| EP1044116A2 (en) | 1997-12-31 | 2000-10-18 | Gentex Corporation | Vehicle vision system |
| US6163361A (en) * | 1999-04-23 | 2000-12-19 | Eastman Kodak Company | Digital camera including a printer for receiving a cartridge having security control circuitry |
| JP3298851B2 (en) * | 1999-08-18 | 2002-07-08 | 松下電器産業株式会社 | Multi-function vehicle camera system and image display method of multi-function vehicle camera |
| US20040100565A1 (en) * | 2002-11-22 | 2004-05-27 | Eastman Kodak Company | Method and system for generating images used in extended range panorama composition |
-
2004
- 2004-07-26 JP JP2007523514A patent/JP4543147B2/en not_active Expired - Lifetime
- 2004-07-26 WO PCT/US2004/023849 patent/WO2006022630A1/en active Application Filing
- 2004-07-26 CN CN2004800431488A patent/CN1985266B/en not_active Expired - Lifetime
- 2004-07-26 EP EP04779083A patent/EP1771811A4/en not_active Withdrawn
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5978521A (en) * | 1997-09-25 | 1999-11-02 | Cognex Corporation | Machine vision methods using feedback to determine calibration locations of multiple cameras that image a common object |
| CN1252978C (en) * | 1998-01-23 | 2006-04-19 | 夏普公司 | Image processing device and image processing method |
| WO2003107274A2 (en) * | 2002-06-12 | 2003-12-24 | Silicon Optix Inc. | System and method for electronic correction of optical anomalies |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105898338A (en) * | 2015-12-18 | 2016-08-24 | 乐视致新电子科技（天津）有限公司 | Panorama video play method and device |
| WO2018094697A1 (en) * | 2016-11-25 | 2018-05-31 | 深圳市窝窝头科技有限公司 | Fast three-dimensional space projection and photographing visual identification system |
| CN111149356A (en) * | 2017-06-27 | 2020-05-12 | 布鲁姆斯技术有限公司 | Method for projecting immersive audiovisual content |
Also Published As
| Publication number | Publication date |
|---|---|
| JP2008507449A (en) | 2008-03-13 |
| EP1771811A1 (en) | 2007-04-11 |
| WO2006022630A1 (en) | 2006-03-02 |
| EP1771811A4 (en) | 2010-06-09 |
| CN1985266A (en) | 2007-06-20 |
| JP4543147B2 (en) | 2010-09-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN1985266B (en) | Panoramic vision system and method | |
| TWI287402B (en) | Panoramic vision system and method | |
| US11601601B2 (en) | Optical array for high-quality imaging in harsh environments | |
| US9445011B2 (en) | Dynamic rearview mirror adaptive dimming overlay through scene brightness estimation | |
| US10564267B2 (en) | High dynamic range imaging of environment with a high intensity reflecting/transmitting source | |
| US20150109444A1 (en) | Vision-based object sensing and highlighting in vehicle image display systems | |
| US8390686B2 (en) | Surveillance camera apparatus and surveillance camera system | |
| US20150042799A1 (en) | Object highlighting and sensing in vehicle image display systems | |
| JP4491453B2 (en) | Method and apparatus for visualizing the periphery of a vehicle by fusing infrared and visual images depending on the periphery | |
| US20150145950A1 (en) | Multi field-of-view multi sensor electro-optical fusion-zoom camera | |
| US20020130953A1 (en) | Enhanced display of environmental navigation features to vehicle operator | |
| JP2020056839A (en) | Imaging apparatus | |
| JP2005509129A5 (en) | ||
| JP2020057869A (en) | Imaging apparatus | |
| CN106991415A (en) | Image processing method and device for vehicle-mounted fisheye camera | |
| DE102013220839B4 (en) | A method of dynamically adjusting a brightness of an image of a rear view display device and a corresponding vehicle imaging system | |
| KR20070049109A (en) | Panoramic vision system and method | |
| US11780368B2 (en) | Electronic mirror system, image display method, and moving vehicle | |
| TW202019152A (en) | Vehicle vision auxiliary system | |
| CN112825552B (en) | Panoramic lens, optical imaging method thereof and image processing method | |
| TWM560413U (en) | Real-time imaging system for vehicle rear viewing | |
| GB2360413A (en) | Wide angle parabolic imaging and image mapping apparatus | |
| TWM659102U (en) | Automatic switching device for display view | |
| CN118544952A (en) | Intelligent electronic rearview mirror visual field display method, device, equipment and medium | |
| CN117129002A (en) | Vehicle navigation system for improving driving safety |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| ASS | Succession or assignment of patent right |
Owner name: GEO SEMICONDUCTOR CO., LTD. Free format text: FORMER OWNER: SILICON OPTIX INC. Effective date: 20100914 |
|
| C41 | Transfer of patent application or patent right or utility model | ||
| COR | Change of bibliographic data |
Free format text: CORRECT: ADDRESS; FROM: CALIFORNIA, THE USA TO: ROOM 1050, NO.2350, MISSION COLLEGE BOULEVARD, SANTA CLARA, CALIFORNIA, CALIFORNIA, THE USA |
|
| TR01 | Transfer of patent right |
Effective date of registration: 20100914 Address after: Room 2350, 1050th College Avenue, Santa Clara, California, USA Patentee after: Geo Semiconductor Inc. Address before: california Patentee before: Silicon Optix Inc. |
|
| CX01 | Expiry of patent term |
Granted publication date: 20100505 |
|
| CX01 | Expiry of patent term |