US8446268B2 - System for displaying views of vehicle and its surroundings - Google Patents

System for displaying views of vehicle and its surroundings Download PDF

Info

Publication number
US8446268B2
US8446268B2 US12/576,721 US57672109A US8446268B2 US 8446268 B2 US8446268 B2 US 8446268B2 US 57672109 A US57672109 A US 57672109A US 8446268 B2 US8446268 B2 US 8446268B2
Authority
US
United States
Prior art keywords
view
obstacle
vehicle
alternative
suitably
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active, expires
Application number
US12/576,721
Other languages
English (en)
Other versions
US20100259372A1 (en
Inventor
Fujie Hideshiro
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hyundai Motor Japan R&D Center Inc
Hyundai Motor Co
Kia Corp
Original Assignee
Hyundai Motor Japan R&D Center Inc
Hyundai Motor Co
Kia Motors Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hyundai Motor Japan R&D Center Inc, Hyundai Motor Co, Kia Motors Corp filed Critical Hyundai Motor Japan R&D Center Inc
Assigned to KIA MOTORS CORPORATION, HYUNDAI MOTOR COMPANY, HYUNDAI MOTOR JAPAN R&D CENTER INC. reassignment KIA MOTORS CORPORATION ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: HIDESHIRO, FUJIE
Publication of US20100259372A1 publication Critical patent/US20100259372A1/en
Application granted granted Critical
Publication of US8446268B2 publication Critical patent/US8446268B2/en
Active legal-status Critical Current
Adjusted expiration legal-status Critical

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • H04N7/181—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4038—Image mosaicing, e.g. composing plane images from plane sub-images
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/10—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of camera system used
    • B60R2300/105—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of camera system used using multiple cameras
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/30—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing
    • B60R2300/301—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing combining image information with other obstacle sensor information, e.g. using RADAR/LIDAR/SONAR sensors for estimating risk of collision
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/30—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing
    • B60R2300/303—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing using joined images, e.g. multiple camera images
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/30—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing
    • B60R2300/304—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing using merged images, e.g. merging camera image with stored images
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/40—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the details of the power supply or the coupling to vehicle components
    • B60R2300/402—Image calibration
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/60—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by monitoring and displaying vehicle exterior scenes from a transformed perspective
    • B60R2300/607—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by monitoring and displaying vehicle exterior scenes from a transformed perspective from a bird's eye viewpoint
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/70—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by an event-triggered choice to display a specific image among a selection of captured images
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/80—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement
    • B60R2300/8093—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement for obstacle warning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30248—Vehicle exterior or interior
    • G06T2207/30252—Vehicle exterior; Vicinity of vehicle
    • G06T2207/30261—Obstacle

Definitions

  • the present invention relates, in general, to technology for displaying views of a vehicle, such as an automobile, and its surroundings. More particularly, the present invention relates to technology that is capable of, when displaying a combined bird's eye view of a vehicle and its surroundings based on views of the vehicle and surroundings acquired from a plurality of cameras for photographing a vehicle and its surroundings, displaying a three-dimensional (3D) object, where the 3D object may, for example be an object or obstacle which may disturb driving, or a human body which may be harmed while driving; hereinafter referred to as the “obstacle”) in the bird's eye view without suitably causing a sense of incongruity.
  • 3D three-dimensional
  • An example of a system for combining views from a plurality of cameras on a vehicle with an alternative view of the vehicle and providing a bird's eye view of the vehicle taken from a virtual viewpoint above the vehicle to a driver includes, for example, the “Around View Monitor” of Nissan Motor Co. Ltd.
  • an obstacle is viewed as being laid down flat on a road surface. Furthermore, in this type of technology, it is suitably impossible to display the part of an obstacle, that is, the part of an object, having a height which is suitably outside the photographable range of the cameras. Further, in employing this type of technology, there also arises the sense of incongruity in which an object having a height is viewed as extending out of a vehicle in the form of a rectilinear line which connects the location of a camera with the location of the object on a road surface.
  • Japanese Unexamined Patent Publication No. 2003-189293 incorporated by reference in its entirety herein, is directed to technology that, when part of an obstacle is detected in the range in which a camera cannot photograph it (the side behind the obstacle which is viewed from the camera), determines the non-photographed part using the location of the edge of the detected obstacle facing the vehicle and filling in the part with an alternative view.
  • this technology is related with the display of a top view, and does not disclose an improvement in the display of a bird's eye view of an area outside a vehicle, for example, an area behind or beside the vehicle.
  • dynamic representation is not included in an alternative view of an obstacle, the ability to call attention to the motion of an obstacle, such as a human, is low.
  • Japanese Unexamined Patent Publication No. Hei 7-223488 incorporated by reference in its entirety herein, is directed to technology that, when an obstacle is detected, displays a schematic figure (a static alternative view) at a location where the obstacle is detected according to the type of detected obstacle (e.g., a human, a bicycle, a vehicle, or the like).
  • a schematic figure a static alternative view
  • the type of detected obstacle e.g., a human, a bicycle, a vehicle, or the like.
  • this type of display of an obstacle is not accompanied with an indication of the presence of a motion or the direction of a posture, and still exhibits a sense of incongruity.
  • Japanese Unexamined Patent Publication No. 2007-295043 incorporated by reference in its entirety herein, is directed to technology that, in the case where a detected obstacle (a human) is present outside the range of a displayed view, illustrates the movement of the obstacle using alternative views based on top views, but the alternative views are suitably limited to the top views acquired from directly above the obstacle. Meanwhile, in the case where an obstacle is present within the range of a displayed screen, the obstacle is displayed as it is. Accordingly, neither of the two cases can overcome the sense of incongruity.
  • the present invention provides a system for suitably displaying views of a vehicle and its surroundings, which is capable of suitably eliminating a sense of incongruity (for example, such as the sense of an obstacle being viewed as having been laid down flat on a road surface) from the display of a conventional combined bird's eye view using a low-cost method, and which is suitably capable of allowing a driver to intuitively and rapidly locate an obstacle outside a vehicle, thus suitably assisting the driver to ensure safety around the vehicle.
  • a sense of incongruity for example, such as the sense of an obstacle being viewed as having been laid down flat on a road surface
  • the present invention provides a system for displaying views of a vehicle and its surroundings preferably including one or more cameras for suitably photographing a vehicle and its surroundings; obstacle detection means for suitably detecting an obstacle outside the vehicle; a memory unit for previously storing alternative views corresponding to the obstacle; and a view processing unit for suitably creating a bird's eye view from a virtual viewpoint outside the vehicle based on views suitably acquired by the cameras; wherein when the obstacle detection means suitably detects the obstacle, the view processing unit pauses a screen on which the obstacle appears, suitably selects an alternative view corresponding to the paused obstacle, reads the alternative view from the memory unit, suitably changes a direction and inclination of the selected alternative view in line with the virtual viewpoint, and then overlaps the changed alternative view on the bird's eye view.
  • the view processing unit suitably measures the size of the paused obstacle, suitably detects whether a motion is present, and then suitably selects a moving image or still image alternative view corresponding to the measured size.
  • the memory unit suitably stores a plurality of still image alternative views and/or moving image alternative views of different sizes for the same type of obstacle.
  • the view processing unit prior to the overlapping of the alternative view, suitably eliminates a view of the obstacle and performs compensation using information about the neighboring pixels of the view of the obstacle.
  • the view processing unit after the overlapping of the alternative view, suitably inserts a gradation effect view around the overlapped alternative view, or reads the alternative view, previously associated with the gradation effect view, from the memory unit and then overlaps them.
  • the view processing unit if the size of the obstacle detected by the obstacle detection means is suitably greater than a predetermined value, does not overlap the alternative view.
  • the view processing unit overlaps the alternative view along a rectilinear line that suitably connects the center of the vehicle with the location of the obstacle on a road surface.
  • the memory unit preferably stores a plurality of models for the same type of obstacle depending on distances from the vehicle, the models including models for the cases where part of the obstacle falls within the photographable range of the cameras, and the view processing unit suitably compares a view of the detected obstacle with the models, pauses the obstacle, and then selects an overall alternative view of the obstacle.
  • a view processing unit suitably transforms views from photographing means into a bird's eye view taken from above a vehicle or from beside a vehicle, selects an alternative view suitable for the size of a 3D obstacle outside the vehicle and the motion of the vehicle from a memory unit based on information from obstacle detection means, and then overwrites the bird's eye view, so that the 3D obstacle located outside the vehicle is suitably displayed without causing a sense of incongruity and a driver can easily locate the obstacle.
  • vehicle or “vehicular” or other similar term as used herein is inclusive of motor vehicles in general such as passenger automobiles including sports utility vehicles (SUV), buses, trucks, various commercial vehicles, watercraft including a variety of boats and ships, aircraft, and the like, and includes hybrid vehicles, electric vehicles, plug-in hybrid electric vehicles, hydrogen-powered vehicles and other alternative fuel vehicles (e.g. fuels derived from resources other than petroleum).
  • motor vehicles in general such as passenger automobiles including sports utility vehicles (SUV), buses, trucks, various commercial vehicles, watercraft including a variety of boats and ships, aircraft, and the like, and includes hybrid vehicles, electric vehicles, plug-in hybrid electric vehicles, hydrogen-powered vehicles and other alternative fuel vehicles (e.g. fuels derived from resources other than petroleum).
  • SUV sports utility vehicles
  • plug-in hybrid electric vehicles e.g. fuels derived from resources other than petroleum
  • a hybrid vehicle is a vehicle that has two or more sources of power, for example both gasoline-powered and electric-powered.
  • FIGS. 1A and 1B are block diagrams showing the overall configurations of systems for displaying views of a vehicle and its surroundings according to preferred embodiments of the present invention, wherein FIG. 1A shows a system in which separate obstacle detection means is provided and FIG. 1B shows a system in which a set of photographing means and view processing means include obstacle detection means;
  • FIGS. 2A and 2B are diagrams showing the locations of virtual viewpoints around a vehicle, wherein FIG. 2A shows the virtual viewpoints viewed from above the vehicle and FIG. 2B shows the virtual viewpoints viewed from beside the vehicle;
  • FIGS. 3A to 3D are views of a vehicle and an obstacle beside the vehicle viewed from locations corresponding to virtual viewpoints, wherein FIG. 3A is a view from a side viewpoint in the case where the height of the obstacle is relatively low, FIG. 3B is a view from a top viewpoint common to the case where the height of the obstacle is high and the case where the height of the obstacle is low, FIG. 3C is a view from a side viewpoint in the case where the height of the obstacle is high, and FIG. 3D is a view from a back viewpoint in the case where the height of the obstacle is high;
  • FIGS. 4A to 4D are diagrams illustrating the flow of, in the case where the height of the obstacle is relatively low, transforming a view captured by photographing means (a side camera) into a bird's eye view from a side virtual viewpoint, wherein FIG. 4A is an input view, FIG. 4B is a rotated view, FIG. 4C is a projectively transformed view, and FIG. 4D is an output view;
  • FIGS. 5A to 5D are diagrams illustrating the flow of, in the case where the height of an obstacle is relatively high, transforming a view captured by photographing means (a side camera) into a bird's eye view from a side virtual viewpoint, wherein FIG. 5A is an input view, FIG. 5B is a rotated view, FIG. 5C is a projectively transformed view, and FIG. 5D is an output view;
  • FIG. 6 is a diagram illustrating the flow of transforming views captured by photographing means (four front, rear, right and left side cameras) into a bird's eye view from a top virtual viewpoint;
  • FIG. 7 is a diagram illustrating the flow of, when a view captured by a side camera is transformed into a bird's eye view from a side virtual viewpoint, replacing a view of an obstacle with an alternative view according to the present invention
  • FIG. 8 is a flowchart of the view processing of FIG. 7 ;
  • FIG. 9 is a diagram showing examples of the alternative view of FIG. 7 ;
  • FIGS. 10A and 10B are drawings showing a comparison in the top view between an example using an alternative view which is viewed from the point of view of inclination and an example using an alternative view the angle of which has been corrected;
  • FIGS. 11A and 11B are diagrams showing a range within which obstacles can be photographed depending on the distance to a vehicle and examples of the selection of an alternative view, respectively;
  • FIGS. 12A to 12D show an example of a top view illustrating the effect of using an alternative view to which the gradation effect has been applied, wherein FIG. 12A shows a conventional view (the angle of which has not been corrected), FIG. 12B shows a view overwritten with an alternative view the angle of which has been corrected, FIG. 12C shows a view using an alternative view to which the gradation effect has been applied, and FIG. 12D is an ideal view;
  • FIG. 13 is a diagram illustrating an alternative view to which the gradation effect has been applied.
  • FIG. 14 is a diagram illustrating the monoscopic ranging principle based on the motion stereo method.
  • the present invention provides, in one aspect, a system for displaying views of a vehicle and its surroundings comprising one or more cameras for photographing a vehicle and its surroundings, obstacle detection means for detecting an obstacle outside the vehicle, a memory unit; and a view processing unit.
  • the memory unit is used for previously storing alternative views corresponding to the obstacle.
  • the view processing unit is used for creating a bird's eye view from a virtual viewpoint outside the vehicle based on views acquired by the cameras.
  • the view processing unit pauses the screen on which the obstacle appears, selects an alternative view corresponding to the paused obstacle, reads the alternative view from the memory unit, changes a direction and inclination of the selected alternative view in line with the virtual viewpoint, and then overlaps the changed alternative view on the bird's eye view.
  • the present invention also features a motor vehicle comprising a system for displaying views of a vehicle and its surroundings of the aspects described herein.
  • the system for displaying views of a vehicle and its surroundings preferably includes one or more cameras 11 (e.g., photographing means) for suitably photographing a vehicle and its surroundings, obstacle detection means 12 for suitably detecting an obstacle outside the vehicle, a memory unit 13 for suitably storing the data of alternative views, a view processing unit 16 for suitably creating a view in which views captured by the cameras are transformed into a bird's eye view and an obstacle (a 3D object) in the bird's eye view has been replaced with an alternative view, and a monitor 15 (a display unit) for suitably displaying a view created by the view processing unit 16 .
  • cameras 11 e.g., photographing means
  • obstacle detection means 12 for suitably detecting an obstacle outside the vehicle
  • a memory unit 13 for suitably storing the data of alternative views
  • a view processing unit 16 for suitably creating a view in which views captured by the cameras are transformed into a bird's eye view and an obstacle (a 3D object) in the bird's eye view has been replaced with an
  • the memory unit 13 and the view processing unit 16 are generally included in an Electronic Control Unit (ECU) 14 .
  • ECU Electronic Control Unit
  • the obstacle detection means 12 may be suitably constructed of any one of a laser, a sonar and a radar, as shown in FIG. 1A .
  • the obstacle detection means 12 is preferably included in the photographing means 11 and the view processing unit 16 , so that views captured by the photographing means 11 are suitably analyzed by the view processing unit 16 and then an obstacle is detected, for example as shown in FIG. 1B .
  • the cameras 11 are generally mounted on mirrors that are suitably disposed on the front right and left and rear right and left sides of a vehicle.
  • the monitor 15 may preferably be separately provided in a vehicle, the monitor 15 may be preferably also used as a car navigation monitor, so that the monitor 15 may be automatically transformed into a bird's eye view monitor in the case of stopping or low-speed driving.
  • virtual viewpoints that are preferably considered when a bird's eye view of a vehicle 20 and its surroundings are suitably created may be virtual viewpoints 21 , 22 , 23 and 24 above the vehicle 20 , on a side of the vehicle 20 , in front of the vehicle 20 and behind the vehicle 20 , and these viewpoints may be suitably referred to as top, side, front and back viewpoints, respectively.
  • the above views must suitably correspond to the views of FIGS. 3C and 3D , respectively, relative to the road surface views 55 and 50 .
  • the front and back surfaces of the obstacle 25 refer to the surface of the obstacle 25 facing the vehicle 20 (that is, the camera 11 mounted on the side door mirror) and its opposite surface, respectively.
  • the upper part of the obstacle does suitably fall within the photographable range of the camera, so that there arises the sense of incongruity in which the obstacle 25 which must stand while showing its back surface is generally viewed as having been suitably turned upside down while showing its front surface, or is generally viewed as having been completely laid down flat, that is, as having fallen over, or in which only the lower part of the obstacle 28 is suitably viewed, as in an output view shown in FIG. 5D , thereby suitably provoking a driver's misjudgment.
  • the cause of the phenomenon will now be described with reference to the flow of the creation of a bird's eye view, shown in FIGS. 4A to 4D , for a preferred example where the height of an obstacle is relatively low.
  • FIG. 4A shows an input view which is captured by a camera mounted on a side door mirror according to certain exemplary embodiments of the present invention.
  • FIG. 4B shows “a view from a camera side” that is preferably considered to be “a view from a side viewpoint,” so that the view has been suitably rotated by 180 degrees, and the upper side of the view is preferably a vehicle side and the lower side of the view is preferably a side viewpoint side.
  • the obstacle 26 is viewed as having been suitably turned upside down.
  • the vehicle side (the upper side of the view) has been suitably reduced and the side outside the vehicle (the lower side) has been suitably extended using a projective transformation.
  • the obstacle 27 is greatly deformed, so that the sense of incongruity is further suitably increased.
  • a view from a side viewpoint has been completed by suitably combining the projectively transformed view of FIG. 4C with an alternative view of the vehicle.
  • the obstacle 28 is viewed as having been turned upside down (or as having been completely laid down flat), so that the shape of the obstacle 28 is greatly deformed, with the result that the sense of incongruity is suitably serious and a driver's misjudgment may be caused.
  • the cause of the phenomenon for the case where the height of an obstacle is relatively high can preferably be understood in the same way based on the flow of creating a bird's eye view shown in FIGS. 5A to 5D .
  • the lower part of the obstacle 25 can be suitably photographed but the upper part thereof cannot be suitably photographed, so that the lower part 28 of the obstacle 25 which has been turned upside down and deformed is shown in FIG. 5D , which is the bird's eye view which is output.
  • the same problem occurs in the case where a bird's eye view from a top viewpoint is created using views from four cameras.
  • FIG. 6 the general sequence of creating a bird's eye view is illustrated.
  • respective original views 601 a - b captured by four cameras 603 a - b are suitably geometrically transformed into rectangular views based on respective sides of a vehicle on which the cameras have been suitably mounted, necessary view information is cut out using respective trapezoidal masks, trapezoidal views 602 a - d are suitably attached to the four peripheries of the alternative view of the vehicle, and smoothing is performed on the overlapping portions of the trapezoidal views, thereby creating a bird's eye view 605 from a top viewpoint.
  • the above-described rotation by 180 degrees is not necessary, so that the obstacle 25 is neither suitably reversed nor suitably inverted.
  • the deformation phenomenon of having been completely laid down flat occurs as well, and the sense of incongruity felt by a driver is suitably serious.
  • the camera 11 that is suitably mounted on a side door mirror preferably captures a view of a vehicle and its surroundings.
  • the obstacle detection means 12 detects whether an obstacle is suitably present outside the vehicle.
  • upper and lower views suitably correspond to the presence and absence of the obstacle 25 , respectively.
  • the obstacle detection means 12 in the case where the obstacle detection means 12 is suitably constructed of the photographing means 11 and the view processing unit 16 as shown in FIG. 1B , the obstacle detection means 12 suitably analyzes the captured camera view and then detects the obstacle 25 .
  • the obstacle detection means 12 is preferably separately provided as shown in FIG. 1A , for example, in the case where the obstacle detection means 12 is preferably a radar, the obstacle 25 is detected using said means.
  • step S 2 the process suitably proceeds directly to step S 10 , described herein.
  • the size of the obstacle is suitably measured, and at step S 4 , whether the motion of the obstacle is present is suitably detected.
  • the obstacle detection means 12 is preferably a radar
  • the measurement of the size of the obstacle and the detection of the presence of the motion of the obstacle are preferably performed by scanning the area for the obstacle using the radar.
  • the application of the motion stereo principle to camera views may preferably be considered. Accordingly, by using this principle, suitably capturing the views, suitably measuring the distance to the obstacle, suitably recognizing the shape of the obstacle, and suitably determining the motion of the obstacle can be performed.
  • the motion stereo method is preferably used to estimate the distance to an obstacle and the shape of the obstacle based on view data acquired at time t and view data acquired at time t+ ⁇ t.
  • moving image alternative views stored in the memory unit 13 are suitably searched for at step S 5 , and an alternative view 35 corresponding to the measured size is preferably selected from among the retrieved views at step S 7 .
  • step S 4 still image alternative views stored in the memory unit 13 are suitably searched for at step S 6 , and an alternative view 35 corresponding to the measured size is preferably selected from among the retrieved views at step S 7 .
  • a moving image or still image alternative view is suitably determined at step S 7 .
  • the alternative view 35 is suitably adjusted such that the direction and inclination of the alternative view 35 are preferably set for a side viewpoint (a viewpoint in the case where the vehicle is viewed from above in the lateral direction), which is the viewpoint of the bird's eye view in the present embodiment.
  • the direction is suitably set for the rear surface of the obstacle (for example, a standing human), and the inclination is suitably set for a vertical state (90 degrees).
  • the obstacle such as a pole
  • the inclination is set to the angle of inclination.
  • the “direction and inclination” may be suitably calculated whenever necessary, or an optimal view may be preferably selected from among corresponding alternative views stored in the memory unit.
  • the photographable range of the camera 11 generally extends from the location of a side door mirror to a remote location on a road surface.
  • part of the obstacle 25 is suitably outside the photographable range of the camera, so that it may be impossible to suitably accurately measure the size of the obstacle.
  • a case where an alternative view is suitably changed on the way such as the case where an object approaches a location near a vehicle from a remote location, may be considered.
  • the system for displaying views of a vehicle and its surroundings suitably prepares various types of models, including a model for representing part of an obstacle, for the same type of obstacle, depending the height of the obstacle 25 (human) (an adult and a child) and the distance to a vehicle, for example as shown in FIG. 11B .
  • FIG. 11A in the case of an adult obstacle 25 , four stage models (1. a model below the neck, 2. a model below the chest, 3. a model below the waist, and 4. a model below the knee) are suitably provided as captured shape recognition models depending on the distance to a vehicle, and an alternative view for the overall adult acquired by compensating for the part outside the photographable range is provided, as shown in FIG. 11B .
  • an obstacle detected by the obstacle detection means is suitably determined to encompass a predetermined portion of a vehicle, a wall or the like, for example, 50% or more of any one of the front, rear, right and left photographable ranges outside a vehicle, the following view processing is not performed, but the display of the phenomenon is suitably maintained.
  • a road surface view 52 is suitably acquired by eliminating the pixels of the obstacle 25 (a view that has been determined to be a view of the obstacle 25 ) detected at step S 2 from a view, including the obstacle 25 and the road surface view 51 , that are suitably acquired by the camera 11 at step S 1 .
  • a road surface view 53 is suitably created by performing compensation, reproduction and overwriting based on the chromatic information of the pixels of a surrounding road surface part.
  • pixel compensation technology such as, but not limited to, nearest neighbor interpolation, is suitably applied to the supplementation and the reproduction.
  • traffic signs such as lane lines drawn on a road surface
  • traffic signs are suitably determined not to be obstacles, and are considered to constitute part of a road surface view.
  • the sense of incongruity of the road surface view can be suitably reduced by adding a view having a gradation effect close to pixel compensation to the alternative view 35 itself.
  • a corresponding gradation effect view may be suitably stored in the memory unit in association with each alternative view in the form of a set of the gradation effect view and the alternative view.
  • the gradation effect view 45 is preferably added to the alternative view 35 , thus suitably forming an alternative view.
  • the gradation effect view 45 has a simple contour composed of smooth closed curves, such as a circle and an ellipse, and a single color in gray scale, and hides a deformed obstacle view resulting from an original obstacle view.
  • step S 8 the elimination of the pixels of the obstacle
  • step S 9 the interpolating reproduction of the pixels of a road surface
  • step S 9 may be omitted and only step S 8 may preferably be performed, and in other preferred embodiments, both steps S 8 and S 9 may be suitably performed without omission.
  • step S 10 bird's eye view processing is suitably performed.
  • the road surface view 54 is suitably obtained by rotating the road surface view 53 by 180 degrees so that the vehicle 20 is preferably located in the upper part of a screen, and then the road surface view 55 is suitably created by performing a projective transformation on the road surface view 54 so that the road surface view 54 has a natural perspective in such a manner that a front side is close and a vehicle side is remote.
  • the locations 26 a , 27 a and 28 a of the obstacle on the road surface are preferably preserved through steps S 9 and S 10 .
  • step S 11 view overlapping is performed.
  • an alternative view of the vehicle 20 viewed from a virtual viewpoint is suitably combined with the road surface view 55 created at step S 10 .
  • a bird's eye view in which the vehicle and the obstacle have been viewed from a virtual viewpoint is completed by further overlapping the alternative view 35 of the obstacle 25 determined at step S 7 , on the view obtained by combining the alternative view (which is also denoted by reference numeral 20 ) of the vehicle 20 and the road surface view 55 together.
  • the complete bird's eye view is output by displaying it on the monitor 15 .
  • the alternative view 35 of the obstacle is suitably lined up with the location 28 of the road surface view 55 on a road surface and is then combined in an overwriting manner.
  • FIGS. 7 and 8 The features and advantages of a system for displaying views of a vehicle and its surroundings according to preferred embodiments of the present invention have been described in the sequence of FIGS. 7 and 8 with reference to the case where a bird's eye view from the virtual viewpoint 22 beside the vehicle (a side viewpoint) is suitably created using a side door mirror camera, as shown in FIG. 2 .
  • the sequence of FIGS. 7 and 8 according to the present invention may be suitably applied to both the creation of a bird's eye view from the virtual viewpoint 23 in front of the vehicle (the front viewpoint) in FIG. 2 and the creation of a bird's eye view from the virtual viewpoint 24 behind the vehicle (the back viewpoint) in FIG. 2 in the completely same manner.
  • the sequence of FIGS. 7 and 8 may be suitably applied to each of the views captured by four cameras respectively mounted on the front right, front left, rear right and rear left sides of the vehicle, as shown in FIG. 6 .
  • step S 9 180-degree rotation is not necessary, and the road surface view 53 is immediately and projectively transformed to the road surface view 55 .
  • the obstacle is, for example, a human or a pole.
  • the alternative view 36 viewed from the point of view of inclination is suitably displayed in the direction that connects the front camera 11 with the location 27 a of the obstacle on a road surface.
  • the alternative view 36 suitably exhibits the sense of incongruity of being viewed as having been completely laid down flat.
  • the direction and inclination of the alternative view 37 are suitably set for the top viewpoint 21 (for example, a viewpoint above the center of the vehicle), which is the viewpoint of the bird's eye view in the present embodiment.
  • the direction is suitably set for the front view of the obstacle (for example, a standing human), and the inclination is suitably set for the direction that connects the center of the vehicle (the top viewpoint 21 ) with the location 27 a of the obstacle on a road surface.
  • the sense of incongruity of being viewed as having been completely laid down flat has been suitably eliminated.
  • FIGS. 12A to 12D show an example of a top view illustrating the effect of using an alternative view to which the gradation effect has been suitably applied.
  • the view of FIG. 12A is a bird's eye view that is suitably acquired by using the sequence shown in FIG. 6 .
  • a view of an obstacle 25 b is considerably large in the direction that connects the front camera 11 with the location 27 a of the obstacle 25 b on a road surface, and the obstacle 25 b is viewed as having been completely laid down flat on the road surface.
  • FIG. 12B shows a view overwritten with an alternative view 37 in inclined form which has been suitably tilted in the direction that connects the center 21 of the vehicle with the location 27 a of the obstacle on a road surface.
  • the alternative view 37 itself has no sense of incongruity, a considerable part of the view of the obstacle 25 b is not suitably hidden by the alternative view 37 but is suitably exposed by the alternative view 37 , thus resulting in confusion and a sense of incongruity.
  • FIG. 12C a case where the gradation effect view 45 for the alternative view 35 , such as that shown in FIG. 13 , has been suitably applied to the alternative view 37 .
  • the view of the obstacle 25 b can be actually hidden, the resulting view can suitably compare with an ideal view shown in FIG. 12D , and the sense of incongruity can preferably be practically reduced.
  • the view of FIG. 4C can be obtained at low cost, it is substantially equivalent to the view of FIG. 4D from the viewpoint of alerting a driver.
  • FIG. 9 Examples of alternative views stored in the memory unit are shown in FIG. 9 .
  • an alternative view is suitably represented in a bird's eye view in the form of a moving image alternative view and an obstacle is an animal or a human
  • still image alternative views showing the alternate motions of both hands or both legs may be alternately displayed, thus creating the impression of walking.
  • alternative views preferably stored in the memory unit are, for example, but not limited to, still image alternative views including a pole, a vending machine, a bench, etc.
  • the obstacle detection means may be any device as long as it can suitably detect the distance from a vehicle to an obstacle, the size of an obstacle, and whether a motion is present.
  • the obstacle detection means may be a laser or an ultrasonic sensor, although is not only limited as such.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Mechanical Engineering (AREA)
  • Closed-Circuit Television Systems (AREA)
  • Image Processing (AREA)
  • Traffic Control Systems (AREA)
  • Fittings On The Vehicle Exterior For Carrying Loads, And Devices For Holding Or Mounting Articles (AREA)
US12/576,721 2009-04-14 2009-10-09 System for displaying views of vehicle and its surroundings Active 2031-09-08 US8446268B2 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2009097685A JP5165631B2 (ja) 2009-04-14 2009-04-14 車両周囲画像表示システム
JP2009-097685 2009-04-14

Publications (2)

Publication Number Publication Date
US20100259372A1 US20100259372A1 (en) 2010-10-14
US8446268B2 true US8446268B2 (en) 2013-05-21

Family

ID=42933929

Family Applications (1)

Application Number Title Priority Date Filing Date
US12/576,721 Active 2031-09-08 US8446268B2 (en) 2009-04-14 2009-10-09 System for displaying views of vehicle and its surroundings

Country Status (3)

Country Link
US (1) US8446268B2 (ja)
JP (1) JP5165631B2 (ja)
KR (1) KR101491170B1 (ja)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130124061A1 (en) * 2011-11-10 2013-05-16 GM Global Technology Operations LLC System and method for determining a speed of a vehicle
US20140009589A1 (en) * 2011-02-10 2014-01-09 Daimler Ag Vehicle having a device for detecting the surroundings of said vehicle
US20140132527A1 (en) * 2012-11-14 2014-05-15 Avisonic Technology Corporation Method for controlling display of vehicular image by touch panel and vehicular image system thereof
US20150307024A1 (en) * 2014-04-25 2015-10-29 Hitachi Construction Machinery Co., Ltd. Vehicle peripheral obstacle notification system
US20170320437A1 (en) * 2015-01-27 2017-11-09 Bayerische Motoren Werke Aktiengesellschaft Measurement of a Dimension on a Surface
DE102016117518A1 (de) 2016-09-16 2018-03-22 Connaught Electronics Ltd. Angepasstes Zusammenfügen von Einzelbildern zu einem Gesamtbild in einem Kamerasystem für ein Kraftfahrzeug
US9990550B2 (en) 2014-09-19 2018-06-05 Bendix Commercial Vehicle Systems Llc Wide baseline object detection stereo system
US20190025854A1 (en) * 2017-07-20 2019-01-24 Mohsen Rohani Method and system for vehicle localization

Families Citing this family (30)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103180522B (zh) * 2010-10-22 2016-01-13 日立建机株式会社 作业机械的周边监视装置
EP2720458A4 (en) 2011-06-09 2015-02-25 Aisin Seiki IMAGING DEVICE
US9019347B2 (en) 2011-10-13 2015-04-28 Aisin Seiki Kabushiki Kaisha Image generator
JP5870608B2 (ja) 2011-10-13 2016-03-01 アイシン精機株式会社 画像生成装置
DE102011116169A1 (de) * 2011-10-14 2013-04-18 Continental Teves Ag & Co. Ohg Vorrichtung zur Unterstützung eines Fahrers beim Fahren eines Fahrzeugs oder zum autonomen Fahren eines Fahrzeugs
DE102011084596A1 (de) * 2011-10-17 2013-04-18 Robert Bosch Gmbh Verfahren zum Assistieren eines Fahrers in einer fremden Umgebung
DE102011088332B4 (de) 2011-12-13 2021-09-02 Robert Bosch Gmbh Verfahren zur Verbesserung der Objektdetektion bei Multikamerasystemen
US9381812B2 (en) 2012-02-29 2016-07-05 Toyota Motor Engineering & Manufacturing North America, Inc. Method to graphically display a vehicle condition
US9280826B2 (en) * 2013-12-10 2016-03-08 GM Global Technologies Operations LLC Distance determination using a monoscopic imager in a vehicle
JP6232994B2 (ja) * 2013-12-16 2017-11-22 ソニー株式会社 画像処理装置、および画像処理方法、並びにプログラム
DE102014107235A1 (de) * 2014-05-22 2015-11-26 Dr. Ing. H.C. F. Porsche Aktiengesellschaft Verfahren zur Darstellung einer Fahrzeugumgebung auf einer Anzeigevorrichtung; eine Anzeigevorrichtung; ein System aus einer Mehrzahl an Bilderfassungseinheiten und einer Anzeigevorrichtung; ein Computerprogramm
JP2016013793A (ja) * 2014-07-03 2016-01-28 株式会社デンソー 画像表示装置、画像表示方法
US20160176340A1 (en) * 2014-12-17 2016-06-23 Continental Automotive Systems, Inc. Perspective shifting parking camera system
JP6519408B2 (ja) * 2015-08-27 2019-05-29 株式会社Ｊｖｃケンウッド 車両用表示装置および車両用表示方法
WO2017108221A1 (en) * 2015-12-21 2017-06-29 Robert Bosch Gmbh Dynamic image blending for multiple-camera vehicle systems
JP6794193B2 (ja) * 2016-09-02 2020-12-02 株式会社小松製作所 作業機械の画像表示システム
KR102632361B1 (ko) * 2016-12-23 2024-02-02 삼성전기주식회사 어라운드 뷰 영상 제공 시스템
JP6730177B2 (ja) * 2016-12-28 2020-07-29 株式会社デンソーテン 画像生成装置および画像生成方法
JP2018110328A (ja) * 2017-01-04 2018-07-12 株式会社デンソーテン 画像処理装置および画像処理方法
DE102017201000A1 (de) * 2017-01-23 2018-07-26 Robert Bosch Gmbh Verfahren zum Kombinieren einer Vielzahl von Kamerabildern
JP6787209B2 (ja) * 2017-03-23 2020-11-18 日産自動車株式会社 表示装置の制御方法及び表示装置
US10740972B2 (en) * 2017-04-28 2020-08-11 Harman International Industries, Incorporated System and method for presentation and control of augmented vehicle surround views
JP6700222B2 (ja) 2017-06-09 2020-05-27 株式会社Ｓｏｋｅｎ 画像作成装置
JP2019151304A (ja) * 2018-03-06 2019-09-12 アイシン精機株式会社 周辺監視装置
DE102018108751B4 (de) 2018-04-12 2023-05-04 Motherson Innovations Company Limited Verfahren, System und Vorrichtung zum Erhalten von 3D-Information von Objekten
JP7000980B2 (ja) * 2018-05-08 2022-01-19 トヨタ自動車株式会社 駐車支援装置
DE102018214875A1 (de) * 2018-08-31 2020-03-05 Audi Ag Verfahren und Anordnung zum Erzeugen einer Umgebungsrepräsentation eines Fahrzeugs und Fahrzeug mit einer solchen Anordnung
JP7115300B2 (ja) * 2018-12-26 2022-08-09 株式会社デンソー 画像生成装置及び画像表示システム
DE102019219017A1 (de) * 2019-12-05 2021-06-10 Robert Bosch Gmbh Anzeigeverfahren zur Darstellung eines Umgebungsmodells eines Fahrzeugs, Computerprogramm, Steuergerät und Fahrzeug
JP2024513233A (ja) * 2021-04-07 2024-03-22 ストーンリッジ インコーポレイテッド オフセットカメラビューを含む車両ディスプレイ

Citations (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5969969A (en) * 1992-09-30 1999-10-19 Hitachi, Ltd. Vehicle driving support system which is responsive to environmental conditions
KR20010112433A (ko) 1999-04-16 2001-12-20 마츠시타 덴끼 산교 가부시키가이샤 화상처리장치 및 감시시스템
US20040090314A1 (en) * 2002-08-27 2004-05-13 Suzuki Motor Corporation Alarm information providing apparatus for vehicle
US6956469B2 (en) * 2003-06-13 2005-10-18 Sarnoff Corporation Method and apparatus for pedestrian detection
US20060119472A1 (en) * 2004-11-09 2006-06-08 Shoichi Tsuboi Driving support apparatus and driving support method
US20070003108A1 (en) * 2005-05-20 2007-01-04 Nissan Motor Co., Ltd. Image processing device and method for parking support
US20070057816A1 (en) * 2005-09-12 2007-03-15 Aisin Aw Co., Ltd. Parking assist method and parking assist apparatus
US20070206833A1 (en) * 2006-03-02 2007-09-06 Hitachi, Ltd. Obstacle detection system
US20070222565A1 (en) * 2006-03-27 2007-09-27 Mazda Motor Corporation Pedestrian detecting device for vehicle
JP2007295043A (ja) 2006-04-20 2007-11-08 Matsushita Electric Ind Co Ltd 車両周囲監視装置
US7298247B2 (en) * 2004-04-02 2007-11-20 Denso Corporation Vehicle periphery monitoring system
JP2008011187A (ja) 2006-06-29 2008-01-17 Matsushita Electric Ind Co Ltd 表示制御装置
US20080012940A1 (en) * 2006-07-06 2008-01-17 Nissan Motor Co., Ltd. Vehicle image display system and image display method
JP2008030705A (ja) 2006-07-31 2008-02-14 Aisin Aw Co Ltd 車両周辺画像表示システム

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH08305999A (ja) * 1995-05-11 1996-11-22 Hitachi Ltd 車載用カメラシステム
JP3606853B2 (ja) * 2001-09-07 2005-01-05 松下電器産業株式会社 車両周囲状況表示装置
JP2006040008A (ja) * 2004-07-28 2006-02-09 Auto Network Gijutsu Kenkyusho:Kk 運転支援装置
JP4907883B2 (ja) * 2005-03-09 2012-04-04 株式会社東芝 車両周辺画像表示装置および車両周辺画像表示方法

Patent Citations (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5969969A (en) * 1992-09-30 1999-10-19 Hitachi, Ltd. Vehicle driving support system which is responsive to environmental conditions
KR20010112433A (ko) 1999-04-16 2001-12-20 마츠시타 덴끼 산교 가부시키가이샤 화상처리장치 및 감시시스템
US20040090314A1 (en) * 2002-08-27 2004-05-13 Suzuki Motor Corporation Alarm information providing apparatus for vehicle
US6956469B2 (en) * 2003-06-13 2005-10-18 Sarnoff Corporation Method and apparatus for pedestrian detection
US7298247B2 (en) * 2004-04-02 2007-11-20 Denso Corporation Vehicle periphery monitoring system
US20060119472A1 (en) * 2004-11-09 2006-06-08 Shoichi Tsuboi Driving support apparatus and driving support method
US20070003108A1 (en) * 2005-05-20 2007-01-04 Nissan Motor Co., Ltd. Image processing device and method for parking support
US20070057816A1 (en) * 2005-09-12 2007-03-15 Aisin Aw Co., Ltd. Parking assist method and parking assist apparatus
US20070206833A1 (en) * 2006-03-02 2007-09-06 Hitachi, Ltd. Obstacle detection system
US20070222565A1 (en) * 2006-03-27 2007-09-27 Mazda Motor Corporation Pedestrian detecting device for vehicle
JP2007295043A (ja) 2006-04-20 2007-11-08 Matsushita Electric Ind Co Ltd 車両周囲監視装置
JP2008011187A (ja) 2006-06-29 2008-01-17 Matsushita Electric Ind Co Ltd 表示制御装置
US20080012940A1 (en) * 2006-07-06 2008-01-17 Nissan Motor Co., Ltd. Vehicle image display system and image display method
JP2008030705A (ja) 2006-07-31 2008-02-14 Aisin Aw Co Ltd 車両周辺画像表示システム

Cited By (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140009589A1 (en) * 2011-02-10 2014-01-09 Daimler Ag Vehicle having a device for detecting the surroundings of said vehicle
US20130124061A1 (en) * 2011-11-10 2013-05-16 GM Global Technology Operations LLC System and method for determining a speed of a vehicle
US20140132527A1 (en) * 2012-11-14 2014-05-15 Avisonic Technology Corporation Method for controlling display of vehicular image by touch panel and vehicular image system thereof
US9195390B2 (en) * 2012-11-14 2015-11-24 Avisonic Technology Corporation Method for controlling display of vehicular image by touch panel and vehicular image system thereof
US20150307024A1 (en) * 2014-04-25 2015-10-29 Hitachi Construction Machinery Co., Ltd. Vehicle peripheral obstacle notification system
US9463741B2 (en) * 2014-04-25 2016-10-11 Hitachi Construction Machinery Co., Ltd. Vehicle peripheral obstacle notification system
US9990550B2 (en) 2014-09-19 2018-06-05 Bendix Commercial Vehicle Systems Llc Wide baseline object detection stereo system
US20170320437A1 (en) * 2015-01-27 2017-11-09 Bayerische Motoren Werke Aktiengesellschaft Measurement of a Dimension on a Surface
US10611307B2 (en) * 2015-01-27 2020-04-07 Bayerische Motoren Werke Aktiengesellschaft Measurement of a dimension on a surface
DE102016117518A1 (de) 2016-09-16 2018-03-22 Connaught Electronics Ltd. Angepasstes Zusammenfügen von Einzelbildern zu einem Gesamtbild in einem Kamerasystem für ein Kraftfahrzeug
US20190025854A1 (en) * 2017-07-20 2019-01-24 Mohsen Rohani Method and system for vehicle localization
US10579067B2 (en) * 2017-07-20 2020-03-03 Huawei Technologies Co., Ltd. Method and system for vehicle localization

Also Published As

Publication number Publication date
JP2010251939A (ja) 2010-11-04
KR20100113959A (ko) 2010-10-22
KR101491170B1 (ko) 2015-02-06
US20100259372A1 (en) 2010-10-14
JP5165631B2 (ja) 2013-03-21

Similar Documents

Publication Publication Date Title
US8446268B2 (en) System for displaying views of vehicle and its surroundings
JP7245295B2 (ja) 車両・被牽引車両コンビの周辺シーンを表示するための方法、並びに、装置
US8576285B2 (en) In-vehicle image processing method and image processing apparatus
US9672432B2 (en) Image generation device
EP2763407B1 (en) Vehicle surroundings monitoring device
US8502860B2 (en) Electronic control system, electronic control unit and associated methodology of adapting 3D panoramic views of vehicle surroundings by predicting driver intent
JP4323377B2 (ja) 画像表示装置
CN104204847B (zh) 用于可视化车辆的周围环境的方法和装置
US20150084755A1 (en) Driver assistance system for displaying surroundings of a vehicle
EP2631696B1 (en) Image generator
US20090015675A1 (en) Driving Support System And Vehicle
EP2660104A2 (en) Apparatus and method for displaying a blind spot
US9019347B2 (en) Image generator
JP5516998B2 (ja) 画像生成装置
CN101763640A (zh) 车载多目摄像机环视系统的在线标定处理方法
US11055541B2 (en) Vehicle lane marking and other object detection using side fisheye cameras and three-fold de-warping
US20210004614A1 (en) Surround View System Having an Adapted Projection Surface
US20200238907A1 (en) Display control apparatus
US11562576B2 (en) Dynamic adjustment of augmented reality image
US11580695B2 (en) Method for a sensor-based and memory-based representation of a surroundings, display device and vehicle having the display device
KR20190067578A (ko) 오버랩 촬영 영역을 가지는 이종 카메라를 이용한 충돌 경고 장치 및 방법
JP2005269010A (ja) 画像生成装置、画像生成プログラム、及び画像生成方法
JP2021013072A (ja) 画像処理装置および画像処理方法
JP2012116400A (ja) コーナーポール投影装置
CN103377372A (zh) 一种环视合成图重叠区域划分方法和环视合成图表示方法

Legal Events

Date Code Title Description
AS Assignment

Owner name: HYUNDAI MOTOR JAPAN R&D CENTER INC., JAPAN

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:HIDESHIRO, FUJIE;REEL/FRAME:023353/0370

Effective date: 20090908

Owner name: KIA MOTORS CORPORATION, KOREA, REPUBLIC OF

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:HIDESHIRO, FUJIE;REEL/FRAME:023353/0370

Effective date: 20090908

Owner name: HYUNDAI MOTOR COMPANY, KOREA, REPUBLIC OF

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:HIDESHIRO, FUJIE;REEL/FRAME:023353/0370

Effective date: 20090908

FEPP Fee payment procedure

Free format text: PAYOR NUMBER ASSIGNED (ORIGINAL EVENT CODE: ASPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

STCF Information on status: patent grant

Free format text: PATENTED CASE

FEPP Fee payment procedure

Free format text: PAYOR NUMBER ASSIGNED (ORIGINAL EVENT CODE: ASPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

Free format text: PAYER NUMBER DE-ASSIGNED (ORIGINAL EVENT CODE: RMPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

FPAY Fee payment

Year of fee payment: 4

MAFP Maintenance fee payment

Free format text: PAYMENT OF MAINTENANCE FEE, 8TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1552); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

Year of fee payment: 8