Classifications

• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V20/00—Scenes; Scene-specific elements
  • G06V20/50—Context or environment of the image
  • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N7/00—Television systems
  • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
  • H04N7/181—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T3/00—Geometric image transformations in the plane of the image
  • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
  • G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T5/00—Image enhancement or restoration
  • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T5/00—Image enhancement or restoration
  • G06T5/90—Dynamic range modification of images or parts thereof
  • G06T5/92—Dynamic range modification of images or parts thereof based on global image properties
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
  • H04N23/70—Circuitry for compensating brightness variation in the scene
  • H04N23/741—Circuitry for compensating brightness variation in the scene by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/20—Special algorithmic details
  • G06T2207/20172—Image enhancement details
  • G06T2207/20208—High dynamic range [HDR] image processing
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/20—Special algorithmic details
  • G06T2207/20212—Image combination
  • G06T2207/20221—Image fusion; Image merging

Definitions

• the invention relates to a method and a device for generating an image of a vehicle environment.
• Driver assistance systems that support the driver in performing driving maneuvers.
• Some of these driver assistance systems contain camera surround view systems that allow the driver of the vehicle to see the surroundings of the vehicle.
• Such camera surround view systems include several vehicle cameras which deliver real images of the vehicle environment, which are combined in particular by a data processing unit of the camera surround view system to form an image of the surroundings of the vehicle. The image of the vehicle surroundings is then advantageously displayed to the driver on a display unit.
• Adjacent cameras of surround view systems usually have overlapping fields of view. Since more information is available in these areas than is required, the images in these areas are usually mixed by alpha blending. This can lead to strong transitions in the mixing areas if the brightness in the adjacent images varies too much.
• one known approach is to equalize the brightness in the images before mixing. This is done in particular through image harmonization. Image harmonization has some disadvantages, however. For example, it is computationally intensive; the sampling must be close to perfect; the brightness cannot be easily adjusted in some color formats to adjust; and the sample pattern must be random enough not to match a pattern in the images. Overall, the results are heavily dependent on the current situation.
• the object of the invention is to provide a method and a device with which the existing problems based on the differences in brightness between neighboring cameras can be eliminated.
• Embodiments are the subject of the subclaims.
• a device is the subject of the independent claim.
• the present invention relates to a method for generating an image of a vehicle environment, comprising the steps:
• the method according to the invention ensures, in particular, that the very complex and relatively error-prone image harmonization can be dispensed with, since in particular the differences in brightness in the individual images are eliminated.
• the multiple vehicle cameras each capture at least a partial area of the vehicle environment.
• the vehicle cameras or the viewing areas of the vehicle cameras point in particular in different directions, with adjacent vehicle cameras normally having viewing areas that overlap at least in certain areas.
• HDR images are understood to mean in particular images with a high dynamic range. These are also known to the person skilled in the art as high dynamic range images. Such images make it possible for large differences in brightness to be reproduced in great detail in the camera's field of view.
• a single, entire HDR image is made from the multiple, individual HDR images.
• the joining of the individual images to form the overall image works in a manner known to the person skilled in the art, preferably by stitching. This also applies in particular to the areas of overlap.
• LDR images are understood to mean in particular images with a low dynamic range. These are also known to the person skilled in the art as low dynamic range images.
• the LDR image is obtained in particular by compressing the dynamic range of the HDR image.
• the individual HDR images are transmitted to a control unit.
• the control unit preferably creates the entire HDR image from the multiple, individual HDR images. In particular finds that Merging of the individual HDR images into a single, complete HDR image takes place in the control unit.
• the calculation and / or generation of the individual HDR images takes place in the individual vehicle cameras.
• the calculation of the individual HDR images takes place outside the vehicle cameras.
• the raw data of the images or the image data can be sent to a device that can be located, for example, in the vehicle, but also outside the vehicle, where the calculation then takes place.
• the LDR image is calculated from the HDR image using tone mapping.
• the calculation is not limited to any specific tone mapping process.
• the use of global operators is basically conceivable.
• a function is used here which assigns a dynamically compressed value to each HDR value and which is then applied to each pixel.
• the pixels of the HDR image or the output image are processed independently of one another.
• the HDR image is linearly compressed; a factor is used for all pixels.
• the use of local or frequency-based operators is also possible.
• image regions with high contrast are compressed more strongly and image regions with low contrast less strongly.
• the neighborhood, in particular neighboring pixels is preferably included.
• each vehicle camera records several exposures per image.
• the individual HDR images are preferably reconstructed from an exposure series, advantageously from an exposure series of images with a low dynamic range, in particular low dynamic ranks.
• exposure series is understood to mean, in particular, a graduated series of exposures.
• the same motif is preferably imaged, in particular several times, with different exposure settings.
• Each image region is preferably correctly exposed in at least one of the individual images.
• the change in exposure can take place over the duration of the exposure time.
• the change in exposure can also be controlled via the f-number or by changing the sensitivity setting.
• Exposure series are advantageously recorded to calculate the individual HDR images.
• the entire LDR image is displayed on a display unit.
• the display unit can be a display, a display and / or a head-up display.
• the display unit can be part of the vehicle. It is also possible, however, for the display unit to be part of a mobile terminal.
• cameras of a surround view system supply the image data of the vehicle environment. These are in particular four cameras, which are ideally arranged on different sides of the vehicle.
• the cameras are preferably fisheye cameras.
• the invention relates to a device for generating an image of a vehicle environment, comprising:
• Vehicle cameras which are attached in particular to a vehicle body of a vehicle and are designed in such a way that they generate HDR images or can calculate HDR images, and
• control unit that is designed to control the multiple Combine individual HDR images into a complete HDR image and calculate an LDR image from this.
• the device according to the invention is preferably suitable for carrying out a method according to the invention.
• the device comprises a display unit which outputs the LDR image calculated by the control device.
• the vehicle cameras are advantageously designed to be identical.
• identical construction is to be understood in particular to the effect that the vehicle cameras have the same sensor and / or the same optics. It is advantageous if all vehicle cameras use the same values for exposure time (s), sensitivity (s), aperture, AD converter and / or ISP.
• the vehicle cameras are preferably designed as fisheye cameras.
• the vehicle cameras are arranged on different sides of the vehicle. It is advantageous if exactly four cameras are used, in particular a vehicle camera being arranged on a front side, a vehicle camera on a rear side, a vehicle camera on the right side and a vehicle camera on a left side of the vehicle body.
• the vehicle cameras are in particular cameras of a surround view system.
• FIG. 1 a schematic representation of a flow chart of a method according to the invention for generating an image of a vehicle environment in one embodiment
• Fig. 2 a schematic representation of an inventive
• FIG. 1 shows a schematic representation of a flow chart of a method according to the invention for generating an image of a vehicle environment in one embodiment.
• a first step S1 several vehicle cameras 12, 14,
• the cameras 12, 14, 16, 18 are arranged in particular on a vehicle body of a vehicle, the vehicle cameras 12, 14, 16, 18 or the viewing areas of the vehicle cameras 12, 14, 16, 18 preferably pointing in different directions. Adjacent vehicle cameras 12, 14, 16, 18 in particular partially have a
• a second step S2 individual HDR images are calculated and / or generated from image data or images from vehicle cameras 12, 14, 16, 18.
• the individual HDR images are preferably calculated in each of the individual vehicle cameras 12, 14, 16, 18.
• the individual HDR images are advantageously obtained in that they are reconstructed from an exposure series, preferably from an exposure series of images with a low dynamic range.
• each vehicle camera 12, 14, 16, 18 takes in particular several per image
• a third step S3 the multiple, individual HDR images are combined to form an overall HDR image.
• the individual HDR images can be combined in a control device 20.
• an image with a low dynamic range is generated from the entire HDR image. calculated.
• the LDR image is preferably calculated from the HDR image using tone mapping.
• the entire LDR image can then be displayed on a display unit 22.
• FIG. 2 shows a schematic representation of a device 10 according to the invention for generating an image of a vehicle environment in one embodiment.
• the device comprises several vehicle cameras 12, 14, 16, 18, which are designed such that they can generate HDR images or that HDR images can be calculated therefrom.
• the vehicle cameras 12, 14, 16, 18 are preferably attached to a vehicle body of a vehicle.
• the vehicle cameras 12, 14, 16, 18 are arranged on different sides of the vehicle.
• the vehicle cameras 12, 14, 16, 18 are ideally designed to be identical. In particular, they have the same sensor and / or the same optics.
• the vehicle cameras 12, 14, 16, 18 can be designed as fisheye cameras. It is advantageous if the vehicle cameras 12, 14, 16, 18 are cameras of a surround view system.
• the device 10 according to the invention further comprises a control device 20.
• the control device 20 is designed to combine the multiple, individual HDR images to form an overall HDR image and then to calculate an LDR image therefrom.
• the device 10 according to the invention can also include a display unit 22 which outputs or displays the LDR image calculated by the control device 20.

Landscapes

• Engineering & Computer Science (AREA)
• Physics & Mathematics (AREA)
• General Physics & Mathematics (AREA)
• Theoretical Computer Science (AREA)
• Multimedia (AREA)
• Signal Processing (AREA)
• Closed-Circuit Television Systems (AREA)
• Studio Devices (AREA)
• Automation & Control Theory (AREA)
• Image Processing (AREA)
• Human Computer Interaction (AREA)
• Transportation (AREA)
• Mechanical Engineering (AREA)
• Image Analysis (AREA)

Applications Claiming Priority (2)

Publications (1)

Family

ID=70779404

Family Applications (1)

Country Status (6)

Family Cites Families (16)

• 2019
  • 2019-05-08 DE DE102019206581.3A patent/DE102019206581A1/de active Pending
• 2020
  • 2020-05-05 CN CN202080033410.XA patent/CN113812144A/zh active Pending
  • 2020-05-05 WO PCT/DE2020/200030 patent/WO2020224730A1/de unknown
  • 2020-05-05 JP JP2021561823A patent/JP2022529657A/ja active Pending
  • 2020-05-05 EP EP20727143.8A patent/EP3967030A1/de active Pending
  • 2020-05-05 US US17/595,073 patent/US20220215668A1/en active Pending

Also Published As

Similar Documents

Legal Events