WO2018130600A1 - Method for operating a camera with a first operating mode and a second operating mode, camera and motor vehicle - Google Patents

Method for operating a camera with a first operating mode and a second operating mode, camera and motor vehicle Download PDF

Info

Publication number
WO2018130600A1
WO2018130600A1 PCT/EP2018/050626 EP2018050626W WO2018130600A1 WO 2018130600 A1 WO2018130600 A1 WO 2018130600A1 EP 2018050626 W EP2018050626 W EP 2018050626W WO 2018130600 A1 WO2018130600 A1 WO 2018130600A1
Authority
WO
WIPO (PCT)
Prior art keywords
operating mode
histogram
image
camera
range
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/EP2018/050626
Other languages
French (fr)
Inventor
Peadar Conneely
Patrick Eoghan Denny
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Connaught Electronics Ltd
Original Assignee
Connaught Electronics Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Connaught Electronics Ltd filed Critical Connaught Electronics Ltd
Publication of WO2018130600A1 publication Critical patent/WO2018130600A1/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/70—Circuitry for compensating brightness variation in the scene
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/70—Circuitry for compensating brightness variation in the scene
    • H04N23/71—Circuitry for evaluating the brightness variation

Definitions

  • the invention relates to a method for operating a camera of a motor vehicle.
  • the camera is operated in a first operating mode.
  • an image of an environmental region of the motor vehicle is captured by the camera.
  • a histogram of the image is determined and a histogram range is determined in the histogram.
  • the invention also relates to a camera formed for executing the method as well as to a motor vehicle with a corresponding camera.
  • US 2014/01 1 1637 A1 shows an image capturing apparatus for capturing an image outside of a motor vehicle. Therein, the camera is operated depending on exposure.
  • the camera is for example operated with different exposure settings for day and for night.
  • the exposure time of the camera is usually increased in the operation in the night.
  • the light sensitivity of the image sensor of the camera is also accordingly adapted.
  • the exposure setting of the camera cannot be arbitrarily adapted.
  • the simultaneous imaging of bright and dark objects in an image is also problematic.
  • bright objects require another exposure setting than dark objects to achieve a detailed imaging of the respective objects.
  • this object is solved by a method as well as by a camera and a motor vehicle having the features according to the respective independent claims.
  • a camera of a motor vehicle is operated.
  • the camera is operated in a first operating mode and an image of an environmental region of the motor vehicle is captured by the camera in the first operating mode.
  • a histogram of the image is determined and a histogram range is determined in the histogram.
  • all of the frequency values of intensity value classes of the histogram in the histogram range are compared to a frequency limit value, and it is changed from the first operating mode to a second operating mode of the camera different from the first operating mode if the compared frequency values in the histogram range are less than the frequency limit value.
  • the second operating mode a further image is captured by the camera.
  • the camera is operated with an identical exposure setting in the first operating mode and in the second operating mode.
  • the respective intensity value class each denotes a bin or a column of the histogram.
  • the intensity value class can for example include one or more intensity values of the image.
  • a height of the respective column of the histogram is then described by the respective frequency value.
  • the camera is operated in the first operating mode such that a bright object is imaged with a finely graduated intensity value interval.
  • the brightness differences between the intensity values are lower in bright objects than it is the case in the second operating mode.
  • low brightness differences of the bright object are recognizable in the image.
  • the bright object is headlights of a further motor vehicle in the environmental region.
  • the second operating mode it is placed importance on clearly representing the environmental region dark compared to the bright object.
  • the darker environmental region is then for example imaged with a finely graduated intensity value interval.
  • dark objects in the environmental region can be more clearly recognized and for example associated with a class of obstacles for the motor vehicle.
  • the change from the first operating mode to the second operating mode is effected depending on the histogram of the image captured in the first operating mode. It is changed if all of the frequency values in the histogram range are less than the frequency limit value. Then, it can be assumed that a bright object is not located in the image. As a result, it is changed to the second operating mode to be able to more clearly, thus more detailed, image the dark environmental region. Instead, it can occur that the bright object is no longer imaged as clearly as in the first operating mode in the second operating mode.
  • the exposure setting of the camera is identical during the first operating mode and during the second operating mode. Thereby, the image and the further image can be imaged with higher quality, thus more detailed and lower in noise than if the exposure setting would be adapted.
  • the exposure setting of the first operating mode is in particular maintained.
  • an exposure time of the camera is in particular preset.
  • light sensitivity of an image sensor of the camera or a gain setting of the image sensor can also be encompassed by the exposure setting.
  • the exposure time could also be increased since in particular less bright areas are imaged.
  • the exposure time can for example also be adapted within a tolerance interval from the first operating mode to the second operating mode.
  • moved objects are also more blurred imaged with longer exposure time, which would result in lower quality of the further image.
  • the motor vehicle for example moves with the camera, exactly the role of moved objects is assigned to many objects - actually static in the earth-fixed coordinate system.
  • Both the first operating mode and the second operating mode are provided in particular as a night operating mode in contrast to a day operating mode of the camera not further described.
  • the histogram range is determined within 20 %, in particular 15 %, of the intensity value range of the histogram with the greatest intensity values, and thus the histogram range is determined within in particular 20 %, especially 15 % of the greatest intensity values of the entire intensity value range of the histogram.
  • the histogram range is preferably determined in the range of the histogram with the brightest intensity values. Thereby, it can for example be determined if a bright object is located in the image based on the histogram range.
  • the histogram range thus extends preferably only within the last fifth of the intensity value range of the histogram.
  • the histogram range also extends only within the last 15 % of the intensity value range. Thereby, the change from the first operating mode to the second operating mode can be even more reliably effected since it is changed only in presence of bright objects falling within the last 20 %, in particular within the last 15 % of the intensity value range.
  • the image is captured or generated in the first operating mode with an, in particular local, first tone mapping method and the further image is captured in the second operating mode with an, in particular local, second tone mapping method different from the first tone mapping method, wherein a dynamic range of the further image is less severely reduced by the second tone mapping method than a dynamic range of the image captured by the first tone mapping method.
  • the first tone mapping method and/or the second tone mapping method are present as a local tone mapping method.
  • the tone mapping method which is also referred to as dynamic compression method, the dynamic range of high-contrast images or HDR images (HDR - high dynamic range), thus of digital images with high brightness range, is compressed.
  • Tone mapping operators used for the respective tone mapping method can for example be divided into operator classes cited in the following. So-called global operators use a function, which assigns a dynamic-compressed value to each HDR value and which is applied to each pixel. In contrast thereto, this function is varied for each pixel according to local adaptation level in local operators. Frequency-based operators use another technology, in which the dynamic range of image regions is reduced according to local frequency.
  • the image is captured in the first operating mode with an, in particular local, first tone mapping method and the first tone mapping method, in particular any tone mapping method, is deactivated in capturing the further image in the second operating mode.
  • the further image is again captured with less noise and sharper than the image analogously to the advantageous description of the second tone mapping method.
  • a plurality of adjacent image sensor elements of an image sensor of the camera is combined to an image sensor element block of the image sensor and the further image is captured depending on the image sensor element block in the second operating mode.
  • the image sensor element block in particular includes at least two image sensor elements.
  • image sensor elements of the image sensor of the camera are combined.
  • image sensor element blocks higher light sensitivity per virtual image sensor element is achieved, wherein the signal to noise ratio improves since the noise is statistically distributed.
  • image resolution is also reduced corresponding to the number of the combined image sensor elements and the image becomes coarser.
  • image sensor elements are for example combined within a line and/or a column. This combination occurs by addition of the brightness values, either in analog way by physical charge addition and charge transport on the image sensor itself towards the local line amplifier and/or digitally by addition of the digitalized values. Analog binning increases the frame rate of the camera since less image sensor elements have to be read out and digitalized.
  • the image sensor elements are output with half of the horizontal frequency and the vertical frequency is maintained. The capture of an image then takes only half as long due to the half line number. Thereby, the exposure time is halved per virtual image sensor element, however, it has already the theoretically four-fold charge amount in advance.
  • the adjacent image sensor elements of a line are analogously combined, but the adjacent lines are for example also digitally combined.
  • a further advantage of the method is reducing the bandwidth upon transmission to the following processing system, in particular if very high-resolution sensors are used, the pixel number of which is not required for the specific application - but in which it depends on high dynamic range and later combining would be unnecessary anyway.
  • the combination can be effected not solely by the chip matrix, but additionally also by a digital signal processor downstream the analog-digital-converter.
  • sensors can for example be externally configured by digital interface to for example adjust the desired image resolution and the pre-filtering.
  • permanent brightness adjustment can also be performed to adapt a binning ratio of a prevailing exposure of the environmental region.
  • a number of the image sensor elements combined to the image sensor element block in the second operating mode is determined based on a total number of the image sensor elements of the image sensor of the camera and/or a resolution of a display for outputting the further image. Based on the total number of the image sensor elements and/or the resolution of the display, the number of the image sensor elements of the image sensor element block is more adequately determined. Thus, the number is thereby determined neither too low nor too excessive for the present situation. With a too low number and a consequent too small image sensor block, the effect of the pixel binning becomes too low, while the pixel number is unnecessarily severely reduced with excessive number and consequent too large image sensor element block.
  • a further histogram range is determined within 20 % to 80 %, in particular from 30 % to 70 %, related to the intensity values of the intensity value range of the histogram and the change to the second operating mode is suppressed if a frequency value of an intensity value class in the further histogram range of the histogram is greater than a further frequency limit value.
  • a temperature of the image sensor of the camera can be inferred.
  • the further histogram range for example exhibits at least one frequency value, which is greater than the further frequency limit value, if the temperature of the image sensor reaches or has exceeded a certain temperature.
  • a disadvantageous consequence of increased temperature of the image sensor is increasingly occurring noise in the image.
  • the change to the second operating mode is suppressed, in particular until the frequency value, in particular all of the frequency values in the further histogram range, is or are again less than or equal to the frequency limit value and thus it can be assumed that the temperature is again at a level, at which the image noise is acceptable for the image quality.
  • the change to the second operating mode can thus be effected depending on temperature without the temperature of the image sensor having to be explicitly measured by a temperature sensor of the camera.
  • the image sensor temperature can also be measured by a temperature sensor and be incorporated in the decision on the change between the first operating mode and the second operating mode.
  • a further histogram of the further image is determined and a further histogram range is determined within 20 % to 80 %, in particular from 30 % to 70 %, related to the intensity values of the intensity value range of the further histogram, wherein it is changed from the second operating mode to the first operating mode if a frequency value of an intensity value class in the further histogram range of the further histogram is greater than a further frequency limit value.
  • the further histogram range is preferably also determined in the further histogram of the further image. Thereby, it can be changed back from the second operating mode to the first operating mode if at least one frequency value within the further histogram range exceeds the further frequency limit value there due to increased image sensor temperature. With increased image sensor temperature, the quality of the further image would no longer be sufficiently enough and it is preferably again changed to the first operating mode.
  • a temperature of an image sensor of the camera is determined and the change to the second operating mode is suppressed if the temperature of the image sensor is greater than a temperature limit value.
  • the advantage of the operation depending on temperature is present here too as already previously described.
  • the change can also be performed depending on the evaluation of the further histogram range.
  • the temperature of the image sensor is determined or measured by a temperature sensor of the camera. The determination of the temperature by means of the temperature sensor is advantageous since the temperature can thereby be precisely determined and thus the operation of the camera depending on temperature in the first operating mode and in the second operating mode is more reliably effected.
  • the environmental region is imaged lower in noise by the operation depending on temperature.
  • a further histogram of the further image is determined and the histogram range is determined in the further histogram, wherein it is changed back from the second operating mode to the first operating mode if a frequency value of an intensity value class of the further histogram in the histogram range of the further histogram is greater than the frequency limit value.
  • a frequency value of an intensity value class of the further histogram in the histogram range of the further histogram is greater than the frequency limit value.
  • an imaging of a bright object in the further image is assumed, and thus the requirements for the change from the first operating mode to the second operating mode do no longer exist, it is preferably changed from the second operating mode back to the first operating mode.
  • the environmental region is further detailed and clearly imaged adequate for the objects disposed in the environmental region.
  • the change from the second operating mode back to the first operating mode is performed depending on a hysteresis.
  • the hysteresis is also referred to as a switching differential.
  • the change is no longer performed depending on the fixed frequency limit value. Instead, the change from the second operating mode back into the first operating mode takes place only if the frequency value is greater than the frequency limit value plus a hysteresis value.
  • the hysteresis value can be, for example, up to 10 % of the frequency limit value or more.
  • the changeover from the first operating mode into the second operating mode can also be carried out as a function of a hysteresis.
  • the change to the second operating mode takes place only when the frequency value is less than the frequency limit minus the hysteresis value.
  • the hysteresis-dependent change is advantageous since this prevents a high-frequency reciprocation between the operating modes if the frequency value changes slightly within short time around the frequency limit value and alternately falls below it by less than for example 10 % and exceeds by less than for example 10 %.
  • the change to the second operating mode is also suppressed as a function of the hysteresis if the frequency value of the intensity value class in the further histogram range of the histogram is greater than the further frequency limit value plus a further hysteresis value.
  • the further hysteresis value can also be, for example, 10% of the further frequency limit value.
  • a light from a motor vehicle lighting of a further motor vehicle present in the environmental region is imaged in the image and the imaged light is uniquely associated with a part of a first light source of the motor vehicle lighting and uniquely associated with a further part of a second light source of the motor vehicle lighting.
  • the further motor vehicle is preferably recognized in that the motor vehicle lighting is recognized as two light sources spaced from each other, namely the first light source and the second light source. Two light sources disposed in a certain distance from each other and appearing extremely similar with respect to their shape and light color, are for example an indication of the headlights of the further motor vehicle. It is advantageous that the further motor vehicle is thereby reliably recognized in the image.
  • light sources imaged in the image can preferably be classified to the effect if they belong to a further motor vehicle or an object external to motor vehicle in the environmental region.
  • the association or the detection of the motor vehicle lighting as two spaced apart light sources is carried out in particular with a method of machine vision.
  • the camera is operated with an identical gamma value in the first operating mode and in the second operating mode.
  • This compensation factor of an imaging system for the perception of the human eye is designated by the gamma value.
  • This gamma correction performed based on the gamma value is in particular a correction function for transferring a physically proportionally increasing quantity to a quantity non-linearly increasing according to the human perception.
  • the gamma value is preferably kept unchanged in the first operating mode and in the second operating mode to be able to provide the further image with higher image quality.
  • black zone clipping and/or restriction of a dynamic range of the further image are deactivated upon capturing the further image in the second operating mode.
  • a part of the intensity value range of the image is combined to a single value, usually a value at the respective end of the intensity value range, thus for example 0 or 255, with an 8-bit intensity value range.
  • Black zone clipping describes the combination of dark intensity values of the intensity value range adjacent to the intensity value of 0.
  • the dynamic range or the dynamic span in particular denotes the quotient of greatest and least brightness value capable of being differentiated from noise and granularity, respectively.
  • the quotient can be indicated as f-stops in logarithm to the base of 2.
  • the dynamic range of a scene in the environmental region often greatly exceeds the dynamic range of the camera.
  • a high-contrast image can for example be captured by the camera with a single capture or be created from an exposure series of images with low dynamic range.
  • the individual captures are combined with each other such that the contrast of the scene is completely captured and absolute brightness values are stored.
  • a tone mapping method or a dynamic compression is in particular applied.
  • the invention also relates to a camera for a motor vehicle, which is formed to perform a method according to the invention.
  • the camera has a motor vehicle fixing element, by which the camera can be fixed to a motor vehicle.
  • the camera is a component of a driver assistance system of the motor vehicle, for example of an ambient vision system (CMS - camera monitoring system), by which an environmental region of the motor vehicle can be at least partially captured.
  • CMS ambient vision system
  • the exterior mirrors of the motor vehicle are preferably replaced by cameras.
  • An image captured by the camera can then for example be used to recognize objects relevant in terms of traffic in the environmental region such as for example a further motor vehicle.
  • the invention also relates to a motor vehicle with a camera according to the invention.
  • implementations and feature combinations are also to be considered as disclosed, which thus do not have all of the features of an originally formulated independent claim. Moreover, implementations and feature combinations are to be considered as disclosed, in particular by the implementations set out above, which extend beyond or deviate from the feature combinations set out in the relations of the claims.
  • FIG. 1 a schematic representation of an embodiment of a motor vehicle according to the invention with a camera
  • Fig. 2 a flow diagram of an embodiment of a method according to the invention for operating the camera with a change from a first operating mode of the camera to a second operating mode of the camera;
  • Fig. 3 a schematic representation of a histogram of an image captured in the first operating mode with a histogram range and a further histogram range;
  • Fig. 4 a schematic representation of the image captured in the first operating
  • Fig. 5 a schematic representation of a further image captured in the second
  • Fig. 6 a further schematic representation of the image captured in the first
  • Fig. 7 a further schematic representation of the further image captured in the
  • Fig. 1 shows an embodiment of a motor vehicle 1 with at least one camera 2.
  • the camera 2 is disposed at a front 3 of the motor vehicle 1 .
  • the arrangement of the camera 2 is variously possible, thus, the camera 2 can for example also be disposed at a rear 4 of the motor vehicle 1 or on a left side 5 of the motor vehicle 1 or on a right side 6 of the motor vehicle 1 .
  • the camera 2 is disposed at the motor vehicle 1 such that an environmental region 7 of the motor vehicle 1 is at least partially captured.
  • the camera 2 has an image sensor 8 with image sensor elements 9.
  • the image sensor 8 is preferably formed as a CCD (charge-coupled device) sensor or as a CMOS
  • the camera 2 is preferably formed as a video camera, which captures an image sequence of frames.
  • the camera 2 includes an evaluation unit 10.
  • the evaluation unit 10 is formed separately from the camera 2 or else integrated in the camera 2.
  • the image sequence is evaluated and/or corrections are performed on the respective frames, for example by applying filters, by the evaluation unit 10.
  • Fig. 2 shows a flow diagram 1 1 of a method for operating the camera 2.
  • the camera 2 is initialized for operation at low brightness, in particular night operation.
  • the night operation differs from day operation of the camera 2 in particular by a different exposure setting of the camera 2.
  • an exposure time is preset by the exposure setting.
  • a low brightness is in particular present if the environmental region 7 is at least partially less illuminated than it is the case with normal solar radiation.
  • the camera 2 is set in a first operating mode 12 by step S1 .
  • the first operating mode 12 an image 13 of at least a part of the environmental region 7 is captured.
  • a histogram 14 - shown in Fig. 3 - is determined from the, in particular entire, image 13.
  • the camera 2 is changed to a second operating mode 15 in a step S2.
  • a further image 16 is then captured.
  • a further histogram 17 is determined from the further image 16. Depending on the further histogram 17, it is changed back from the second operating mode 15 to the first operating mode 12 in a step S3.
  • An exposure setting 18 of the camera 2 is identical in the first operating mode 12 and in the second operating mode 15.
  • an exposure time of the camera 2 is preset by the exposure setting 18.
  • the exposure setting 18 can also include an aperture setting of the camera 2 or a light sensitivity of the image sensor 8 and a signal gain setting of the image sensor 8, respectively.
  • the image 13 is captured with a first tone mapping method 19 and the further image 16 is captured with a second tone mapping method 20 in the second operating mode 15.
  • a dynamic range of the further image 16 is less severely reduced than a dynamic range of the image 13 is reduced in the first tone mapping method 19.
  • a tone mapping method is not performed at all in the second operating mode 15.
  • the further image 16 is generated without tone mapping method.
  • the first tone mapping method 19 is deactivated in the second operating mode 15.
  • a plurality of adjacent image sensor elements 9 of the image sensor 8 is combined to an image sensor element block of the image sensor 8 as a so-called pixel binning in the second operating mode 15.
  • a number of the image sensor elements 9 combined in the second operating mode 15 is determined based on a total number of the image sensor elements 9 and/or a resolution of a display 21 of the motor vehicle 1 according to the embodiment.
  • the display 21 is coupled to the camera 2.
  • the further image 16 is output on the display 21 .
  • Fig. 3 shows the histogram 14.
  • Intensity value classes 22 are recorded on an abscissa A of the histogram 14.
  • a respective intensity value class 22 is for example given by an intensity value 23 of the image 13 - also shown in Fig. 4 to Fig. 7.
  • the respective intensity value class 22 can also include multiple intensity values 23.
  • the intensity value 23 is within an intensity value range 24 of the image 13.
  • the intensity value range 24 for example extends from 0 to 255 with 8 bits of color depth.
  • the image 13 can also have any other color depth, for example 9 bits, 10 bits, 12 bits or 16 bits.
  • the image 13 can also have multiple color channels. Furthermore, for the sake of clarity, only presence of a single color channel or intensity value channel of the image 13 is assumed.
  • the maximally possible amount of gradations or color gradations is usually indicated in bits and thereby designates the color depth of the image 13. These gradations present a scale, on which the actual color information is stored. Thus, the color depth is the mathematical basis of the actual color information.
  • the respective intensity value class 22 each designates a bin or a column of the histogram 14.
  • the intensity value class 22 can for example include one or more intensity values 23 of the image 13 and can be variously preset.
  • the intensity value range 24 preferably includes all of the intensity values 23 of the image 13.
  • frequency values 25 are recorded on an ordinate O of the histogram 14. By the respective frequency values 25 it is described, how often the respective intensity value class 22 occurs in the image 13.
  • a histogram range 26 is determined during the first operating mode 12.
  • the histogram range 26 extends only over 15 % of the intensity value range 24 of the histogram 14 with the greatest intensity values or the intensity value classes with the greatest intensity values according to the embodiment. Thus, the histogram range 26 in particular does not extend over the entire intensity value range 24 of the histogram 14.
  • All of the frequency values 25 within the histogram range 26 are compared to a frequency limit value 27. If the frequency values 25 within the histogram range 26 are less than the frequency limit value 27, it is changed to the second operating mode 15.
  • the histogram range 26 is then also determined.
  • the histogram range 26 is in the same position in the histogram 14 and in the further histogram 17 and extends over the same intensity value range 24 or over the same intensity value classes 22.
  • At least one frequency value, in particular all of the frequency values, of the further histogram 17 is compared to the frequency limit value 27 within the histogram range 26. If at least one frequency value of the further histogram 17 within the histogram range 26 is equal to or greater than the frequency limit value 27, it is changed back to the first operating mode 12.
  • the change between the operating modes 12, 15 takes place in particular depending on a hysteresis.
  • the frequency limit value 27 is supplemented or reduced, in particular, with a hysteresis value, whereby high-frequency oscillation between the first operating mode 12 and the second operating mode 15 is suppressed.
  • the second operating mode 15 is changed back to the first operating mode 12 only if the frequency value 25 is greater than the frequency limit value 27 plus a hysteresis value.
  • the hysteresis value is, for example, 10 % of the frequency limit value 27.
  • the changeover from the first operating mode 12 to the second operating mode 15 is also carried out as a function of a hysteresis.
  • the change to the second operating mode 15 takes place only when the frequency value 25 is smaller than the frequency limit value 27 minus the hysteresis value.
  • a further histogram range 28 of the histogram 14 is determined according to the embodiment.
  • the further histogram range 28 extends from 30 % to 70 % of the intensity value range 24 of the histogram 14 in the embodiment.
  • At least one of the frequency values 25, in particular all of the frequency values 25, within the further histogram range 28 is compared to a further frequency limit value 29. If one of the frequency values 26 within the further histogram range 28 is greater than the further frequency limit value 29, the change from the first operating mode 12 to the second operating mode 15 is suppressed. In this case, the change is thus not performed even if all of the frequency values 25 within the histogram range 26 are less than the frequency limit value.
  • the further histogram range 28 is preferably also determined in the further histogram 17. Then, it is changed back from the second operating mode 15 to the first operating mode 12 if one of the frequency values 25 within the further histogram range 28 of the further histogram 17 is greater than the further frequency limit value 29.
  • a current temperature of the image sensor 8 is determined.
  • the temperature is for example determined by a temperature sensor of the camera 2. If the temperature is greater than a temperature limit value, the change to the second operating mode 15 is suppressed or it is changed back from the second operating mode 15 to the first operating mode 12.
  • Fig. 4 shows the image 13.
  • a light 30 of a motor vehicle lighting 31 of a further motor vehicle 32 is imaged in the image 13.
  • the further motor vehicle 32 is in the environmental region 7 of the motor vehicle 1 at the time of the capture of the image 13.
  • the imaged light 30 can be uniquely associated with a part 33 of a first light source 34 of the motor vehicle lighting 31 and uniquely associated with a further part 35 of a second light source 36 of the motor vehicle lighting 31 .
  • Fig. 5 shows the further image 16.
  • the light 30 of the motor vehicle lighting 31 is imaged.
  • the imaged light 30 cannot be uniquely associated with the first light source 34 or the second light source 36.
  • the light 30 appears as if it is only output by a single light source.
  • Assessment based on the depicted light sources 34, 36 if the further motor vehicle 32 is in the further image 16, is not possible as reliably as in the image 13.
  • Fig. 6 shows a further embodiment of the image 13 captured in the first operating mode
  • a test pattern is captured to be able to assess the image quality of the image 13. Compared to the further image 16, the edges 37 of the image 13 are more blurred or the edges of the higher frequencies fall flatter and the image 13 has higher image noise.
  • Fig. 7 shows a further embodiment of the further image 16 captured in the second operating mode 15. The test pattern from Fig. 6 is again captured. Compared to the image
  • the edges 38 of the further image 16 are sharper or the edges of the higher frequencies fall steeper and the further image 16 has lower image noise. This becomes possible by the omission of the second tone mapping method 20 among other things.
  • a larger part of the intensity value range 24 is used for imaging the darker areas of the environmental region 7 than it would be the case with the first tone mapping method 19 or the second tone mapping method 20.
  • the temperature-dependent operation of the camera 2 in particular results in less noise in the respective image 13, 16.
  • the method it is allowed changing back and forth between an operating mode, in particular the first operating mode 12, for more reliably recognizing oncoming motor vehicles and an operating mode, in particular the second operating mode 15, with higher image quality, in particular sharper edges and less noise. Furthermore, by the heuristic method, in particular by the histogram-dependent change in steps S2 and S3, a point of time for the respective change appropriate to the situation is reliably determined.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Studio Devices (AREA)

Abstract

The invention relates to a method for operating a camera (2) of a motor vehicle (1), in which the camera (2) is operated in a first operating mode (12) and an image (13) of an environmental region (7) of the motor vehicle (1) is captured by the camera (2) in the first operating mode (12), wherein a histogram (14) of the image (13) is determined and a histogram range (26) is determined in the histogram (14), wherein all of the frequency values (25) of intensity value classes (22) of the histogram (14) in the histogram range (26) are compared to a frequency limit value (27) and it is changed (S2) from the first operating mode (12) to a second operating mode (15) of the camera (2) different from the first operating mode (12) if the compared frequency values (25) in the histogram range (26) are less than the frequency limit value (27), wherein a further image (16) is captured by the camera (2) in the second operating mode (15) and the camera (2) is operated with an identical exposure setting (18) in the first operating mode (12) and in the second operating mode (15).

Description

Method for operating a camera with a first operating mode and a second operating mode, camera and motor vehicle
The invention relates to a method for operating a camera of a motor vehicle. The camera is operated in a first operating mode. In the first operating mode, an image of an environmental region of the motor vehicle is captured by the camera. Furthermore, a histogram of the image is determined and a histogram range is determined in the histogram. The invention also relates to a camera formed for executing the method as well as to a motor vehicle with a corresponding camera.
Methods for operating a camera of a motor vehicle are known from the prior art. Thus, US 2014/01 1 1637 A1 for example shows an image capturing apparatus for capturing an image outside of a motor vehicle. Therein, the camera is operated depending on exposure.
In further known methods, the camera is for example operated with different exposure settings for day and for night. Thus, the exposure time of the camera is usually increased in the operation in the night. Usually, the light sensitivity of the image sensor of the camera is also accordingly adapted.
However, it is disadvantageous that the exposure setting of the camera cannot be arbitrarily adapted. The simultaneous imaging of bright and dark objects in an image is also problematic. Usually, bright objects require another exposure setting than dark objects to achieve a detailed imaging of the respective objects.
It is the object of the invention to demonstrate a solution, how a camera of a motor vehicle can be operated such that an environmental region of the motor vehicle can be imaged lower in noise and in more detailed manner.
According to the invention, this object is solved by a method as well as by a camera and a motor vehicle having the features according to the respective independent claims.
Advantageous implementations of the invention are the subject matter of the dependent claims, of the description and of the figures.
In a method according to the invention, a camera of a motor vehicle is operated. The camera is operated in a first operating mode and an image of an environmental region of the motor vehicle is captured by the camera in the first operating mode. A histogram of the image is determined and a histogram range is determined in the histogram. As an essential idea of the invention, it is provided that all of the frequency values of intensity value classes of the histogram in the histogram range are compared to a frequency limit value, and it is changed from the first operating mode to a second operating mode of the camera different from the first operating mode if the compared frequency values in the histogram range are less than the frequency limit value. In the second operating mode, a further image is captured by the camera. The camera is operated with an identical exposure setting in the first operating mode and in the second operating mode.
By the method according to the invention, it becomes possible to operate a camera of a motor vehicle such that a bright object relevant in terms of traffic is more clearly and more detailed imaged, respectively, in particular under poor lighting conditions, thus for example at night, and an environmental region dark compared thereto is more clearly and more detailed imaged, respectively.
The respective intensity value class each denotes a bin or a column of the histogram. The intensity value class can for example include one or more intensity values of the image. A height of the respective column of the histogram is then described by the respective frequency value.
This becomes possible due to the first operating mode and the second operating mode. Thus, the camera is operated in the first operating mode such that a bright object is imaged with a finely graduated intensity value interval. By the more finely graduated intensity value interval, the brightness differences between the intensity values are lower in bright objects than it is the case in the second operating mode. Thus, low brightness differences of the bright object are recognizable in the image. As a result, it can then for example be recognized if the bright object is headlights of a further motor vehicle in the environmental region. In the second operating mode, it is placed importance on clearly representing the environmental region dark compared to the bright object. Hereto, the darker environmental region is then for example imaged with a finely graduated intensity value interval. Thereby, dark objects in the environmental region can be more clearly recognized and for example associated with a class of obstacles for the motor vehicle.
The change from the first operating mode to the second operating mode is effected depending on the histogram of the image captured in the first operating mode. It is changed if all of the frequency values in the histogram range are less than the frequency limit value. Then, it can be assumed that a bright object is not located in the image. As a result, it is changed to the second operating mode to be able to more clearly, thus more detailed, image the dark environmental region. Instead, it can occur that the bright object is no longer imaged as clearly as in the first operating mode in the second operating mode.
The exposure setting of the camera is identical during the first operating mode and during the second operating mode. Thereby, the image and the further image can be imaged with higher quality, thus more detailed and lower in noise than if the exposure setting would be adapted. For the second operating mode, thus, the exposure setting of the first operating mode is in particular maintained. By the exposure setting, an exposure time of the camera is in particular preset. However, light sensitivity of an image sensor of the camera or a gain setting of the image sensor can also be encompassed by the exposure setting.
In the second operating mode, the exposure time could also be increased since in particular less bright areas are imaged. Thus, the exposure time can for example also be adapted within a tolerance interval from the first operating mode to the second operating mode. However, moved objects are also more blurred imaged with longer exposure time, which would result in lower quality of the further image. However, if the motor vehicle for example moves with the camera, exactly the role of moved objects is assigned to many objects - actually static in the earth-fixed coordinate system.
Both the first operating mode and the second operating mode are provided in particular as a night operating mode in contrast to a day operating mode of the camera not further described.
Preferably, it is provided that the histogram range is determined within 20 %, in particular 15 %, of the intensity value range of the histogram with the greatest intensity values, and thus the histogram range is determined within in particular 20 %, especially 15 % of the greatest intensity values of the entire intensity value range of the histogram. Thus, the histogram range is preferably determined in the range of the histogram with the brightest intensity values. Thereby, it can for example be determined if a bright object is located in the image based on the histogram range. The histogram range thus extends preferably only within the last fifth of the intensity value range of the histogram. However, In particular, the histogram range also extends only within the last 15 % of the intensity value range. Thereby, the change from the first operating mode to the second operating mode can be even more reliably effected since it is changed only in presence of bright objects falling within the last 20 %, in particular within the last 15 % of the intensity value range.
Preferably, it is provided that the image is captured or generated in the first operating mode with an, in particular local, first tone mapping method and the further image is captured in the second operating mode with an, in particular local, second tone mapping method different from the first tone mapping method, wherein a dynamic range of the further image is less severely reduced by the second tone mapping method than a dynamic range of the image captured by the first tone mapping method. Preferably, the first tone mapping method and/or the second tone mapping method are present as a local tone mapping method. By the tone mapping method, which is also referred to as dynamic compression method, the dynamic range of high-contrast images or HDR images (HDR - high dynamic range), thus of digital images with high brightness range, is compressed. Thus, the contrast range of a high-contrast image is reduced in the tone mapping to be able to present it without loss on output apparatuses conceived for lower dynamic ranges. Tone mapping operators used for the respective tone mapping method can for example be divided into operator classes cited in the following. So-called global operators use a function, which assigns a dynamic-compressed value to each HDR value and which is applied to each pixel. In contrast thereto, this function is varied for each pixel according to local adaptation level in local operators. Frequency-based operators use another technology, in which the dynamic range of image regions is reduced according to local frequency. Finally, there are also gradient-based operators, which mitigate the brightness gradients of the starting image for each pixel to generate an LDR image (LDR - low dynamic range) or image with low brightness range. In contrast to global operators, local operators can process a higher number of HDR images since they can represent a greater dynamic range without losing details. In local operators, it is assumed that the human brightness perception does not adapt to the entire image, but only to smaller regions. Overall, the further image is therefore captured with less noise and sharper than the image.
Preferably, it is provided that the image is captured in the first operating mode with an, in particular local, first tone mapping method and the first tone mapping method, in particular any tone mapping method, is deactivated in capturing the further image in the second operating mode. Thereby, the further image is again captured with less noise and sharper than the image analogously to the advantageous description of the second tone mapping method. Preferably, it is provided that a plurality of adjacent image sensor elements of an image sensor of the camera is combined to an image sensor element block of the image sensor and the further image is captured depending on the image sensor element block in the second operating mode. The image sensor element block in particular includes at least two image sensor elements. By the pixel binning, pixel information of the further image, in particular in the charge domain of the image sensor, is combined. In particular, the adjacent image sensor elements of the image sensor of the camera are combined. By the formation of image sensor element blocks, higher light sensitivity per virtual image sensor element is achieved, wherein the signal to noise ratio improves since the noise is statistically distributed. Conversely, however, the image resolution is also reduced corresponding to the number of the combined image sensor elements and the image becomes coarser. In pixel binning, image sensor elements are for example combined within a line and/or a column. This combination occurs by addition of the brightness values, either in analog way by physical charge addition and charge transport on the image sensor itself towards the local line amplifier and/or digitally by addition of the digitalized values. Analog binning increases the frame rate of the camera since less image sensor elements have to be read out and digitalized. In imaging of for example 4:1 , the image sensor elements are output with half of the horizontal frequency and the vertical frequency is maintained. The capture of an image then takes only half as long due to the half line number. Thereby, the exposure time is halved per virtual image sensor element, however, it has already the theoretically four-fold charge amount in advance. Alternatively, the adjacent image sensor elements of a line are analogously combined, but the adjacent lines are for example also digitally combined. A further advantage of the method is reducing the bandwidth upon transmission to the following processing system, in particular if very high-resolution sensors are used, the pixel number of which is not required for the specific application - but in which it depends on high dynamic range and later combining would be unnecessary anyway. Additionally, the combination can be effected not solely by the chip matrix, but additionally also by a digital signal processor downstream the analog-digital-converter. Such sensors can for example be externally configured by digital interface to for example adjust the desired image resolution and the pre-filtering. However, permanent brightness adjustment can also be performed to adapt a binning ratio of a prevailing exposure of the environmental region.
Preferably, it is further provided that a number of the image sensor elements combined to the image sensor element block in the second operating mode is determined based on a total number of the image sensor elements of the image sensor of the camera and/or a resolution of a display for outputting the further image. Based on the total number of the image sensor elements and/or the resolution of the display, the number of the image sensor elements of the image sensor element block is more adequately determined. Thus, the number is thereby determined neither too low nor too excessive for the present situation. With a too low number and a consequent too small image sensor block, the effect of the pixel binning becomes too low, while the pixel number is unnecessarily severely reduced with excessive number and consequent too large image sensor element block.
Preferably, it is provided that a further histogram range is determined within 20 % to 80 %, in particular from 30 % to 70 %, related to the intensity values of the intensity value range of the histogram and the change to the second operating mode is suppressed if a frequency value of an intensity value class in the further histogram range of the histogram is greater than a further frequency limit value. Based on the further histogram range disposed in the middle range of the intensity value range of the histogram, a temperature of the image sensor of the camera can be inferred. Thus, the further histogram range for example exhibits at least one frequency value, which is greater than the further frequency limit value, if the temperature of the image sensor reaches or has exceeded a certain temperature. A disadvantageous consequence of increased temperature of the image sensor is increasingly occurring noise in the image. For this case, the change to the second operating mode is suppressed, in particular until the frequency value, in particular all of the frequency values in the further histogram range, is or are again less than or equal to the frequency limit value and thus it can be assumed that the temperature is again at a level, at which the image noise is acceptable for the image quality. Based on the further histogram range, the change to the second operating mode can thus be effected depending on temperature without the temperature of the image sensor having to be explicitly measured by a temperature sensor of the camera. Additionally, however, the image sensor temperature can also be measured by a temperature sensor and be incorporated in the decision on the change between the first operating mode and the second operating mode.
Preferably, it is provided that a further histogram of the further image is determined and a further histogram range is determined within 20 % to 80 %, in particular from 30 % to 70 %, related to the intensity values of the intensity value range of the further histogram, wherein it is changed from the second operating mode to the first operating mode if a frequency value of an intensity value class in the further histogram range of the further histogram is greater than a further frequency limit value. The further histogram range is preferably also determined in the further histogram of the further image. Thereby, it can be changed back from the second operating mode to the first operating mode if at least one frequency value within the further histogram range exceeds the further frequency limit value there due to increased image sensor temperature. With increased image sensor temperature, the quality of the further image would no longer be sufficiently enough and it is preferably again changed to the first operating mode.
In an embodiment, it can be provided that a temperature of an image sensor of the camera is determined and the change to the second operating mode is suppressed if the temperature of the image sensor is greater than a temperature limit value. The advantage of the operation depending on temperature is present here too as already previously described. In addition to determining the temperature, the change can also be performed depending on the evaluation of the further histogram range. Preferably, the temperature of the image sensor is determined or measured by a temperature sensor of the camera. The determination of the temperature by means of the temperature sensor is advantageous since the temperature can thereby be precisely determined and thus the operation of the camera depending on temperature in the first operating mode and in the second operating mode is more reliably effected. Furthermore, the environmental region is imaged lower in noise by the operation depending on temperature.
Preferably, it is provided that a further histogram of the further image is determined and the histogram range is determined in the further histogram, wherein it is changed back from the second operating mode to the first operating mode if a frequency value of an intensity value class of the further histogram in the histogram range of the further histogram is greater than the frequency limit value. In case that in the further image due to exceeding of the frequency limit value by a frequency value within the histogram range of the further image, an imaging of a bright object in the further image is assumed, and thus the requirements for the change from the first operating mode to the second operating mode do no longer exist, it is preferably changed from the second operating mode back to the first operating mode. Thereby, the environmental region is further detailed and clearly imaged adequate for the objects disposed in the environmental region.
In particular, the change from the second operating mode back to the first operating mode is performed depending on a hysteresis. The hysteresis is also referred to as a switching differential. Thus, the change is no longer performed depending on the fixed frequency limit value. Instead, the change from the second operating mode back into the first operating mode takes place only if the frequency value is greater than the frequency limit value plus a hysteresis value. The hysteresis value can be, for example, up to 10 % of the frequency limit value or more. In addition or alternatively, the changeover from the first operating mode into the second operating mode can also be carried out as a function of a hysteresis. The change to the second operating mode takes place only when the frequency value is less than the frequency limit minus the hysteresis value. The hysteresis-dependent change is advantageous since this prevents a high-frequency reciprocation between the operating modes if the frequency value changes slightly within short time around the frequency limit value and alternately falls below it by less than for example 10 % and exceeds by less than for example 10 %.
In particular, the change to the second operating mode is also suppressed as a function of the hysteresis if the frequency value of the intensity value class in the further histogram range of the histogram is greater than the further frequency limit value plus a further hysteresis value. The further hysteresis value can also be, for example, 10% of the further frequency limit value.
Preferably, it is provided that a light from a motor vehicle lighting of a further motor vehicle present in the environmental region is imaged in the image and the imaged light is uniquely associated with a part of a first light source of the motor vehicle lighting and uniquely associated with a further part of a second light source of the motor vehicle lighting. In the first operating mode, the further motor vehicle is preferably recognized in that the motor vehicle lighting is recognized as two light sources spaced from each other, namely the first light source and the second light source. Two light sources disposed in a certain distance from each other and appearing extremely similar with respect to their shape and light color, are for example an indication of the headlights of the further motor vehicle. It is advantageous that the further motor vehicle is thereby reliably recognized in the image. Thus, light sources imaged in the image can preferably be classified to the effect if they belong to a further motor vehicle or an object external to motor vehicle in the environmental region. The association or the detection of the motor vehicle lighting as two spaced apart light sources is carried out in particular with a method of machine vision.
In a further embodiment, it can be provided that the camera is operated with an identical gamma value in the first operating mode and in the second operating mode. A
compensation factor of an imaging system for the perception of the human eye is designated by the gamma value. This gamma correction performed based on the gamma value is in particular a correction function for transferring a physically proportionally increasing quantity to a quantity non-linearly increasing according to the human perception. The gamma value is preferably kept unchanged in the first operating mode and in the second operating mode to be able to provide the further image with higher image quality.
Furthermore, it can be provided that black zone clipping and/or restriction of a dynamic range of the further image are deactivated upon capturing the further image in the second operating mode. By clipping, a part of the intensity value range of the image is combined to a single value, usually a value at the respective end of the intensity value range, thus for example 0 or 255, with an 8-bit intensity value range. Black zone clipping describes the combination of dark intensity values of the intensity value range adjacent to the intensity value of 0. As a result, a larger part of the intensity value range remains for imaging the remaining, brighter brightness values of the environmental region. The dynamic range or the dynamic span in particular denotes the quotient of greatest and least brightness value capable of being differentiated from noise and granularity, respectively. Therein, the quotient can be indicated as f-stops in logarithm to the base of 2. The dynamic range of a scene in the environmental region often greatly exceeds the dynamic range of the camera. A high-contrast image can for example be captured by the camera with a single capture or be created from an exposure series of images with low dynamic range. Herein, the individual captures are combined with each other such that the contrast of the scene is completely captured and absolute brightness values are stored. In order to be able to output a high-contrast image by a medium with lower dynamic range, for example a display, a tone mapping method or a dynamic compression is in particular applied.
The invention also relates to a camera for a motor vehicle, which is formed to perform a method according to the invention.
In particular, the camera has a motor vehicle fixing element, by which the camera can be fixed to a motor vehicle.
Preferably, the camera is a component of a driver assistance system of the motor vehicle, for example of an ambient vision system (CMS - camera monitoring system), by which an environmental region of the motor vehicle can be at least partially captured. In the ambient vision system (CMS), the exterior mirrors of the motor vehicle are preferably replaced by cameras. An image captured by the camera can then for example be used to recognize objects relevant in terms of traffic in the environmental region such as for example a further motor vehicle. Furthermore, the invention also relates to a motor vehicle with a camera according to the invention.
The preferred embodiments presented with respect to the method according to the invention and the advantages thereof correspondingly apply to the camera according to the invention as well as to the motor vehicle according to the invention.
Both the specific values of parameters specified in the documents and the specification of parameters and the ratio of parameters to each other for characterizing specific characteristics of the camera are also to be considered as encompassed by the scope of the invention even within the scope of deviations, for example due to measurement errors, system errors, DIN tolerances etc., such that specifications concerning an equality of values of mentioned parameters are also to be considered as equal within the scope of an indication of "substantially" in this context.
Further features of the invention are apparent from the claims, the figures and the description of figures. The features and feature combinations mentioned above in the description as well as the features and feature combinations mentioned below in the description of figures and/or shown in the figures alone are usable not only in the respectively specified combination, but also in other combinations without departing from the scope of the invention. Thus, implementations are also to be considered as
encompassed and disclosed by the invention, which are not explicitly shown in the figures and explained, but arise from and can be generated by separated feature combinations from the explained implementations. Implementations and feature combinations are also to be considered as disclosed, which thus do not have all of the features of an originally formulated independent claim. Moreover, implementations and feature combinations are to be considered as disclosed, in particular by the implementations set out above, which extend beyond or deviate from the feature combinations set out in the relations of the claims.
Now, the invention is explained in more detail based on a preferred embodiment as well as with reference to the attached drawings.
There show: Fig. 1 a schematic representation of an embodiment of a motor vehicle according to the invention with a camera;
Fig. 2 a flow diagram of an embodiment of a method according to the invention for operating the camera with a change from a first operating mode of the camera to a second operating mode of the camera;
Fig. 3 a schematic representation of a histogram of an image captured in the first operating mode with a histogram range and a further histogram range;
Fig. 4 a schematic representation of the image captured in the first operating
mode with a motor vehicle lighting of a further motor vehicle, which can be uniquely associated with a first light source of the motor vehicle lighting and uniquely associated with a second light source of the motor vehicle lighting;
Fig. 5 a schematic representation of a further image captured in the second
operating mode with a motor vehicle lighting of the further motor vehicle, which cannot be uniquely associated with the first light source and not uniquely associated with the second light source;
Fig. 6 a further schematic representation of the image captured in the first
operating mode with more blurred edges compared to the further image; and
Fig. 7 a further schematic representation of the further image captured in the
second operating mode with sharper edges compared to the image.
In the figures, identical or functionally identical elements are provided with the same reference characters.
Fig. 1 shows an embodiment of a motor vehicle 1 with at least one camera 2. The camera 2 is disposed at a front 3 of the motor vehicle 1 . However, the arrangement of the camera 2 is variously possible, thus, the camera 2 can for example also be disposed at a rear 4 of the motor vehicle 1 or on a left side 5 of the motor vehicle 1 or on a right side 6 of the motor vehicle 1 . According to the embodiment, the camera 2 is disposed at the motor vehicle 1 such that an environmental region 7 of the motor vehicle 1 is at least partially captured.
The camera 2 has an image sensor 8 with image sensor elements 9. The image sensor 8 is preferably formed as a CCD (charge-coupled device) sensor or as a CMOS
(complementary metal-oxide-semiconductor) sensor. Furthermore, the camera 2 is preferably formed as a video camera, which captures an image sequence of frames.
According to the embodiment, the camera 2 includes an evaluation unit 10. Therein, the evaluation unit 10 is formed separately from the camera 2 or else integrated in the camera 2. The image sequence is evaluated and/or corrections are performed on the respective frames, for example by applying filters, by the evaluation unit 10.
Fig. 2 shows a flow diagram 1 1 of a method for operating the camera 2. In a first step S1 , the camera 2 is initialized for operation at low brightness, in particular night operation. The night operation differs from day operation of the camera 2 in particular by a different exposure setting of the camera 2. In particular, an exposure time is preset by the exposure setting. A low brightness is in particular present if the environmental region 7 is at least partially less illuminated than it is the case with normal solar radiation.
According to the embodiment, the camera 2 is set in a first operating mode 12 by step S1 . In the first operating mode 12, an image 13 of at least a part of the environmental region 7 is captured. A histogram 14 - shown in Fig. 3 - is determined from the, in particular entire, image 13. Depending on the histogram 14 of the image 13, the camera 2 is changed to a second operating mode 15 in a step S2.
In the second operating mode 15, a further image 16 is then captured. A further histogram 17 is determined from the further image 16. Depending on the further histogram 17, it is changed back from the second operating mode 15 to the first operating mode 12 in a step S3.
An exposure setting 18 of the camera 2 is identical in the first operating mode 12 and in the second operating mode 15. In particular, an exposure time of the camera 2 is preset by the exposure setting 18. However, the exposure setting 18 can also include an aperture setting of the camera 2 or a light sensitivity of the image sensor 8 and a signal gain setting of the image sensor 8, respectively. In the first operating mode 12, the image 13 is captured with a first tone mapping method 19 and the further image 16 is captured with a second tone mapping method 20 in the second operating mode 15. In the second tone mapping method 20, a dynamic range of the further image 16 is less severely reduced than a dynamic range of the image 13 is reduced in the first tone mapping method 19.
In an alternative embodiment, a tone mapping method is not performed at all in the second operating mode 15. Thus, the further image 16 is generated without tone mapping method. Accordingly, the first tone mapping method 19 is deactivated in the second operating mode 15.
Furthermore, a plurality of adjacent image sensor elements 9 of the image sensor 8 is combined to an image sensor element block of the image sensor 8 as a so-called pixel binning in the second operating mode 15.
A number of the image sensor elements 9 combined in the second operating mode 15 is determined based on a total number of the image sensor elements 9 and/or a resolution of a display 21 of the motor vehicle 1 according to the embodiment. The display 21 is coupled to the camera 2. Preferably, the further image 16 is output on the display 21 .
Fig. 3 shows the histogram 14. Intensity value classes 22 are recorded on an abscissa A of the histogram 14. A respective intensity value class 22 is for example given by an intensity value 23 of the image 13 - also shown in Fig. 4 to Fig. 7. However, the respective intensity value class 22 can also include multiple intensity values 23. The intensity value 23 is within an intensity value range 24 of the image 13. The intensity value range 24 for example extends from 0 to 255 with 8 bits of color depth. However, the image 13 can also have any other color depth, for example 9 bits, 10 bits, 12 bits or 16 bits. The image 13 can also have multiple color channels. Furthermore, for the sake of clarity, only presence of a single color channel or intensity value channel of the image 13 is assumed. The maximally possible amount of gradations or color gradations is usually indicated in bits and thereby designates the color depth of the image 13. These gradations present a scale, on which the actual color information is stored. Thus, the color depth is the mathematical basis of the actual color information.
The respective intensity value class 22 each designates a bin or a column of the histogram 14. The intensity value class 22 can for example include one or more intensity values 23 of the image 13 and can be variously preset. The intensity value range 24 preferably includes all of the intensity values 23 of the image 13.
Furthermore, frequency values 25 are recorded on an ordinate O of the histogram 14. By the respective frequency values 25 it is described, how often the respective intensity value class 22 occurs in the image 13.
In the histogram 14, a histogram range 26 is determined during the first operating mode 12. The histogram range 26 extends only over 15 % of the intensity value range 24 of the histogram 14 with the greatest intensity values or the intensity value classes with the greatest intensity values according to the embodiment. Thus, the histogram range 26 in particular does not extend over the entire intensity value range 24 of the histogram 14.
All of the frequency values 25 within the histogram range 26 are compared to a frequency limit value 27. If the frequency values 25 within the histogram range 26 are less than the frequency limit value 27, it is changed to the second operating mode 15.
In the second operating mode 15, in the further histogram 17 of the further image 16, the histogram range 26 is then also determined. The histogram range 26 is in the same position in the histogram 14 and in the further histogram 17 and extends over the same intensity value range 24 or over the same intensity value classes 22. At least one frequency value, in particular all of the frequency values, of the further histogram 17 is compared to the frequency limit value 27 within the histogram range 26. If at least one frequency value of the further histogram 17 within the histogram range 26 is equal to or greater than the frequency limit value 27, it is changed back to the first operating mode 12.
The change between the operating modes 12, 15 takes place in particular depending on a hysteresis. Thus, the frequency limit value 27 is supplemented or reduced, in particular, with a hysteresis value, whereby high-frequency oscillation between the first operating mode 12 and the second operating mode 15 is suppressed. Thus, for example, the second operating mode 15 is changed back to the first operating mode 12 only if the frequency value 25 is greater than the frequency limit value 27 plus a hysteresis value. The hysteresis value is, for example, 10 % of the frequency limit value 27. In addition or alternatively, the changeover from the first operating mode 12 to the second operating mode 15 is also carried out as a function of a hysteresis. The change to the second operating mode 15 takes place only when the frequency value 25 is smaller than the frequency limit value 27 minus the hysteresis value.
Additionally, a further histogram range 28 of the histogram 14 is determined according to the embodiment. The further histogram range 28 extends from 30 % to 70 % of the intensity value range 24 of the histogram 14 in the embodiment. At least one of the frequency values 25, in particular all of the frequency values 25, within the further histogram range 28 is compared to a further frequency limit value 29. If one of the frequency values 26 within the further histogram range 28 is greater than the further frequency limit value 29, the change from the first operating mode 12 to the second operating mode 15 is suppressed. In this case, the change is thus not performed even if all of the frequency values 25 within the histogram range 26 are less than the frequency limit value.
Furthermore, the further histogram range 28 is preferably also determined in the further histogram 17. Then, it is changed back from the second operating mode 15 to the first operating mode 12 if one of the frequency values 25 within the further histogram range 28 of the further histogram 17 is greater than the further frequency limit value 29.
Additionally or alternatively to determining the further histogram range 28, a current temperature of the image sensor 8 is determined. The temperature is for example determined by a temperature sensor of the camera 2. If the temperature is greater than a temperature limit value, the change to the second operating mode 15 is suppressed or it is changed back from the second operating mode 15 to the first operating mode 12.
Fig. 4 shows the image 13. A light 30 of a motor vehicle lighting 31 of a further motor vehicle 32 is imaged in the image 13. The further motor vehicle 32 is in the environmental region 7 of the motor vehicle 1 at the time of the capture of the image 13. Based on the image 13, the imaged light 30 can be uniquely associated with a part 33 of a first light source 34 of the motor vehicle lighting 31 and uniquely associated with a further part 35 of a second light source 36 of the motor vehicle lighting 31 .
Fig. 5 shows the further image 16. In the further image 16, the light 30 of the motor vehicle lighting 31 is imaged. Based on the further image 16, the imaged light 30 cannot be uniquely associated with the first light source 34 or the second light source 36. The light 30 appears as if it is only output by a single light source. Assessment based on the depicted light sources 34, 36 if the further motor vehicle 32 is in the further image 16, is not possible as reliably as in the image 13.
Fig. 6 shows a further embodiment of the image 13 captured in the first operating mode
12. A test pattern is captured to be able to assess the image quality of the image 13. Compared to the further image 16, the edges 37 of the image 13 are more blurred or the edges of the higher frequencies fall flatter and the image 13 has higher image noise.
Fig. 7 shows a further embodiment of the further image 16 captured in the second operating mode 15. The test pattern from Fig. 6 is again captured. Compared to the image
13, the edges 38 of the further image 16 are sharper or the edges of the higher frequencies fall steeper and the further image 16 has lower image noise. This becomes possible by the omission of the second tone mapping method 20 among other things. Now, a larger part of the intensity value range 24 is used for imaging the darker areas of the environmental region 7 than it would be the case with the first tone mapping method 19 or the second tone mapping method 20. In addition, the temperature-dependent operation of the camera 2 in particular results in less noise in the respective image 13, 16.
By the method, it is allowed changing back and forth between an operating mode, in particular the first operating mode 12, for more reliably recognizing oncoming motor vehicles and an operating mode, in particular the second operating mode 15, with higher image quality, in particular sharper edges and less noise. Furthermore, by the heuristic method, in particular by the histogram-dependent change in steps S2 and S3, a point of time for the respective change appropriate to the situation is reliably determined.

Claims

Claims
1 . Method for operating a camera (2) of a motor vehicle (1 ), in which the camera (2) is operated in a first operating mode (12) and an image (13) of an environmental region (7) of the motor vehicle (1 ) is captured by the camera (2) in the first operating mode (12), wherein a histogram (14) of the image (13) is determined and a histogram range (26) is determined in the histogram (14),
characterized in that
all of the frequency values (25) of intensity value classes (22) of the histogram (14) in the histogram range (26) are compared to a frequency limit value (27) and it is changed (S2) from the first operating mode (12) to a second operating mode (15) of the camera (2) different from the first operating mode (12) if the compared frequency values (25) in the histogram range (26) are less than the frequency limit value (27), wherein a further image (16) is captured by the camera (2) in the second operating mode (15) and the camera (2) is operated with an identical exposure setting (18) in the first operating mode (12) and in the second operating mode (15).
2. Method according to claim 1 ,
characterized in that
the histogram range (26) is determined within 20 %, in particular 15 %, of the intensity value range (24) of the histogram (14) with the greatest intensity values (23).
3. Method according to claim 1 or 2,
characterized in that
the image (13) is captured in the first operating mode (12) with an, in particular local, first tone mapping method (19) and the further image (16) is captured in the second operating mode (15) with a second tone mapping method (20) different from the first tone mapping method (19), wherein a dynamic range of the further image (16) is less severely reduced by the second tone mapping method (20) than a dynamic range of the image (13) captured with the first tone mapping method (19).
4. Method according to claim 1 or 2,
characterized in that
the image (13) is captured in the first operating mode (12) with an, in particular local, first tone mapping method (19), and upon capturing the further image (16) in the second operating mode (15), the first tone mapping method (19), in particular any tone mapping method (19, 20), is deactivated.
5. Method according to any one of the preceding claims,
characterized in that
in the second operating mode (15), a plurality of adjacent image sensor elements (9) of an image sensor (8) of the camera (2) is combined to an image sensor element block of the image sensor (8) and the further image (16) is captured depending on the image sensor element block.
6. Method according to claim 5,
characterized in that
a number of the image sensor elements (9) combined to the image sensor element block in the second operating mode (15) is determined based on a total number of the image sensor elements (9) of the image sensor (8) of the camera (2) and/or a resolution of a display (21 ) for outputting the further image (16).
7. Method according to any one of the preceding claims,
characterized in that
a further histogram range (28) is determined within 20 % to 80 %, in particular from 30 % to 70 %, of the intensity value range (24) of the histogram (14) and the change to the second operating mode (15) is suppressed if a frequency value (25) of an intensity value class (22) in the further histogram range (28) of the histogram (14) is greater than a further frequency limit value (29).
8. Method according to any one of the preceding claims,
characterized in that
a further histogram (17) of the further image (16) is determined and a further histogram range (28) is determined within 20 % to 80 %, in particular from 30 % to 70 %, of the intensity value range (24) of the further histogram (17), wherein it is changed (S3) from the second operating mode (15) to the first operating mode (12), if a frequency value (25) of an intensity value class (22) in the further histogram range (28) of the further histogram (17) is greater than a further frequency limit value (29).
9. Method according to any one of the preceding claims,
characterized in that
a temperature of an image sensor (8) of the camera (2) is determined and the change to the second operating mode (15) is suppressed if the temperature of the image sensor (8) is greater than a temperature limit value.
10. Method according to any one of the preceding claims,
characterized in that
a further histogram (17) of the further image (16) is determined and the histogram range (26) is determined in the further histogram (17), wherein it is changed back from the second operating mode (15) to the first operating mode (12) if a frequency value (25) of an intensity value class (22) of the further histogram (17) in the histogram range (26) of the further histogram (17) is greater than the frequency limit value (27).
1 1 . Method according to any one of the preceding claims,
characterized in that
a light (30) of a motor vehicle lighting (31 ) of a further motor vehicle (32) present in the environmental region (7) is imaged in the image (13) and the imaged light (30) is uniquely associated with a part (33) of a first light source (34) of the motor vehicle lighting (31 ) and uniquely associated with a further part (35) of a second light source (36) of the motor vehicle lighting (31 ).
12. Method according to any one of the preceding claims,
characterized in that
the camera (2) is operated with an identical gamma value in the first operating mode (12) and in the second operating mode (15).
13. Method according to any one of the preceding claims, characterized in that
in capturing the further image (16) in the second operating mode (15), black zone clipping and/or restriction of a dynamic range of the further image (16) are deactivated.
14. Camera (2) for a motor vehicle (1 ), which is formed to execute a method according to any one of the preceding claims.
15. Motor vehicle (1 ) with a camera (2) according to claim 14.
PCT/EP2018/050626 2017-01-12 2018-01-11 Method for operating a camera with a first operating mode and a second operating mode, camera and motor vehicle Ceased WO2018130600A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
DE102017100531.5A DE102017100531B4 (en) 2017-01-12 2017-01-12 Method for operating a camera with a first operating mode and a second operating mode, camera and motor vehicle
DE102017100531.5 2017-01-12

Publications (1)

Publication Number Publication Date
WO2018130600A1 true WO2018130600A1 (en) 2018-07-19

Family

ID=60943039

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP2018/050626 Ceased WO2018130600A1 (en) 2017-01-12 2018-01-11 Method for operating a camera with a first operating mode and a second operating mode, camera and motor vehicle

Country Status (2)

Country Link
DE (1) DE102017100531B4 (en)
WO (1) WO2018130600A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US12288317B2 (en) 2021-07-07 2025-04-29 Samsung Electronics Co., Ltd. Method and system for enhancing image quality by multi-frame blending

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110292246A1 (en) * 2010-05-25 2011-12-01 Apple Inc. Automatic Tone Mapping Curve Generation Based on Dynamically Stretched Image Histogram Distribution
US20130128044A1 (en) * 2011-11-17 2013-05-23 GM Global Technology Operations LLC Vision-based scene detection
US20140111637A1 (en) 2012-10-22 2014-04-24 GM Global Technology Operations LLC Dynamic Rearview Mirror Adaptive Dimming Overlay Through Scene Brightness Estimation
US20150350513A1 (en) * 2014-05-30 2015-12-03 Apple Inc. Constant Bracket High Dynamic Range (cHDR) Operations

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110292246A1 (en) * 2010-05-25 2011-12-01 Apple Inc. Automatic Tone Mapping Curve Generation Based on Dynamically Stretched Image Histogram Distribution
US20130128044A1 (en) * 2011-11-17 2013-05-23 GM Global Technology Operations LLC Vision-based scene detection
US20140111637A1 (en) 2012-10-22 2014-04-24 GM Global Technology Operations LLC Dynamic Rearview Mirror Adaptive Dimming Overlay Through Scene Brightness Estimation
US20150350513A1 (en) * 2014-05-30 2015-12-03 Apple Inc. Constant Bracket High Dynamic Range (cHDR) Operations

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US12288317B2 (en) 2021-07-07 2025-04-29 Samsung Electronics Co., Ltd. Method and system for enhancing image quality by multi-frame blending

Also Published As

Publication number Publication date
DE102017100531B4 (en) 2024-12-24
DE102017100531A1 (en) 2018-07-12

Similar Documents

Publication Publication Date Title
KR101367637B1 (en) Monitoring device
EP2833618B1 (en) Method for activating and deactivating an image correction function, camera system and motor vehicle
JP6564421B2 (en) Image processing apparatus, imaging apparatus, image processing method, and program
US9307207B2 (en) Glaring reduction for dynamic rearview mirror
JP4706466B2 (en) Imaging device
US10091422B2 (en) Image processing device and recording medium
KR101713684B1 (en) Monitoring camera of complex filtering type robust to change of visibility state and image monitoring system having the same
JP4985394B2 (en) Image processing apparatus and method, program, and recording medium
DE102013115000A1 (en) Method for generating glare-reduced image from images captured by camera device of subject vehicle, involves generating glare-reduced image based on modified identified glaring region upon resulting high dynamic range image
JP4952499B2 (en) Image processing device
US20080267522A1 (en) Image processor, image processing method and computer readable medium for image processing program
WO2014181743A1 (en) Image processing device and image processing method
EP2843937B1 (en) Method for adapting a gamma curve of a camera system of a motor vehicle, camera system and motor vehicle
JP2013223152A (en) Image pickup device
EP4422197B1 (en) Method for generating a high dynamic range image
Deegan LED flicker: Root cause, impact and measurement for automotive imaging applications
KR101786553B1 (en) Monitoring camera having auto-focusing function based on complex filtering robust to change of visibility state and image monitoring system having the same
US20180370455A1 (en) Convertible roof opening detection for mirror camera
KR101822344B1 (en) Motor vehicle camera device with histogram spreading
US20110149130A1 (en) Photographing Apparatus and Photographing Method
JP2018074191A (en) In-vehicle image display system, in-vehicle image display method, and program
JP6944350B2 (en) Captured image display system, electronic mirror system and captured image display method
KR20150118003A (en) Image processing apparatus and image processing method
KR102282464B1 (en) Image processing apparatus and image processing method
JP5520863B2 (en) Image signal processing device

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18700167

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18700167

Country of ref document: EP

Kind code of ref document: A1