EP3437315A1 - Capteur d'images pour une caméra de surveillance multifonctionnelle destinée à surveiller un conducteur d'un véhicule, dispositif de capteur d'images, caméra de surveillance et procédé pour produire une image infrarouge et une image d'intensité - Google Patents

Capteur d'images pour une caméra de surveillance multifonctionnelle destinée à surveiller un conducteur d'un véhicule, dispositif de capteur d'images, caméra de surveillance et procédé pour produire une image infrarouge et une image d'intensité

Info

Publication number
EP3437315A1
EP3437315A1 EP17711586.2A EP17711586A EP3437315A1 EP 3437315 A1 EP3437315 A1 EP 3437315A1 EP 17711586 A EP17711586 A EP 17711586A EP 3437315 A1 EP3437315 A1 EP 3437315A1
Authority
EP
European Patent Office
Prior art keywords
image
intensity
infrared
pixels
image sensor
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
EP17711586.2A
Other languages
German (de)
English (en)
Inventor
Hoang Trinh
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Robert Bosch GmbH
Original Assignee
Robert Bosch GmbH
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Robert Bosch GmbH filed Critical Robert Bosch GmbH
Publication of EP3437315A1 publication Critical patent/EP3437315A1/fr
Ceased legal-status Critical Current

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/10—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from different wavelengths
    • H04N23/11—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from different wavelengths for generating image signals from visible and infrared light wavelengths
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/56—Cameras or camera modules comprising electronic image sensors; Control thereof provided with illuminating means

Definitions

  • An image sensor for a multi-functional observation camera for observing a driver for a vehicle image sensor device, observation camera, and method for generating an infrared image and an intensity image
  • the approach is based on a device or a method according to the preamble of the independent claims. Subject of the present approach is also a computer program.
  • CFA imagers ie image sensors with a special Bayer matrix, each pixel within one of many macro pixels, which typically have a 2x2 matrix grouping of pixels, is each provided with a color filter z.
  • a red color filter for example, lets only red light pass through, etc.
  • Driver surveillance camera systems typically with single or dual or multi-camera systems, consist of a camera module with the image sensor, an active near-infrared illumination unit, also called the I R module, and a determination device .
  • a camera module with the image sensor
  • an active near-infrared illumination unit also called the I R module
  • a determination device e.g., a camera module with the image sensor
  • the image sensor of the camera monochrome and the camera optics is provided with a special band-pass filter to sensitive only for their own infrared lighting and thus insensitive to stray light, z. As sunlight to be.
  • an image sensor and an image sensor device further a method for generating an infrared image and an intensity image, further an observation camera, the this method is used, and finally a corresponding
  • the advantages that can be achieved with the presented approach are that both an infrared image and an intensity image can be generated by the sensor unit.
  • the provided intensity image can be
  • the image sensor for a multi-functional observation camera for observing a driver of a vehicle is presented.
  • the image sensor has a plurality of macro pixels, wherein a macro pixel has at least one infrared pixel.
  • the image sensor such as a semiconductor-based image sensor, may include one of the plurality of macro-pixels.
  • the macro-pixels can form a sensor surface of the image sensor and be arranged adjacent to one another like a matrix.
  • Each of the macro pixels may comprise a field of pixels, also referred to as pixels.
  • Each pixel may be configured to detect a light intensity.
  • the intensity pixel can also be referred to as a grayscale pixel.
  • An intensity pixel is designed to be one
  • an intensity pixel may be formed to detect at least the light intensity of light in the visible range.
  • the intensity pixels have no color filter.
  • An infrared pixel is designed to detect only infrared light incident on the infrared pixel.
  • the infrared pixels have a filter that allows only infrared light to pass.
  • the presented approach enables the combination of infrared pixels and intensity pixels in an image sensor to generate both an infrared image and an intensity image, also called gray scale image.
  • each of the macro pixels may have more infrared pixels than intensity pixels, for example, three infrared pixels and a single intensity pixel. In this way, a macro pixel can be made up of four equal-sized pixels that can form a square adjacent to one another. According to others
  • each of the macro pixels may have a different number of infrared pixels and / or more than one intensity pixel.
  • a macro pixel may have another suitable shape, such as a honeycomb shape.
  • An image sensor device has the following features:
  • One named image sensor and a determiner configured to determine an infrared image using infrared image signals of the infrared pixels and an intensity image using intensity image signals of the intensity pixels.
  • the determination device of the presented image sensor device can be, for example, an unprocessed raw image, also called CFA image, which is a common image of both the infrared image signals and the
  • the determination device is designed to generate the infrared image by a bilinear interpolation.
  • bilinear interpolation positions associated with the intensity pixels in the infrared image are filled by interpolated pixels.
  • the determination device can be designed to be used for
  • the determining means may be further configured to provide the intensity image as a video image for video telephony.
  • the video image for video telephony can be interesting and useful, in particular for highly automated mobile vehicles.
  • the determination device can also be designed to provide a dazzling signal indicating dazzling of the driver using the intensity image.
  • the glare signal can be determined using an image evaluation method by which, for example, a very high light intensity regions of the intensity image can be evaluated.
  • the glare signal can for example be used to control an adaptive sun visor. A video-based detection of glare on the face is possible because there is no optical filtering of the sunlight in the intensity pixel.
  • An observation camera for observing a driver has one of the presented image sensor devices and a near-infrared illumination unit for providing an infrared light for illuminating the driver.
  • the presented observation camera can be used as a substitute for known
  • Driver observation cameras are used, with the difference that the presented observation camera using only a single image sensor can advantageously generate both the infrared image and the intensity image.
  • a method for generating an infrared image and an intensity image using one of the presented image sensor devices comprises the following steps:
  • Observation camera be executable.
  • a computer program product or computer program with program code which can be stored on a machine-readable carrier or storage medium such as a semiconductor memory, a hard disk memory or an optical memory and for carrying out, implementing and / or controlling the steps of the method for generating an infrared image and an intensity image according to any one of the above
  • Embodiments is used, in particular when the program product or program is executed on a computer or a device.
  • FIG. 1 shows a schematic representation of an observation camera according to an embodiment
  • FIG. 2 is a schematic plan view of an image sensor according to a
  • FIG. 3 is a schematic representation of an infrared image according to an embodiment
  • FIG. 4 is a schematic representation of an intensity image according to an exemplary embodiment
  • FIG. 6 is a flowchart of a method for generating an infrared image and an intensity image according to an exemplary embodiment.
  • the observation camera 100 may be arranged to observe a driver of a vehicle in a vehicle and has an image sensor device 105 and a proximity sensor.
  • the near-infrared illumination unit 110 is configured to provide an infrared light for illuminating the driver
  • the image sensor device 105 has an image sensor 112 and a determination device 115.
  • the image sensor 112 has a plurality of macro pixels 120 with four pixels according to this exemplary embodiment.
  • the image sensor 112 has three macro-pixels 120 whose four equally sized pixels are each arranged relative to one another in such a way that the macro-pixels 120 have a square shape.
  • an image sensor 112 has at least several hundred of macro-pixels 120.
  • each macro-pixel 120 has at least one infrared pixel I R and one intensity pixel I.
  • Each of the infrared pixels IR is configured to have an intensity of incident on the infrared pixel I R
  • Each of the intensity pixels I is configured to be one
  • intensity-depicting intensity image signal 130 To detect light intensity of, for example, light in the visible region and provide an intensity-depicting intensity image signal 130.
  • intensity-depicting intensity image signal 130 To detect light intensity of, for example, light in the visible region and provide an intensity-depicting intensity image signal 130.
  • only one infrared image signal 125 and one intensity image signal 130 are shown for the sake of clarity, however, all other infrared pixels IR shown also generate one each
  • Infrared image signal 135 and all other intensity pixels I shown also each generate an intensity image signal 130.
  • the determining device 115 is configured to use the infrared image signals 125 of the infrared pixels IR, an infrared image 135 and under Using the intensity image signals 130 of the intensity pixels I to generate an intensity image 140 and to provide an interface of the camera 100 or a processing device integrated in the camera 100.
  • the camera 100 is suitable according to an embodiment to previous
  • SW functions or algorithms such as head and eye tracking, so a head and eye tracking, face identification, so a face recognition, Driver
  • Modeling ie a driver modeling (fatigue, distraction detection) and gaze control, so a view control, are given.
  • the advantage here is that the system costs remain low, since no second observation camera is required for the mentioned additional functions.
  • the observation camera 100 thereby becomes multifunctional.
  • Video telephony with a normal gray value image, ie the intensity image 140 advantageously corresponds to the human
  • a video image can only be offered when the vehicle is at a standstill. That the driver moves the face or the head down during video telephony in order to see the video image and thus an unfavorable image perspective on the chin and nostrils of the driver is no longer present, can be easily understood.
  • the video telephony can be made available to the driver in partially or highly automated mobile vehicles, since the driver can look down here to the observation camera 100, so that in this case a video image free of nostrils is possible. This typical look down on a display is also very common in smartphone
  • FIG. 2 shows a schematic plan view of an image sensor 112 according to an exemplary embodiment. It may be the reference to FIG. 1
  • the image sensor 112 shown here has a larger number of macro pixels 120.
  • each of the exemplary 36 macro-pixels 120 has three infrared pixels I R and one intensity pixel I.
  • FIG. 3 shows a schematic representation of an infrared image 135 according to a
  • Embodiment This may be the case with reference to FIG. 1
  • Embodiments are associated with the intensity pixels of FIG. 2
  • the determination device described with reference to FIG. 1 is designed to carry out a bilinear interpolation which is necessary for this purpose. How the bilinear interpolation can be performed is described below with reference to FIG. 5.
  • FIG. 4 shows a schematic representation of an intensity image 140 according to an exemplary embodiment. This may be the intensity image 140 described with reference to FIG. According to this embodiment, the intensity pixels I shown in FIG. 2 are brought together and shown without the infrared pixels. A resolution of the illustrated Intensity image 140 thus has only one quarter of the resolution of the infrared image shown with reference to FIG. 3.
  • FIG. 5 shows a schematic representation of a bilinear interpolation 500, as can be carried out by the determination device described with reference to FIG. 1 for determining the infrared image.
  • the procedure is described here by way of example with reference to an image sensor 502 of the type R3I, but can be applied correspondingly for an image sensor according to the approach described here.
  • the image sensor 502 has a plurality of macro pixels 120 each having three pixels I i
  • This image sensor 502 is configured to provide a plurality of driver assistance functions, such as a
  • Lane detection for z Lane Keeping Support, to enable object recognition and traffic sign recognition.
  • the red pixel R supports the video-based detection of red signs that z. For example, show a speed limit.
  • a raw image 505, or CFA image is decomposed into a gray scale image, or intensity image 510, and a red image 515, respectively.
  • the fourth missing pixel 520 in the macro pixel is supplemented by bilinear interpolation 500.
  • the red image 515 has only a quarter of the resolution of the intensity image 510.
  • the bilinear interpolation 500 is also called debayering and means that a colored raster image is reconstructed from the incomplete color values of an image sensor 502 superimposed with mosaic color filters.
  • the bilinear interpolation 500 four adjacent pixels I of the missing pixel 520, gradient-based, or up to eight adjacent pixels I, gradient + orientation-based, for interpolation, in the x-y direction and diagonal, are used for this purpose.
  • FIG. 6 shows a flow chart of a method 600 for generating an infrared image and an intensity image according to an exemplary embodiment.
  • the infrared image and the intensity image are generated using one of the image sensor devices described with reference to the preceding figures.
  • a step 605 of reading in the infrared image signals of the infrared pixels and the intensity image signals of the intensity pixels are read.
  • a step 610 of generating the infrared image is acquired using the
  • an exemplary embodiment comprises a "and / or" link between a first feature and a second feature, then this is to be read so that the embodiment according to one embodiment, both the first feature and the second feature and according to another embodiment either only first feature or only the second feature.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Studio Devices (AREA)
  • Closed-Circuit Television Systems (AREA)
  • Image Analysis (AREA)

Abstract

L'invention concerne un capteur d'images (112) pour une caméra de surveillance multifonctionnelle (100) destinée à surveiller un conducteur d'un véhicule. Le capteur d'images (112) comporte une pluralité de macro-pixels (120), un macro-pixel (120) comprenant au moins un pixel infrarouge (IR) et un pixel d'intensité (I).
EP17711586.2A 2016-03-29 2017-03-08 Capteur d'images pour une caméra de surveillance multifonctionnelle destinée à surveiller un conducteur d'un véhicule, dispositif de capteur d'images, caméra de surveillance et procédé pour produire une image infrarouge et une image d'intensité Ceased EP3437315A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
DE102016205095.8A DE102016205095A1 (de) 2016-03-29 2016-03-29 Bildsensor für eine multifunktionale Beobachtungskamera zum Beobachten eines Fahrers für ein Fahrzeug, Bildsensoreinrichtung, Beobachtungskamera und Verfahren zum Erzeugen eines Infrarot-Bilds und eines Intensität-Bilds
PCT/EP2017/055400 WO2017167557A1 (fr) 2016-03-29 2017-03-08 Capteur d'images pour une caméra de surveillance multifonctionnelle destinée à surveiller un conducteur d'un véhicule, dispositif de capteur d'images, caméra de surveillance et procédé pour produire une image infrarouge et une image d'intensité

Publications (1)

Publication Number Publication Date
EP3437315A1 true EP3437315A1 (fr) 2019-02-06

Family

ID=58358554

Family Applications (1)

Application Number Title Priority Date Filing Date
EP17711586.2A Ceased EP3437315A1 (fr) 2016-03-29 2017-03-08 Capteur d'images pour une caméra de surveillance multifonctionnelle destinée à surveiller un conducteur d'un véhicule, dispositif de capteur d'images, caméra de surveillance et procédé pour produire une image infrarouge et une image d'intensité

Country Status (4)

Country Link
EP (1) EP3437315A1 (fr)
CN (1) CN108886587B (fr)
DE (1) DE102016205095A1 (fr)
WO (1) WO2017167557A1 (fr)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102019111914A1 (de) * 2019-05-08 2020-11-12 Deutsche Institute Für Textil- Und Faserforschung Denkendorf Verfahren zur Auswertung einer Infrarotsignatur

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE10046309C1 (de) * 2000-09-19 2001-10-31 Siemens Ag Sensoranordnung zur Bilderkennung
DE10220825A1 (de) * 2002-05-08 2003-07-03 Audi Ag Bildaufnahmevorrichtung für Fahrerassistenzsysteme
CN1698381A (zh) * 2003-05-08 2005-11-16 西门子公司 检测物体或人员的方法和装置
JP2007245911A (ja) * 2006-03-15 2007-09-27 Omron Corp 監視装置および方法、記録媒体、並びに、プログラム
JP6308760B2 (ja) * 2012-12-20 2018-04-11 キヤノン株式会社 光電変換装置および光電変換装置を有する撮像装置
CN104224204B (zh) * 2013-12-24 2016-09-07 烟台通用照明有限公司 一种基于红外检测技术的驾驶员疲劳检测系统

Also Published As

Publication number Publication date
CN108886587B (zh) 2022-02-01
CN108886587A (zh) 2018-11-23
DE102016205095A1 (de) 2017-10-05
WO2017167557A1 (fr) 2017-10-05

Similar Documents

Publication Publication Date Title
EP1503580B1 (fr) Dispositif de capteur comportant une pluralité de capteurs de types differents
DE10046309C1 (de) Sensoranordnung zur Bilderkennung
DE112011103083T5 (de) Umgebungsbild-Anzeigesystem und Umgebungsbild-Anzeigeverfahren für ein Fahrzeug
EP1665815A2 (fr) Procede pour creer un modele d'image en trois dimensions destine a des procedes d'imagerie a effet de profondeur spatiale et dispositif pour afficher un modele d'image en trois dimensions
DE102011106072A1 (de) Schattenentfernung in einem durch eine fahrzeugbasierte kamera erfassten bild unter verwendung einer optimierten ausgerichteten linearen achse
DE102015217253A1 (de) Umfelderfassungseinrichtung für ein Fahrzeug und Verfahren zum Erfassen eines Bilds mittels einer Umfelderfassungseinrichtung
DE10203421C1 (de) Automobiles Infrarot-Nachtsichtgerät und automobiles Display
DE102014205519A1 (de) Verfahren und Vorrichtung zum Anpassen einer Anzeige eines autostereoskopischen Displays für ein Fahrzeug
DE102009001122B4 (de) 1 - 15Kameraanordnung und Verfahren zum Ermitteln von Bildsignalen mit Farbwerten
DE102009014437B4 (de) Objekterkennungssystem und -verfahren
DE102007034608A1 (de) Kamera
DE102017011562A1 (de) Verfahren zur Umwandlung eines Rohbilds in ein Farbbild
DE10220825A1 (de) Bildaufnahmevorrichtung für Fahrerassistenzsysteme
EP3437315A1 (fr) Capteur d'images pour une caméra de surveillance multifonctionnelle destinée à surveiller un conducteur d'un véhicule, dispositif de capteur d'images, caméra de surveillance et procédé pour produire une image infrarouge et une image d'intensité
WO2014063701A1 (fr) Agencement de capteur pour capture d'images
EP3073446B1 (fr) Procede de representation d'un environnement d'un vehicule automobile
DE102004033625A1 (de) Verfahren zur Darstellung des insbesonderen vorderen Sichtfeldes aus einem Kraftfahrzeug
DE102015206796A1 (de) Sichtfeldanzeigevorrichtung für ein Fahrzeug und Verfahren zum Darstellen eines Bildes in einem Sichtbereich einer Person
WO2015074915A1 (fr) Ensemble de filtres et procédé de fabrication d'un ensemble de filtres
EP3089105A1 (fr) Procede de generation d'une image de contraste de la nature d'un objet et dispositifs correspondants
WO2015090997A1 (fr) Système de détection optique
DE102019123740B4 (de) Verfahren zum Bereitstellen einer Rück- und/oder Seitenansicht eines Fahrzeugs, virtuelleSpiegelvorrichtung und Fahrzeug
DE102020215419A1 (de) Bildsensor für eine Kamera und Verfahren zum Erzeugen eines Kamerabilds
DE102005041241A1 (de) Verfahren und System zum Betreiben eines Kamerasystems eines Fahrzeugs
EP2819408A2 (fr) Procédé et dispositif destinés à adapter une représentation d'une image sur un écran

Legal Events

Date Code Title Description
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: UNKNOWN

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE

PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20181029

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

DAV Request for validation of the european patent (deleted)
DAX Request for extension of the european patent (deleted)
RAP1 Party data changed (applicant data changed or rights of an application transferred)

Owner name: ROBERT BOSCH GMBH

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

17Q First examination report despatched

Effective date: 20210125

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

REG Reference to a national code

Ref country code: DE

Ref legal event code: R003

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE APPLICATION HAS BEEN REFUSED

18R Application refused

Effective date: 20221230