WO2020189182A1 - Dispositif de surveillance de condition pour conducteur - Google Patents

Dispositif de surveillance de condition pour conducteur Download PDF

Info

Publication number
WO2020189182A1
WO2020189182A1 PCT/JP2020/007141 JP2020007141W WO2020189182A1 WO 2020189182 A1 WO2020189182 A1 WO 2020189182A1 JP 2020007141 W JP2020007141 W JP 2020007141W WO 2020189182 A1 WO2020189182 A1 WO 2020189182A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
crop
driver
area
face
Prior art date
Application number
PCT/JP2020/007141
Other languages
English (en)
Japanese (ja)
Inventor
智也 山下
Original Assignee
株式会社デンソー
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 株式会社デンソー filed Critical 株式会社デンソー
Publication of WO2020189182A1 publication Critical patent/WO2020189182A1/fr

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
    • B60K35/00—Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R11/00—Arrangements for holding or mounting articles, not otherwise provided for
    • B60R11/04—Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Definitions

  • This disclosure relates to a driver's condition monitoring device.
  • the position of the driver's face moves according to the posture of the driver, it is necessary to set the shooting range by the camera to be relatively large so as to include the moving range of the driver's face. Therefore, the number of pixels in one frame of the imaged image output from the camera is increased, which increases the processing load of the condition monitoring device.
  • a video camera or the like provided with a subject tracking function is provided.
  • This subject tracking function cuts out an image to be photographed from an image and follows it. If this subject tracking function is used in a camera, it is sufficient to process an image having a small number of pixels obtained by cutting out the driver's face from the imaged image, so that the processing of the condition monitoring device is compared with the case of processing the entire imaged image. The load can be reduced.
  • An object of the present disclosure is to provide a driver's condition monitoring device capable of correctly identifying a crop area corresponding to a crop image input from an image detection unit.
  • the image detection unit outputs a crop image cut out from the imaged image based on the set crop area and crop information indicating the crop area.
  • the condition monitoring unit sets a crop area in the image detection unit so that the center position of the driver's face becomes the center position of the crop image based on the crop image and the crop information input from the image detection unit, and the crop image. Monitor the driver's condition based on.
  • FIG. 1 is a block diagram schematically showing an overall configuration according to an embodiment.
  • FIG. 2 is a diagram schematically showing a driver, a DSM, and an image processing ECU.
  • FIG. 3 is a diagram showing register values.
  • FIG. 4 is a diagram showing a crop area.
  • FIG. 5 is a diagram showing an embedded register value in the first row of the crop area.
  • FIG. 6 is a diagram showing a crop area.
  • FIG. 7 is a diagram showing the movement of the crop area.
  • FIG. 8 is a diagram showing an image resized from a crop image.
  • FIG. 9 is a diagram showing a crop area in a state where the center position of the driver's face is greatly moved.
  • FIG. 10 is a flowchart showing the operation of the image processing ECU.
  • a DSM1 (corresponding to an image detection unit) is installed in a portion of the vehicle in front of the driver.
  • the DSM 1 is connected to the image processing ECU 3 (corresponding to the condition monitoring unit) via, for example, an in-vehicle network 2 using CAN (controller area network, registered trademark) as a communication protocol, and outputs the captured image to the image processing ECU 3. ..
  • CAN controller area network, registered trademark
  • the image processing ECU 3 detects the driver's line-of-sight direction based on the image input from the DSM 1, and calls attention to the driver when it is determined that there is a problem in driving.
  • An image processing device is composed of the DSM 1 and the image processing ECU 3.
  • the DSM1 is mainly composed of the image detection IC4 shown in FIG.
  • the image detection IC 4 is composed of a two-dimensional image sensor 5 and a signal processing circuit 6.
  • An object to be imaged is imaged on the image sensor 5 by an optical system (not shown), and a frame showing the imaged image is output at a frame rate of, for example, 30 fps.
  • the pixel of the image sensor 5 is, for example, 1600 pixels ⁇ 1300 pixels, but the pixel is not limited to this.
  • the driver's face is set to form an image on the image sensor 5.
  • the signal processing circuit 6 outputs an image generated by performing predetermined signal processing on the signal input from the image sensor 5.
  • the signal processing circuit 6 has a plurality of registers 7 for setting its own function. By setting a value in a predetermined register 7 from the image processing ECU 3, a predetermined function of the signal processing circuit 6 is enabled / disabled, and details of each function are set.
  • the predetermined functions include shutter speed and gain adjustment.
  • the image detection IC4 used in this embodiment is manufactured by Omnivision, and has a crop function in addition to shutter speed and gain adjustment as functions of the image detection IC4.
  • the crop function is a function of cutting out a part of an image image taken by the image sensor 5 and outputting it as a crop image. Since the number of pixels of the crop image is smaller than that of the imaged image, the processing load of the image processing ECU 3 can be reduced.
  • register 7 related to the crop function in order to enable / disable the crop function, enable / disable the register value embedding function, set the start coordinates indicating the crop area, the vertical size, and the horizontal size. It is provided in.
  • the image detection IC 4 cuts out the pixels in the crop area from the imaged image captured by the image sensor 5 to create and output a crop image.
  • the register value embedding function is enabled, the start coordinates set in the register 7 are embedded in the crop image as crop information and output.
  • the crop image input from the image detection IC 4 should correspond to the crop area.
  • the internal clock of the image detection IC 4 operates independently of the image processing ECU 3, it operates asynchronously with the image processing ECU 3. Therefore, even if the crop area is set in the image detection IC 4 from the image processing ECU 3, there is a delay until the crop image corresponding to the crop area is output from the image detection IC 4.
  • the register value embedding function as a function of the image detection IC4. That is, as described above, since the image detection IC4 manufactured by Omnivision has a function of embedding the register value in the pixel data, the embedding function is used.
  • the register value embedding function will be described below.
  • the predetermined register 7 of the image detection IC 4 is set to output the register value indicating the crop area shown in FIG. 4, the start coordinates of the crop area are embedded in the pixel data corresponding to the first row indicated by the broken line in the crop image. It is supposed to be included.
  • the image processing ECU 3 can acquire the start coordinates of the crop area corresponding to the input crop image by extracting the resist value embedded in the first line of the crop image.
  • the image processing ECU 3 When the image processing ECU 3 is activated in response to the ON of the ignition switch, the image processing ECU 3 is initially set with respect to the DSM 1 as shown in FIG. 10 (S1). In the initial setting, the register 7 for setting the crop function and the register 7 for setting the embedding of the register value in the crop image shown in FIG. 3 are set to be valid respectively. Further, the crop area is set in the register 7 for setting the crop area.
  • the initial value is set for the crop area as shown in FIG.
  • the initial value of the crop area is set so that the center position of the crop area is located at the center position of the pixel area.
  • the image detection IC4 cuts out the crop area set from the pixel area of the image sensor 5, and embeds the start coordinates of the crop area set in the register 7 in the first line of the crop image corresponding to the crop area to obtain a crop image. Output. In this case, since the driver's face is imaged on the image sensor 5, the driver's face is output as a crop image.
  • the image processing ECU 3 performs face detection from the crop image input from DSM1 (S2). To detect a face, for example, feature points such as eyes, nose, and mouth are extracted. At this time, if the driver's posture is out of order, the center position of the face has moved from the center position of the crop image (S3: NO), so that the center position of the face is the crop image as shown in FIG. Change the crop area setting so that it is in the center position (S4). In this case, since the size of the crop area is fixed in the present embodiment, the start coordinates set at addresses 3810 to 3813 of the register 7 built in the image detection IC 4 are changed. That is, when the center position of the face is moving as shown in FIG. 7, a value obtained by adding the movement amount to the start coordinate by the movement amount is set as the new start coordinate.
  • the image detection IC4 embeds the changed start coordinate in the crop image corresponding to the crop area of the changed start coordinate and outputs the changed start coordinate.
  • a crop image of 1280 ⁇ 960 size is cropped from a pixel size of 1600 ⁇ 1300 size.
  • the pitch of the pixels is the same, and it is possible to determine the details of the driver's face.
  • a crop image was used to recognize the direction of the line of sight.
  • the number of pixels is large, so that the processing load of the image processing ECU 3 is large.
  • the image processing ECU 3 resizes by thinning out the pixel data of the crop image (S8).
  • This resized size is the minimum required size and is VGA (640 ⁇ 480) size as shown in FIG.
  • the image processing ECU 3 identifies the driver by face recognition based on the resized crop image.
  • face recognition it is sufficient if the driver's face can be identified, so that the face can be recognized even if the size is VGA.
  • the steering wheel, seat, rearview mirror, etc. stored in advance for each driver can be automatically adjusted.
  • the image processing ECU 3 can correctly detect the center position of the driver's face and recognize the face in all the frames without dropping the frame.
  • the crop area is dynamically changed so as to follow the driver's face position, and only the face area is cut out to process the image data of the minimum size. Since the data is transferred to the ECU 3, the processing load of the image processing ECU 3 can be significantly reduced.
  • the image processing ECU 3 provides a crop area to the DSM 1 so that the center position of the driver's face becomes the center position of the crop image based on the crop image input from the DSM 1 and the start coordinates of the crop area embedded in the crop image. Since the setting and the line-of-sight direction of the driver are recognized based on the crop image, the line-of-sight direction of the driver can be correctly recognized even when the driver's posture is lost. Since the driver's face is recognized based on the resized crop image by thinning out the pixel data, the processing load of the image processing IC3 can be further reduced.
  • the image detection sensor may be configured to independently output crop information without embedding it in the crop image.
  • crop information in addition to the starting coordinates of the crop area, the vertical size and the horizontal size may be output.
  • the driver's face recognition may be performed without resizing the crop image.
  • the state of the driver is not limited to the direction of the driver's line of sight, and for example, the degree of drowsiness of the driver may be determined.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Mechanical Engineering (AREA)
  • Chemical & Material Sciences (AREA)
  • Combustion & Propulsion (AREA)
  • Transportation (AREA)
  • Image Processing (AREA)
  • Instrument Panels (AREA)
  • Closed-Circuit Television Systems (AREA)
  • Studio Devices (AREA)
  • Fittings On The Vehicle Exterior For Carrying Loads, And Devices For Holding Or Mounting Articles (AREA)

Abstract

La présente invention comprend : une unité de détection d'image (4) qui fournit en sortie une image rognée découpée à partir d'une image formée sur la base d'une zone de rognage définie et d'informations de rognage indiquant la zone de rognage ; et une unité de surveillance de condition (3) qui établit la zone de rognage dans l'unité de détection d'image de sorte que la position centrale du visage d'un conducteur devienne la position centrale de l'image rognée sur la base de l'image rognée et des informations de rognage fournies en entrée à partir de l'unité de détection d'image et surveille une condition du conducteur sur la base de l'image rognée.
PCT/JP2020/007141 2019-03-18 2020-02-21 Dispositif de surveillance de condition pour conducteur WO2020189182A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2019050303A JP2020155835A (ja) 2019-03-18 2019-03-18 運転者の状態監視装置
JP2019-050303 2019-03-18

Publications (1)

Publication Number Publication Date
WO2020189182A1 true WO2020189182A1 (fr) 2020-09-24

Family

ID=72520206

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2020/007141 WO2020189182A1 (fr) 2019-03-18 2020-02-21 Dispositif de surveillance de condition pour conducteur

Country Status (2)

Country Link
JP (1) JP2020155835A (fr)
WO (1) WO2020189182A1 (fr)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH03254291A (ja) * 1990-03-02 1991-11-13 Toyota Motor Corp 車両運転者監視装置
JP2008205961A (ja) * 2007-02-21 2008-09-04 Canon Inc 画像処理装置、その制御方法、及びプログラム

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH03254291A (ja) * 1990-03-02 1991-11-13 Toyota Motor Corp 車両運転者監視装置
JP2008205961A (ja) * 2007-02-21 2008-09-04 Canon Inc 画像処理装置、その制御方法、及びプログラム

Also Published As

Publication number Publication date
JP2020155835A (ja) 2020-09-24

Similar Documents

Publication Publication Date Title
CN109842753B (zh) 摄像头防抖系统、方法、电子设备和存储介质
US8149280B2 (en) Face detection image processing device, camera device, image processing method, and program
US11558553B2 (en) Electronic device for stabilizing image and method for operating same
US8063938B2 (en) Photographed image process changeover apparatus of a video telephone function
CN109951638B (zh) 摄像头防抖系统、方法、电子设备和计算机可读存储介质
KR20190032061A (ko) 이미지 보정을 수행하는 전자 장치 및 그 동작 방법
JP6669240B1 (ja) 記録制御装置、記録制御システム、記録制御方法、および記録制御プログラム
US11708025B2 (en) Vehicle vision system with smart camera video output
JP2010069921A (ja) 車線認識装置
US10462346B2 (en) Control apparatus, control method, and recording medium
US20180270444A1 (en) Image recording system, image recording method and storage medium recording image recording program
US20200369207A1 (en) Image processing device, image display system, and image processing method
JP7070082B2 (ja) 車載カメラ
JP4235510B2 (ja) デジタルカメラ及び固体撮像装置
WO2020189182A1 (fr) Dispositif de surveillance de condition pour conducteur
WO2016121406A1 (fr) Appareil de traitement d'image, système de traitement d'image, véhicule, appareil d'imagerie et procédé de traitement d'image
US10430935B2 (en) Imaging apparatus, imaging system, movable body, and chip
KR20210023859A (ko) 화상 처리 장치, 이동 장치 및 방법, 그리고 프로그램
KR20210011556A (ko) 차량 영상 제어 장치, 그를 포함한 시스템 및 그 방법
WO2015115103A1 (fr) Dispositif de traitement d'image, système d'appareil de prise de vues et procédé de traitement d'image
KR102609415B1 (ko) 영상시스템 및 그 작동방법
JP7018561B2 (ja) 表示制御装置、表示制御システム、表示制御方法、及び表示制御プログラム
JP6766433B2 (ja) 車両用表示制御装置、車両用表示システム、車両用表示制御方法およびプログラム
JP2002182260A (ja) 撮像装置
JP2014216694A (ja) 高解像度化処理付き追尾雲台装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20773998

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20773998

Country of ref document: EP

Kind code of ref document: A1