EP1629334A1

EP1629334A1 - An optical driver assistance system for a motor vehicle

Info

Publication number: EP1629334A1
Application number: EP04732660A
Authority: EP
Inventors: Martin John Thompson
Original assignee: TRW Ltd
Current assignee: TRW Ltd
Priority date: 2003-05-30
Filing date: 2004-05-13
Publication date: 2006-03-01
Anticipated expiration: 2024-05-13
Also published as: EP1629334B1; DE602004002870T2; GB2402208A; WO2004107072A1; GB0312484D0; DE602004002870D1

Abstract

An optical driver assistance system for a motor vehicle is disclosed. In order to meet challenges relating to packaging, EMC compatibility and functionality, the system comprises a low level unit incorporating both an optical sensor and (50) electronics (52) for carrying out low level compression processing of the data output from the sensor. This data is passed via a communications link to a separately formed high level unit (54) whose electronics perform high level processing. Moderate bandwidth is required of the communications link.

Description

DESCRIPTION

AN OPTICAL DRIVER ASSISTANCE SYSTEM

FOR A MOTOR VEHICLE

The present invention relates to an optical driver assistance system for use in a motor vehicle.

Optical systems for assisting the driver have been known for some years. Typically, optical data from a forward-looking electronic camera or other optical assembly is processed to yield information about vehicle road position, approaching curves etc. and used typically to provide information to the driver. For instance, such technology may be used to provide the driver with a lane departure warning (LDW). Such a system may, additionally or alternatively, actually influence control of the vehicle, e.g. by applying a torque to the steering wheel.

The raw data from the optical assembly has a large bandwidth. For example if the assembly outputs 30 frames per second with a 640 x 480 resolution and 10 bits ^' per pixel, the rate of data output is 92 megabits per second (Mbps). In motor vehicle applications the space available, at the mounting site of the optical assembly, is typically limited. As an example the volume available to receive the optical assembly in a current project is only 30mm x 30mm x 30mm. The processor power required to process the optical data is also significant. The design of a suitable architecture for an optical driver assistance system consequently presents a problem.

One option would be to attempt to fit both the optical assembly and the electronics for image processing into a single unit small enough to meet motor industry requirements. This is attractive from the point of view of electromagnetic compatibility (EMC). However development of such a unit is expected to be

expensive and problematic given the small available space, requiring customised

integrated circuits. Additionally, providing a power supply that is robust enough for

the usual automotive environment and small enough for the purpose is believed to be

a problem, as also is the dissipation of sufficient heat from such a small unit.

There is the alternative of packaging and mounting the optical assembly

separately from an electronic control unit (ECU) carrying out the image processing. This provides for a very small optical assembly which can thus be mounted in the available space in the vehicle, its site being dictated of course by the need to provide

this unit with a view of the road. The ECU can be mounted elsewhere in the vehicle

so that its size is less critical. The problem with this approach is that it necessitates

transmitting the video data from the optical assembly to the ECU at the necessary

clock rates. Achieving this data transfer with suitable electromagnetic compatibility has proved to be impractical.

The raw data rates output from the sensor pose a high risk in designing a

sensor that will meet automotive requirements. This design problem is all the more

difficult since the serialised pixel clock frequency (90MHz) falls in the middle of the FM radio band. Not only that, but a LDW sensor would most likely be mounted at the front of a vehicle in close proximity to the vehicle FM aerial.

The use of low-voltage differential signalling drivers (LVDS) using a

serialised data (as espoused by the Camera Link consortium) is at present the only practical way to do this over a low wire-count loom. Optical methods could be used

but these have been discounted on cost grounds. Work has been undertaken to try and assess the suitability of Camera Link to work within the automotive EMC

requirements. The results have shown that even with its optimised performance,

Camera Link cannot meet the statutory requirements. In addition, the ability of

LVDS to survive in the automotive electrical environment is unproven and

considered unlikely to be good enough. Connectors of sufficient signal integrity to

carry this volume of data are also unlikely to be available in automotive specification and cost.

As a counter-example, the Media Oriented System Transport (MOST), which

is also an automotive communication bus, has a bandwidth of only 25Mbps, and uses an optical physical layer.

In accordance with the present invention there is an optical driver assistance

system for a motor vehicle comprising an optical sensor and electronics for

processing the optical sensor's output, wherein the system comprises a low level unit

incorporating the optical sensor and electronics for carrying out low level

compression processing of the data output from the optical sensor, a high level unit

comprising electronics for carrying out higher level processing, and a communications link in the form of a high speed CAN physical layer with custom

protocol for transfer of compressed data from the low level unit to the high level unit.

The bandwidth required for transfer of the low level processed data is very much smaller than would be required to transfer the raw data from the optical sensor. Consequently a suitably robust and economical communications link can be utilised.

This architecture thus overcomes the relevant problems without any compromise in

terms of performance, while also enabling the packaging requirements to be met. Specific embodiments of the present invention will now be described, by way

of example only, with reference to the accompanying drawings in which:

Figure 1 is a block diagram of the circuitry of a "smart camera" used in a

system embodying the present invention;

Figure 2 is a block diagram of the circuitry of a high level processing unit

used in the same system.; and

Figure 3 is a block diagram of the functional elements of a lane departure

warning system.

Suitable algorithms for processing image data from a camera and providing

a lane departure warning are known. The illustrated system can be broken down into

three main functional elements as illustrated in Figure 3, wherein the visual scene is

sensed by an optical assembly 50. The image, in the form of pixel data, is passed to

a low level processing function 52. The intention herein is not to limit the scope of the invention by reference to any specific algorithm. However in the current

algorithm the low level processing function carries out a process which can be broken

down into three stages. Firstly edges-transitions from dark to light or vice versa - in the image are identified. Secondly a thresholding function selects among these.

Thirdly a point pairing function identifies the two sides of a feature such as a white line.

The resulting feature data is then processed by a high level processing

function 54 which carries out functions such as curve fitting and the geometry

required to obtain a lane offset.

The embodiment illustrated in Figures 1 and 2 is a driver assistance system, and more specifically a lane departure warning system, whose physical construction

comprises two separate units -(a) a smart camera unit, comprising a camera as such

and a processor for carrying out low leve compression processing of the raw data and (b) a high level processor unit. The smart camera unit can be compact and so meet

motor industry packaging requirements. The high level processor unit can be

mounted at a site in the vehicle remote from the smart camera where sufficient space is available. The data output from the smart camera unit is sufficiently compressed

in the low level processor to be passed to the high level processor through a serial communications link while meeting EMC requirements. For example, in a typical

embodiment the compression can be approximately ¹⁰⁰/₁, ie 100 Mbits ^•=>^■ 1Mbit. The

smart camera unit sends a set of feature points to the high level processing unit. In

the illustrated embodiment the high-level processing unit provides electrical power

to the smart camera unit and also feeds back various calibration/tune parameters to

it

Looking at Figure 1, the smart camera unit's processor could for example take

the form of a Field Programmable Gate Array (FPGA) but for cost reasons a Digital Signal Processor (DSP) 2 has been chosen in the illustrated embodiment. The low

level processing of the output from optical sensor 4 requires around 3.5 M cycles per frame (around 105 M cycles per second, given the frame rate of 30 per second). The

image processing algorithms utilised are designed to operate with fixed point arithmetic and an external frame buffer is not required. Hence they are well suited

to implementation in a low cost DSP or FPGA. The particular DSP chosen for the level unit (DSP 2) is an Analog Devices Blackfin BF351 processor which is economical and compact but fast enough for the purpose, providing in excess of 600

MIPS (millions of instructions per second). It must be emphasised that any of a wide

range of processors could be used.

The smart camera unit 1 receives its power supply from the separate high

level processing unit at 6, through a linear regulator 8. Image processing software is

stored in a flash memory 10 and external clock, reset and watchdog functions 12 are

connected to the DSP 2. By generating an intermediate voltage to transmit to the

smart camera unit (in the high level processing unit 30 to be described below), it is made possible to reduce the physical size of the smart camera unit 1 since it can be

isolated from the more severe transients that occur on the main vehicle battery supply.

Communication with the high level processing unit requires bandwidth of the

order of 1 Mbps. A standard high speed CAN (controller area network) physical

layer is used, being easily able to meet the bandwidth requirement and well proven to withstand the particular electrical problems of the automotive environment. The

CAN physical layer is utilised (Figure 1, item 14) with a standard RS232- like UART

implemented on-board the DSP as indicated at 16. A low cost, high reliability serial

communications link (SLINK) results.

Some of the bandwidth is required to send data from the high level processing algorithms to the low level processing algorithms and to the imaging element. The

necessary bi-directional communications can be achieved by the time division

multiplexing of the usage of the point-to-point link. There is also the possibility, for

future embodiments, of booting the processor 2 of the smart camera module through this link, allowing the flash memory 10 to be dispensed with.

Looking now at the high level processing unit 30 illustrated in Figure 2, the

current high level processing algorithms use around 6 million floating point

operations per second (MFLOPS). They are not at present suitable for

implementation in a fixed point architecture. Allowing for expansion and diagnostic

functions, development is being based upon a requirement for twice this processor

speed - 12 MFLOPS. Suitable DSPs and microcontrollers are readily available. For the illustrated embodiment a DSP already known and tested in automotive

applications, the ADSP-21161, is preferably used. In fact the high level processing

unit 30 can be used to handle additional functions such as ACC (autonomous cruise

control) radar.

The high level processing unit 30 communicates with the low level processing unit 1 through the SLINK formed by the CAN physical layer 34 and a standard

UART (universal asynchronous receiver/transmitter) 36. Communication with other

systems, typically for signalling to the driver but potentially also for influencing vehicle control, is via a CAN link layer or a microprocessor with CAN, 38 and the

CAN physical layer 40. An on/off function is provided at 42.

The power supply 44 is integrated into the high level processing unit, receiving the vehicle's 12 or 24 volt supply at 44 which is then filtered and transient

protected at 46 and passed via a switcher 48 to the smart camera unit 1 and to the circuitry of the high level processing unit 30.

Whereas it is preferred for DSP 2 and DSP 32 to be of different characteristics, as exemplified by the Blackfm BF 351 and ADSP-21161 referred to hereinbefore, it is also posible for DSP 2 and DSP 32 to be of the same type, such as a TMS 320 C 6711, although this is currently less preferred for cost and size reasons.

Furthermore, the functions of both DSP 2 and DSP 32 can be implemented in FPGA.

Claims

1. An optical driver assistance system for a motor vehicle, comprising an optical sensor and electronics for processing the optical sensor's output, wherein the system

comprises a low level unit incorporating the optical sensor and electronics for

carrying out low level compression processing of data output from the optical sensor,

a separately formed high level unit comprising electronics for carrying out high level processing, and a communications link in the form of a high speed CAN physical

layer for transfer of compressed data from the low level circuit to the high level unit.

2. An optical driver assistance system as claimed in claim 1 , wherein the high

level unit provides a regulated power supply to the low level unit.

3. An optical driver assistance system as claimed in claim 1 or claim 2 wherein

calibration data or other data is passed from the high level unit to the low level unit.

4. An optical driver assistance system as claimed in any preceding claim which is a lane departure warning system.

5. An optical driver assistance system as claimed in any of claims 1 to 4,

wherein the level of compression of data output from the optical sensor is

approximately times 100.

6. An optical driver assistance system substantially as herein described with

reference to, and as illustrated in, the accompanying drawings.