GB2402208A

GB2402208A - Optical driver assistance system

Info

Publication number: GB2402208A
Application number: GB0312484A
Authority: GB
Inventors: Martin John Thompson
Original assignee: TRW Ltd
Current assignee: TRW Ltd
Priority date: 2003-05-30
Filing date: 2003-05-30
Publication date: 2004-12-01
Also published as: EP1629334A1; GB0312484D0; WO2004107072A1; DE602004002870T2; DE602004002870D1; EP1629334B1

Abstract

An optical driver assistance system for a motor vehicle comprises a low-level unit incorporating an optical sensor 50 and low-level processing 52, a separate high-level processing unit 54, and a communications link between the low and high-level units. Using the pixel information, the low-level unit analyses edge transitions and performs point pairing to identify two sides of a feature, e.g. a white line. The high-level processing calculates the geometry required to achieve a lane offset. The high-level unit may provide a regulated power supply to the low-level unit, and may also pass calibration data to the low-level unit. The optical driver assistance system may be a lane departure warning system.

Description

AN OPTICAL DRIVER ASSISTANCE SYSTEM FO!] A MOTOR VEHICLE The present

invention relates to an optical driver assistance system for use in a motor vehicle.

Optical systems for assisting the driver have been known for some years.

Typically, optical data from a forward-looking electronic camera or other optical assembly is processed to yield information about vehicle road position, approaching curves etc. and used typically to provide information to the driver.

For instance, such technology may be used to provide the driver with a lane departure warning (LDW). Such a system may, additionally or alternatively, actually influence control of the vehicle, e.g. by applying a torque to the steering wheel.

The raw data from the optical assembly has a large bandwidth. For example if the assembly outputs 30 frames per second with a 640 x 480 resolution and 10 bits per pixel, the rate of data output is 92 megabits per second (Mbps). In motor vehicle applications the space available, at the mounting site of the optical assembly, is typically limited. As an example the volume available to receive the optical assembly in a current project is only 30mm x 30mrn x 30mm. The processor power required to process the optical data is also significant. The design of a suitable architecture for an optical driver assistance system consequently presents a problem.

One option would be to attempt to fit both the optical assembly and the electronics for image processing into a single unit small enough to meet motor industry requirements. This is attractive from the point of view of electromagnetic compatibility (EMC). However development of such a unit is expected to be expensive and problematic given the small available space, requiring customised integrated circuits. Additionally, providing a power supply that is robust enough for the usual automotive environment and small enough for the purpose is believed to be a problem, as also is the dissipation of sufficient heat from such a small unit.

There is the alternative of packaging and mounting the optical assembly separately from an electronic control unit (ECU) carrying out the image processing. This provides for a very small optical assembly which can thus be mounted in the available space in the vehicle, its site being dictated of course by the need to provide this unit with a view of the road. The ECU can be mounted elsewhere in the vehicle so that its size is less critical. The problem with this approach is that it necessitates transmitting the video data from the optical assembly to the ECU at the necessary clock rates. Achieving this data transfer with suitable electromagnetic compatibility has proved to be impractical.

The raw data rates output from the sensor pose a high risk in designing a sensor that will meet automotive requirements. This design problem is all the more cliff cult since the serialised pixel clock frequency (9OMHz) falls in the middle of the FM radio band. Not only that, but a LDW sensor would most likely be mounted at the front of a vehicle in close proximity to the vehicle FM aerial.

The use of low-voltage differential signalling drivers (LVDS) using a serialised data (as espoused by the Camera Link consortium) is at present the only practical way to do this over a low wire-count loom. Optical methods could be used but these have been discounted on cost grounds. Work has been undertaken to try and assess the suitability of Camera Link to work within the automotive EMC requirements. The results have shown that even with its optimised performance, Camera Link cannot meet the statutory requirements. In addition, the ability of LVDS to survive in the automotive electrical environment is unproven and considered unlikely to be good enough. Connectors of sufficient signal integrity to carry this volume of data are also unlikely to be available in

automotive specification and cost.

As a counter-example, the Media Oriented System Transport (MOST), which is also an automotive communication bus, has a bandwidth of only 25Mbps, and uses an optical physical layer.

In accordance with the present invention there is an optical driver assistance system for a motor vehicle comprising an optical sensor and electronics for processing the optical sensor's output, wherein the system comprises a low level unit incorporating the optical sensor and electronics for carrying out low level processing of the data output from the optical sensor, a high level unit comprising electronics for carrying out higher level processing, and a communications link for transfer of low level processed data from the low level unit to the high level unit.

The bandwidth required for transfer of the low level processed data is very much smaller than would be required to transfer the raw data from the optical sensor. Consequently a suitably robust and economical communications link can be utilised. This architecture thus overcomes the relevant problems without any compromise in terms of performance, while also enabling the packaging requirements to be met.

Specific embodiments of the present invention will now be described, by way of example only, with reference to the accompanying drawings in which: Figure 1 is a block diagram of the circuitry of a "smart camera" used in a system embodying the present invention; Figure 2 is a block diagram of the circuitry of a high level processing unit used in the same system.; and Figure 3 is a block diagram of the functional elements of a lane departure warning system.

Suitable algorithms for processing image data from a camera and providing a lane departure warning are known. The system can be broken down in accordance with the invention, into three main functional elements as illustrated in Figure 3, wherein the visual scene is sensed by an optical assembly 50. The image, in the form of pixel data, is passed to a low level processing function 52.

The intention herein is not to limit the scope of the invention by reference to any specific algorithm. However in the current algorithm the low level processing function carries out a process which can be broken down into three stages. Firstly edges-transitions from dark to light or vice versa - in the image are identified.

Secondly a thresholding function selects among these. Thirdly a point pairing function identifies the two sides of a feature such as a white line.

The resulting feature data is then processed by a high level processing function 54 which carries out functions such as curve fitting and the geometry required to obtain a lane offset.

The embodiment illustrated in Figures 1 and 2 is a driver assistance system, and more specifically a lane departure warning system, whose physical construction comprises two separate units -(a) a smart camera unit, comprising a camera as such and a processor for carrying out low level processing of the raw data and (b) a high level processor unit. The smart camera unit can be compact and so meet motor industry packaging requirements. The high level processor unit can be mounted at a site in the vehicle remote from the smart camera where sufficient space is available. The data output from the smart camera is sufficiently compressed to be passed to the high level processor through a serial communications link while meeting EMC requirements. The smart camera unit sends a set of feature points to the high level processing unit. In the illustrated embodiment the high-level processing unit provides electrical power to the smart camera unit and also feeds back various calibration/tune parameters to it Looking at Figure 1, the smart camera unit's processor could for example take the form of a Field Programmable Gate Array (FPGA) but for cost reasons a Digital Signal Processor (DSP) 2 has been chosen in the illustrated embodiment, the particular unit being a TMS320C67 1 1. The low level processing of the output from optical sensor 4 requires around 3.5 M cycles per frame (around 105 M cycles per second, given the frame rate of 30 per second). The image processing algorithms utilised are designed to operate with fixed point arithmetic and an external frame buffer is not required. Hence they are well suited to implementation in a low cost DSP or FPGA. The particular DSP chosen is an Analog Devices Blacken BF35 1 processor which is economical and compact but fast enough for the purpose, providing in excess of 600 Mbps. It must be emphasised that any of a wide range of processors could be used.

The smart camera unit 1 receives its power supply from the separate high level processing unit at 6, through a linear regulator 8. Image processing software is stored in a flash memory 10 and external clock, reset and watchdog functions 12 are connected to the DSP 2. By generating an intermediate voltage to transmit to the smart camera unit (in the high level processing unit 30 to be described below), it is made possible to reduce the physical size of the smart camera unit 1 since it can be isolated from the more severe transients that occur on the main vehicle battery supply.

Communication with the high level processing unit requires bandwidth of the order of 1 Mbps. A standard high speed CAN (controller area network) is used, being easily able to meet the bandwidth requirement and well proven to withstand the particular electrical problems of the automotive environment. The CAN physical layer is utilised (Figure 1, item 14) with a standard RS232- like UART implemented on-board the DSP as indicated at 16. A low cost, high reliability serial communications link (SLINK) results.

Some of the bandwidth is required to send data from the high level processing algorithms to the low level processing algorithms and to the imaging element. The necessary bi-directional communications can be achieved by the time division multiplexing of the usage of the point-topoint link. There is also the possibility, for future embodiments, of booting the processor 2 of the smart camera module through this link, allowing the flash memory 10 to be dispensed with.

Looking now at the high level processing unit 30 illustrated in Figure 2, the current high level processing algorithms use around 6 million floating point operations per second (MFLOPS). They are not at present suitable for implementation in a fixed point architecture. Allowing for expansion and diagnostic functions, development is being based upon a requirement for twice this processor speed - 12 MFLOPS. Suitable DSPs and microcontrollers are readily available. For the illustrated embodiment a DSP already known and tested in automotive applications, the ADSP-2 1161, is used. In fact the high level processing unit 30 can be used to handle additional functions such as ACC (autonomous cruise control) radar.

The high level processing unit 30 communicates with the low level processing unit 1 through the SLINK formed by the CAN physical layer 34 and a standard UART (universal asynchronous receiver/transmitter) 36.

Communication with other systems, typically for signalling to the driver but potentially also for influencing vehicle control, is via a CAN link layer or a microprocessor with CAN, 38 and the CAN physical layer 40. An on/off function is provided at 42.

The power supply 44 is integrated into the high level processing unit, receiving the vehicle's 12 or 24 volt supply at 44 which is then filtered and transient protected at 46 and passed via a switcher 48 to the smart camera unit] and to the circuitry of the high level processing unit 30.

Claims

1. An optical driver assistance system for a motor vehicle, comprising an optical sensor and electronics for processing the optical sensor's output, wherein the system comprises a low level unit incorporating the optical sensor and electronics for carrying out low level processing of data output from the optical sensor, a separately formed high level unit comprising electronics for carrying out high level processing, and a communications links for transfer of low level processor data from the low level circuit to the high level unit.

2. An optical driver assistance system as claimed in claim I, wherein the high level unit provides a regulated power supply to the low level unit.

3. An optical driver assistance system as claimed in claim 1 or claim 2 wherein calibration data or other data is passed from the high level unit to the low level unit.

4. An optical driver assistance system as claimed in any preceding claim which is a lane departure warning system.

5. An optical driver assistance system substantially as herein described with reference to, and as illustrated in, the accompanying drawings.