GB2529408A - Display system and method - Google Patents

Display system and method Download PDF

Info

Publication number
GB2529408A
GB2529408A GB1414632.8A GB201414632A GB2529408A GB 2529408 A GB2529408 A GB 2529408A GB 201414632 A GB201414632 A GB 201414632A GB 2529408 A GB2529408 A GB 2529408A
Authority
GB
United Kingdom
Prior art keywords
image
trailer
vehicle
display system
image data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
GB1414632.8A
Other versions
GB2529408B (en
GB201414632D0 (en
Inventor
Giovanni Strano
Jeremy Greenwood
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Jaguar Land Rover Ltd
Original Assignee
Jaguar Land Rover Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Jaguar Land Rover Ltd filed Critical Jaguar Land Rover Ltd
Priority to GB1414632.8A priority Critical patent/GB2529408B/en
Publication of GB201414632D0 publication Critical patent/GB201414632D0/en
Priority to CN201580044317.8A priority patent/CN106573577B/en
Priority to US15/504,821 priority patent/US10701299B2/en
Priority to GB1514673.1A priority patent/GB2530649B/en
Priority to PCT/EP2015/068970 priority patent/WO2016026870A1/en
Priority to EP15754161.6A priority patent/EP3183875B1/en
Priority to AU2015306153A priority patent/AU2015306153B2/en
Priority to EP20168870.2A priority patent/EP3709635B1/en
Publication of GB2529408A publication Critical patent/GB2529408A/en
Priority to ZA2017/01208A priority patent/ZA201701208B/en
Priority to AU2017272148A priority patent/AU2017272148A1/en
Priority to US15/853,436 priority patent/US10701300B2/en
Application granted granted Critical
Publication of GB2529408B publication Critical patent/GB2529408B/en
Priority to US16/448,515 priority patent/US11553153B2/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60RVEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R1/00Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B60R1/20Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B60R1/22Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle
    • B60R1/23Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle with a predetermined field of view
    • B60R1/26Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle with a predetermined field of view to the rear of the vehicle
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00Television systems
    • H04N7/18Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • H04N7/181Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60RVEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R1/00Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B60R1/02Rear-view mirror arrangements
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00Geometric image transformations in the plane of the image
    • G06T3/40Scaling of whole images or parts thereof, e.g. expanding or contracting
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00Geometric image transformations in the plane of the image
    • G06T3/40Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4038Image mosaicing, e.g. composing plane images from plane sub-images
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60RVEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R1/00Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B60R1/12Mirror assemblies combined with other articles, e.g. clocks
    • B60R2001/1253Mirror assemblies combined with other articles, e.g. clocks with cameras, video cameras or video screens
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60RVEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/30Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing
    • B60R2300/303Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing using joined images, e.g. multiple camera images
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60RVEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/40Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the details of the power supply or the coupling to vehicle components
    • B60R2300/406Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the details of the power supply or the coupling to vehicle components using wireless transmission
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30248Vehicle exterior or interior
    • G06T2207/30252Vehicle exterior; Vicinity of vehicle

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Mechanical Engineering (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Processing (AREA)
  • Closed-Circuit Television Systems (AREA)

Abstract

The invention relates to a display system, 1, for generating a composite view of a region behind a vehicle, V, towing a trailer, T. A first camera, C1, mounted in a rear-facing orientation to the vehicle outputs first image data whilst a second camera, C2, mounted in a rear-facing orientation to the trailer outputs second image data. An image processor combines the first and second image data to generate a composite image. The composite image may be formed by overlaying a semi-transparent image of the first image onto the second image to create a ghost or phantom image of the trailer overlaid on top of the image of the road behind the trailer. The processor may identify and match common features present in the first and second images in order to correctly overlay the images and the images may be subject to image processing such as rotation, cropping, scaling, skew correction and/or translation. Also disclosed is a system in which a graphical representation of a trailer is overlaid onto an image showing the view behind the trailer to produce an augmented image for display.

Description

DISPLAY SYSTEM AND METHOD
TECHNICAL FIELD
The present disclosure relates to a display system and method and particularly, but not exclusively, to a display system and method for a vehicle. Aspects of the invention relate to a system, to a method, and to a vehicle.
BACKGROUND
It is common practice to tow a trailer behind a vehicle. A particular problem encountered while towing is the reduced visibility behind the vehicle. In particular, the trailer presents an obstacle which partially obscures the driver's field of view behind the vehicle. Large and small trailers typically reduce visibility and create a blind spot behind the vehicle. The resulting lack of visibility can, for example, compromise the following manoeuvres: lane changes, overtaking, parking, joining a motorway (highway) and pulling out of junctions.
In order to mitigate the aforementioned problems, it is common practice to fit a so-called towing mirror to the side mirrors of the vehicle. The towing mirrors extend laterally outwardly to provide improved visibility along the side of the trailer. However, towing mirrors have several disadvantages, notably they increase the vehicle width and may reduce manoeuvrability in confined spaces. Moreover, towing mirrors are not well suited to providing a view of other vehicles positioned directly behind the trailer. Towing mirrors can also suffer from vibration and hence blurring of the image.
It is against this background that the present invention has been conceived. At least in certain embodiments, the present invention seeks to overcome or ameliorate at least some of the aforementioned problems associated with prior art devices.
SUMMARY OF THE INVENTION
Aspects of the present invention relate to display system for a vehicle towing a trailer, to a method of generating a composite image, and to a vehicle.
According to a further aspect of the present invention there is provided a display system for a vehicle, the display system comprising: a first camera configured to be mounted in a rear-facing orientation to the vehicle and to output first image data; a second camera configured to be mounted in a rear-facing orientation to a trailer towed by the vehicle and to output second image data; and processor means for receiving said first image data and said second image data and for generating a composite image therefrom.
According to another aspect of the invention for which protection is sought there is provided a processor or control module for a vehicle, the processor or control module being configured and/or arranged to receive first image data from a first camera disposed in a rear-facing orientation relative to the vehicle and second image data from a second camera disposed in a rear-facing orientation relative to a trailer towed by the vehicle, and to generate a composite image from said first and second image data.
The processor or control module may be configured to output the composite image to a display system, such as a screen or digital display of the vehicle.
The display system may comprise a rear-view display system. The resulting composite image may be a rear-facing composite image which, at least in certain embodiments, can facilitate towing the trailer. In use, the first camera may be mounted to the vehicle and at least a portion of the first image may comprise a view of the front of the trailer. The second camera can be mounted to the rear of the trailer such that the second image is unobstructed.
However, the second image does not provide the driver with an indication of the relative position and/or orientation of the trailer. As such, the driver of the vehicle may find it difficult to judge the relative position of obstructions and/or other vehicles to the trailer using the second image. By combining the first image data and the second image data, the view of the front of the trailer from the first image can be composited with the unobstructed second view.
The first image can, for example, be displayed as a transparent or semi-transparent layer which overlays the second image (for example to provide a phantom or ghost image of the trailer). The composite image data can be output to a display screen to display the composite image.
The image processor can be configured to combine said first image data and said second image data such that at least a portion of one of said first image and said second image is overlaid onto the other of said first image and said second image. At least said portion of one of said first image and said second image can be overlaid as a transparent or semi-transparent image onto the other of said first image and said second image. In one implementation, the image processor can be configured to overlay at least a portion of said first image onto said second image to form the composite image data.
The image processor can be configured to select the portion of said first image or of said second image to be overlaid in dependence on an image mask. The image mask can be predefined or can be generated in dependence on the first image data. The first image data can be analysed to identify the trailer in the first image. The image mask can correspond to the trailer identified in the first image. The analysis of the first image can, for example, comprise performing edge detection or feature identification.
The image processor can be configured to combine said first image data and said second image data such that a portion of said first image or of said second image is substituted with at least a portion of the other of said first image and said second image.
The image processor can be configured to analyse said first image data and said second image data to identify one or more features common to both said first image and said second image. The image processor can be configured to match (align) said first image and said second image in dependence on the identified one or more features. The image processor can be configured to perform one or more of the following transforms to said first image and/or said second image: image rotation, scaling, cropping, magnification, skew correction and translation. These transforms could, for example, compensate for different viewing angles and/or imaging properties of the first and second cameras.
The image processor can be configured to generate said composite image in dependence on positional data defining the relative position of the first and second cameras. The positional data can comprise longitudinal positional data and/or vertical positional data and/or lateral positional data. A longitudinal offset between the first and second cameras can be used to facilitate registration of the first and second images. Alternatively, or in addition, the image processor can compensate for a lateral offset and/or a vertical offset between said first and second cameras.
The first camera can comprise a first wireless transmitter for transmitting the first image data to the image processor and/or the second camera can comprise a second wireless transmitter for transmitting the second image data to the image processor. Alternatively, a wired connection can be established between the image processor and the first camera and br the second camera.
The image processor can be configured to augment the composite image with one or more of the following: a graphical representation of the trailer being towed by the vehicle; a graphical representation of a projected path for the trailer during a reversing operation; a vehicle control instruction(s) to guide the trailer during a reversing operation; and proximity information for the trailer.
The image processor can be a digital image processor. Alternatively, the image processor can be an analogue image processor configured to generate the composite image data.
The image processor can be configured to output the composite image data for display on a display screen. The display screen could, for example, be disposed in a centre console, an instrument cluster, or a rear-view mirror of the vehicle. Alternatively, the image processor can be configured to output the composite image data wirelessly, for example to be displayed on a network device, such as a cellular telephone or a so-called smart-phone", a personal computer or a tablet computer.
According to a further aspect of the present invention there is provided a rig comprising a vehicle and a trailer, wherein the rig comprises a display system as described herein. The first camera can be mounted to the vehicle and the second camera can be mounted to the trailer.
According to a further aspect of the present invention there is provided a display system for providing a view of a region behind a vehicle, the display system comprising: a camera for mounting in a rear-facing orientation to a trailer, the camera being operable to output image data corresponding to an image; an image processor for receiving said image data; wherein the image processor is configured to overlay a graphical representation of the trailer onto the image data to form an augmented image for display.
According to a further aspect of the present invention there is provided a method of generating a composite image of a region behind a vehicle towing a trailer, the method comprising: using a first camera disposed on the vehicle to generate a first image; using a second camera disposed on the trailer to generate a second image; using an image processor to combine said first and second images to form a composite image.
The method can comprise overlaying at least a portion of one of said first image and said second image onto the other of said first image and said second image. The image to be overlaid onto the other image can be modified to be rendered in a transparent or semi-transparent form. At least said portion of one of said first image and said second image can be overlaid as a semi-transparent image onto the other of said first image and said second image. An image mask can be used to select the portion of said first image or of said second image to be overlaid. The image mask can be generated in dependence on the first image data.
The process of combining said first and second images can comprise substituting a portion of said first image or of said second image with at least a portion of the other of said first image and said second image.
The method can comprise identifying one or more features common to both said first image and said second image. The method can comprise matching said first image and said second image in dependence on the identified one or more features. The method can comprise performing image registration to match (align) the first and second images with each other. The image registration can be performed substantially in real-time by the image processor.
The method can comprise performing one or more of the following transforms to said first image and/or to said second image: image rotation, scaling, cropping, magnification, skew correction and translation.
The composite image can be generated in dependence on positional data defining the relative position of the first and second cameras.
The method can comprise augmenting the composite image with one or more of the following: a graphical representation of the trailer being towed by the vehicle: a graphical representation of a projected path for the trailer during a reversing operation; a vehicle control instruction(s) to guide the trailer during a reversing operation: and proximity information for the trailer.
The method can comprise displaying the composite image, for example on a display screen.
The display screen can be provided in a centre console or an instrument cluster.
Alternatively, or in addition, the display screen can be incorporated into a rear view mirror.
The term processor is used herein to refer to one or more electronic processors. Similarly, the term system memory is used herein to refer to one or more storage devices. The processor can be a general purpose computational device configured to execute a set of software instructions to perform the method(s) described herein.
Within the scope of this application it is expressly intended that the various aspects, embodiments, examples and alternatives set out in the preceding paragraphs, in the claims and/or in the following description and drawings, and in particular the individual features thereof, may be taken independently or in any combination. That is, all embodiments and/or features of any embodiment can be combined in any way and/or combination, unless such features are incompatible. The applicant reserves the right to change any originally filed claim or file any new claim accordingly, including the right to amend any originally filed claim to depend from and/or incorporate any feature of any other claim although not originally claimed in that manner.
BRIEF DESCRIPTION OF THE DRAWINGS
One or more embodiments of the present invention will now be described, by way of example only, with reference to the accompanying figures, in which: Figure 1 shows a vehicle and a trailer incorporating a rear-view display system in accordance with an embodiment of the present invention; Figure 2 shows a schematic representation of the components of the rear-view display system shown in Figure 1; Figure 3A shows a first image from a first camera disposed on the vehicle; Figure 3B shows a second image from a second camera disposed on the trailer; Figure 3C shows a composite image generated by combining the first and second images shown in Figures 3A and 33; and Figure 4 shows an augmented composite image generated by the rear-view display system shown in Figure 1.
DETAILED DESCRIPTION
A rear-view display system 1 in accordance with an embodiment of the present invention will now be described with reference to the accompanying figures. The rear-view display system 1 is intended for use in a vehicle V towing a trailer T (referred to in combination as a rig) to generate a composite image for providing improved visibility for the vehicle driver of the region behind the vehicle V. The vehicle V has a first longitudinal axis Xl, and the trailer T has a second longitudinal axis X2, as shown in Figure 1. The terms "front" and "rear" are used herein in their conventional sense when defining the relative position of features on the vehicle V and the trailer T. The terms "rear-facing" and "rear-view" are used herein to refer to a position or orientation which is in a direction towards the back of the vehicle V or the trailer T. As shown in Figure 1, the rear-view display system 1 is in the form of a vehicle-mounted unit and a trailer-mounted unit. The vehicle-mounted unit comprises a central processing unit 3 connected to a first camera Cl. As shown in Figure 2, the central processing unit 3 comprises an image processor 5 coupled to system memory 7, and a wireless receiver 9 having a first antenna 11. The image processor 5 can be a digital image processor. The image processor 5 is configured to execute a set of software instructions held in the system memory 7. The image processor 5 is connected to a communication bus 13, such as the vehicle CAN bus, for communicating with other vehicle systems. The vehicle V comprises a display screen 15 on which the images received from the first and second cameras Cl, C2 can be selectively displayed. The trailer-mounted unit comprises a second camera C2 connected to a wireless transmitter 17 having a second antenna 19.
The first and second cameras Cl, C2 are both rear-facing digital cameras for generating video images. The first and second cameras Cl, C2 each comprise a wide-angle lens to provide a viewing angle of approximately 130°. The first camera Cl is mounted centrally at the rear of the vehicle V above a rear license plate (not shown) and, in the present embodiment, can selectively function as a reversing camera to provide a parking aid when there is no trailer T coupled to the vehicle V. The second camera C2 is mounted centrally at the rear of the trailer T. The first camera Cl has a first field of view FOV1 (shown in Figure 1) having a line of sight (i.e. a centreline) substantially coincident with the first longitudinal axis Xl. The second camera C2 has a second field of view FOV2 (shown in Figure 1) having a line of sight (i.e. a centreline) substantially coincident with the second longitudinal axis X2, for example above a license plate on the trailer I. The first and second cameras Cl, 02 are arranged at approximately the same vertical height above ground level. In alternate arrangements, the first and second cameras 01, C2 can be offset from each other in a vertical and/or transverse direction. The image processor 5 can be configured to correct for any such positional offset. Equally, the image processor 5 can be configured to correct for an angular offset between the first and second cameras Cl, C2, for example if one or both of the first and second cameras Cl, C2 is angularly offset from the respective first and second longitudinal axes Xl, X2.
The first camera Cl outputs a first signal Si comprising first image data corresponding to a first image IMO1. The first signal Si is published to the communication bus 13 and provides a first input for the image processor 5. The second camera C2 outputs a second signal S2 comprising the second image data corresponding to a second image IMG2. The second signal S2 is transmitted by the wireless transmitter 17 to the wireless receiver 9 to provide a second input for the image processorS. The wireless transmitter 17 can transmit the second signal 62 using a suitable wireless communication standard, such as Wi-Fi®.
As outlined above, the first camera Cl is rear-facing and, when the vehicle V is towing the trailer I, the first image IMO1 contains the trailer T which partially obstructs the view afforded to the driver in a rearwards direction. The second camera 02 is mounted to the trailer T and the second image 1M02 provides an unobstructed view of the scene behind the trailer T. The image processor 5 is configured to combine the first image data and the second image data to generate composite image data. The composite image data corresponds to a composite image IMG3 formed from said first and second images IMG1, IMG2.
The image processor 5 modifies the first image data such that the first image IMO1 will be rendered as a semi-transparent image. For example, the image processor 5 can modify an alpha channel or an index transparency of the first image data-The first image data can be modified to provide a transparency of 25%, for example. The first image data and the second image data are combined to form a composite image IMG3. The image processor 5 can be configured to perform other image-manipulation techniques, for example to modify the colour and/or contrast of one or more image, for example to highlight an outline of the trailer T. The image processor 5 could also be configured to implement an edge-recognition algorithm to identify an outline of the trailer T for display in the composite image IMG3. The resulting composite image IMG3 comprises the second image IMG2 as a background element over which the first image IMG1 is overlaid. Significantly, the trailer T (which is present in the first image IMG1, but not the second image IMG2) is partially visible in the composite image IMG3 to facilitate determining its relative position to objects and/or other vehicles. The image processor 5 can perform additional blending or smoothing functions to obscure the transition between the first and second images IMG1, IMG2 in the composite image 1M03. The rear-view display system 1 could be configured to allow the user to select the modification to the first image data, for example to adjust a transparency setting of the first image data, to allow customisation of the composite image IMG3.
When the vehicle V and the trailer I are aligned with each other (such that the first and second longitudinal axes Xl, X2 are coincident), the first and second images IMG1, IMG2 are in respect of the same scene albeit offset from each other due to the longitudinal offset between the first and second cameras Cl, 02. Moreover, there may be a lateral offset and/or a vertical offset due to the positioning of the first and second cameras Cl, C2 on the vehicle V and the trailer T. Accordingly, the first image IMG1 cannot be overlaid directly onto the second image IMG2. Rather, the image processors is configured to implement an image matching procedure to align the first image IMG1 with the second image IMG2. The image processor 5 can implement a cross-correlation technique automatically to match features common to both the first image IMO1 and the second image IMG2. The image processor 5 can, for example, use a Hough transform feature extraction technique. The relative position and/or orientation of the identified features can be compared in the first and second images IMO1, 1M02. Alternatively, or in addition, the image processor 5 can identify the centroid of one or more image features and compare their relative position in the first and second images IMG1, IMG2. Other image analysis techniques that could be included are point mapping and/or edge-recognition. The image processor 5 can optionally perform image processing techniques with respect to time in order to track movement of the identified features in the first and second images. The image processor 5 could retrieve vehicle dynamic data published to the communication bus 13 to refine analysis of the image data.
For example, the image processor 5 could optionally utilise the vehicle speed and/or turning angle during analysis of the image data. The image processor 5 could also be configured to determine the relative orientation of the trailer T by analysing a target 21 provided on the front of the trailer T. In the illustrated arrangement, the target 21 comprises three circles arranged in a triangular formation. The image processor 5 could also be configured to compensate for differences in the first and second images IMG1, IMG2 due to different specifications of the first and second cameras Cl, 02.
The image processor 5 utilises the results of the cross-correlation to perform image registration such that the first and second images IMG1, IMG2 are aligned with each other.
The image registration can comprise one or more of the following transforms: image rotation, scaling, cropping, magnification (zooming), skew correction and translation. It will be appreciated that the transform(s) required to match the first and second images IMO1, 1M02 depend on the relative positioning of the first and second cameras Cl, C2. The transform(s) could be applied in two dimensions (2D) or three dimensions (3D), for example to compensate for an angular offset between the first and second longitudinal axes Xl, X2 (in a horizontal plane and/or a vertical plane). As outlined above, the first image IMG1 data is modified such that the first image IMO1 appears semi-transparent when displayed. The first image IMG1 is then overlaid onto the second image IMG2 to form the composite image 1M03 (for example using alpha-compositing techniques). The image processor 5 can optionally edit the first image IMG1 such that only a portion thereof is overlaid onto the second image IMG2. A cropping function could be applied to a predefined region of the first image IMO1, for example having a predefined width and height. Alternatively, the image processor 5 could be configured to identify the trailer T within the first image IMG1 to generate a mask which is overlaid onto the second image 1M02.
The operation of the rear-view display system 1 in accordance with an embodiment of the present invention will now be described with reference to Figures 3A, 3B and 3C. The first camera Cl generates first image data corresponding to a first image IMG1 which includes the trailer T being towed behind the vehicle V, as shown in Figure 3A. The second camera C2 generates second image data corresponding to a second image 1M32 which is an unobstructed view of the region behind the trailer T, as shown in Figure 3B. The image processor S implements an image matching procedure to match (align) the first and second images IMG1, IMG2, as described herein. The image processor 5 modifies the first image data such that the first image IMG1 will be displayed as a partially transparent image. The first image data and the second image data is then combined to form composite image data.
The composite image data corresponds to a composite image 1M03 comprising the first image IMG1 overlaid onto the second image 1M32, as shown in Figure 3C. The trailer T appears as a ghost' image or grind of the composite image IMG3. The resulting composite image IMG3 provides a clear view of the region behind the trailer T whilst retaining sufficient details of the trailer T to enable the driver to determine its relative position to obstacles and other vehicles.
The composite image IMG3 is output for display on a display screen, such as a liquid-crystal display, provided in the vehicle. The display screen can, for example, be provided in a centre console or in an instrument cluster. Alternatively, or in addition, the display screen can be incorporated into a rear view mirror, for example to display the composite image IMG3 alongside a reflective image.
If the angular offset between the first and second longitudinal axes Xl, X2 becomes excessive, for example during a turning manoeuvre, the image processor 5 may be unable to match the first and second images. The image processor 5 could be configured to activate one or more side-mounted cameras (on the vehicle V and/or the trailer T) to obtain different images for processing. If this function is not available, the image processor 5 can output a notification to the vehicle driver that the rear-view display function is not available.
The rear-view display system 1 described herein can be modified to display additional information. By way of example, the composite image IMG3 can be augmented by displaying a projected path P which the trailer T will follow during a reversing procedure. The projected path P can be overlaid onto the composite image 1M03 when the vehicle V is placed in reverse and the trailer T is detected. The driver can use the projected path P to control the trailer T when the vehicle V is being reversed. An illustration of an augmented composite image IMG4 is shown in Figure 4 by way of example. Moreover, the rear-view display system 1 could be modified to display driver instructions, for example to provide vehicle control instructions (steering angle and/or transmission selection) for guiding the trailer T to an identified parking area. Further details of a vehicle control system for controlling a vehicle to facilitate parking a trailer are known from the applicant's earlier UK patent application GB1312036.5 filed on 04th July 2013), the contents of which are incorporated herein in their entirety by reference.
In the illustrated example, the vehicle V is an automobile (for example a sports utility vehicle) and the trailer T is a caravan. It will be appreciated, however, that the apparatus and method(s) described herein are not limited in this respect and could be implemented in other types and categories of vehicle and to other types of trailer. The apparatus and method(s) could, for example, be applied to an articulated vehicle, a semi-trailer or a tractor-trailer It will be appreciated that further changes and modifications can be made to the apparatus and method described herein without parting from the scope of the present invention. A rear-view display system 1 has been described herein with reference to a single first camera Cl disposed on the vehicle V and a single second camera C2 disposed on the trailer T. It will be appreciated that more than one rear-facing camera could be provided on the vehicle V and/or the trailer T. By providing more than one camera, the composite image 1M03 may provide a greater sense of depth perception.
The rear-view display system 1 could be configured to overlay the first image IMG1 onto the second image 1M02 only when the vehicle is parked or during a parking operation, for example when the transmission of the vehicle V is placed in reverse.
Furthermore, rather than overlaying the first image onto the second image, a graphical representation of the trailer (for example a silhouette or an outline of the trailer T, or a graphical indicia) could be overlaid onto the second image. Alternatively, a mask of the trailer I could be generated based on a static image generated by the first camera Cl and overlaid onto the second image generated by the second camera C2.
The rear-view display system 1 has been described with reference to a first camera Cl mounted to the rear of the vehicle V. In an alternate arrangement, the rear-view display system 1 can comprise a first side-mounted camera mounted to a first side of the vehicle and/or a second side-mounted camera mounted to a second side of the vehicle V. The first and second side-mounted cameras would be rearward-facing, for example mounted to the side (wing) mirrors. The first side-mounted camera and/or the second side-mounted camera could be used in conjunction with the vehicle-mounted first camera Cl and/or the trailer-mounted second camera C2. In particular, the image processor 5 can be arranged to combine the first image IMG1 and/or the second image 1M02 with an image generated by the first side-mounted camera and/or the second side-mounted camera. The image processor 5 could utilise the image(s) from the first side-mounted camera and/or the second side-mounted camera to provide a partially transparent overlay on the first image IMG1 and/or the second image IMG2. In this arrangement, the partially transparent overlay can comprise a side view of the vehicle V, for example a view of a rear quarter of the vehicle V. It will be appreciated that this arrangement could be used when the vehicle V is not towing a trailer I, so the second camera C2 could be omitted. The resulting composite image 1M03 could be used during parking or reversing manoeuvres to provide a substantially uninterrupted view of the area behind the vehicle V. By displaying the image from the first side-mounted camera and/or the second side-mounted camera as a partially transparent overlay, the driver can be provided with an indication of the extent of the vehicle V to facilitate judgement of distances and/or position. The composite image IMG3 could be output to a display provided, for example, in the centre console, the instrument cluster, the rear view mirror or a side mirror display.
Further aspects of the present invention are set out in the following sequence of numbered paragraphs: 1. A display system for generating a composite view of a region behind a vehicle towing a trailer, the display system comprising: a first camera for outputting first image data corresponding to a first image, the first camera being configured to be mounted in a rear-facing orientation to the vehicle; a second camera for outputting second image data corresponding to a second image, the second camera being configured to be mounted in a rear-facing orientation to the trailer; an image processor for receiving said first image data and said second image data; wherein the image processor is configured to combine said first image data and said second image data to generate composite image data corresponding to a composite image.
2. A display system as described in paragraph 1, wherein the image processor is configured to combine said first image data and said second image data such that at least a portion of one of said first image and said second image is overlaid onto the other of said first image and said second image.
3. A display system as described in paragraph 2, wherein at least said portion of one of said first image and said second image is overlaid as a semi-transparent image onto the other of said first image and said second image.
4. A display system as described in paragraph 2, wherein the image processor is configured to select the portion of said first image or of said second image to be overlaid in dependence on an image mask.
5 A display system as described in paragraph 4, wherein the image mask is generated in dependence on the first image data.
6. A display system as described in paragraph 1, wherein the image processor is configured to combine said first image data and said second image data such that a portion of said first image or of said second image is substituted with at least a portion of the other of said first image and said second image.
7. A display system as described in paragraph 1, wherein the image processor is configured to analyse said first image data and said second image data to identify one or more features common to both said first image and said second image.
8. A display system as described in paragraph 7, wherein the image processor is configured to match said first image and said second image in dependence on the identified one or more features.
9. A display system as described in paragraph 1, wherein the image processor is configured to perform one or more of the following transforms to said first image and/or said second image: image rotation, scaling, cropping, magnification, skew correction and translation.
10. A display system as described in paragraph 1, wherein the image processor is configured to generate said composite image in dependence on positional data defining the relative position of the first and second cameras.
11. A display system as described in paragraph 1, wherein the second camera comprises a second wireless transmitter for transmitting the second image data to the image processor.
12. A display system as described in paragraph 1, wherein the image processor is configured to augment the composite image with one or more of the following: a graphical representation of the trailer being towed by the vehicle; a graphical representation of a projected path for the trailer during a reversing operation; a vehicle control instruction(s) to guide the trailer during a reversing operation; and proximity information for the trailer.
13. A display system as described in paragraph 1, wherein the image processor is configured to output the composite image data for display on a display screen.
14. A rig comprising a vehicle and a trailer, wherein the rig comprises a display system as claimed in any one of the preceding claims, the first camera being mounted to the vehicle and the second camera being mounted to the trailer.
15. A display system for providing a view of a region behind a vehicle towing a trailer comprising: a camera for mounting in a rear-facing orientation to a trailer, the camera being operable to output image data corresponding to an image; an image processor for receiving said image data; wherein the image processor is configured to overlay a graphical representation of the trailer onto the image data to form an augmented image for display.
16. A method of generating a composite image of a region behind a vehicle towing a trailer, the method comprising: using a first camera disposed on the vehicle to generate a first image; using a second camera disposed on the trailer to generate a second image; using an image processor to combine said first and second images to form a composite image.
17. A method as described in paragraph 16, wherein combining said first and second images comprises overlaying at least a portion of one of said first image and said second image onto the other of said first image and said second image.
18. A method as described in paragraph 16, wherein at least said portion of one of said first image and said second image is overlaid as a semi-transparent image onto the other of said first image and said second image.
19. A method as described in paragraph 17, wherein an image mask is used to select the portion of said first image or of said second image to be overlaid.
20. A method as described in paragraph 19, wherein the image mask is generated in dependence on the first image data.
21. A method as described in paragraph 16, wherein combining said first and second images comprises substituting a portion of said first image or of said second image with at least a portion of the other of said first image and said second image.
22. A method as described in paragraph 16 comprising identifying one or more features common to both said first image and said second image.
23. A method as described in paragraph 22 comprising matching said first image and said second image in dependence on the identified one or more features.
24. A method as described in paragraph 16 comprising performing one or more of the following transforms to said first image and/or to said second image: image rotation, scaling, cropping, magnification, skew correction and translation.
25. A method as described in paragraph 16, wherein said composite image is generated in dependence on positional data defining the relative position of the first and second cameras.
26. A method as described in paragraph 16 comprising augmenting the composite image with one or more of the following: a graphical representation of the trailer being towed by the vehicle; a graphical representation of a projected path for the trailer during a reversing operation; a vehicle control instruction(s) to guide the trailer during a reversing operation; and proximity information for the trailer.

Claims (29)

  1. CLAIMS: 1. A display system for generating a composite view of a region behind a vehicle towing a trailer, the display system comprising: a first camera for outputting first image data corresponding to a first image, the first camera being configured to be mounted in a rear-facing orientation to the vehicle; a second camera for outputting second image data corresponding to a second image, the second camera being configured to be mounted in a rear-facing orientation to the trailer; an image processor for receiving said first image data and said second image data; wherein the image processor is configured to combine said first image data and said second image data to generate composite image data corresponding to a composite image.
  2. 2. A display system as claimed in 1, wherein the image processor is configured to combine said first image data and said second image data such that at least a portion of one of said first image and said second image is overlaid onto the other of said first image and said second image.
  3. 3. A display system as claimed in claim 2, wherein at least said portion of one of said first image and said second image is overlaid as a semi-transparent image onto the other of said first image and said second image.
  4. 4. A display system as claimed in claim 2 or claim 3, wherein the image processor is configured to select the portion of said first image or of said second image to be overlaid in dependence on an image mask.
  5. 5. A display system as claimed in claim 4, wherein the image mask is generated in dependence on the first image data.
  6. 6. A display system as claimed in claim 1, wherein the image processor is configured to combine said first image data and said second image data such that a portion of said first image or of said second image is substituted with at least a portion of the other of said first image and said second image.
  7. 7. A display system as claimed in any one of the preceding claims, wherein the image processor is configured to analyse said first image data and said second image data to identify one or more features common to both said first image and said second image.
  8. 8. A display system as claimed in claim 7, wherein the image processor is configured to match said first image and said second image in dependence on the identified one or more features.
  9. 9. A display system as claimed in any one of the preceding claims, wherein the image processor is configured to perform one or more of the following transforms to said first image and/or said second image: image rotation, scaling, cropping, magnification, skew correction and translation.
  10. 10. A display system as claimed in any one of the preceding claims, wherein the image processor is configured to generate said composite image in dependence on positional data defining the relative position of the first and second cameras.
  11. 11. A display system as claimed in any one of the preceding claims, wherein the second camera comprises a wireless transmitter for transmitting the second image data to the image processor.
  12. 12. A display system as claimed in any one of the preceding claims, wherein the image processor is configured to augment the composite image with one or more of the following: a graphical representation of the trailer being towed by the vehicle; a graphical representation of a projected path for the trailer during a reversing operation; a vehicle control instruction(s) to guide the trailer during a reversing operation; and proximity information for the trailer.
  13. 13. A display system as claimed in any one of the preceding claims, wherein the image processor is configured to output the composite image data for display on a display screen.
  14. 14. A rig comprising a vehicle and a trailer, wherein the rig comprises a display system as claimed in any one of the preceding claims, the first camera being mounted to the vehicle and the second camera being mounted to the trailer.
  15. 15. A display system for providing a view of a region behind a vehicle towing a trailer, the display system comprising: a camera for mounting in a rear-facing orientation to a trailer, the camera being operable to output image data corresponding to an image; an image processor for receiving said image data; wherein the image processor is configured to overlay a graphical representation of the trailer onto the image data to form an augmented image for display.
  16. 16. A method of generating a composite image of a region behind a vehicle towing a trailer, the method comprising: using a first camera disposed on the vehicle to generate a first image; using a second camera disposed on the trailer to generate a second image; using an image processor to combine said first and second images to form a composite image.
  17. 17. A method as claimed in claim 16, wherein combining said first and second images comprises overlaying at least a portion of one of said first image and said second image onto the other of said first image and said second image.
  18. 18. A method as claimed in claim 16, wherein at least said portion of one of said first image and said second image is overlaid as a semi-transparent image onto the other of said first image and said second image.
  19. 19. A method as claimed in claim 17 or claim 18, wherein an image mask is used to select the portion of said first image or of said second image to be overlaid.
  20. 20. A method as claimed in claim 19, wherein the image mask is generated in dependence on the first image data.
  21. 21. A method as claimed in claim 16, wherein combining said first and second images comprises substituting a portion of said first image or of said second image with at least a portion of the other of said first image and said second image.
  22. 22. A method as claimed in any one of claims 16 to 21 comprising identifying one or more features common to both said first image and said second image.
  23. 23. A method as claimed in claim 22 comprising matching said first image and said second image in dependence on the identified one or more features.
  24. 24. A method as claimed in any one of claims 16 to 23 comprising performing one or more of the following transforms to said first image and/or to said second image: image rotation, scaling, cropping, magnification, skew correction and translation.
  25. 25. A method as claimed in any one of claims 16 to 24, wherein said composite image is generated in dependence on positional data defining the relative position of the first and second cameras.
  26. 26. A method as claimed in any one of claims 16 to 25 comprising augmenting the composite image with one or more of the following: a graphical representation of the trailer being towed by the vehicle; a graphical representation of a projected path for the trailer during a reversing operation; a vehicle control instruction(s) to guide the trailer during a reversing operation; and proximity information for the trailer.
  27. 27. A processor or control module for a vehicle configured and/or arranged to receive first image data from a first camera disposed in a rear-facing orientation relative to the vehicle and second image data from a second camera disposed in a rear-facing orientation relative to a trailer towed by the vehicle, and to generate a composite image from said first and second image data.
  28. 28. A vehicle having a display system as claimed in any of claims 1-15 or a processor or control module as claimed in claim 27.
  29. 29. A display system, a method or a vehicle and trailer combination substantially as herein described with reference to the accompanying figures.
GB1414632.8A 2014-08-18 2014-08-18 Display system and method Active GB2529408B (en)

Priority Applications (12)

Application Number Priority Date Filing Date Title
GB1414632.8A GB2529408B (en) 2014-08-18 2014-08-18 Display system and method
EP15754161.6A EP3183875B1 (en) 2014-08-18 2015-08-18 Display system and method
EP20168870.2A EP3709635B1 (en) 2014-08-18 2015-08-18 Display system and method
US15/504,821 US10701299B2 (en) 2014-08-18 2015-08-18 Display system and method
GB1514673.1A GB2530649B (en) 2014-08-18 2015-08-18 Display system and method
PCT/EP2015/068970 WO2016026870A1 (en) 2014-08-18 2015-08-18 Display system and method
CN201580044317.8A CN106573577B (en) 2014-08-18 2015-08-18 Display system and method
AU2015306153A AU2015306153B2 (en) 2014-08-18 2015-08-18 Display system and method
ZA2017/01208A ZA201701208B (en) 2014-08-18 2017-02-17 Display system and method
AU2017272148A AU2017272148A1 (en) 2014-08-18 2017-12-04 Display system and method
US15/853,436 US10701300B2 (en) 2014-08-18 2017-12-22 Display system and method
US16/448,515 US11553153B2 (en) 2014-08-18 2019-06-21 Display system and method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
GB1414632.8A GB2529408B (en) 2014-08-18 2014-08-18 Display system and method

Publications (3)

Publication Number Publication Date
GB201414632D0 GB201414632D0 (en) 2014-10-01
GB2529408A true GB2529408A (en) 2016-02-24
GB2529408B GB2529408B (en) 2019-01-30

Family

ID=51662579

Family Applications (1)

Application Number Title Priority Date Filing Date
GB1414632.8A Active GB2529408B (en) 2014-08-18 2014-08-18 Display system and method

Country Status (1)

Country Link
GB (1) GB2529408B (en)

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB2559758A (en) * 2017-02-16 2018-08-22 Jaguar Land Rover Ltd Apparatus and method for displaying information
EP3418159A3 (en) * 2017-06-20 2019-02-13 Valeo Schalter und Sensoren GmbH Method for operating a park assisting device of a motor vehicle with combined view of transparent trailer and environment
GB2569680A (en) * 2017-11-02 2019-06-26 Jaguar Land Rover Ltd Display method and apparatus
GB2580377A (en) * 2019-01-08 2020-07-22 Jaguar Land Rover Ltd Image capture device and system for a vehicle rig
WO2020177970A1 (en) * 2019-03-02 2020-09-10 Jaguar Land Rover Limited Imaging system and method
WO2020187978A1 (en) 2019-03-19 2020-09-24 Jaguar Land Rover Limited Image processing system and method
WO2020212287A1 (en) * 2019-04-19 2020-10-22 Jaguar Land Rover Limited Imaging system and method
EP3736768A1 (en) * 2019-05-07 2020-11-11 Alpine Electronics, Inc. Image processing apparatus, image processing system, image processing method, and program
US10901416B2 (en) * 2018-07-19 2021-01-26 Honda Motor Co., Ltd. Scene creation system for autonomous vehicles and methods thereof
GB2611615A (en) * 2019-03-02 2023-04-12 Jaguar Land Rover Ltd Imaging system and method
EP4373076A1 (en) * 2022-11-16 2024-05-22 Volvo Truck Corporation Activation of vehicle convoy rear cameras

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006256544A (en) * 2005-03-18 2006-09-28 Aisin Seiki Co Ltd Reverse drive supporting system
EP1892150A2 (en) * 2006-08-21 2008-02-27 Sanyo Electric Co., Ltd. Image processor and vehicle surrounding visual field support device
EP2181898A1 (en) * 2007-09-03 2010-05-05 Sanyo Electric Co., Ltd. Driving assistance system and connected vehicles
US20140085472A1 (en) * 2012-09-26 2014-03-27 Magna Electronics Inc. Trailer angle detection system

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006256544A (en) * 2005-03-18 2006-09-28 Aisin Seiki Co Ltd Reverse drive supporting system
EP1892150A2 (en) * 2006-08-21 2008-02-27 Sanyo Electric Co., Ltd. Image processor and vehicle surrounding visual field support device
EP2181898A1 (en) * 2007-09-03 2010-05-05 Sanyo Electric Co., Ltd. Driving assistance system and connected vehicles
US20140085472A1 (en) * 2012-09-26 2014-03-27 Magna Electronics Inc. Trailer angle detection system

Cited By (27)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018149665A1 (en) * 2017-02-16 2018-08-23 Jaguar Land Rover Limited Apparatus and method for displaying information
US20200023772A1 (en) * 2017-02-16 2020-01-23 Jaguar Land Rover Limited Apparatus and method for displaying information
US11420559B2 (en) 2017-02-16 2022-08-23 Jaguar Land Rover Limited Apparatus and method for generating a composite image from images showing adjacent or overlapping regions external to a vehicle
GB2559758B (en) * 2017-02-16 2021-10-27 Jaguar Land Rover Ltd Apparatus and method for displaying information
GB2559758A (en) * 2017-02-16 2018-08-22 Jaguar Land Rover Ltd Apparatus and method for displaying information
EP3418159A3 (en) * 2017-06-20 2019-02-13 Valeo Schalter und Sensoren GmbH Method for operating a park assisting device of a motor vehicle with combined view of transparent trailer and environment
GB2569680A (en) * 2017-11-02 2019-06-26 Jaguar Land Rover Ltd Display method and apparatus
US11027667B2 (en) 2017-11-02 2021-06-08 Jaguar Land Rover Limited Display method and apparatus
GB2569680B (en) * 2017-11-02 2021-04-21 Jaguar Land Rover Ltd Display method and apparatus
US10901416B2 (en) * 2018-07-19 2021-01-26 Honda Motor Co., Ltd. Scene creation system for autonomous vehicles and methods thereof
GB2580377B (en) * 2019-01-08 2021-03-17 Jaguar Land Rover Ltd Image capture device and system for a vehicle rig
GB2580377A (en) * 2019-01-08 2020-07-22 Jaguar Land Rover Ltd Image capture device and system for a vehicle rig
US11833968B2 (en) 2019-03-02 2023-12-05 Jaguar Land Rover Limited Imaging system and method
GB2611615A (en) * 2019-03-02 2023-04-12 Jaguar Land Rover Ltd Imaging system and method
GB2611615B (en) * 2019-03-02 2023-11-08 Jaguar Land Rover Ltd Imaging system and method
GB2582262B (en) * 2019-03-02 2022-09-28 Jaguar Land Rover Ltd Imaging system and method
CN113508574A (en) * 2019-03-02 2021-10-15 捷豹路虎有限公司 Imaging system and method
WO2020177970A1 (en) * 2019-03-02 2020-09-10 Jaguar Land Rover Limited Imaging system and method
WO2020187978A1 (en) 2019-03-19 2020-09-24 Jaguar Land Rover Limited Image processing system and method
US11993208B2 (en) 2019-03-19 2024-05-28 Jaguar Land Rover Limited Image processing system and method
WO2020212287A1 (en) * 2019-04-19 2020-10-22 Jaguar Land Rover Limited Imaging system and method
GB2583704A (en) * 2019-04-19 2020-11-11 Jaguar Land Rover Ltd Imaging system and method
GB2583704B (en) * 2019-04-19 2023-05-24 Jaguar Land Rover Ltd Imaging system and method
US12088931B2 (en) 2019-04-19 2024-09-10 Jaguar Land Rover Limited Imaging system and method
US11132817B2 (en) 2019-05-07 2021-09-28 Alpine Electronics, Inc. Image processing apparatus, image processing system, and image processing method for generating a vehicle see-through rear view image
EP3736768A1 (en) * 2019-05-07 2020-11-11 Alpine Electronics, Inc. Image processing apparatus, image processing system, image processing method, and program
EP4373076A1 (en) * 2022-11-16 2024-05-22 Volvo Truck Corporation Activation of vehicle convoy rear cameras

Also Published As

Publication number Publication date
GB2529408B (en) 2019-01-30
GB201414632D0 (en) 2014-10-01

Similar Documents

Publication Publication Date Title
US11553153B2 (en) Display system and method
GB2529408A (en) Display system and method
JP7010221B2 (en) Image generator, image generation method, and program
WO2017037267A1 (en) Display system and method
JP6145497B2 (en) Visual system
JP5132249B2 (en) In-vehicle imaging device
JP7247173B2 (en) Image processing method and apparatus
US11027667B2 (en) Display method and apparatus
US11833968B2 (en) Imaging system and method
CN113614779A (en) Image processing system and method
GB2548598B (en) Display system and method
JP2016136326A (en) Information display apparatus and information display method
GB2611615A (en) Imaging system and method
GB2597866A (en) Image processing method and apparatus