GB2580377A

GB2580377A - Image capture device and system for a vehicle rig

Info

Publication number: GB2580377A
Application number: GB1900218.7A
Authority: GB
Inventors: Freeman-Powell Aaron
Original assignee: Jaguar Land Rover Ltd
Current assignee: Jaguar Land Rover Ltd
Priority date: 2019-01-08
Filing date: 2019-01-08
Publication date: 2020-07-22
Anticipated expiration: 2039-01-08
Also published as: GB2580377B

Abstract

A trailer image capture device 104 is attachable to a trailer 220 and is configured to capture image data from the vicinity of the trailer 108 and comprise: a measurement unit (114, Fig.1) which identifies an orientation of the trailer image capture device and generates a device orientation signal, and a transmitter which transmits the trailer image data. In a first embodiment trailer camera ceases transmission of the trailer image data dependent on the device orientation signal. In a second embodiment the trailer image capture device is part of a system further comprising a vehicle image capture device 106 attachable to a vehicle 210 which is configured to capture image data from the vicinity of the vehicle 110 and transmits the vehicle image data; the transmitter further transmits the device orientation signal; a processor 102 receives the transmitted trailer image data, device orientation signal, and vehicle image data and selectively combines that trailer image data and vehicle data dependent on the device orientation signal. The trailer image data may only provided when it is helpful to a driver of the vehicle; when the trailer door/ramp is open the trailer image data may not be incorporated.

Description

Intellectual Property Office Application No. GII1900218.7 RTM Date:5 July 2019 The following terms are registered trade marks and should be read as such wherever they occur in this document: Wi-Fi Intellectual Property Office is an operating name of the Patent Office www.gov.uk/ipo Image Capture Device and System for a Vehicle Rig

TECHNICAL FIELD

The present disclosure relates to a device, system and method for a vehicle rig and particularly, but not exclusively, to an image capture device, an imaging system and a method for selectively combining image data for use in providing a user of a vehicle with a view from a rear of a trailer coupled to the vehicle. Aspects of the invention relate to a device, a system, a vehicle rig, a method, and to a non-transitory computer-readable medium.

BACKGROUND

It is common practice to tow a trailer, such as a caravan or a horsebox, behind a vehicle. However, the towed trailer can at least partially obscure the driver's view behind the vehicle; in some instances, a blind spot can be created behind the trailer, preventing the driver from seeing behind the vehicle at all. The resulting lack of visibility can, for example, compromise manoeuvres such as lane changes, overtaking, parking, and pulling out of junctions.

It is known to provide a camera on a rear of a trailer being towed by a vehicle, in order to provide a driver of a vehicle with a composite view of a region behind the trailer. Such an arrangement is described in published UK patent application GB2530649, for example.

However, in operation, rear cameras may not always capture the required field of view. It is an aim of the present invention to address one or more of the disadvantages of the trailer

cameras associated with the prior art.

SUMMARY OF THE INVENTION

Aspects and embodiments of the invention provide an image capture device attachable to a trailer coupled to a vehicle, an imaging system, a vehicle rig comprising an imaging system, a method for selectively combining image data, and a non-transitory computer-readable medium for causing performance of a method for selectively combining data, as claimed in the appended claims.

According to an aspect of the present invention, there is provided a trailer image capture device attachable to a trailer of a vehicle rig and configured to capture image data from the vicinity of the trailer. The trailer image capture device comprises: a measurement unit configured to identify an orientation of the trailer image capture device and generate a device orientation signal in dependence on the identified orientation of the trailer image capture device; and a transmitter configured to transmit the image data to a further vehicle system. The trailer image capture device is configured to cease transmission of the image data in dependence on the device orientation signal.

According to an aspect of the present invention, there is provided an imaging system. The imaging system comprises a trailer image capture device attachable to a trailer coupled to a vehicle, the trailer image capture device configured to capture trailer image data from the vicinity of the trailer and comprising: a measurement unit configured to identify an orientation of the trailer image capture device and generate a device orientation signal in dependence on the identified orientation of the trailer image capture device, and a transmitter configured to transmit the device orientation signal and the trailer image data. The imaging system also comprises: a vehicle image capture device attachable to the vehicle, the vehicle image capture device arranged to capture vehicle image data from the vicinity of the vehicle and transmit the vehicle image data; and a processor communicatively coupled to the trailer image capture device and the vehicle image capture device. The processor is configured to: receive the transmitted vehicle image data; receive the transmitted trailer image data and device orientation signal; and selectively combine the trailer image data and the vehicle image data in dependence on the device orientation signal to form combined image data for use in generating an image for output to a display device. The trailer image capture device may be configured to cease transmission of the trailer image data in dependence on the device orientation signal.

According to an aspect of the present invention, there is provided a vehicle rig comprising any aspect or embodiment of the imaging system or the trailer image capture device.

According to an aspect of the present invention, there is provided a method for selectively combining image data. The method comprises: capturing, with a trailer image capture device attached to a trailer coupled to a vehicle, trailer image data from the vicinity of the trailer; identifying an orientation of the trailer image capture device; generating a device orientation signal in dependence on the identified orientation of the trailer image capture device; capturing, with a vehicle image capture device attached to the vehicle, vehicle image data from the vicinity of the vehicle; transmitting the trailer image data, the device orientation signal, and the vehicle image data; receiving, at a processor associated with the vehicle, the trailer image data, the device orientation signal, and the vehicle image data; and selectively combining, at the processor, the trailer image data into the vehicle image data in dependence on the device orientation signal to form a combined image data for use in generating an image for output to a display device.

According to an aspect of the present invention, there is provided a non-transitory computer readable medium comprising computer readable instructions that, when executed by a processor, causes performance of any aspect or embodiment of the method.

The trailer image capture device may be located on a door or a ramp of the trailer. Such a device may, for example, be retrofit or removably mounted by a customer, and may therefore not be at an optimum position for combination with the vehicle image data. It may therefore be useful to know the orientation (for example, the angle of roll, pitch and/or yaw) of the trailer image capture device in order to correct the captured trailer image data for future use. However, when the door or ramp is opened, for example, the orientation of the trailer image capture device can change such that the required field of view is no longer being captured by the trailer image capture device. Instead, the trailer image capture device may be orientated towards the ground, or towards a side of the trailer, and a dark area may therefore be present in the composite image. In such circumstances, instead of correcting the trailer image data in dependence on the orientation before combining it into the vehicle image data, the trailer image data can be disregarded and not combined with the vehicle image data. By selectively combining the trailer image data and the vehicle image data in dependence on the device orientation signal, instances of display of a confusing or misleading combined (or composite) image can therefore be reduced. The utility and performance of the system may therefore be improved.

Optionally, the device orientation signal is indicative of a change in orientation of the trailer image capture device. The method may comprise, or the processor may be configured to, combine the trailer image data in dependence on a determination that the change in orientation of the trailer image capture device satisfies a predetermined criterion. Optionally, the device orientation signal is indicative of a rate of change of orientation of the trailer image capture device. The method may comprise, or the processor may be configured to, combine the trailer image data in dependence on a determination that the rate of change of orientation of the trailer image capture device satisfies a predetermined criterion.

For example, a change in orientation of the trailer image capture device may indicate that the door or ramp has opened, since a change in angle of the trailer image capture device has exceeded a predetermined threshold. Additionally or alternatively, a rate of change in orientation of the trailer image capture device may indicate that the door or ramp has opened, since the change in angle of the trailer image capture device with time has exceeded a predetermined threshold. The trailer and vehicle image data may therefore be combined so long as the change in orientation, or rate of change of orientation, is below a threshold indicative of a door or ramp of the trailer opening. By using such criteria for selectively combining the trailer image data and the vehicle image data, trailer image data which is not of the required view can be disregarded and not presented to a user as part of a combined or composite image. The utility and performance of the system may therefore be improved.

Optionally, the trailer image capture device comprises its own power source. For example, the trailer image capture device may be battery powered. When loading and/or unloading the trailer, for example, when the door or ramp of the trailer is opened, the trailer image capture device may be drawing full power in order to capture the trailer image data, which data is being disregard and not combined with the vehicle image data to form combined image data. Power may therefore be wasted. The method may comprise, or the trailer image capture device may be configured to, cease transmission in dependence on a determination that the change in orientation satisfies a predetermined criterion. The method may comprise, or the trailer image capture device may be configured to, cease transmission in dependence on a determination that the rate of change of orientation of the image capture device satisfies a predetermined criterion. For example, transmission may cease if the change, or the rate of change, in orientation exceeds a threshold. By ceasing transmission of the trailer image data in dependence on the device orientation signal, battery life of the trailer image capture device may therefore be improved.

Optionally, the method comprises, or the processor is configured to: receive a vehicle orientation signal indicative of an orientation of the vehicle; compare the received device orientation signal and the received vehicle orientation signal to determine a difference in the orientation of the trailer image capture device relative to the vehicle; and combine the trailer image data and the vehicle image data in dependence on a determination that the difference in the orientation satisfies a predetermined criterion. For example, the trailer and vehicle image data may be combined so long as the difference in the orientation is below a threshold. Optionally, the method comprises, or the processor is configured to: receive a vehicle speed signal indicative of a speed of the vehicle; and combine the trailer image data and the vehicle image data in dependence on a determination that the speed of the vehicle satisfies a predetermined criterion. For example, the trailer and vehicle image data may be combined so long as the speed of the vehicle is above a threshold.

By considering the speed of the vehicle in the determination of whether or not to combine the vehicle and trailer image data, it can be determined whether or not it is plausible that a door or ramp of the trailer has been opened. This may provide a secondary check and improve resilience of the system. The orientation of the vehicle may be used to correct the captured trailer image data before combining it into the vehicle image data, but can also be used as a reference point for the device orientation signal; for example, the change of orientation of the trailer image capture device with respect to the vehicle can be considered in determining whether or not to combine the vehicle and trailer image data By using such criteria for selectively combining the trailer image data and the vehicle image data, trailer image data which is not of the required view can be disregarded and not presented to a user as part of a combined or composite image. The utility and performance of the system may therefore be improved.

Optionally, the method comprises, or the processor is configured to: generate a standby signal in dependence on a determination that the received device orientation signal satisfies a first predetermined criterion; and transmit the standby signal to the trailer image capture device, wherein the trailer image capture device comprises a receiver configured to receive the standby signal, the standby signal configured to cause the trailer image capture device to enter a standby state in which no trailer image data is transmitted. Optionally, in the standby state, no trailer image data is captured. Optionally, the method comprises, or the processor is configured to: generate a wake signal in dependence on a determination that the received device orientation signal satisfies a second predetermined criterion, and transmit the wake signal to the trailer image capture device; wherein the receiver of the trailer image capture device is configured to receive the wake signal, the wake signal configured to cause the image capture device to exit the standby state.

By entering and exiting the trailer image capture device into and out of a standby state in dependence on the device orientation signal satisfying predetermined criteria, the power drawn by the trailer image capture device can be reduced, since the trailer image capture device can be prevented from transmitting (and optionally from capturing) trailer image data in circumstances where said data is to be disregarded by the processor. Power usage may therefore be reduced, and thus battery lifetime of the trailer image capture device may be improved.

Optionally, the trailer and/or vehicle image capture device(s) comprise a camera. Optionally, the trailer and/or vehicle image capture device(s) comprise attachment means for attaching the respective image capture devices, which attachment means may be a clamp or a screw plate and one or more screws. Optionally, the measurement unit comprises an inertial measurement unit, i.e. one or more gyroscopes and one or more accelerometers. Optionally, the transmitter and/or receiver comprise one or more antenna.

Optionally, the processor comprises one or more electronic processors having an electrical input for receiving the device orientation signal, the trailer image data and the vehicle image data, and an electronic memory device electrically coupled to the electronic processor and having instructions stored therein, wherein the processor being configured to perform the method of any aspect or embodiment comprises the one or more electronic processors being configured to access the memory device and execute the instructions stored therein such that it is operable to perform said method.

Optionally, the vehicle rig comprises a vehicle and a trailer, wherein the trailer image capture device is coupled to the trailer and the vehicle image capture device is coupled to a rear of the vehicle.

Optionally, the vehicle comprises a display device, for example an electronic display, wherein the method comprises, or the processor is configured to, generate an image in dependence on the combined image data and to output the image, the display device configured to render the image output by the processor for display (or to display the image).

Image data captured from a camera coupled to a trailer of a vehicle rig is combined with image data captured from a vicinity of a vehicle of the vehicle rig to form a composite image of the surroundings. The combination of the two different types of image data is performed selectively, in dependence on an orientation of the camera coupled to the trailer, to help ensure that image data from the surroundings of the trailer is only provided in situations where it is helpful to a driver of the vehicle. In situations where the orientation of the camera is such that the image data could be confusing or misleading, the image data from the surroundings of the trailer is not incorporated into the image data from the vicinity of the vehicle.

Within the scope of this application it is expressly intended that the various aspects, embodiments, examples and alternatives set out in the preceding paragraphs, in the claims and/or in the following description and drawings, and in particular the individual features thereof, may be taken independently or in any combination. That is, all embodiments and/or features of any embodiment can be combined in any way and/or combination, unless such features are incompatible. For example, the processor can be configured to selectively combine the trailer image data and the vehicle image data to form combined image data in dependence on the device orientation signal alone, or on any suitable combination of the device orientation signal and any of the above described criteria, or any other criteria not described herein. The applicant reserves the right to change any originally filed claim or file any new claim accordingly, including the right to amend any originally filed claim to depend from and/or incorporate any feature of any other claim although not originally claimed in that manner.

BRIEF DESCRIPTION OF THE DRAWINGS

One or more embodiments of the invention will now be described, by way of example only, with reference to the accompanying drawings, in which: Figure 1 shows a block diagram of an imaging system in accordance with an embodiment of the invention; Figure 2 shows a block diagram of a vehicle rig, the rig comprising a vehicle and a trailer, where the imaging system of Figure 1 is installed in the rig; Figure 3 provides schematic illustrations of trailers -Figure 3A shows an example trailer with a door, and Figure 3B shows an example trailer with a ramp; Figure 4 shows a schematic representation of an imaging system in accordance with an embodiment of the invention; Figure 5 shows a flow chart of a method in accordance with an embodiment of the invention; 35 and Figure 6 shows a vehicle in accordance with an embodiment of the invention.

DETAILED DESCRIPTION

S

An imaging system 100 in accordance with an embodiment of the present invention is described herein with reference to the accompanying Figures 1 and 2.

With reference to Figure 1, the (imaging) system 100 comprises a processor 102, a trailer image capture device 104 attachable to a trailer, and a vehicle image capture device 106 attachable to a vehicle. Processor 102 is communicatively coupled to the trailer image capture device 104 and the vehicle image capture device 106 through any suitable wired or wireless connection. In the group of embodiments described herein, communication between the trailer image capture device 104 and the processor 102 is over a wireless connecting using a suitable wireless communication standard, such as Wi-Fi, or using a suitable proprietary wireless communication technology. Communication between the vehicle image capture device 106 and the processor 102 may be over a wireless connection using a suitable wireless communication standard, such as Wi-Fi, or using a suitable proprietary wireless communication technology, or may be over a wired connection such as Ethernet or LAN.

Each of trailer image capture device 104 and vehicle image capture device 106 may be a camera or a video camera, or any other device configured to substantially continuously capture image data. In particular, devices (or cameras) 104, 106 may be digital cameras or video recorder cameras configured to generate video images of the surroundings of a vehicle which hosts system 100. Image data captured by the cameras 104, 106 may be individual frames of a video or an entire segment of video, as appropriate. Trailer image capture device, or camera, 104 and vehicle image capture device, or camera, 106 need not be the same type of device.

The trailer camera 104 comprises a measurement unit 114 configured to identify an orientation of the trailer camera. The measurement unit 114 of this embodiment comprises an accelerometer and a gyroscope to determine the roll, pitch and yaw of the trailer camera 104. The measurement unit 114 generates a device orientation signal in dependence on the identified orientation of the trailer image capture device. The device orientation signal can indicate one or more angular orientations of the trailer camera 104 (i.e., an angle of roll, pitch and/or yaw), a rate of change of one or more of the angular orientations of the trailer camera 104, an acceleration in one or more axis (optionally, in three axis) of the trailer camera 104, and/or or any other suitable device orientation information which can be measured by the measurement unit 114 (which may include one or more other sensors, as appropriate). The trailer camera 104 also comprises a wireless transceiver 116 comprising a transmitter having an antenna and a wireless receiver having an antenna. The transmitter is configured to wirelessly transmit the device orientation signal and any image data captured by the trailer camera 104 to the processor 102 or a further vehicle system, as required.

Imaging system 100 is intended for use with a vehicle rig comprising a vehicle and a trailer coupled to the vehicle. With reference to Figure 2, the system 100 is provided as part of rig 200, the rig comprising a host vehicle 210 arranged to tow a trailer 220. The vehicle 210 in the present embodiment is an automobile or a utility vehicle. However, it will be appreciated that the imaging system 100 may be incorporated into other types of vehicle, as required.

The terms "front" and "rear" are used herein in their conventional sense when defining the relative position of features on the vehicle 210 and the trailer 220. The terms "rear-facing" and "rear-view" are used herein to refer to a position or orientation which is in a direction towards the back of the vehicle 210 or the trailer 220.

As shown in Figure 2, the vehicle camera 106 of system 100 is provided as a vehicle-mounted device mounted to vehicle 210, and the trailer camera 104 of system 100 is provided as a trailer-mounted device mounted to trailer 220. The processor 102 is provided within the vehicle 220.

Once trailer camera 104 is coupled or mounted to trailer 220, the measurement unit 114 can be arranged to transmit calibration data to the processor 102 indicative of a position or orientation of the trailer camera 104 when installed correctly on the trailer. Collection of this calibration data can be initiated or instigated by a driver from vehicle 210, for example, using processor 102 which may be accessed or controlled via a graphical user interface (GUI).

The vehicle 210 comprises a display screen 230 (which may be a liquid-crystal display, an LED, or Organic LED display), on which image data received from the respective image capture devices 104, 106 and processed by processor 102 can be selectively displayed. The display screen 230 could, for example, be disposed in a centre console, an instrument cluster, or a rear-view mirror of the vehicle (for example to display an image alongside a reflective image). Alternatively, the processor 102 can be configured to output the combined image data wirelessly, for example to be displayed on a network device, such as a cellular telephone or a so-called "smart-phone", a personal computer or a tablet computer.

The vehicle 210 in this embodiment also comprises a measurement unit 118, which may be associated with the vehicle camera 106 or with another vehicle system. The measurement unit 118 may comprise a plurality of different sensors distributed around vehicle 210, for example, one or more cameras, accelerometers, or gyroscopes. The vehicle measurement unit 118 is communicatively coupled to the processor 102 and configured to provide an indication of one or more of an angular orientation of the vehicle 210, a rate of change of orientation of the vehicle 210, or a speed or an acceleration of the vehicle 210. The measurement unit 118 can also comprise a sensor suitable for providing processor 102 with data indicative of a hitch angle, that is, an angle between the vehicle 210 and the trailer 220.

The vehicle camera 106 is preferably mounted centrally at the rear of the vehicle 210, optionally above a rear license plate (not shown). The trailer camera 104 is mounted at a rear of the trailer 220, preferably mounted centrally on the trailer. It will be understood that, depending on the geometry of the vehicle 210 and/or trailer 220 a central positioning is not always possible; the vehicle camera 106 and trailer camera 104 are therefore preferably mounted substantially centrally. Substantially centrally as used herein comprises centrally, or within a small offset from a centre (for example, of up to a few hundred millimetres, optionally of up to 200 mm). Both cameras 104, 106 are rear-ward facing devices, and each camera comprises a wide-angle lens to provide a wide respective viewing angle, or field of view, 108, 110 (the extent of which is denoted by the solid lines in Figure 2). Images of the surroundings of the rig 200 within fields of view 108, 110 are captured by the respective image capture devices, or cameras, 104, 106 and the captured image data provided to processor 102.

In particular, the vehicle camera 106 is arranged to capture images from field of view 110, which can be provided to a driver of the vehicle 210. Vehicle camera 106 can selectively function as a reversing camera to provide a parking aid when there is no trailer 220 coupled to the vehicle 220. However, when trailer 220 is coupled to the vehicle 210 to form rig 200, portion 112 of viewing angle 110 is occluded by trailer 220 (the extent of portion 112 is denoted by the dashed lines in Figure 2). In particular, the vehicle image data captured by vehicle camera 106 corresponds to field of view 110 and includes image(s) of trailer 220 being towed behind the vehicle 210, thereby obstructing the view of the environment behind, or to the rear of, the trailer. To overcome this issue, image data captured by trailer camera 104 from field of view 108 can be combined into, or with, image data from vehicle camera 106 to provide a user of vehicle 210 with an image of the vehicle surroundings in field of view 112.

In particular, the trailer camera 104 captures trailer image data corresponding to an unobstructed view of the region behind the trailer 220 within field of view 108. Processor 102 receives the image data from each of cameras 104, 106 and implements an image matching procedure to match (align) the respective image data; this alignment can be based on one or more of the device orientation signal from the measurement unit 114, the trailer image capture device 104 calibration data, and data from the vehicle measurement unit 118. The processor then combines the aligned image data to form composite image data comprising the vehicle image data overlaid onto the trailer image data. The trailer 220 can appear as a ghost or semi-transparent image in the composite image which is generated in dependence on the composite image data. This can provide a clear view of the region behind the trailer 220, whilst retaining sufficient details of the trailer 220 to enable the driver to determine its relative position to obstacles and other vehicles. The composite image is output for display on display screen 230 provided in the vehicle.

In order to ensure the vehicle and trailer images can be properly aligned and facilitate provision of the composite image, the trailer camera 104 is required to be mounted substantially centrally (both vertically and horizontally) at a rear of the trailer 220. For example, the trailer camera 104 may need to be mounted horizontally centrally on the rear of the trailer, at an approximate height of 500 mm above the ground. As a result, the trailer camera 104 may need to be mounted or located on an opening door or a ramp of the trailer 220. This positioning of the trailer camera 104 can lead to operational issues when the door or ramp is opened, as described with reference to accompanying Figure 3.

With reference to Figure 3A, when a door 340 of trailer 220 is opened, the trailer camera 104 mounted on the door 340 is no longer rear-ward facing, but instead may be forward facing (i.e. facing towards the vehicle towing the trailer 220), or may be facing at least part of the trailer 220 itself. In such circumstances, it may be advantageous to disregard the trailer image data and not combine it with the captured vehicle image data. Moreover, in such circumstances it may be advantageous to cease transmission and/or capturing of the trailer image data in the first place in order to save power and increase a battery lifetime of the trailer camera 104. In order to determine whether or not to combine the trailer and vehicle image data, and/or whether or not to cease transmission of the trailer image data from the trailer image capture device or camera 104, the device orientation signal generated by the measurement unit 114 of the camera 104 is used. In other words, the trailer and vehicle image data is selectively combined in dependence on the device orientation signal. This can improve utility of the system, since in situations where the trailer image data does not provide useful information, it can be disregarded and not combined into the vehicle image data.

In some embodiments, the device orientation signal can indicate that an angular change in the pitch angle (angular change around the x-axis of axes 350a) of the trailer camera 104, in the yaw angle (angular change around the y-axis of axes 350a) of the trailer camera 104, or in the roll angle (angular change around the z-axis of axes 350a, orientated into the page in Figure 3A and towards a front of the trailer) of the trailer camera 104 satisfies a predetermined criterion. Additionally and/or alternatively, the device orientation signal can indicate that a rate of change in one or more of the pitch, yaw or roll angles satisfies a predetermined criterion.

With reference to Figure 3B, trailer 220 comprises a rear opening ramp 360 (here shown in an open position); trailer 220 may be a horsebox, vehicle transporter, or box trailer, for example. When ramp 360 of trailer 220 is opened, the trailer camera 104 mounted on the ramp 360 is no longer rear-ward facing, but instead may be facing the ground or surface on which the trailer 220 rests. In such circumstances, it may be advantageous to disregard the trailer image data and not combine it with the captured vehicle image data. Moreover, in such circumstances it may be advantageous to cease transmission and/or capturing of the trailer image data in the first place in order to save power and increase a battery lifetime of the trailer camera 104. In order to determine whether or not to combine the trailer and vehicle image data, and/or whether or not to cease transmission of the trailer image data from the trailer image capture device or camera 104, the device orientation signal generated by the measurement unit 114 of the camera 104 is used.

In some embodiments, the device orientation signal can indicate that an angular change in the pitch angle (angular change around the x-axis of axes 350b, orientated into the page in Figure 3B and towards a front of the trailer) of the trailer camera 104, in the yaw angle (angular change around the y-axis of axes 350b) of the trailer camera 104, or in the roll angle (angular change around the z-axis of axes 350b) of the trailer camera 104 satisfies a predetermined criterion. Additionally and/or alternatively, the device orientation signal can indicate that a rate of change in one or more of the pitch, yaw or roll angles satisfies a predetermined criterion.

When trailer 220 comprises the ramp 360, opening of the ramp 360 causes a rotation in the ramp, and thus a rotation in the trailer camera 104 removably coupled to the ramp 360, around the x-axis; a change in pitch angle of the trailer camera 104 can thus be measured by the measurement unit 114. Such a change in the pitch angle is unlikely in normal driving or towing operations of trailer 220, so can be indicative of an opening of the ramp 360. If the change in pitch angle exceeds a predetermined threshold, the processor 102 can stop combining the trailer image data with the vehicle image data. For example, if the trailer camera 104 is pitched through an angle of greater than or equal to 80 degrees (optionally, greater than or equal to 50, 60, 70, 80, 90, 110, or 120 degrees) to a trailer ramp open position, the processor may inhibit use of the trailer image data captured by the trailer camera 104. Any image generated by the processor 102, or by another vehicle system, for output to display 230 may therefore comprise only the vehicle image data, with an opaque trailer image being presented to a driver in place of the trailer image data. When the change in pitch angle does not exceed the predetermined threshold (or otherwise satisfies a predetermined criterion), the processor combines the trailer image data into the vehicle image data to form the combined image data.

Similarly, system 100 is also arranged to detect and act on movement of the trailer camera 104 in the opposite direction; when a change in trailer camera pitch angle indicative of a change from the trailer ramp open position to a trailer ramp closed position is detected, and a predetermined criterion satisfied, the processor 102 can use the trailer image data to provide a combined or composite image to a driver in which the trailer image data is incorporated into the vehicle image data.

When trailer 220 comprises door 340, opening of the door 340 causes a rotation in the door, and thus in the trailer camera 104 removably coupled to the door 340, around the y-axis; a change in yaw angle of the trailer camera 104 can thus be measured by the measurement unit 114. In order to distinguish from turning of the rig 200, the rate of change of yaw angle of the trailer camera 104 may be considered. If the rate in change in yaw angle exceeds a predetermined threshold, the processor 102 can stop combining the trailer image data with the vehicle image data. For example, if the trailer camera 104 experiences a rate of change in yaw angle of more than 50 degrees/second (optionally, greater than or equal to 40 degrees/second, 50 degrees/second, 60 degrees/second, 70 degrees/second, or 80 degrees/second), the processor may inhibit use of the trailer image data captured by the trailer camera 104. This is because such rates of change in the orientation of the trailer camera 104 are indicative of an opening of the door 340, rather than indicative of one or more turning manoeuvres of the rig 200 which would be expected to occur in normal driving or towing operations of trailer 220. When rate of change of orientation does not exceed the predetermined threshold (or otherwise satisfies a predetermined criterion), the processor combines the trailer image data into the vehicle image data to form the combined image data.

By selectively combining the trailer image data into the vehicle image data in the manner described above, confusing and misleading composite images may be prevented. In particular, when the ramp or door was opened the trailer camera 104 would not be capturing the required field of view (i.e. the road or area around the trailer 220) and the resulting composite image would display a portion of another scene from a different field of view within the vehicle image data. For example, in the case of a horsebox or other trailer with a ramp, the trailer camera 104 is likely to be looking at the ground and will be in a dark environment (from the shadow of the ramp); similarly, in the case of trailer with opening doors, the trailer camera 104 is likely to be looking in front of the vehicle or at the trailer itself. Without selectively combining the vehicle and image data, a dark area or an erroneous field of view would be present in the composite image presented to a driver. This occurrence of this may be reduced with the present system.

In some embodiments, system 100 receives calibration data from the measurement unit 114 indicative of the trailer camera being installed correctly (i.e. with the ramp or door of the trailer closed); subsequent use of the device orientation signal can take account of the initial orientation of the trailer camera 104 from the calibration data to detect any change in orientation. This can improve the robustness of the system.

It will be understood that Figures 3A and 3B represent example arrangements of trailer 220 and trailer camera 104, and that any suitable orientation and/or rate of change of orientation of the trailer camera 104 can be used by the processor 102 to selectively combine the trailer image data and the vehicle image data in dependence on the device orientation signal to form combined image data for use in generating an image for output to a display device.

The change, or rate of change, in orientation of the trailer camera 104 determined from the device orientation signal can be verified by comparing, at the processor 102, the device orientation signal from trailer camera 104 with the vehicle orientation, as determined by the vehicle measurement unit 118. In particular, processor 102 is configured to receive a vehicle orientation signal indicative of an orientation of the vehicle from the vehicle measurement unit 118 and compare the device orientation signal received from the trailer camera 104 with the vehicle orientation signal received from the vehicle measurement unit 118 to determine a difference in the orientation of the trailer image capture device relative to the vehicle.

The processor may then combine the trailer image data and the vehicle image data in dependence on a determination that the difference in orientation of the trailer camera 104 relative to the vehicle 210 satisfies a predetermined criterion. For example, if the difference in orientation of the trailer camera 104 relative to the vehicle 210 exceeds a threshold (optionally, if the difference is greater than or equal to 40, 50, 60, 70, 80, 90, 110, or 120 degrees), this may indicate opening of a door or ramp of trailer 220 to which the trailer camera 104 is coupled (and thus that the processor 102 should not combine the vehicle and image data, i.e. that the predetermined criterion is not met). This arrangement may improve the robustness of the system, for example preventing a determination that the ramp 360 has opened when the entire rig is inclined, or that the door 340 has opened when the entire rig is turning at high speed (and so there is a large rate of change of orientation).

Additionally and/or alternatively, the change, or rate of change, in orientation of the trailer camera 104 determined from the device orientation signal can be verified by comparing, at the processor 102, the device orientation signal from trailer camera 104 with the hitch angle, as determined by the vehicle measurement unit 118 and/or any other suitable sensors. In particular, processor 102 is configured to receive the hitch angle between the trailer and vehicle. Where the rotation of the trailer image capture device 104 relative to the hitch angle exceeds a threshold, or where the rotation of the trailer image capture device 104 exceeds a jack-knife angle, this may indicate opening of a door of trailer 220 to which the trailer camera 104 is coupled (and thus that the processor 102 should not combine the vehicle and image data, i.e. that the predetermined criterion is not met). This arrangement may improve the robustness of the system, for example preventing a determination that the door 340 has opened when trailer jack-knifes. Safety of the rig using system 100 may therefore be improved.

Additionally and/or alternatively, the selective combining of the trailer image data with the vehicle image data in dependence on the device orientation signal can be verified by considering a vehicle speed signal indicative of a speed of the vehicle 210. If the vehicle 210 has a speed below a predetermined threshold (for example, a speed less than or equal to 1km/hour, 2km/hour, 3 km/hour, or 4 km/hour), the processor 102 may, in combination with the device orientation, stop combining the trailer image data with the vehicle image data. Any image generated by the processor 102, or by another vehicle system, for output to display 230 may therefore comprise only the vehicle image data, with an opaque trailer image being presented to a driver in place of the trailer image data. However, if the speed of the vehicle is above the predetermined threshold (i.e. if the predetermined criterion is met), this may indicate that the rig 200 is moving and that it is unlikely that the door has been opened or the ramp lowered, for example. Processor 102 may therefore continue to combine the trailer image data and the vehicle image data. This may improve the robustness of the system by, for example, preventing selective combination in dependence on a determination that the ramp 360 or door 340 has opened when the vehicle is in fact moving (and so the likelihood of the door or ramp actually having been opened is small).

As described with reference to accompanying Figure 4, the trailer image capture device or camera 104 is configured to cease transmission of trailer image data, and optionally to cease capture of trailer image data in dependence on the device orientation signal. When system 100 is installed in a rig 200, the trailer image capture device may be controlled by processor 102 to cease transmission and/or capture of the trailer image data based on the same criteria which causes processor 102 to decide whether or not to combine the trailer image data into the vehicle image data. The image capture device may cease transmission (and optionally capture) of the trailer image data independently of the processor 102, i.e. in direct response to the device orientation signal from the measurement unit for example, or may be controlled by processor 102 to cease transmission (and optionally capture), as described below.

In some embodiments, transceiver 116 of trailer image capture device or camera 104 is configured to output a signal 470 comprising the device orientation signal 470b and the trailer image data, which signal 470 is received at vehicle 210 by a receiver and provided to the processor 102 for processing. The processor 102 is configured to generate a standby signal 472 in dependence on a determination that the device orientation signal received in signal 470 satisfies a first predetermined criterion. The first predetermined criterion may be any suitable criterion or any combination of criteria described above, as required by the intended application of system 100, and may be based on (optionally the reverse of) the predetermined criterion used to determine whether or not to combine the trailer image data with the vehicle image data. For example, the first predetermined criterion may comprise the change in orientation or rate of change of orientation exceeding a threshold and/or the vehicle speed being below a threshold.

The standby signal 472 is transmitted from the vehicle 210 to the receiver antenna of transceiver 116. Once received at the trailer camera 104, the standby signal is configured to cause the trailer image capture device to enter a standby state in which no trailer image data is transmitted to the vehicle 210. Optionally, in the standby state of the trailer image capture device or camera 104, no trailer image data is captured. In the standby state, the trailer camera 104 is still configured to transmit the device orientation signal 470b to the vehicle 210 without the trailer image data.

The processor 102 continues to receive the device orientation signal 470b from the trailer camera 104 via the receiver of vehicle 210, even when the trailer camera 104 is in the standby state. Processor 102 is configured to generate a wake signal 474 in dependence on a determination that the received device orientation signal satisfies a second predetermined criterion. The second predetermined criterion may be the reverse of the first predetermined criteria; for example, if the first predetermined criterion is to generate a standby signal when the device orientation signal indicates the change in orientation in a positive direction exceeds a threshold, the second predetermined criterion may be to generate a wake signal when the device orientation signal indicates the change in orientation exceeds the same threshold but in a negative direction. Alternatively, any other thresholds or criterion for the first and second predetermined criterion may be used.

The wake signal 474 is transmitted from the vehicle 210 to the receiver antenna of transceiver 116. Once received at the trailer camera 104, the wake signal is configured to cause the trailer image capture device to exit the standby state in which no trailer image data is transmitted to the vehicle 210. Optionally, the trailer camera 104 resumes capturing trailer image data of the surroundings of the trailer 210 once the trailer camera 104 has exited the standby state. The trailer camera 104 then resumes transmitting the trailer image data for selective combination by the processor 102 with the vehicle image data.

When loading or unloading the trailer 220, if the system 100 in the vehicle 220 is active, the battery powered trailer camera 104 will be drawing full operating power, whilst not providing full system functionality, since processor 102 is not combining the trailer image data with the vehicle image data. The trailer camera 104 is therefore put into a low power, or standby, state during the loading/unloading of the trailer 220 to maximise operating duration when the trailer 220 ramp or door is closed and full operation of system 100 is resumed. By entering and exiting the trailer image capture device into and out of a standby state in this way, the power drawn by the trailer image capture device 104 can be reduced, since the trailer image capture device can be prevented from transmitting (and optionally from capturing) trailer image data in circumstances where the trailer image data is to be disregarded. Power usage may be reduced and thus battery lifetime of the trailer image capture device 104 may be improved.

A method in accordance with an embodiment of the present invention is described herein with reference to the accompanying Figure 5. Method 500 is a method for selectively combining image data, optionally using an imaging system 100 of a vehicle rig comprising a vehicle and a trailer coupled to the vehicle, for example, vehicle 210 and trailer 220.

The method 500 comprises identifying 510 an orientation a trailer image capture device 104 attached to a trailer 220 and generating 520 a device orientation signal in dependence on the identified orientation of the trailer image capture device or camera 104 (i.e. generating a device orientation signal which is indicative of an orientation, or of a change of orientation, of the trailer camera 104). The device orientation signal (e.g. signal 470b of Figure 4) is transmitted (operation 530) from the trailer camera 104 to be received, directly or indirectly, by processor 102 installed within vehicle 210.

If the trailer image capture device, or trailer camera, is in a standby state, operations 510 to 530 may be repeated without any further operations occurring. The standby state can be determined in dependence on the device orientation signal. If the camera is not in a standby state, or only optionally comprises a standby state, the method comprises capturing 540, with the trailer image capture device, trailer image data from the vicinity of the trailer 220. The method further comprises transmitting 550 the captured trailer image data from the trailer camera. Operation 540, and optionally operation 550, do not occur in a standby state of the trailer camera; this may facilitate a reduction in power use and thus improve battery life of the trailer camera 104. Optionally, operation 540 may be performed even when the trailer camera is in a standby state, and only operation 550 is performed in dependence on the state of the trailer image capture device or camera 104. Optionally, a standby state of the trailer camera 104 is not considered and both operations 540 and 550 are performed at all times.

Method 500 further comprises capturing 560, with a vehicle image capture device attached to vehicle 210, vehicle image data from the vicinity of the vehicle and transmitting 570 the vehicle image data, directly or indirectly, to processor 102 installed within vehicle 210. At operation 580, the device orientation signal, vehicle image data, and trailer image data (if captured and/or transmitted) are received at the processor 102 of system 100. Receipt of the different data at operation 580 can occur at different times, and the data can be transmitted in different signals, for example. Alternatively, operations 530 and 550 may occur at the same time such that the device orientation signal and captured trailer image data are transmitted at the same time, optionally as part of the same signal.

Method 500 further comprises selectively combining 590, at the processor, the trailer image data into the vehicle image data in dependence on the device orientation signal to form combined image data. The combined image data can be output to a further vehicle system for use in generating an image for output to a display device, such as display 230 of vehicle 210. Alternatively, processor 102 can generate an image for output to a display screen using the combined image data. The display screen could, for example, be disposed in a centre console, an instrument cluster, or a rear-view mirror of the vehicle, or the combined image data may be output wirelessly to another system or electronic device for display.

In situations where a device orientation signal and/or expected trailer image data is not received by processor 102, for example due to a loss of communication signal between transceiver 116 of the trailer camera 104 and the processor 102, the processor will not combine trailer image data into the vehicle image data. This may improve safety of the system, since it can help prevent misleading images being displayed to a driver.

A vehicle in accordance with an embodiment of the present invention is described herein with reference to the accompanying Figure 6. This vehicle may comprise any of the above described features of vehicle 210, and may be arranged to tow a trailer 220 to form a rig 200, as described above. This vehicle comprises imaging system 100 (where the trailer image capture device, or camera, 104 is coupled to trailer 220 to be towed by this vehicle and the vehicle image capture device 106 is coupled to this vehicle), which system may be arranged in such a way so as to implement method 500. In one embodiment, the vehicle comprises a display which is configured to display an image generated in dependence on the combined image data, as described above.

It will be appreciated that various changes and modifications can be made to the present invention without departing from the scope of the present application.

Claims

CLAIMS1. A trailer image capture device attachable to a trailer of a vehicle rig, the trailer image capture device configured to capture image data from the vicinity of the trailer and 5 comprising: a measurement unit configured to identify an orientation of the trailer image capture device and generate a device orientation signal in dependence on the identified orientation of the trailer image capture device; and a transmitter configured to transmit the image data to a further vehicle system, the trailer image capture device configured to cease transmission of the image data in dependence on the device orientation signal.
2. A trailer image capture device according to claim 1, wherein the trailer image capture device comprises a camera.
3. A trailer image capture device according to claim 1 or claim 2, wherein the measurement unit comprises an inertial measurement unit.
4. A trailer image capture device according to any preceding claim, the trailer image capture device comprising attachment means for attaching the trailer image capture device to the trailer.
5. A trailer image capture device according to any preceding claim, wherein the device orientation signal is indicative of a change in orientation of the trailer image capture device and the trailer image capture device is configured to cease transmission in dependence on a determination that the change in orientation of the trailer image capture device satisfies a predetermined criterion.
6. A trailer image capture device according to any preceding claim, wherein the device orientation signal is indicative of a rate of change of orientation of the trailer image capture device and the trailer image capture device is configured to cease transmission in dependence on a determination that the rate of change of orientation of the trailer image capture device satisfies a predetermined criterion.
7. An imaging system comprising: a trailer image capture device attachable to a trailer of a vehicle rig, the trailer image capture device configured to capture trailer image data from the vicinity of the trailer and comprising: a measurement unit configured to identify an orientation of the trailer image capture device and generate a device orientation signal in dependence on the identified orientation of the trailer image capture device, and a transmitter configured to transmit the device orientation signal and the trailer image data; a vehicle image capture device attachable to a vehicle of the vehicle rig, the vehicle image capture device arranged to capture vehicle image data from the vicinity of the vehicle and transmit the vehicle image data; and a processor communicatively coupled to the trailer image capture device and the vehicle image capture device and configured to: receive the transmitted vehicle image data; receive the transmitted trailer image data and device orientation signal; and selectively combine the trailer image data and the vehicle image data in dependence on the device orientation signal to form combined image data for use in generating an image for output to a display device.
8. An imaging system according to claim 7, wherein the device orientation signal is indicative of a change in orientation of the trailer image capture device and the processor is configured to combine the trailer image data in dependence on a determination that the change in orientation of the trailer image capture device satisfies a predetermined criterion.
9. An imaging system according to claim 7 or claim 8, wherein the device orientation signal is indicative of a rate of change of orientation of the trailer image capture device and the processor is configured to combine the trailer image data in dependence on a determination that the rate of change of orientation of the trailer image capture device satisfies a predetermined criterion.
10. An imaging system according to any one of claims 7 to 9, wherein the processor is configured to: receive a vehicle orientation signal indicative of an orientation of the vehicle; compare the received device orientation signal and the received vehicle orientation signal to determine a difference in the orientation of the trailer image capture device relative to the vehicle; and combine the trailer image data and the vehicle image data in dependence on a determination that the difference in the orientation satisfies a predetermined criterion.
11. An imaging system according to any one of claims 7 to 10, wherein the processor is configured to: receive a vehicle speed signal indicative of a speed of the vehicle; and combine the trailer image data and the vehicle image data in dependence on a determination that the speed of the vehicle satisfies a predetermined criterion.
12. An imaging system according to any one of claims 7 to 11, the processor configured to: generate a standby signal in dependence on a determination that the received device orientation signal satisfies a first predetermined criterion; and transmit the standby signal to the trailer image capture device; wherein the trailer image capture device comprises a receiver configured to receive the standby signal, the standby signal configured to cause the trailer image capture device to enter a standby state in which no trailer image data is transmitted.
13. An imaging system according to claim 12, wherein the processor is configured to: generate a wake signal in dependence on a determination that the received device orientation signal satisfies a second predetermined criterion; and transmit the wake signal to the trailer image capture device; wherein the receiver of the trailer image capture device is configured to receive the wake signal, the wake signal configured to cause the trailer image capture device to exit the standby state.
14. An imaging system according to any one of claims 7 to 13, wherein said processor comprises one or more electronic processors having an electrical input for receiving the device orientation signal, the trailer image data and the vehicle image data, and an electronic memory device electrically coupled to the electronic processor and having instructions stored therein, wherein the processor being configured to selectively combine the trailer image data and the vehicle image data comprises the one or more electronic processors being configured to access the memory device and execute the instructions stored therein such that it is operable to perform the operations to selectively combine the received trailer image data and the received vehicle image data in dependence on the device orientation signal.
15. A vehicle rig comprising an imaging system according to any one of claims 7 to 14.
16. A vehicle rig according to claim 15, the vehicle rig further comprising a vehicle and a trailer, wherein the trailer image capture device is coupled to the trailer and the vehicle image capture device is coupled to a rear of the vehicle.
17. A vehicle rig according to claim 15 or claim 16, wherein the processor is configured to generate an image in dependence on the combined image data and to output the image, the vehicle comprising a display device configured to display the image output by the processor.
18. A method for selectively combining image data, the method comprising: capturing, with a trailer image capture device attached to a trailer of a vehicle rig, trailer image data from the vicinity of the trailer; identifying an orientation of the trailer image capture device; generating a device orientation signal in dependence on the identified orientation of the trailer image capture device; capturing, with a vehicle image capture device attached to a vehicle of the vehicle rig, vehicle image data from the vicinity of the vehicle; transmitting the trailer image data, the device orientation signal, and the vehicle image data; receiving, at a processor associated with the vehicle, the trailer image data, the device orientation signal, and the vehicle image data; and selectively combining, at the processor, the trailer image data into the vehicle image data in dependence on the device orientation signal to form combined image data for use in generating an image for output to a display device.
19. A method according to claim 18, the method comprising: determining, in dependence on the device orientation signal, a change in orientation of the trailer image capture device; and combining the trailer image data into the vehicle image data in dependence on a determination that the change in orientation satisfies a predetermined criterion.
20. A method according to claim 18 or claim 19, the method comprising: determining, in dependence on the device orientation signal, a rate of change of orientation of the trailer image capture device; and combining the trailer image data into the vehicle image data in dependence on a determination that the rate of change of orientation satisfies a predetermined criterion.
21. A method according to any one of claims 18 to 20, the method comprising: receiving a vehicle orientation signal indicative of an orientation of the vehicle; comparing the received device orientation signal and the received vehicle orientation signal to determine a difference in the orientation of the trailer image capture device relative to the vehicle; and combining the trailer image data into the vehicle image data in dependence on a determination that the difference in the orientation satisfies a predetermined criterion.
22. A method according to any one of claims 18 to 21, the method comprising: receiving a vehicle speed signal indicative of a speed of the vehicle; and combining the trailer image data into the vehicle image data in dependence on a determination that the speed of the vehicle satisfies a predetermined criterion.
23. A method according to any one of claims 18 to 22, the method comprising: generating, at the processor, a standby signal in dependence on a determination that the received device orientation signal satisfies a first predetermined criterion; transmitting the standby signal; receiving, at the trailer image capture device, the standby signal; and entering, in response to receiving the standby signal, the trailer image capture device into a standby state in which no trailer image data is transmitted.
24. A method according to any one of claims 18 to 23, the method comprising: generating an image in dependence on the combined image data; and outputting the image to a display device for display.
25. A non-transitory computer readable medium comprising computer readable instructions that, when executed by a processor, cause performance of the method of any of claims 18 to 24.