WO2019172117A1

WO2019172117A1 - Sensor system, and image data generating device

Info

Publication number: WO2019172117A1
Application number: PCT/JP2019/008084
Authority: WO
Inventors: 重之渡邉; 裕一綿野
Original assignee: 株式会社小糸製作所
Priority date: 2018-03-05
Filing date: 2019-03-01
Publication date: 2019-09-12
Also published as: JPWO2019172117A1; JP7288895B2

Abstract

A LiDAR sensor unit (11) detects information relating to the exterior of a vehicle, and outputs a signal (S1) corresponding to said information. A signal processing device (12) processes the signal (S1) to generate LiDAR data corresponding to the information. The signal processing device (12) acquires reference height information (H) indicating a reference height determined on the basis of a pitch angle of the vehicle. On the basis of the reference height information (H), the signal processing device (12) generates the LiDAR data using the signal (S1) associated with a region corresponding to the reference height, within a detection zone of the LiDAR sensor unit (11).

Description

Sensor system and image data generation device

The present disclosure relates to a sensor system mounted on a vehicle.

The present disclosure also relates to an image data generation device mounted on a vehicle.

In order to realize a vehicle driving support technology, it is necessary to mount a sensor for detecting information outside the vehicle on the vehicle body. Examples of such a sensor include a LiDAR (Light Detection and Ranging) sensor and a camera (see, for example, Patent Document 1). As vehicle driving support technology becomes more sophisticated, the load on information processing required also increases.

In this specification, “driving support” means a control process that at least partially performs at least one of driving operation (steering operation, acceleration, deceleration), monitoring of driving environment, and backup of driving operation. In other words, this means including partial driving assistance such as a collision damage reducing brake function and a lane keeping assist function to a fully automatic driving operation.

Instead of side mirrors and rearview mirrors, an electronic mirror technology is known in which an image taken by a camera mounted on a vehicle is displayed on a display device installed in the vehicle interior. Patent Document 2 discloses an example of such an electronic mirror technique.

Japanese Patent Application Publication No. 2010-185769 Japanese Patent Application Publication No. 2017-047868

The first problem in the present disclosure is to suppress an increase in load related to signal processing required for driving support of a vehicle.

The second problem in the present disclosure is to maintain the detection accuracy of a plurality of sensors required for driving support of the vehicle.

The third problem in the present disclosure is to provide an electronic mirror technology with improved rear visibility.

One aspect for achieving the first problem is a sensor system mounted on a vehicle,
At least one sensor unit that detects information outside the vehicle and outputs a signal corresponding to the information;
A signal processing device that processes the signal to generate data corresponding to the information;
With
The signal processing device includes:
Obtaining reference height information indicating a reference height determined based on the pitch angle of the vehicle;
Based on the reference height information, the data is generated using the signal associated with the region corresponding to the reference height in the detection range of the sensor unit.

For example, when information outside the vehicle at the reference height is required for driving support, it is preferable that the detection reference direction of the sensor unit in the vertical direction of the vehicle matches the reference height. However, since the detection reference direction of the sensor unit changes in the vertical direction of the vehicle according to the change in the pitch angle of the vehicle, the detection range of the sensor unit may be set to have redundancy in the vertical direction of the vehicle. It is common.

According to the configuration as described above, the signal processing device specifies the region using reference height information separately determined based on the pitch angle of the vehicle. Therefore, it is possible to easily and highly reliably specify a region including information necessary for driving support regardless of the pitch angle of the vehicle. Furthermore, since only the signal associated with the region that is part of the detection range having redundancy is subjected to data generation processing for acquiring external information, it relates to signal processing required for vehicle driving support. Increase in load can be suppressed.

The above sensor system can be configured as follows.
A leveling adjustment mechanism that adjusts the detection reference direction of the sensor unit in the vertical direction of the vehicle based on the reference height information is provided.

According to such a configuration, since the change in the detection reference direction of the sensor unit according to the change in the pitch angle of the vehicle can be suppressed, the change in the position of the region including the information required for driving support can be suppressed. Therefore, it is possible to further suppress the processing load of the signal processing device for specifying the region.

The above sensor system can be configured as follows.
The at least one sensor unit comprises:
A first sensor unit that detects first information outside the vehicle and outputs a first signal corresponding to the first information;
A second sensor unit that detects second information outside the vehicle and outputs a second signal corresponding to the second information;
Contains
The signal processing device processes the first signal to generate first data corresponding to the first information, and processes the second signal to generate second data corresponding to the second information. ,
The signal processing device processes, as the first signal, a signal associated with a region corresponding to the reference height in the detection range of the first sensor unit based on the reference height information. A signal associated with a region corresponding to the reference height in the detection range of the two sensor units is processed as the second signal.

In the above configuration, based on the common reference height information, the generation of the first data based on the first signal output from the first sensor unit and the second signal based on the second signal output from the second sensor unit. Two data are generated. That is, the generated first data and second data both include information associated with the reference height. Therefore, the integrated use of both data for driving assistance becomes easy. Both the first data and the second data are generated using only signals associated with a region that is a part of the detection range of each sensor unit. Therefore, even if both data are used in an integrated manner, an increase in the processing load of the signal processing device can be suppressed.

The above sensor system can be configured as follows.
The signal processing device associates the data with map information.

According to such a configuration, the data and map information generated by the signal processing device can be used for driving support in an integrated manner. As described above, since an increase in signal processing load related to data generation can be suppressed, an increase in processing load in integrated driving support combining the data and map information can also be suppressed as a whole.

In this case, the above sensor system can be configured as follows.
The data is associated with information corresponding to the reference height in the map information.

According to such a configuration, map information can be converted into two-dimensional information, so that the amount of data used for integrated driving support can be reduced. Therefore, it is possible to further suppress an increase in processing load in integrated driving support.

The above sensor system can be configured as follows.
A lamp housing that partitions a lamp chamber that houses the lamp unit;
The sensor unit is disposed in the lamp chamber.

Since the lamp unit has a function of supplying light to the outside of the vehicle, the lamp unit is generally arranged in a place where there is little shielding in the vehicle. By arranging the sensor unit in such a place, information outside the vehicle can be efficiently acquired.

Also, when the height detection information is acquired from the vehicle auto leveling system, the height detection information can be shared with the lamp unit. In this case, efficient system design is possible.

The above sensor system can be configured as follows.
The at least one sensor unit is at least one of a LIDAR sensor unit, a camera unit, and a millimeter wave sensor unit.

While these sensor units are useful for acquiring information outside the vehicle, it is known that the amount of data corresponding to the acquired information is very large. However, as described above, only a signal associated with an area that is a part of the detection range of the sensor unit is subjected to a data generation process for acquiring external information. Therefore, while using these sensor units, it is possible to suppress an increase in load related to signal processing required for driving support of the vehicle.

One aspect for achieving the second problem is a sensor system mounted on a vehicle,
A plurality of sensor units that detect information outside the vehicle and each output a signal corresponding to the information;
A signal processing device for acquiring the signal;
With
The plurality of sensor units are:
A first sensor unit that detects first information outside the vehicle and outputs a first signal corresponding to the first information;
A second sensor unit that detects second information outside the vehicle and outputs a second signal corresponding to the second information;
Contains
The signal processing device includes:
Based on the first signal and the second signal, determine whether the first information and the second information include the same reference target,
When it is determined that the first information and the second information include the same reference target, the position of the reference target specified by the first information and the reference specified by the second information A difference from the position of the target is detected.

* Several sensor units mounted on a vehicle may be displaced due to vibration during driving or passage of time. The occurrence of such misalignment leads to a phenomenon that the position of the identified reference target is different among the plurality of sensor units while the same reference target is detected by the plurality of sensor units. Therefore, by detecting the difference, it is possible to grasp that a positional deviation has occurred in at least one of the plurality of sensor units.

For example, when the magnitude of the difference exceeds a predetermined value, the user can be notified. The user can take appropriate measures to correct the misalignment. Therefore, it is possible to maintain the detection accuracy of the plurality of sensor units required for driving support of the vehicle.

The above sensor system can be configured as follows.
The signal processing device includes:
Obtaining the time-dependent change of the position of the reference target specified by the first information and the position of the reference target specified by the second information;
A sensor unit that requires correction is specified based on the change over time.

Only the fact that there is a displacement in at least one of the first sensor unit and the second sensor unit is determined based on the difference in the position of the reference target that is specified by performing the above process only once. is there. As described above, the sensor unit that needs to be corrected can be specified by monitoring the change with time of the position of the reference target specified by each sensor unit. Therefore, it is possible to more easily maintain the detection accuracy of the plurality of sensor units required for vehicle driving assistance.

Alternatively, the above sensor system can be configured as follows.
The plurality of sensor units include a third sensor unit that detects third information outside the vehicle and outputs a third signal corresponding to the third information,
The signal processing device includes:
Based on the first signal, the second signal, and the third signal, determine whether the first information, the second information, and the third information include the same reference target,
When it is determined that the first information, the second information, and the third information include the same reference target, the position of the reference target specified by the first information, the second information Based on the difference between the position of the reference target specified by the above and the position of the reference target specified by the third information, the sensor unit that needs to be corrected is specified.

According to such a configuration, it is possible to identify a sensor unit that needs to be corrected based on the principle of majority decision, for example, without repeating the process of specifying the position of the reference target. Therefore, it is possible to more easily maintain the detection accuracy of the plurality of sensor units required for vehicle driving assistance.

The reference target is preferably one that can provide a reference for the position in the height direction. This is because the position reference in the height direction tends to be relatively small in variation due to the running state of the vehicle, and it is easy to suppress an increase in processing load in the signal processing device.

The above sensor system can be configured as follows.
A lamp housing that partitions a lamp chamber that houses the lamp unit;
At least one of the plurality of sensor units is disposed in the lamp chamber.

The lamp unit is generally arranged in a place with little shielding because of the function of supplying light to the outside of the vehicle. By arranging the sensor unit in such a place, information outside the vehicle can be efficiently acquired.

The above sensor system can be configured as follows.
The at least a plurality of sensor units is at least one of a LIDAR sensor unit, a camera unit, and a millimeter wave sensor unit.

In the present specification, the “sensor unit” means a component unit of a component that has a desired information detection function and can be circulated by itself.

In the present specification, the “lamp unit” means a structural unit of a part that is provided with a desired lighting function and can be circulated by itself.

One aspect for achieving the third problem is an image data generation device mounted on a vehicle,
An input interface to which a signal corresponding to the image output from at least one camera that acquires an image behind the driver's seat of the vehicle is input;
A processor that generates first image data corresponding to a first monitoring image displayed on the display device based on the signal;
An output interface for outputting the first image data to the display device;
With
The processor is
Determining a reference object included in the first monitoring image;
Determining whether the reference object is included in a predetermined area in the first monitoring image;
When it is determined that the reference object is not included in the predetermined area, the second monitoring is performed such that a second monitoring image including the reference object in the predetermined area is displayed on the display device. Generate second image data corresponding to the image,
The second image data is output to the display device through the output interface.

According to such a configuration, either the first image data or the second image data is generated according to the position of the reference object, and the first monitoring image or the second monitoring image including the reference object in a predetermined region. Can be continuously displayed on the display device. Therefore, it is possible to avoid a situation in which the driver is prevented from visually recognizing the rear depending on the state of the vehicle. That is, it is possible to provide an electronic mirror technology with further improved rear visibility.

The above image data generation device can be configured as follows.
The processor is
Generating the first image data based on the signal corresponding to the first portion of the image;
The second image data is generated based on the signal corresponding to the second portion of the image.

When an entire original image acquired by a camera is displayed on a display device, it is inevitable that an object displayed in a monitoring image becomes small while a wide range can be visually recognized. According to the above configuration, since the first monitoring image and the second monitoring image are generated by clipping the minimum necessary area from the original image, it is possible to achieve both securing the visual field and preventing the deterioration of the visibility.

Alternatively, the above image data generation device can be configured as follows.
The processor is
Generating the first image data based on the image when the angle of view of the camera is the first angle of view;
Change the angle of view of the camera so that the second angle of view is wider than the first angle of view,
The second image data is generated based on the image when the angle of view of the camera is the second angle of view.

When using a camera with a wide angle of view from the beginning, it is inevitable that an object displayed in the monitoring image displayed on the display device becomes small while a wide range can be visually recognized. According to the configuration as described above, since the second monitoring image is generated by expanding the angle of view only when necessary, it is possible to achieve both securing the visual field and preventing the visibility from being lowered.

Alternatively, the above image data generation device can be configured as follows.
The processor is
Generating the first image data based on the image when the optical axis of the camera is in the first direction;
Changing the direction of the optical axis of the camera to a second direction different from the first direction;
The second image data is generated based on the image when the optical axis of the camera is in the second direction.

When using a camera with a wide angle of view from the beginning, it is inevitable that an object displayed in the monitoring image displayed on the display device becomes small while a wide range can be visually recognized. According to the above configuration, it is possible to continue to generate a monitoring image in which the reference object is included in a predetermined range without changing the angle of view. Since the angle of view only needs to be set to such an extent that an object displayed in the monitoring image can be properly visually recognized, it is possible to achieve both securing a visual field and preventing a reduction in visibility.

Alternatively, the above image data generation device can be configured as follows.
The at least one camera includes a first camera and a second camera;
The processor is
Generating the first image data based on the signal acquired from the first camera;
The second image data is generated based on the signal acquired from the second camera.

When using a camera with a wide angle of view from the beginning, it is inevitable that an object displayed in the monitoring image displayed on the display device becomes small while a wide range can be visually recognized. According to the above configuration, it is possible to continue to generate a monitoring image in which the reference object is included in a predetermined range without changing the angle of view. Since the angle of view of each camera has only to be determined to such an extent that an object displayed in the monitoring image can be properly visually recognized, both the securing of the visual field and the prevention of deterioration in visibility can be achieved.

The above image data generation device can be configured as follows.
The reference object can be designated by the user via the first monitoring image.

According to such a configuration, it is possible to increase flexibility and flexibility in setting a reference object for generating the second monitoring image.

The reference object may be a part of a rear part of the vehicle or a part of a vehicle located behind the vehicle.

The structure of the sensor system which concerns on 1st embodiment is illustrated. It is a figure explaining the operation example of the sensor system of FIG. It is a figure which illustrates the position of the sensor system in vehicles. The structure of the sensor system which concerns on 2nd embodiment is illustrated. 5 illustrates processing performed by the sensor system of FIG. 6 illustrates a configuration in which the sensor unit in the sensor system of FIG. 4 is disposed in the lamp chamber. The functional structure of the image data generation apparatus which concerns on 3rd embodiment is illustrated. An example of a vehicle on which the above-described image data generation device is mounted is shown. The operation | movement flow of said image data generation apparatus is illustrated. The operation result of said image data generation apparatus is illustrated. The 1st example of operation | movement of said image data generation apparatus is shown. The 2nd example of operation | movement of said image data generation apparatus is shown. The 3rd example of operation | movement of said image data generation apparatus is shown. The 4th example of operation | movement of said image data generation apparatus is shown.

DETAILED DESCRIPTION Hereinafter, exemplary embodiments will be described in detail with reference to the accompanying drawings. In each drawing used in the following description, the scale is appropriately changed to make each member a recognizable size.

In the accompanying drawings, an arrow F indicates the forward direction of the illustrated structure. Arrow B indicates the backward direction of the illustrated structure. Arrow L indicates the left direction of the illustrated structure. Arrow R indicates the right direction of the illustrated structure. “Left” and “right” used in the following description indicate the left and right directions viewed from the driver's seat.

FIG. 1 schematically shows a configuration of a sensor system 1 according to the first embodiment. The sensor system 1 is mounted on a vehicle.

The sensor system 1 includes a LiDAR sensor unit 11. The LiDAR sensor unit 11 has a configuration for emitting invisible light and a configuration for detecting return light as a result of reflection of the invisible light at least on an object existing outside the vehicle. The LiDAR sensor unit 11 may include a scanning mechanism that sweeps the invisible light by changing the emission direction (that is, the detection direction) as necessary. In the present embodiment, infrared light having a wavelength of 905 nm is used as invisible light.

The LiDAR sensor unit 11 can acquire the distance to the object associated with the return light based on, for example, the time from when the invisible light is emitted in a certain direction until the return light is detected. Further, by accumulating such distance data in association with the detection position, information related to the shape of the object associated with the return light can be acquired. In addition to or instead of this, information related to attributes such as the material of the object associated with the return light can be acquired based on the difference between the waveforms of the emitted light and the return light.

The LiDAR sensor unit 11 is configured to output a signal S1 corresponding to information outside the vehicle detected as described above.

The sensor system 1 includes a signal processing device 12. The signal processing device 12 can be realized by a general-purpose microprocessor that operates in cooperation with a general-purpose memory. As the general-purpose microprocessor, a CPU, an MPU, and a GPU can be exemplified. As the general-purpose memory, ROM and RAM can be exemplified. In this case, the ROM can store a computer program that realizes processing to be described later. The general-purpose microprocessor designates at least a part of a program stored on the ROM, expands it on the RAM, and executes the above-described processing in cooperation with the RAM. The signal processing device 12 may be realized by a dedicated integrated circuit such as a microcontroller, an ASIC, or an FPGA that can execute a computer program that realizes processing to be described later. The signal processing device 12 may be realized by a combination of a general-purpose microprocessor and a dedicated integrated circuit.

Specifically, the signal processing device 12 is configured to process the signal S1 output from the LiDAR sensor unit 11 and generate LiDAR data corresponding to the detected information outside the vehicle. LiDAR data is used for driving support of a vehicle.

The signal processing device 12 is configured to acquire the reference height information H from an auto leveling system mounted on the vehicle. The auto leveling system is a system that adjusts the direction of the optical axis of the headlamp in the vertical direction of the vehicle based on the pitch angle of the vehicle. In the process in which the auto leveling system adjusts the direction of the optical axis of the headlamp, a reference height (for example, the direction of the adjusted optical axis) is determined. The reference height information H indicates this reference height.

Based on the acquired reference height information H, the signal processing device 12 is configured to generate LiDAR data using the signal S1 associated with the region corresponding to the reference height in the detection range of the LiDAR sensor unit 11. Has been. This operation will be described in detail with reference to FIG.

2A shows an example in which the sensor system 1 is mounted on the front portion of the vehicle 100. FIG. In this case, the LiDAR sensor unit 11 is configured to detect information at least ahead of the vehicle 100. An arrow D indicates the detection reference direction of the LiDAR sensor unit 11 in the vertical direction of the vehicle 100.

(B) of FIG. 2 shows the detection range A of the LiDAR sensor unit 11. The symbol h represents the reference height indicated by the reference height information H. The signal S1 output from the LiDAR sensor unit 11 may include information within the detection range A. The signal processing device 12 generates LiDAR data corresponding to information detected in the region P using the signal S1 associated with the region P that is a part of the detection range A. The region P is a region corresponding to the reference height h.

In this specification, the meaning of the expression “corresponding to the reference height h” is not limited to the case where the reference height h determined by the auto leveling system matches the height of the region P in the vertical direction of the vehicle 100. . As long as the reference height h and the height of the region P have a predetermined correspondence, the two may be different.

The signal processing device 12 may acquire only the signal corresponding to the region P as the signal S1, and may use the signal S1 for data generation processing, or after acquiring the signal S1 corresponding to the entire detection range A, Only the corresponding signal may be extracted and used for data generation processing.

FIG. 2C shows a state in which the front end of the vehicle 100 is tilted upward from the rear end. The detection reference direction D of the LiDAR sensor unit 11 is directed upward from the reference height h. Therefore, as shown in FIG. 2D, the detection range A of the LiDAR sensor unit 11 moves upward from the state shown in FIG. Even in this case, the signal processing device 12 uses the signal S1 associated with the region P that is a part of the detection range A based on the acquired reference height information H, and converts the information detected in the region P into Corresponding LiDAR data is generated.

FIG. 2E shows a state in which the front end of the vehicle 100 is inclined downward from the rear end. The detection reference direction D of the LiDAR sensor unit 11 is directed downward from the reference height h. Therefore, as shown in FIG. 2F, the detection range A of the LiDAR sensor unit 11 moves downward from the state shown in FIG. Even in this case, the signal processing device 12 uses the signal S1 associated with the region P that is a part of the detection range A based on the acquired reference height information H, and converts the information detected in the region P into Corresponding LiDAR data is generated.

For example, when information outside the vehicle 100 at the reference height h is required for driving support, the detection reference direction D of the LiDAR sensor unit 11 in the vertical direction of the vehicle 100 matches the reference height h. Is preferred. However, since the detection reference direction D of the LiDAR sensor unit 11 changes in the vertical direction of the vehicle 100 according to the change in the pitch angle of the vehicle 100, the detection range A of the LiDAR sensor unit 11 is redundant in the vertical direction of the vehicle 100. Generally, it is set to have.

According to the configuration as described above, the signal processing device 12 specifies the region P using the reference height information H separately determined based on the pitch angle of the vehicle 100 by the auto leveling system. Therefore, regardless of the pitch angle of the vehicle 100, the region P including information required for driving assistance can be easily and highly reliably identified. Furthermore, since only the signal S1 associated with the region P that is a part of the detection range A having redundancy is used for data generation processing for acquiring external information, it is required for driving support of the vehicle 100. An increase in load related to signal processing can be suppressed.

As shown in FIG. 1, the sensor system 1 can include a leveling adjustment mechanism 13. The leveling adjustment mechanism 13 is configured to include an actuator that can change the detection reference direction D of the LiDAR sensor unit 11 in the vertical direction of the vehicle 100. In the auto leveling system, a configuration similar to a known mechanism for changing the direction of the optical axis of the headlamp in the vertical direction of the vehicle can be employed.

The leveling adjustment mechanism 13 can be communicably connected to the signal processing device 12. The leveling adjustment mechanism 13 is configured to adjust the detection reference direction D of the LiDAR sensor unit 11 in the vertical direction of the vehicle based on the reference height information H acquired by the signal processing device 12.

For example, as illustrated in FIG. 2C, when the detection reference direction D is directed upward from the reference height h, the signal processing device 12 recognizes the fact based on the reference height information H. . The signal processing device 12 outputs a signal for driving the leveling adjustment mechanism 13 so as to eliminate the deviation from the reference height h in the detection reference direction D. The leveling adjustment mechanism 13 operates based on the signal, and directs the detection reference direction D of the LiDAR sensor unit 11 downward.

On the other hand, when the detection reference direction D is directed downward from the reference height h as shown in FIG. 2E, the signal processing device 12 indicates that the detection reference direction D of the LiDAR sensor unit 11 is upward. The signal which operates the leveling adjustment mechanism 13 so that it may face is output.

According to such a configuration, since the change in the detection reference direction D of the LiDAR sensor unit 11 according to the change in the pitch angle of the vehicle 100 can be suppressed, the change in the position of the region P within the detection range A of the LiDAR sensor unit 11 can be reduced. Can be suppressed. That is, the position of the region P specified by the signal processing device 12 can be the position shown in FIG. 2B regardless of the pitch angle of the vehicle 100. Therefore, the processing load of the signal processing device 12 for specifying the region P can be further suppressed.

As shown in FIG. 1, the sensor system 1 can include a camera unit 14. The camera unit 14 is a device for acquiring image information outside the vehicle. The camera unit 14 is configured to output a signal S2 corresponding to the acquired image information. The camera unit 14 is an example of a sensor unit.

The signal processing device 12 is configured to process the signal S2 output from the camera unit 14 and generate camera data corresponding to the acquired image information outside the vehicle. The camera data is used for driving support of the vehicle.

Based on the acquired reference height information H, the signal processing device 12 is configured to generate camera data using a signal S2 associated with a region corresponding to the reference height h in the field of view of the camera unit 14. ing. That is, the signal processing device 12 generates the camera data corresponding to the image included in the area using the signal S2 associated with the area that is a part of the field of view of the camera unit 14. The field of view of the camera unit 14 is an example of the detection range of the sensor unit.

In this case, the LiDAR sensor unit 11 is an example of a first sensor unit. The information outside the vehicle 100 detected by the LiDAR sensor unit 11 is an example of first information. The signal S1 output from the LiDAR sensor unit 11 is an example of a first signal. LiDAR data generated by the signal processing device 12 is an example of first data.

In this case, the camera unit 14 is an example of a second sensor unit. An image outside the vehicle 100 acquired by the camera unit 14 is an example of second information. The signal S2 output from the camera unit 14 is an example of a second signal. The camera data generated by the signal processing device 12 is an example of second data.

In the above configuration, generation of LiDAR data based on the signal S1 output from the LiDAR sensor unit 11 and generation of camera data based on the signal S2 output from the camera unit 14 based on the common reference height information H And is made. That is, the generated LiDAR data and camera data both include information associated with the reference height h. Therefore, the integrated use of both data for driving assistance becomes easy. Moreover, both LiDAR data and camera data are generated using only signals associated with a region that is a part of the detection range of each sensor unit. Therefore, even if both data are used in an integrated manner, an increase in processing load on the signal processing device 12 can be suppressed.

The above description relating to the leveling adjustment mechanism 13 can also be applied to the camera unit 14.

In the above example, data output from a plurality of sensor units of different types is used for driving support in an integrated manner. However, data output from a plurality of sensor units of the same type arranged at relatively distant positions in the vehicle may be used for driving support in an integrated manner.

For example, the sensor system 1 shown in FIG. 1 is arranged in at least two of the left front corner LF, the right front corner RF, the left rear corner LB, and the right rear corner RB of the vehicle 100 shown in FIG. Can be done. Here, a case where two sensor systems 1 are arranged at the left front corner LF and the right front corner RF of the vehicle 100 will be described as an example.

In this example, the LiDAR sensor unit 11 disposed in the left front corner LF is an example of a first sensor unit. Information outside the vehicle 100 detected by the LiDAR sensor unit 11 disposed in the left front corner LF is an example of first information. The signal S1 output from the LiDAR sensor unit 11 disposed in the left front corner LF is an example of a first signal. The LiDAR data generated by the signal processing device 12 arranged at the left front corner LF is an example of first data.

In this example, the LiDAR sensor unit 11 disposed in the right front corner RF is an example of the second sensor unit. Information outside the vehicle 100 detected by the LiDAR sensor unit 11 disposed in the right front corner RF is an example of second information. The signal S1 output from the LiDAR sensor unit 11 disposed at the right front corner RF is an example of a second signal. The LiDAR data generated by the signal processing device 12 disposed in the right front corner RF is an example of second data.

The LiDAR data generated by the signal processing device 12 arranged in the left front corner LF and the LiDAR data generated by the signal processing device 12 arranged in the right front corner RF are used by the control device 101 for driving support. . An ECU is an example of the control device 101. The ECU can be realized by a general-purpose microprocessor that operates in cooperation with a general-purpose memory. As the general-purpose microprocessor, a CPU, an MPU, and a GPU can be exemplified. As the general-purpose memory, ROM and RAM can be exemplified. The ECU may be realized by a dedicated integrated circuit such as a microcontroller, ASIC, or FPGA. The ECU may be realized by a combination of a general-purpose microprocessor and a dedicated integrated circuit.

In the above configuration, based on the common reference height information H, generation of LiDAR data based on the signal S1 output from the LiDAR sensor unit 11 disposed at the left front corner LF, and placement at the right front corner RF LiDAR data is generated based on the signal S1 output from the LiDAR sensor unit 11. That is, the LiDAR data generated at two locations includes information associated with the reference height h. Therefore, the integrated use of both data for driving assistance becomes easy. The LiDAR data generated at two locations is generated using only the signal S1 associated with the region P that is part of the detection range A of each LiDAR sensor unit 11. Therefore, even when both data are used in an integrated manner, an increase in processing load on the control device 101 can be suppressed.

As shown in FIG. 1, the signal processing device 12 can be configured to acquire map information M. The map information M can be information used for the navigation system of the vehicle 100, for example. The map information M may be stored in advance in a storage device mounted on the vehicle 100, or may be downloaded from an external network periodically or as necessary.

The signal processing device 12 may be configured to associate the LiDAR data generated based on the signal S1 output from the LiDAR sensor unit 11 with the map information M. For example, when the LiDAR data indicates the presence of an object outside the vehicle 100, it can be determined by collating with the map information M whether the object is a structure such as a guardrail or a traffic sign.

According to such a configuration, LiDAR data and map information M can be used for driving support in an integrated manner. As described above, since an increase in signal processing load related to generation of LiDAR data can be suppressed, an increase in processing load in integrated driving support combining the LiDAR data and map information M can also be suppressed as a whole.

The above map information M can include three-dimensional information. In this case, the signal processing apparatus 12 can associate LiDAR data with information corresponding to the reference height h in the map information M. That is, the two-dimensional map information corresponding to the reference height h can be extracted from the map information M including the three-dimensional information and associated with the LiDAR data.

According to such a configuration, the map information data used for the integrated driving support can be reduced, so that an increase in processing load in the integrated driving support can be further suppressed.

Note that the map information M acquired by the signal processing device 12 may be provided in advance as two-dimensional information.

As shown in FIG. 1, the sensor system 1 can include a left front lamp device 15. The left front lamp device 15 may include a lamp housing 51 and a translucent cover 52. The lamp housing 51 partitions the lamp chamber 53 together with the translucent cover 52. The left front lamp device 15 is mounted on the left front corner LF of the vehicle 100 shown in FIG.

As shown in FIG. 1, the left front lamp device 15 may include a lamp unit 54. The lamp unit 54 is a device that emits visible light to the outside of the vehicle 100. The lamp unit 54 is accommodated in the lamp chamber 53. Examples of the lamp unit 54 include a headlamp unit, a vehicle width lamp unit, a direction indicator lamp unit, and a fog lamp unit.

At least one of the LiDAR sensor unit 11 and the camera unit 14 described above can be accommodated in the lamp chamber 53. Therefore, the translucent cover 52 is formed of a material that transmits not only visible light emitted from the lamp unit 54 but also light having a wavelength with which the sensor unit accommodated in the lamp chamber 53 has sensitivity.

Since the front left lamp device 15 has a function of supplying light to the outside of the vehicle 100, the front left lamp device 15 is generally arranged in a place with a small amount of shielding such as the left front corner FB. By arranging the sensor unit in such a place, information outside the vehicle 100 can be efficiently acquired.

Further, when the height detection information H is acquired from the auto leveling system of the vehicle 100, the height detection information H can be shared with the lamp unit 54. In this case, efficient system design is possible.

Therefore, a right front lamp device having a symmetrical configuration with the left front lamp device 15 can be mounted on the right front corner RF of the vehicle 100 shown in FIG. A left rear lamp device may be mounted on the left rear corner LB of the vehicle 100. In this case, examples of the lamp unit included in the left rear lamp device may include a brake light unit, a taillight unit, a vehicle width light unit, and a reverse light unit. A right rear lamp device having a configuration symmetrical to the left rear lamp device may be mounted on the right rear corner RB of the vehicle 100. In any lamp device, the sensor unit described above can be accommodated in a lamp chamber defined by a lamp housing and a light-transmitting cover.

The first embodiment is merely an example for facilitating understanding of the present disclosure. The configuration according to the first embodiment can be changed or improved as appropriate without departing from the spirit of the present disclosure.

In the first embodiment, an example in which the sensor system 1 includes at least one of the LiDAR sensor unit 11 and the camera unit 14 has been described. However, the sensor system 1 can be configured to include at least one of a LiDAR sensor unit, a camera unit, and a millimeter wave sensor unit. The camera unit can include a visible light camera unit and an infrared camera unit.

The millimeter wave sensor unit has a configuration for transmitting a millimeter wave and a configuration for receiving a reflected wave resulting from the reflection of the millimeter wave by an object existing outside the vehicle 100. Examples of the millimeter wave frequency include 24 GHz, 26 GHz, 76 GHz, and 79 GHz. For example, the millimeter wave sensor unit can acquire the distance to the object associated with the reflected wave based on the time from when the millimeter wave is transmitted in a certain direction until the reflected wave is received. Further, by accumulating such distance data in association with the detection position, it is possible to acquire information related to the motion of the object associated with the reflected wave.

While these sensor units are useful for acquiring information outside the vehicle 100, it is known that the amount of data corresponding to the acquired information is very large. However, as described above, only a signal associated with an area that is a part of the detection range of the sensor unit is subjected to a data generation process for acquiring external information. Therefore, while using these sensor units, it is possible to suppress an increase in load related to signal processing required for driving support of the vehicle 100.

In the first embodiment, the reference height information H is acquired from the auto leveling system. However, if information indicating the reference height h is obtained, the reference height information H may be acquired from a vehicle height sensor or the like.

At least a part of the functions of the signal processing device 12 described above can be realized by the control device 101 shown in FIG.

FIG. 4 shows a configuration of the sensor system 2 according to the second embodiment. The sensor system 2 is mounted on the vehicle 100 shown in FIG.

As shown in FIG. 4, the sensor system 2 includes a plurality of sensor units 20. Each of the plurality of sensor units 20 is a device that detects information outside the vehicle 100 and outputs a signal corresponding to the information. The plurality of sensor units 20 includes a first sensor unit 21 and a second sensor unit 22.

The first sensor unit 21 is configured to detect first information outside the vehicle 100 and output a first signal S1 corresponding to the first information. The first sensor unit 21 can be any one of a LiDAR sensor unit, a camera unit, and a millimeter wave sensor unit.

The second sensor unit 22 is configured to detect second information outside the vehicle 100 and output a second signal S2 corresponding to the second information. The second sensor unit 22 can be any one of a LiDAR sensor unit, a camera unit, and a millimeter wave sensor unit.

The LiDAR sensor unit has a configuration for emitting non-visible light and a configuration for detecting return light as a result of reflection of the non-visible light on at least an object existing outside the vehicle. The LiDAR sensor unit can include a scanning mechanism that sweeps the invisible light by changing the emission direction (that is, the detection direction) as necessary. For example, infrared light having a wavelength of 905 nm can be used as invisible light.

The camera unit is a device for acquiring an image as information outside the vehicle. The image can include at least one of a still image and a moving image. The camera unit may include a camera having sensitivity to visible light, or may include a camera having sensitivity to infrared light.

The millimeter wave sensor unit has a configuration for transmitting a millimeter wave and a configuration for receiving a reflected wave resulting from the reflection of the millimeter wave by an object existing outside the vehicle 100. Examples of the millimeter wave frequency include 24 GHz, 26 GHz, 76 GHz, and 79 GHz.

The first sensor unit 21 and the second sensor unit 22 may be a plurality of sensor units arranged in different areas in the vehicle 100. For example, the first sensor unit 21 and the second sensor unit 22 may be disposed at the left front corner LF and the right front corner RF of the vehicle 100, respectively. Alternatively, the first sensor unit 21 and the second sensor unit 22 may be disposed at the left front corner LF and the left rear corner LB of the vehicle 100, respectively.

Alternatively, the first sensor unit 21 and the second sensor unit 22 may be a plurality of sensor units arranged in substantially the same region in the vehicle 100. For example, both the first sensor unit 21 and the second sensor unit 22 can be disposed at the left front corner LF of the vehicle 100.

The sensor system 2 includes a signal processing device 30. The signal processing device 30 can be arranged at an arbitrary position in the vehicle 100.

The signal processing device 30 can be realized by a general-purpose microprocessor that operates in cooperation with a general-purpose memory. As the general-purpose microprocessor, a CPU, an MPU, and a GPU can be exemplified. As the general-purpose memory, ROM and RAM can be exemplified. In this case, the ROM can store a computer program that realizes processing to be described later. The general-purpose microprocessor designates at least a part of a program stored on the ROM, expands it on the RAM, and executes the above-described processing in cooperation with the RAM. The signal processing device 30 may be realized by a dedicated integrated circuit such as a microcontroller, ASIC, or FPGA that can execute a computer program that realizes processing to be described later. The signal processing device 30 may be realized by a combination of a general-purpose microprocessor and a dedicated integrated circuit.

First, the signal processing device 30 acquires the first signal S1 from the first sensor unit 21. In other words, the signal processing device 30 acquires the first information detected by the first sensor unit 21 by receiving the first signal S1 (STEP 21).

Subsequently, the signal processing device 30 acquires the second signal S <b> 2 from the second sensor unit 22. In other words, the signal processing device 30 acquires the second information detected by the second sensor unit 22 by receiving the second signal S2 (STEP 22).

The order in which the processing in STEP 21 and the processing in STEP 22 are performed may be reversed. The processing of STEP 21 and the processing of STEP 22 may be performed simultaneously.

Next, the signal processing device 30 determines whether the first information and the second information include the same feature based on the first signal S1 and the second signal S2 (STEP 23).

If it is determined that the first information and the second information do not contain the same feature (N in STEP 23), the processing by the signal processing device 30 ends.

If it is determined that the first information and the second information contain the same feature (Y in STEP 23), the signal processing device 30 determines whether the feature can be a reference target (STEP 24).

In this specification, the “reference target” means a feature that can be detected by the sensor unit 20 and can provide reference position information. Examples of the reference target include a license plate, a guardrail, a sound barrier, a traffic light, a traffic sign, and a center line of a preceding vehicle. In other words, the height from the road surface and the distance from the road shoulder are legally determined, and a feature whose position can be identified with relatively high accuracy by detecting its presence can be a reference target.

Even if the information detected by the two sensor units 20 includes the above-described feature, the feature may not always be a reference target. For example, even if a license plate of a preceding vehicle is detected as a feature, if the position is not determined due to a relative position change with the preceding vehicle, it is determined that the license plate cannot be a reference target. That is, it can be a condition that it is determined that the feature can be a reference target when a certain position information can be read from the detected feature exceeds a predetermined value.

If it is determined that the detected feature cannot be a reference target (N in STEP 24), the processing by the signal processing device 30 ends.

When it is determined that the detected feature can be the reference target (Y in STEP 24), the signal processing device 30 determines the reference target specified by the first information and the reference specified by the second information. A difference from the position of the target is detected (STEP 25).

The plurality of sensor units 20 mounted on the vehicle may be displaced due to vibration during traveling or the passage of time. The occurrence of such misalignment is a phenomenon in which the same reference target is detected by a plurality of sensor units 20, but the positions of the specified reference targets are different among the plurality of sensor units 20. Connected. Therefore, by detecting the difference, it is possible to grasp that a positional deviation has occurred in at least one of the plurality of sensor units 20.

For example, when the magnitude of the difference exceeds a predetermined value, the user can be notified. The user can take appropriate measures to correct the misalignment. Therefore, the detection accuracy of the plurality of sensor units 20 required for driving support of the vehicle 100 can be maintained.

As a first specific example of the above processing, the left front LiDAR unit as the first sensor unit 21 is arranged at the left front corner LF of the vehicle 100, and the left rear LiDAR unit as the second sensor unit 22 is The case where it arrange | positions in the left back corner part LB is given.

The front left LiDAR unit acquires information on an object existing in an area including the left side of the vehicle 100. The information is an example of first information. The left front LiDAR unit outputs a first signal S1 corresponding to the first information. The signal processing device 30 acquires the first signal S1 (STEP 21).

The left rear LiDAR unit acquires information on an object existing in an area including the left side of the vehicle 100. The information is an example of second information. The left rear LiDAR unit outputs a second signal S2 corresponding to the second information. The signal processing device 30 acquires the second signal S2 (STEP 22).

The signal processing device 30 performs information processing based on the first signal S1 and the second signal S2, thereby determining whether the same information is included in the first information and the second information (STEP 23). When the guardrail is detected as a feature (Y in STEP23), the signal processing device 30 determines whether the guardrail can be a reference target (STEP24).

For example, when the upper end of the guard rail having a certain height is detected for a predetermined time, it is determined that the upper end of the guard rail can be a reference target (Y in STEP 24). In this case, the signal processing device 30 detects a difference between the position (height) of the upper end of the guard rail identified through the left front LiDAR unit and the position (height) of the upper end of the guard rail identified through the left rear LiDAR unit ( (STEP 25).

When the magnitude of the difference exceeds a predetermined value, a notification indicating that a positional deviation has occurred in at least one of the left front LiDAR unit and the left rear LiDAR unit is made.

As a second specific example of the above processing, the left front camera unit as the first sensor unit 21 is arranged at the left front corner LF of the vehicle 100, and the right front camera unit as the second sensor unit 22 is the front right of the vehicle 100. The case where it arrange | positions to corner RF is mentioned.

The left front camera unit acquires the first image including the front of the vehicle 100. The first image is an example of first information. The left front camera unit outputs a first signal S1 corresponding to the first image. The signal processing device 30 acquires the first signal S1 (STEP 21).

The front right camera unit acquires a second image including the front of the vehicle 100. The second image is an example of second information. The right front camera unit outputs a second signal S2 corresponding to the second image. The signal processing device 30 acquires the second signal S2 (STEP 22).

The signal processing device 30 performs image processing based on the first signal S1 and the second signal S2, thereby determining whether or not the same feature is included in the first image and the second image (STEP 23). When the center line is detected as a characteristic object (Y in STEP 23), the signal processing device 30 determines whether the center line can be a reference target (STEP 24).

For example, when a center line having a constant distance from the shoulder is detected for a predetermined time, it is determined that the center line can be a reference target (Y in STEP 24). In this case, the signal processing device 30 detects a difference between the position of the center line specified through the left front camera unit and the position of the center line specified through the right front camera unit (STEP 25).

If the magnitude of the difference exceeds a predetermined value, a notification indicating that a positional deviation has occurred in at least one of the left front camera unit and the right front camera unit is made.

As a third specific example of the above processing, a case where a left front LiDAR unit as the first sensor unit 21 and a left front camera unit as the second sensor unit 22 are arranged in the left front corner LF of the vehicle 100 is given.

The front left LiDAR unit acquires information on an object existing in an area including the front of the vehicle 100. The information is an example of first information. The left front LiDAR unit outputs a first signal S1 corresponding to the first information. The signal processing device 30 acquires the first signal S1 (STEP 21).

The left front camera unit acquires a second image including the front of the vehicle 100. The second image is an example of second information. The right front camera unit outputs a second signal S2 corresponding to the second image. The signal processing device 30 acquires the second signal S2 (STEP 22).

The signal processing device 30 performs information processing based on the first signal S1 and the second signal S2, thereby determining whether the same information is included in the first information and the second information (STEP 23). When the traffic signal is detected as a characteristic object (Y in STEP 23), the signal processing device 30 determines whether the traffic signal can be a reference target (STEP 24).

For example, when a specific traffic signal is detected for a predetermined time when the vehicle stops at an intersection, it is determined that the traffic signal can be a reference target (Y in STEP 24). In this case, the signal processing device 30 detects the difference between the position of the traffic light identified through the left front LiDAR unit and the position of the traffic light identified through the left front camera unit (STEP 25).

When the magnitude of the difference exceeds a predetermined value, a notification indicating that a positional deviation has occurred in at least one of the left front LiDAR unit and the left front camera unit is made.

It should be noted that the reference target is preferably one that can provide a reference for the position in the height direction. This is because the position reference in the height direction tends to be relatively small in variation due to the traveling state of the vehicle 100 and easily suppress an increase in processing load in the signal processing device 30.

As shown in FIG. 5, the signal processing device 30 can acquire the time-dependent change in the position of the reference target specified based on the first information and the position of the reference target specified based on the second information. (STEP 26). Specifically, the signal processing device 30 repeats the processing from STEP 21 to STEP 25 at a predetermined timing, and changes the position of the reference target specified based on the first information in the latest processing and the first information in the previous processing. The position of the reference target identified based on the comparison is compared. Similarly, the position of the reference target specified based on the second information in the latest process is compared with the position of the reference target specified based on the second information in the previous process. Examples of the predetermined timing include the elapse of a predetermined time from the end of the previous process, the time when the user inputs a process execution instruction, and the like.

In this case, the signal processing device 30 specifies the sensor unit 20 that needs to be corrected based on the acquired temporal change (STEP 27). For example, when the position of the reference target specified by the first sensor unit 21 does not change over time and the position of the reference target specified by the second sensor unit 22 changes over time, the second sensor unit 22 It is determined that there is a cause of misalignment on the side and correction is necessary. The identified sensor unit can be notified to the user.

It is determined that at least one of the first sensor unit 21 and the second sensor unit 22 is displaced based on the difference in the position of the reference target specified by performing the processing from STEP 21 to STEP 25 only once. It is only the fact that. As described above, the sensor unit 20 that needs to be corrected can be specified by monitoring the change with time of the position of the reference target specified by each sensor unit 20. Therefore, the detection accuracy of the plurality of sensor units 20 required for driving support of the vehicle 100 can be more easily maintained.

As shown in FIG. 4, the plurality of sensor units 20 may include a third sensor unit 23. The third sensor unit 23 may be configured to detect third information outside the vehicle 100 and output a third signal S3 corresponding to the third information. The third sensor unit 23 can be any of a LiDAR sensor unit, a camera unit, and a millimeter wave sensor unit.

The third sensor unit 23 can be arranged in a different area in the vehicle 100 from the first sensor unit 21 and the second sensor unit 22. For example, when the first sensor unit 21 and the second sensor unit 22 are disposed at the left front corner LF and the right front corner RF of the vehicle 100, respectively, the third sensor unit 23 is configured to It can be arranged in the rear corner RB. Alternatively, the third sensor unit 23 can be disposed in substantially the same region of the vehicle 100 as at least one of the first sensor unit 21 and the second sensor unit 22.

In this case, as shown in FIG. 5, the signal processing device 30 acquires the third signal S <b> 3 from the third sensor unit 23. In other words, the signal processing device 30 acquires the third information detected by the third sensor unit 23 by receiving the third signal S3 (STEP 28). The order of STEP 21, STEP 22, and STEP 28 is arbitrary. STEP28 may be performed simultaneously with at least one of STEP21 and STEP22.

Next, the signal processing device 30 determines whether the first information, the second information, and the third information include the same feature based on the first signal S1, the second signal S2, and the third signal S3. Determine (STEP 23).

If it is determined that the first information, the second information, and the third information do not include the same feature (N in STEP 23), the processing by the signal processing device 30 ends.

When it is determined that the first information, the second information, and the third information include the same feature (Y in STEP 23), the signal processing device 30 determines whether the feature can be a reference target. (STEP 24).

If it is determined that the detected feature can be the reference target (Y in STEP 24), the signal processing device 30 determines the position of the reference target specified by the first information and the reference specified by the second information. A difference between the position of the target and the position of the reference target specified by the third information is detected (STEP 25).

Subsequently, the signal processing device 30 identifies the sensor unit 20 that needs to be corrected based on the difference identified in STEP 25 (STEP 27). For example, when the position of the reference target specified by the first sensor unit 21 and the second sensor unit 22 is the same and only the position of the reference target specified by the third sensor unit 23 is different, There is a high probability that the sensor unit 23 is displaced. The identified sensor unit 20 can be left to the user.

According to such a configuration, the sensor unit 20 that needs to be corrected can be specified without repeating the process of specifying the position of the reference target. Therefore, the detection accuracy of the plurality of sensor units 20 required for driving support of the vehicle 100 can be more easily maintained.

However, the possibility that the first sensor unit 21 and the second sensor unit 22 are misaligned in the above example cannot be denied. Therefore, as described with reference to STEP 27, by combining the process of monitoring the temporal change in the position of the reference target specified through each sensor unit 20, the determination accuracy of the sensor unit 20 that needs to be corrected can be improved. .

As shown in FIG. 6, the sensor system 2 can include a left front lamp device 40. The left front lamp device 40 may include a lamp housing 41 and a translucent cover 42. The lamp housing 41 partitions the lamp chamber 43 together with the translucent cover 42. The front left lamp device 40 is mounted on the front left corner LF of the vehicle 100 shown in FIG.

As shown in FIG. 6, the left front lamp device 40 may include a lamp unit 44. The lamp unit 44 is a device that emits visible light to the outside of the vehicle 100. The lamp unit 44 is accommodated in the lamp chamber 43. Examples of the lamp unit 44 include a headlamp unit, a vehicle width lamp unit, a direction indicator lamp unit, and a fog lamp unit.

In this case, at least one sensor unit 20 is arranged in the lamp chamber 43. Since the lamp unit 44 has a function of supplying light to the outside of the vehicle 100, the lamp unit 44 is generally disposed in a place with a small amount of shielding such as the left front corner FB. By arranging the sensor unit 20 in such a place, information outside the vehicle 100 can be efficiently acquired.

Therefore, a right front lamp device having a symmetrical configuration with the left front lamp device 40 can be mounted on the right front corner RF of the vehicle 100 shown in FIG. A left rear lamp device may be mounted on the left rear corner LB of the vehicle 100. In this case, examples of the lamp unit included in the left rear lamp device may include a brake light unit, a taillight unit, a vehicle width light unit, and a reverse light unit. A right rear lamp device having a configuration symmetrical to the left rear lamp device may be mounted on the right rear corner RB of the vehicle 100. In any lamp device, at least one sensor unit 20 can be disposed in a lamp chamber defined by a lamp housing.

The second embodiment is merely an example for facilitating understanding of the present disclosure. The configuration according to the second embodiment can be changed or improved as appropriate without departing from the spirit of the present disclosure.

FIG. 7 shows a functional configuration of the image data generation apparatus 301 according to the third embodiment. The image data generation device 301 is mounted on a vehicle.

FIG. 8 shows an example of a vehicle 400 on which the image data generation device 301 is mounted. The vehicle 400 is a towed vehicle having a tractor portion 401 and a trailer portion 402. The tractor portion 401 includes a driver seat 403.

The vehicle 400 includes a camera 404. The camera 404 is a device for acquiring an image behind the driver seat 403. The camera 404 is configured to output a camera signal corresponding to the acquired image.

As shown in FIG. 7, the image data generation apparatus 301 includes an input interface 311. A camera signal CS output from the camera 404 is input to the input interface 311.

The image data generation device 301 further includes a processor 312, an output interface 313, and a communication bus 314. The input interface 311, the processor 312, and the output interface 313 can exchange signals and data via the communication bus 314.

The processor 312 is configured to execute the processing shown in FIG.

First, the processor 312 acquires the camera signal CS input to the input interface 311 (STEP 31). The expression “obtain camera signal CS” means that the camera signal CS input to the input interface 311 is in a state in which processing described later can be performed via an appropriate circuit configuration.

Next, the processor 312 generates the first image data D1 based on the camera signal CS (STEP 32). As shown in FIG. 7, the first image data D <b> 1 is transmitted to the display device 405 mounted on the vehicle 400 via the output interface 313. The display device 405 may be disposed in the passenger compartment of the vehicle 400 or may be disposed at the position of the side door mirror.

The first image data D1 is data corresponding to the first monitoring image I1 displayed on the display device 405. FIG. 10A shows an example of the first monitoring image I1.

That is, an image behind the driver's seat 403 acquired by the camera 404 is constantly displayed on the display device 405. The driver acquires information behind the driver's seat 403 through the first monitoring image I1 displayed on the display device 405.

8, an arrow X indicates the direction of the optical axis of the camera 404. In the automatic towing vehicle, the tractor portion 401 can take a posture that is largely bent with respect to the trailer portion 402 as indicated by a two-dot chain line in FIG. At this time, the optical axis of the camera 404 may face the side wall of the trailer portion 402, which may hinder the driver's rearward visual recognition.

In order to deal with such a situation, as shown in FIG. 9, the processor 312 determines a reference object included in the first monitoring image I1 (STEP 33). The “reference object” needs to be always included in the first monitoring image I1 in order for the driver to continuously acquire information behind the driver's seat 403, and image recognition is compared as a target object. It is defined as a thing that is easy. In this example, as shown in FIG. 10A, the rear end edge 402a of the trailer portion 402 of the vehicle 400 is used as a reference object. The processor 312 determines the trailing edge 402a as a reference object using, for example, an edge extraction technique.

The rear end edge 402a of the trailer portion 402 is an example of a rear portion of the vehicle 400. “Rear part” means a part of the vehicle 400 that is located behind the driver's seat 403.

Next, as shown in FIG. 9, the processor 312 determines whether the reference object determined in STEP 33 is included in a predetermined area in the first monitoring image I1 (STEP 34). The predetermined area may be the entire first monitoring image I1 shown in FIG. 10A. For example, the predetermined area may be the right side of the boundary line BD indicated by the one-dot chain line in the drawing. It may be defined as part of the image I1.

If the rear edge 402a as the reference object is included in the predetermined area (Y in STEP 34), the first monitoring image I1 is continuously displayed on the display device 405. In the example shown in FIG. 10A, when the predetermined area is the entire first monitoring image I1, it is determined that the rear edge 402a as the reference object is included in the predetermined area. In the example shown in FIG. 10A, when the predetermined region is a region on the right side of the boundary line BD, it is determined that the rear edge 402a as the reference object is not included in the predetermined region. (N in STEP 34).

If it is determined that the reference object is not included in the predetermined area in the first monitoring image I1, the processor 312 generates the second image data D2 as shown in FIG. 9 (STEP 35). The second image data D2 is data for causing the display device 405 to display the second monitoring image I2 in which the reference object is included in a predetermined area. As illustrated in FIG. 7, the second image data D2 is transmitted to the display device 405 mounted on the vehicle 400 via the output interface 313. FIG. 10B shows an example of the second monitoring image I2.

Also in the second monitoring image I2, a predetermined area can be defined similarly to the first monitoring image I1. In the illustrated example, a region on the right side of the boundary line BD is a predetermined region. It can be seen that the trailing edge 402a of the trailer portion 402 as a reference object is included in a predetermined region. The driver can continue to acquire information behind the driver seat 403 through the second monitoring image I2 displayed on the display device 405.

According to such a configuration, either the first image data D1 or the second image data D2 is generated according to the position of the reference object, and the first monitoring image I1 or the first image including the reference object in a predetermined region. It is possible to continue displaying the second monitoring image I2 on the display device 405. Therefore, it is possible to avoid a situation in which the driver is prevented from visually recognizing the rear depending on the state of the vehicle 400. That is, it is possible to provide an electronic mirror technology with further improved rear visibility.

Next, some specific examples of the method for generating the second image data D2 as described above will be described with reference to FIGS.

FIG. 11 is a diagram for explaining the first specific example. A symbol I0 indicates the entire image acquired by the camera 404. The first monitoring image I1 and the second monitoring image I2 can be different parts in the image. That is, the processor 312 generates the first image data D1 based on the camera signal CS corresponding to the first portion of the image acquired by the camera 404. Similarly, the processor 312 generates the second image data D2 based on the camera signal CS corresponding to the second portion of the image acquired by the camera 404.

When an entire original image acquired by the camera 404 is displayed on the display device 405, it is inevitable that an object displayed in the monitoring image becomes small while a wide range can be visually recognized. According to the above configuration, since the first monitoring image I1 and the second monitoring image I2 are generated by clipping the minimum necessary area from the original image, it is possible to achieve both securing the visual field and preventing the deterioration of the visibility. .

FIG. 12 is a diagram for explaining a second specific example. In this example, the angle of view of the camera 404 can be changed. Specifically, a known mechanism for changing the angle of view is provided in the camera 404. The angle of view of the camera 404 can be changed by the processor 312 sending a control signal S to the camera 404 through the output interface 313 as shown in FIG.

Specifically, the processor 312 generates the first image data D1 based on the image acquired when the angle of view of the camera 404 is the first angle of view θ1. On the other hand, the processor 312 generates the second image data D2 based on the image acquired when the angle of view of the camera 404 is the second angle of view θ2. The second field angle θ2 is wider than the first field angle θ1. That is, the second monitoring image I2 is displayed on the display device 405 as a wider-angle image.

When the camera 404 is at the position indicated by the solid line in FIG. 12, the trailing edge 402a of the trailer portion 402 serving as the reference object exists in the field of view of the first angle of view θ1. Accordingly, the first monitoring image I1 is generated accordingly. However, when the camera 404 is located at the position indicated by the two-dot chain line in the figure, the rear edge 402a is out of the field of view of the first angle of view θ1. In this case, the processor 312 performs control to expand the angle of view of the camera 404 to the second angle of view θ2. As a result, the rear edge 402a is within the field of view of the second angle of view θ2, and an appropriate second monitoring image I2 is obtained.

When the camera 404 with a wide angle of view is used from the beginning, it is inevitable that an object displayed in the monitoring image displayed on the display device 405 becomes small while a wide range can be visually recognized. According to the configuration as described above, since the second monitoring image I2 is generated by widening the angle of view only when necessary, it is possible to ensure both the field of view and the prevention of deterioration in visibility.

FIG. 13 is a diagram for explaining a third specific example. In this example, the direction of the optical axis of the camera 404 can be changed. Specifically, a known swivel mechanism that changes the direction of the optical axis is provided in the camera 404. The direction of the optical axis of the camera 404 can be changed by the processor 312 sending a control signal S to the camera 404 via the output interface 313 as shown in FIG.

Specifically, the processor 312 generates the first image data D1 based on the image acquired when the optical axis of the camera 404 is oriented in the first direction X1. On the other hand, the processor 312 generates the second image data D2 based on the image acquired when the direction of the optical axis of the camera 404 is in the second direction X2 different from the first direction X1. That is, the imaging target located in the center differs between the first monitoring image I1 and the second monitoring image I2.

When the camera 404 is located at the position indicated by the solid line in FIG. 13, the rear end edge 402a of the trailer portion 402 serving as the reference object exists in the field of view in which the direction of the optical axis is the first direction X1. Accordingly, the first monitoring image I1 is generated accordingly. However, when the camera 404 is located at the position indicated by the two-dot chain line in the figure, the rear end edge 402a is out of the field of view. In this case, the processor 312 performs control to change the direction of the optical axis of the camera 404 from the first direction X1 to the second direction X2. As a result, the rear edge 402a is within the field of view in which the direction of the optical axis is the second direction X2, and an appropriate second monitoring image I2 is obtained.

When the camera 404 with a wide angle of view is used from the beginning, it is inevitable that an object displayed in the monitoring image displayed on the display device 405 becomes small while a wide range can be visually recognized. According to the above configuration, it is possible to continue to generate a monitoring image in which the reference object is included in a predetermined range without changing the angle of view. Since the angle of view only needs to be set to such an extent that an object displayed in the monitoring image can be properly visually recognized, it is possible to achieve both securing a visual field and preventing a reduction in visibility.

FIG. 14 is a diagram for explaining a fourth specific example. In this example, a first camera 404 a and a second camera 404 b are mounted on the vehicle 400 as the camera 404. The direction of the optical axis of the first camera 404a is different from the direction of the optical axis of the second camera 404b.

Specifically, the processor 312 generates the first image data D1 based on the image acquired by the first camera 404a. On the other hand, the processor 312 generates the second image data D2 based on the image acquired by the second camera 404b. Switching of the operating camera can be performed by the processor 312 transmitting the control signal S through the output interface 313.

In FIG. 14, when the camera 404 is at the position indicated by the solid line, the trailing edge 402a of the trailer portion 402 serving as the reference object exists in the field of view of the first camera 404a. Accordingly, the first monitoring image I1 is generated accordingly. However, when the first camera 404a is located at a position indicated by a two-dot chain line in the figure, the rear end edge 402a is out of the field of view. In this case, the processor 312 performs control to switch the camera used for image acquisition from the first camera 404a to the second camera 404b. As a result, the rear edge 402a is within the field of view of the second camera 404b, and an appropriate second monitoring image I2 is obtained.

When the camera 404 with a wide angle of view is used from the beginning, it is inevitable that an object displayed in the monitoring image displayed on the display device 405 becomes small while a wide range can be visually recognized. According to the above configuration, it is possible to continue to generate a monitoring image in which the reference object is included in a predetermined range without changing the angle of view. Since the angle of view of each camera has only to be determined to such an extent that an object displayed in the monitoring image can be properly visually recognized, both the securing of the visual field and the prevention of deterioration in visibility can be achieved.

As shown in FIG. 7, the input interface 311 of the image data generation apparatus 301 can accept an input from the user interface 406. The user interface 406 is provided in the passenger compartment of the vehicle 400 and can accept tactile operation instructions, voice input instructions, line-of-sight input instructions, etc. to buttons, touch panel devices, and the like.

In this case, the reference object used for the determination in STEP 34 in FIG. 9 can be designated by the user via the first monitoring image I1 displayed on the display device 405. For example, when the user interface 406 is a touch panel device provided on the display device 405, the appropriate location included in the first monitoring image I 1 (such as the rear edge 402 a of the trailer portion 402) is touched to make the location a reference object. Can be specified as

According to such a configuration, it is possible to increase flexibility and flexibility in setting a reference object for generating the second monitoring image I2.

The functions of the processor 312 described above can be realized by a general-purpose microprocessor that operates in cooperation with a memory. As the general-purpose microprocessor, a CPU, an MPU, and a GPU can be exemplified. Examples of general-purpose memory include ROM and RAM. In this case, the ROM can store a computer program for executing the above processing. The general-purpose microprocessor can specify at least a part of a program stored in the ROM, expand it on the RAM, and execute the above-described processing in cooperation with the RAM. The functions of the processor 312 described above may be realized by a dedicated integrated circuit such as a microcontroller, ASIC, or FPGA that can execute a computer program that implements processing to be described later. The functions of the processor 312 described above may be realized by a combination of a general-purpose microprocessor and a dedicated integrated circuit.

The third embodiment is merely an example for facilitating understanding of the present disclosure. The configuration according to the third embodiment may be changed or improved as appropriate without departing from the spirit of the present disclosure.

In the third embodiment, a part of the rear portion of the vehicle 400 is designated as the reference object. However, a part of the vehicle located behind the vehicle 400 may be designated as the reference object when the automatic platooning is performed.

The number and position of the cameras 404 mounted on the vehicle 400 can be appropriately determined according to the specifications of the vehicle 400.

As part of the description of the present application, Japanese Patent Application No. 2018-038879 filed on March 5, 2018, Japanese Patent Application No. 2018-049652 filed on March 16, 2018 The contents of Japanese Patent Application No. 2018-051287 filed on Mar. 19, 2018 are incorporated herein by reference.

Claims

A sensor system mounted on a vehicle,
At least one sensor unit that detects information outside the vehicle and outputs a signal corresponding to the information;
A signal processing device that processes the signal to generate data corresponding to the information;
With
The signal processing device includes:
Obtaining reference height information indicating a reference height determined based on the pitch angle of the vehicle;
Based on the reference height information, the data is generated using the signal associated with the region corresponding to the reference height in the detection range of the sensor unit.
Sensor system.
A leveling adjustment mechanism for adjusting a detection reference direction of the sensor unit in the vertical direction of the vehicle based on the reference height information;
The sensor system according to claim 1.
The at least one sensor unit comprises:
A first sensor unit that detects first information outside the vehicle and outputs a first signal corresponding to the first information;
A second sensor unit that detects second information outside the vehicle and outputs a second signal corresponding to the second information;
Contains
The signal processing device processes the first signal to generate first data corresponding to the first information, and processes the second signal to generate second data corresponding to the second information. ,
The signal processing device processes, as the first signal, a signal associated with a region corresponding to the reference height in the detection range of the first sensor unit based on the reference height information. Processing the signal associated with the region corresponding to the reference height in the detection range of the two-sensor unit as the second signal;
The sensor system according to claim 1 or 2.
The signal processing device associates the data with map information;
The sensor system according to any one of claims 1 to 3.
The data is associated with information corresponding to the reference height among the map information.
The sensor system according to claim 4.
A lamp housing that partitions a lamp chamber that houses the lamp unit;
The sensor unit is disposed in the lamp chamber.
The sensor system according to any one of claims 1 to 5.
The at least one sensor unit is at least one of a LIDAR sensor unit, a camera unit, and a millimeter wave sensor unit.
The sensor system according to any one of claims 1 to 6.
A sensor system mounted on a vehicle,
A plurality of sensor units that detect information outside the vehicle and each output a signal corresponding to the information;
A signal processing device for acquiring the signal;
With
The plurality of sensor units are:
A first sensor unit that detects first information outside the vehicle and outputs a first signal corresponding to the first information;
A second sensor unit that detects second information outside the vehicle and outputs a second signal corresponding to the second information;
Contains
The signal processing device includes:
Based on the first signal and the second signal, determine whether the first information and the second information include the same reference target,
When it is determined that the first information and the second information include the same reference target, the position of the reference target specified by the first information and the reference specified by the second information Detect the difference from the target position,
Sensor system.
The signal processing device includes:
Obtaining the time-dependent change of the position of the reference target specified by the first information and the position of the reference target specified by the second information;
Identify a sensor unit that needs to be corrected based on the change over time,
The sensor system according to claim 8.
The plurality of sensor units include a third sensor unit that detects third information outside the vehicle and outputs a third signal corresponding to the third information,
The signal processing device includes:
Based on the first signal, the second signal, and the third signal, determine whether the first information, the second information, and the third information include the same reference target,
When it is determined that the first information, the second information, and the third information include the same reference target, the position of the reference target specified by the first information, the second information Identifying a sensor unit that needs to be corrected based on the difference between the position of the reference target identified by the position of the reference target identified by the third information,
The sensor system according to claim 8.
The reference target provides a reference for a position in the height direction.
The sensor system according to any one of claims 8 to 10.
A lamp housing that partitions a lamp chamber that houses the lamp unit;
At least one of the plurality of sensor units is disposed in the lamp chamber.
The sensor system according to any one of claims 8 to 11.
The at least a plurality of sensor units is at least one of a LIDAR sensor unit, a camera unit, and a millimeter wave sensor unit.
The sensor system according to any one of claims 8 to 12.
An image data generation device mounted on a vehicle,
An input interface to which a signal corresponding to the image output from at least one camera that acquires an image behind the driver's seat of the vehicle is input;
A processor that generates first image data corresponding to a first monitoring image displayed on the display device based on the signal;
An output interface for outputting the first image data to the display device;
With
The processor is
Determining a reference object included in the first monitoring image;
Determining whether the reference object is included in a predetermined area in the first monitoring image;
When it is determined that the reference object is not included in the predetermined area, the second monitoring is performed such that a second monitoring image including the reference object in the predetermined area is displayed on the display device. Generate second image data corresponding to the image,
Outputting the second image data to the display device through the output interface;
Image data generation device.
The processor is
Generating the first image data based on the signal corresponding to the first portion of the image;
Generating the second image data based on the signal corresponding to the second portion of the image;
The image data generation device according to claim 14.
The processor is
Generating the first image data based on the image when the angle of view of the camera is the first angle of view;
Change the angle of view of the camera so that the second angle of view is wider than the first angle of view,
Generating the second image data based on the image when the angle of view of the camera is the second angle of view;
The image data generation device according to claim 14.
The processor is
Generating the first image data based on the image when the optical axis of the camera is in the first direction;
Changing the direction of the optical axis of the camera to a second direction different from the first direction;
Generating the second image data based on the image when the optical axis of the camera is in the second direction;
The image data generation device according to claim 14.
The at least one camera includes a first camera and a second camera;
The processor is
Generating the first image data based on the signal acquired from the first camera;
Generating the second image data based on the signal acquired from the second camera;
The image data generation device according to claim 14.
The reference object can be designated by the user via the first monitoring image.
The image data generation device according to any one of claims 14 to 18.
The reference object is a part of a rear part of the vehicle or a part of a vehicle located behind the vehicle.
The image data generation device according to any one of claims 14 to 19.