WO2021140843A1

WO2021140843A1 - Vehicle-mounted system, and sensing data receiving method and transmitting method

Info

Publication number: WO2021140843A1
Application number: PCT/JP2020/046670
Authority: WO
Inventors: 秀昭田中
Original assignee: 日立Astemo株式会社
Priority date: 2020-01-10
Filing date: 2020-12-15
Publication date: 2021-07-15
Also published as: JP7314314B2; JPWO2021140843A1

Abstract

The objective of the present invention is to provide a vehicle-mounted system that is also capable of supporting V2X (vehicle-to-vehicle communication/road-to-vehicle communication), while utilizing devices such as sensors having inbuilt counters that are simple and inexpensive. This vehicle-mounted system is provided with sensors which output sensing data, an absolute time receiving device, a communication device for receiving sensing data from the outside, and an environment recognizing device for perceiving the environment around a host vehicle, characterized in that: the sensors, the absolute time receiving device, the communication device, and the environment recognizing device each have an inbuilt counter for counting a count value; the absolute time receiving device transmits, to the communication device, the absolute time received from the outside, and correspondence information relating to the count value of the counter at the reception timing; and the communication device converts a time-stamp of the sensing data received from the outside into a corresponding count value from the absolute time, on the basis of the correspondence information, and transmits the count value to the environment recognizing device.

Description

In-vehicle system and sensing data reception method and transmission method

The present invention relates to an in-vehicle system that synchronizes the time between each in-vehicle device, and a method of receiving and transmitting sensing data.

In an in-vehicle system that recognizes the surrounding environment of the own vehicle by integrating the detection results of multiple sensors, it may be necessary to synchronize the time between each device.

FIG. 5 shows the operation of a cognitive system in which the own vehicle equipped with a camera and a radar as sensors detects surrounding objects in the order of the camera and the radar, and then integrates the detection results of both sensors to recognize the surrounding environment of the own vehicle. This is an example.

As in this example, when the cognitive system recognizes the surrounding environment, it detects surrounding objects using multiple sensors of different types and arrangements, thereby expanding the cognitive range and improving reliability and robustness. However, since the detection cycle of an object and the processing time required to output the detection result differ for each sensor, the cognitive system needs to integrate the detection results of multiple sensors (hereinafter referred to as "sensing data"). , A common time axis is needed to indicate when each sensing data was detected. Therefore, in the prior art of FIG. 5, time synchronization is performed between the cognitive system and each sensor to maintain the consistency of the detection results of each sensor.

In this conventional technology, only the sensing data detected by the sensor of the own vehicle is targeted for integration, so the common time axis could be a local index such as the count value of the system clock (counter), but V2X has become widespread in recent years. When data from outside the vehicle is also targeted for integration, such as (vehicle-to-vehicle communication / road-to-vehicle communication), or when data detected by the own vehicle is transmitted to another vehicle, etc., as shown in FIG. 6, GNSS or It is necessary to use a highly versatile index (hereinafter referred to as "absolute time") as a common time axis based on the standard time (year, month, day, hour, minute, second) obtained from a radio clock or the like.

Therefore, in Patent Document 1, a clock circuit is built in each device in the vehicle so that the absolute time can be a common time axis, and each device can self-manage the absolute time (Summary of Patent Document 1, Fig. 2 etc.).

Japanese Unexamined Patent Publication No. 2005-181009

However, the system of Patent Document 1 in which a clock circuit is built in each device such as a sensor has a problem that each device such as a sensor is complicated and expensive as compared with a conventional system in which a counter count value is used as a common time axis. there were.

The present invention has been made in view of such circumstances, and V2X (vehicle-to-vehicle communication / road-to-vehicle distance) is used while using various devices such as a sensor having a built-in simple and inexpensive counter instead of a complicated and expensive clock circuit. The purpose is to provide an in-vehicle system that can also support communication).

In order to solve the above problems, the in-vehicle system of the present invention has a sensor that detects surrounding objects of the own vehicle and outputs sensing data to which a count value is added as a time stamp, and an absolute time receiving device that receives an absolute time from the outside. A communication device that receives sensing data with an absolute time added as a time stamp from the outside, and an environment recognition device that recognizes the surrounding environment of the own vehicle based on the sensing data of both the sensor and the communication device. The sensor, the absolute time receiving device, the communication device, and the environment recognition device each have a built-in counter for counting a count value, and the absolute time receiving device is provided from the outside. The correspondence information between the received absolute time and the count value of the counter at the reception timing is transmitted to the communication device, and the communication device absolutely sets the time stamp of the sensing data received from the outside based on the correspondence information. It was assumed that the time was converted into the corresponding count value and transmitted to the environment recognition device.

Further, another in-vehicle system of the present invention includes a sensor that detects an object around the own vehicle and outputs sensing data to which a count value is added as a time stamp, an absolute time receiving device that receives an absolute time from the outside, and a time. The sensor includes a communication device that transmits sensing data with an absolute time added as a stamp to the outside, and an environment recognition device that recognizes the surrounding environment of the own vehicle based on the sensing data from the sensor. , The absolute time receiving device, the communication device, and the environment recognition device each have a built-in counter for counting a count value, and the absolute time receiving device has an absolute time received from the outside and its reception timing. The correspondence information of the count value of the counter in the above is transmitted to the communication device, and the communication device converts the time stamp of the sensing data received from the sensor from the count value to the corresponding absolute time based on the correspondence information. Then, it was decided to send it to the outside.

According to the in-vehicle system of the present invention, it is possible to support V2X (vehicle-to-vehicle communication / road-to-vehicle communication) while using each device such as a sensor having a built-in simple and inexpensive counter instead of a complicated and expensive clock circuit. it can.

Functional block diagram of the in-vehicle system of one embodiment Flow chart showing control when using only the data detected by the own vehicle Flowchart showing control when using data from outside the vehicle Flowchart showing control when transmitting data outside the vehicle An example of a cognitive system in which the common time axis is the counter count value An example of a cognitive system in which the common time axis is absolute time

The in-vehicle system 100 according to an embodiment of the present invention will be described with reference to FIGS. 1 to 4.

The in-vehicle system 100 of this embodiment is a system that notifies the surrounding environment of the own vehicle to an automatic driving system, a driving support system, or the like, and as shown in FIG. 1, an environment recognition device 1, an absolute time receiving device 2, and a communication device. 3. It is a system composed of a plurality of sensors 4 (sensors 4a to 4n) and a wired or wireless in-vehicle LAN 5 that connects them to each other. The communication interface of each device and the in-vehicle LAN 5 are used for data transmission / reception between the devices, but the intervention thereof will be omitted below.

The environment recognition device 1 is a device that recognizes the surrounding environment of the own vehicle by integrating sensing data at different times detected by a plurality of sensors 4, and has a main counter 11, an integrated processing unit 12, and a communication interface 13. doing. Specifically, the integrated processing unit 12 is a computer provided with hardware such as an arithmetic unit such as a CPU, a main storage device such as a semiconductor memory, and an auxiliary storage device. Then, the arithmetic unit executes the program loaded from the auxiliary storage device to the main storage device to realize each function described later. However, in the following, such well-known techniques in the computer field will be omitted as appropriate. I will explain while.

The absolute time receiving device 2 is a device that receives an absolute time from the outside, and has a slave counter 21, a GNSS receiver 22, a corresponding information generation unit 23, and a communication interface 24. In this embodiment, the configuration in which the absolute time is received from the artificial satellites constituting the GNSS (Global Navigation Satellite System) is illustrated, but as a configuration in which the absolute time is received from the standard radio wave transmission station in the standard time format. In that case, a standard radio wave receiver may be provided instead of the GNSS receiver 22.

The communication device 3 is a device necessary for realizing V2X (vehicle-to-vehicle communication / road-to-vehicle communication), and has a slave counter 31, a communication processing unit 32, a conversion unit 33, and a communication interface 34.

The sensor 4 is, for example, a monocular camera, a stereo camera, a radar, a LIDAR, an infrared sensor, or the like for detecting the surrounding environment of the own vehicle, and has a slave counter 41, a sensing processing unit 42, and a communication interface 43. .. As described above, when a plurality of sensors 4 are mounted, the detection cycle of each sensor and the processing time of the sensing data are different, so that each sensor outputs the detected sensing data at different times. Since the sensor 4 also includes sensors such as a speed sensor and a steering angle sensor that detect the state of the own vehicle, the environment recognition device 1 also obtains vehicle information such as the speed and the direction of travel of the own vehicle. Can be done.

Here, the main counter 11 and the

slave counters

21, 31, and 41 are simple circuits such as a system clock whose count value increments with time, and have high functionality such that absolute time (year, month, day, hour, minute, second) can be managed. Not the one. These counters have the limitation that they can only handle local count values that are closed to the own vehicle, but they also have the advantage of small circuit scale and low cost. Therefore, each device of the present embodiment incorporating these counters can be manufactured at a lower cost than a configuration including a clock circuit capable of managing the absolute time.

When the specific control content of the in-vehicle system 100 of the present embodiment outlined above is used only by the sensing data detected by the own vehicle (conventional example), or when the sensing data received from outside the vehicle is also used (the own vehicle). An example of V2X on the receiving side) and a case of transmitting sensing data outside the vehicle (an example of V2X on the transmitting side of the own vehicle) will be described in detail.

<When using only the sensing data detected by the own vehicle>
FIG. 2 is a flowchart illustrating the control contents when the in-vehicle system 100 recognizes the surrounding environment of the own vehicle by using only the sensing data detected by the own vehicle, and corresponds to the situation described in FIG. It is a thing.

First, in step S1, the environment recognition device 1 reflects the count value of the main counter 11 on the slave counter 21 of the absolute time receiving device 2, the slave counter 31 of the communication device 3, and the slave counter 41 of the sensor 4. As a result, thereafter, the count values of the counters are synchronized.

Next, in step S2, the sensing processing unit 42 of the sensor 4 adds the count value of the slave counter 41 as a time stamp to the sensing data that detects the surrounding environment of the own vehicle, and then integrates the environment recognition device 1. It is transmitted to the unit 12.

In step S3, the integrated processing unit 12 of the environment recognition device 1 integrates the sensing data received from the plurality of sensors 4 in consideration of the difference in the added count values, and recognizes the current surrounding environment of the own vehicle. .. For example, in the case of integrating the camera data captured before 100 counts and the radar data detected before 10 counts, the integrated processing unit 12 determines the position / orientation of the own vehicle after 100 counts in the camera data. By estimating the position and orientation of the own vehicle after 10 counts in the radar data and integrating both sensing data based on the position and orientation of the own vehicle at the same timing, the surrounding environment of the own vehicle at the present time is recognized.

<When using sensing data received from outside the vehicle>
FIG. 3 shows a case where the in-vehicle system 100 recognizes the surrounding environment of the own vehicle by using the sensing data detected by another vehicle or road equipment in addition to the sensing data detected by the own vehicle (the own vehicle is the receiving side). It is a flowchart which illustrates the control content of V2X).

Since the processing of steps S1, S2, and S3 in FIG. 3 is basically the same as the processing of steps S1, S2, and S3 in FIG. 2, duplicate description will be omitted. In FIG. 3, steps S21 and S22 are performed in parallel with the processing of step S2 so that the integrated processing unit 12 of the environment recognition device 1 can recognize the surrounding environment of the own vehicle by using the sensing data received from outside the vehicle. Executes the processing of.

First, in step S21, the GNSS receiver 22 of the absolute time receiver 2 receives the absolute time from the GNSS. Then, the corresponding information generation unit 23 of the absolute time receiving device 2 transmits the corresponding information in which the absolute time is associated with the count value of the slave counter 21 of the receiving timing of the absolute time to the conversion unit 33 of the communication device 3.

Next, in step S22, the communication processing unit 32 of the communication device 3 acquires sensing data to which an absolute time is added as a time stamp from outside the vehicle. Then, the conversion unit 33 of the communication device 3 counts the added absolute time associated with the sensing data in consideration of the correspondence information between the absolute time and the count value received from the correspondence information generation unit 23 of the absolute time reception device 2. It is replaced with a value and transmitted to the integrated processing unit 12 of the environment recognition device 1.

By these processes, the time stamp of the sensing data acquired from the outside of the vehicle is converted from the absolute time to the count value. Therefore, in step S3, the integrated processing unit 12 of the environment recognition device 1 also owns the sensing data acquired from the outside of the vehicle. It can be treated in the same way as the sensing data detected by the vehicle, and both sensing data can be used when recognizing the surrounding environment of the own vehicle.

<When transmitting sensing data outside the vehicle>
FIG. 4 is a flowchart illustrating the control contents when the in-vehicle system 100 transmits the sensing data detected by the own vehicle to the outside of the vehicle (V2X in which the own vehicle is the transmitting side). The method of using the sensing data transmitted to the outside of the vehicle is not particularly limited, but an example of using the sensing data detected by the own vehicle in order for another vehicle to recognize the surrounding environment can be mentioned.

Since the processing of steps S1 and S21 in FIG. 4 is basically the same as the processing of step S1 in FIG. 2 and the processing of step S21 in FIG. 3, duplicate description will be omitted.

In step S4, the sensing processing unit 42 of the sensor 4 adds the count value of the slave counter 41 as a time stamp to the sensing data detected in the surrounding environment of the own vehicle, and then transmits it to the conversion unit 33 of the communication device 3. ..

In step S5, the conversion unit 33 of the communication device 3 receives the count value added as a time stamp to the sensing data received from the sensor 4 from the corresponding information generation unit 23 of the absolute time reception device 2, and the absolute time and the count value. Replace with the absolute time considering the corresponding information of.
Then, the communication processing unit 32 of the communication device 3 transmits the sensing data to which the absolute time is added as a time stamp to the outside of the vehicle.

By these processes, the time stamp of the sensing data detected by the own vehicle is converted from the local count value to the highly versatile absolute time and transmitted to the outside of the vehicle, so the sensing data detected by the own vehicle can be effectively used even outside the vehicle. become able to.

As described above, according to the in-vehicle system 100 of the present embodiment, the time stamp of the sensing data received from outside the vehicle is converted from the absolute time to the local count value and then used for internal processing, which is complicated and expensive. It is possible to support the receiving side of V2X (vehicle-to-vehicle communication / road-to-vehicle communication) while using each device such as a sensor having a built-in simple and inexpensive counter instead of a simple clock circuit.

Further, according to the in-vehicle system 100 of the present embodiment, since the time stamp of the sensing data detected by the own vehicle is converted from the local count value to the absolute time and then transmitted to the outside of the vehicle, it is not a complicated and expensive clock circuit. It is possible to support the transmitting side of V2X (vehicle-to-vehicle communication / road-to-vehicle communication) while using each device such as a sensor having a built-in simple and inexpensive counter.

100 In-vehicle system 1 Environment recognition device 11 Main counter 12 Integrated processing unit 13 Communication interface 2 Absolute time receiver 21 Secondary counter 22 GNSS receiver 23 Corresponding information generator 24 Communication interface 3 Communication device 31 Secondary counter 32 Communication processing unit 33 Conversion unit 34 Communication interface 4 Sensor 41 Secondary counter 42 Sensing processing unit 43 Communication interface 5 In-vehicle network

Claims

A sensor that detects objects around the vehicle and outputs sensing data with a count value added as a time stamp.
An absolute time receiver that receives the absolute time from the outside,
A communication device that receives sensing data with an absolute time added as a time stamp from the outside,
An environment recognition device that recognizes the surrounding environment of the own vehicle based on the sensing data of both the sensor and the communication device, and
It is an in-vehicle system equipped with
The sensor, the absolute time receiving device, the communication device, and the environment recognition device each have a built-in counter for counting a count value.
The absolute time receiving device transmits the correspondence information between the absolute time received from the outside and the count value of the counter at the reception timing to the communication device.
The communication device is an in-vehicle system characterized in that, based on the correspondence information, a time stamp of sensing data received from the outside is converted from an absolute time into a corresponding count value and transmitted to the environment recognition device.
A sensor that detects objects around the vehicle and outputs sensing data with a count value added as a time stamp.
An absolute time receiver that receives the absolute time from the outside,
A communication device that sends sensing data with an absolute time added as a time stamp to the outside,
An environment recognition device that recognizes the surrounding environment of the own vehicle based on the sensing data from the sensor,
It is an in-vehicle system equipped with
The sensor, the absolute time receiving device, the communication device, and the environment recognition device each have a built-in counter for counting a count value.
The absolute time receiving device transmits the correspondence information between the absolute time received from the outside and the count value of the counter at the reception timing to the communication device.
The communication device is an in-vehicle system characterized in that, based on the corresponding information, a time stamp of sensing data received from the sensor is converted from a count value into a corresponding absolute time and transmitted to the outside.
In the in-vehicle system according to claim 1 or 2.
An in-vehicle system characterized in that the count value of the counter of the environment recognition device is reflected in the counter of another device.
In the in-vehicle system according to claim 1 or 2.
The sensor is an in-vehicle system characterized by being any one of a monocular camera, a stereo camera, a radar, a LIDAR, and an infrared sensor.
In the in-vehicle system according to claim 1 or 2.
An in-vehicle system characterized in that the absolute time is a standard time received from an artificial satellite or a standard radio wave transmission station.
A sensor that detects objects around the vehicle and outputs sensing data with a count value added as a time stamp.
An absolute time receiver that receives the absolute time from the outside,
A communication device that receives sensing data with an absolute time added as a time stamp from the outside,
An environment recognition device that recognizes the surrounding environment of the own vehicle based on the sensing data of both the sensor and the communication device, and
It is a method of receiving sensing data in an in-vehicle system equipped with
Generates correspondence information between the absolute time received from the outside and the count value of the counter at the reception timing.
A method for receiving sensing data, which comprises converting a time stamp of sensing data from the outside from an absolute time into a corresponding count value based on the corresponding information.
A sensor that detects objects around the vehicle and outputs sensing data with a count value added as a time stamp.
An absolute time receiver that receives the absolute time from the outside,
A communication device that sends sensing data with an absolute time added as a time stamp to the outside,
An environment recognition device that recognizes the surrounding environment of the own vehicle based on the sensing data from the sensor,
It is a method of transmitting sensing data in an in-vehicle system equipped with
Generates correspondence information between the absolute time received from the outside and the count value of the counter at the reception timing.
A method for transmitting sensing data, which comprises converting a time stamp of sensing data from the sensor from a count value to a corresponding absolute time based on the corresponding information.