JP2019095385A - Sensor system - Google Patents

Abstract

To provide a sensor system that can improve estimation accuracy of a situation.SOLUTION: A sensor system includes: a sensor 1; a light emission unit 3; a sensor controller 2 for controlling the sensor 1 and the light emission unit 3; a camera 4; and estimation units 2 and 5 for estimating a situation, using information acquired by the sensor 1 and an image taken by the camera 4. the sensor controller 2 instructs the light emission unit 3 to emit light and instructs the sensor 1 to operate at the same time. When light emission from the light emission unit 3 is detected from the image taken by the camera 4, the estimation units 2 and 5 estimate the situation, using the image selected with reference to time when the light emission is detected and the information acquired by the sensor 1.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a sensor system comprising a camera.

  This type of sensor system includes, for example, a camera mounted on a vehicle and an ultrasonic sensor, and information obtained from an image captured by the camera and information on the distance to an object obtained by transmission and reception of ultrasonic waves. A sensor system for detecting an obstacle based on the above has been proposed.

  In such a sensor system, collection and processing of an image captured by a camera and collection and processing of results of transmission and reception of ultrasonic waves performed by an ultrasonic sensor are usually performed by separate ECUs. Then, image information and distance information obtained by these processes are collected in one ECU, and detection of an obstacle is performed in this ECU.

  In order to detect an obstacle accurately, it is desirable that the acquisition times of the image information and the distance information coincide. However, the acquisition time of the information collected in the obstacle detection ECU may not coincide with each other due to the cycle of the image processing and the communication between the ECUs.

  With regard to synchronization of a plurality of control units, for example, Patent Document 1 proposes a method of transmitting a message between two controllers, measuring the time taken for the message to reciprocate, and synchronizing the two controllers. .

JP-A-2017-517086

  However, in the method described in Patent Document 1, the accuracy of synchronization decreases when interrupt processing is performed, so when this method is applied to a sensor system configured as described above, the detection accuracy of an obstacle decreases. .

  Also in the sensor system that estimates the situation based on the image information of the camera and the information acquired by a sensor other than the ultrasonic sensor, the estimation accuracy of the situation is also similar due to the time difference between the acquisition of the two pieces of information. descend.

  An object of the present invention is to provide a sensor system capable of improving the estimation accuracy of a situation.

  In order to achieve the above object, the invention according to claim 1 is a sensor system that estimates a situation using an image captured by a camera (4) and information acquired by a sensor (1), 3), a sensor control unit (2) that controls the sensor and the light emitting unit, and an estimation unit (2, 5) that estimates a situation using an image captured by a camera and information acquired by the sensor The sensor control unit issues an operation instruction to the sensor and simultaneously issues a light emission instruction to the light emission unit, and the estimation unit references the time when the light emission is detected when the light emission of the light emission unit is detected from the image captured by the camera. The situation is estimated using the selected image and the information acquired by the sensor.

  As described above, by issuing a light emission instruction to the light emitting unit simultaneously with the operation instruction to the sensor, acquisition of information by the sensor and light emission of the light emitting unit are performed almost simultaneously. That is, among the images captured by the camera, the image in which the light emission of the light emitting unit is detected is captured almost simultaneously with the acquisition of information by the sensor.

  Therefore, when the light emission of the light emitting unit is detected from the image captured by the camera, it is possible to obtain a combination of the image information and the sensor information acquired almost simultaneously, and by estimating the situation at this time, estimation accuracy Improve.

  In addition, the code | symbol in the parenthesis of each said means shows an example of the correspondence with the specific means as described in embodiment mentioned later.

It is a figure showing composition of a sensor system concerning a 1st embodiment. It is a flowchart of the process which camera ECU performs. It is a flowchart of the process which sensor ECU performs. It is a figure which shows the 1st pattern of transmission / reception of an ultrasonic wave. It is a figure which shows the 2nd pattern of transmission / reception of an ultrasonic wave.

  Hereinafter, an embodiment of the present invention will be described based on the drawings. In the following embodiments, parts that are the same as or equivalent to each other will be described with the same reference numerals.

First Embodiment
The first embodiment will be described. The sensor system of the present embodiment estimates a situation using an image captured by a camera and information acquired by a sensor. Here, a sensor system mounted on a vehicle and detecting an object outside the vehicle will be described.

  As shown in FIG. 1, the sensor system includes an ultrasonic sensor 1, a sensor ECU 2, a light emitting unit 3, a camera 4, a camera ECU 5, a time master ECU 6, and an in-vehicle LAN 7.

  The ultrasonic sensor 1 measures the distance to an object by transmitting and receiving ultrasonic waves, and is disposed facing the outer surface of the vehicle. The ultrasonic sensor 1 transmits and receives ultrasonic waves in accordance with a measurement instruction from the sensor ECU 2, and transmits the result of transmission and reception to the sensor ECU 2. Although only one ultrasonic sensor 1 is illustrated in FIG. 1, the sensor system may include only one ultrasonic sensor 1 or a plurality of ultrasonic sensors 1.

  The light emitting unit 3 includes a light source (not shown) that generates visible light or infrared light, and emits light in accordance with a light emission instruction from the sensor ECU 2.

  The sensor ECU 2 controls the ultrasonic sensor 1 and the light emitting unit 3 and corresponds to a sensor control unit. The sensor ECU 2 issues an operation instruction to the ultrasonic sensor 1 and simultaneously issues a light emission instruction to the light emitting unit 3. For example, as shown in FIG. 1, by connecting the light emitting unit 3 on the bus to which the ultrasonic sensor 1 is connected, the signal output from the sensor ECU 2 is simultaneously input to the light emitting unit 3 and the ultrasonic sensor 1 Transmission of the ultrasonic wave by the sound wave sensor 1 and light emission of the light emitting unit 3 are performed almost simultaneously.

  The sensor ECU 2 estimates a situation using an image captured by the camera 4 and information acquired by the ultrasonic sensor 1 and corresponds to an estimation unit. In the present embodiment, the sensor ECU 2 detects an object outside the vehicle using the image captured by the camera 4 and the distance measured by the ultrasonic sensor 1. Then, the sensor ECU 2 performs notification etc. to the driver according to the position of the detected object.

  The camera 4 is for imaging the surroundings of the vehicle. The camera 4 performs imaging in accordance with an imaging instruction from the camera ECU 5, and transmits a captured image to the camera ECU 5. Although only one camera 4 is illustrated in FIG. 1, the sensor system may include only one camera 4 or a plurality of cameras. The camera 4 is installed such that the light emitting unit 3 is included in the range to be imaged.

  The camera ECU 5 is a control unit that controls the camera 4 and performs collection and processing of an image captured by the camera 4. The camera ECU 5 periodically issues an imaging instruction to the camera 4, whereby the imaging by the camera 4 and the transmission of the imaged image to the camera ECU 5 are periodically performed. Then, the camera ECU 5 analyzes the captured image to check whether the light emission of the light emitting unit 3 is captured, and when the light emission is detected, transmits the captured image and the time at which the image was captured to the sensor ECU 2.

  The camera ECU 5 selects one image from among a plurality of captured images of the camera 4 based on the time when light emission of the light emitting unit 3 is detected, and transmits this image to the sensor ECU 2. For example, an image in which the light emission of the light emitting unit 3 is captured, or an image captured a predetermined time before the image in which the light emission is captured is transmitted to the sensor ECU 2. By appropriately setting the selection method of the image based on the time when the light emission is detected, the image information and the distance information used for detecting the object in the sensor ECU 2 are obtained almost simultaneously.

  Further, time information is periodically transmitted from the time master ECU 6 to the sensor ECU 2 and the camera ECU 5, and the camera ECU 5 acquires the time at which the image to be transmitted to the sensor ECU 2 was captured from the time information. Together with the sensor ECU 2.

  The sensor ECU 2 detects an object based on the received image information and distance information on time corresponding to the received time information.

  Each ECU is connected via the in-vehicle LAN 7, and transmission of time information from the time master ECU 6 to the sensor ECU 2 and the camera ECU 5 and transmission of image information and time information from the camera ECU 5 to the sensor ECU 2 are performed in the vehicle It takes place via LAN7.

  The operation of the sensor system will be described. In the present embodiment, the camera ECU 5 and the sensor ECU 2 repeatedly execute the processing shown in FIGS. 2 and 3.

  The camera ECU 5 performs image collection and the like by steps S11 to S13 shown in FIG. In step S11, the camera ECU 5 issues an imaging instruction to the camera 4. Thereby, the surroundings of the vehicle are imaged by the camera 4, and the imaged image is transmitted to the camera ECU 5. Further, the camera ECU 5 acquires time information from the time master ECU 6 and stores the time when the imaging instruction is issued to the camera 4.

  The camera ECU 5 proceeds from step S11 to step S12, analyzes the image transmitted from the camera 4, and determines whether or not the light emission of the light emitting unit 3 is captured. When it is determined that the light emission of the light emitting unit 3 is captured, the camera ECU 5 proceeds to step S13, and transmits the image information to the sensor ECU 2.

  As described above, the camera ECU 5 transmits the image selected on the basis of the time when the light emission of the light emitting unit 3 is detected to the sensor ECU 2. The camera ECU 5 also transmits the time stored in step S11 to the sensor ECU 2 when capturing this image.

  On the other hand, when it is determined in step S12 that the light emission of the light emitting unit 3 is not captured, the camera ECU 5 ends the process.

  The sensor ECU 2 performs distance measurement and object detection by steps S21 to S24 shown in FIG. In step S21, the sensor ECU 2 issues an operation instruction to the ultrasonic sensor 1 and simultaneously issues a light emission instruction to the light emitting unit 3, causes the ultrasonic sensor 1 to transmit an ultrasonic wave, and causes the light emitting unit 3 to emit light. Further, the sensor ECU 2 acquires time information from the time master ECU 6 and stores the time when an instruction is issued to the ultrasonic sensor 1 and the light emitting unit 3.

  The sensor ECU 2 proceeds from step S21 to step S22 to perform wave reception processing. Specifically, the distance to an object outside the vehicle is calculated based on the amplitude and the like of the ultrasonic wave received by the ultrasonic sensor 1. The sensor ECU 2 stores the calculated distance information. The sensor ECU 2 accumulates storage of time and distance information each time steps S21 and S22 are executed.

  The sensor ECU 2 proceeds from step S22 to step S23, and determines whether an image from the camera ECU 5 has been received. If it determines with having received the image, sensor ECU2 will detect an object in step S24.

  Specifically, the sensor ECU 2 selects one corresponding to the time information transmitted from the camera ECU 5 among the information accumulated in steps S21 and S22. Then, the sensor ECU 2 calculates the position of the object in the space outside the vehicle based on the position of the object in the image and the selected distance information, and for example, to the driver if there is a possibility that the object contacts the vehicle. To notify etc.

  On the other hand, when it is determined in step S23 that the image is not received, the sensor ECU 2 ends the process.

  As described above, in the present embodiment, the sensor ECU 2 issues a light emission instruction to the light emitting unit 3 simultaneously with the operation instruction to the ultrasonic wave sensor 1, and transmission and reception of ultrasonic waves by the ultrasonic sensor 1 and light emission from the light emitting unit 3 are substantially It takes place at the same time. That is, among the images captured by the camera 4, the image in which the light emission of the light emitting unit 3 is detected is captured almost at the same time as the ultrasonic sensor 1 transmits and receives the ultrasonic waves.

  Therefore, when the light emission of the light emitting unit 3 is detected from the image captured by the camera 4, it is possible to obtain a combination of the image information and the distance information acquired substantially simultaneously, and by detecting the object at this time, Detection accuracy is improved.

  When the method described in Patent Document 1 is used, the accuracy of synchronization between ECUs is lowered by interrupt processing, and the accuracy of detection of an object is lowered. However, in the present embodiment, image information is selected based on the light emission detection time. It is possible to suppress a decrease in detection accuracy due to processing.

  Moreover, as a method of reducing the deviation of the acquisition time of each information, for example, there is a method of collecting image information of the camera 4 and distance information of the ultrasonic sensor 1 and detecting an object using these with one ECU. Conceivable. However, in such a configuration, internal processing of the image system takes time.

  On the other hand, in the present embodiment, since the image processing is performed by the dedicated camera ECU 5, the time required for the internal processing is shortened compared to the case where the image processing and the collection of distance information are performed by one ECU. be able to.

Second Embodiment
The second embodiment will be described. The present embodiment is the same as the first embodiment except that the operation of the light emitting unit 3 is modified with respect to the first embodiment, and therefore only the parts different from the first embodiment will be described.

  The sensor ECU 2 of the present embodiment instructs the ultrasonic sensor 1 to operate in a plurality of patterns. Specifically, the sensor system includes a plurality of ultrasonic sensors 1 arranged along the outer surface of the vehicle, for example, when the sensor ECU 2 is arranged with four ultrasonic sensors 1 on the front side of the vehicle. Each ultrasonic sensor 1 is operated in two patterns shown in FIG. 4 and FIG. That is, in the first pattern, two out of the four ultrasonic sensors 1 aligned along the front outer surface of the vehicle 10 transmit ultrasonic waves, and in the second pattern, the other two transmit ultrasonic waves. Do.

  The sensor ECU 2 causes the light emitting unit 3 to emit light in a different pattern for each operation instruction, and the camera ECU 5 determines the light emission pattern of the light emitting unit 3 based on the captured image. For example, the sensor ECU 2 causes the light emitting unit 3 to emit light once when operating the plurality of ultrasonic sensors 1 in the first pattern, and emits light twice when emitting the plurality of ultrasonic sensors 1 in the second pattern. Let Then, if light emission of the light emitting unit 3 is detected once from a plurality of images captured by the camera 4 within a predetermined time, the camera ECU 5 determines that the light emitting unit 3 emits light in the first pattern, and light emission is performed twice. If it is detected, it is determined that the light emitting unit 3 emits light in the second pattern. The camera ECU 5 transmits the determination result of the pattern to the sensor ECU 2 together with the image and time information.

  The sensor ECU 2 uses the image captured by the camera 4 and the information acquired by the ultrasonic sensor 1 when the light emission instruction pattern issued by the sensor ECU 2 matches the pattern determination result transmitted from the camera ECU 5. Detect an object.

  As described above, the light emission pattern of the light emitting unit 3 is changed each time the ultrasonic sensor 1 operates, and the light emission pattern is determined based on the captured image of the camera 4 to match the transmission pattern of the ultrasonic wave. It can be selected. This further improves the detection accuracy of the object.

(Other embodiments)
In addition, this invention is not limited to above-described embodiment, In the range described in the claim, it can change suitably.

  For example, in the first embodiment, the sensor ECU 2 detects an object in step S24, but the sensor ECU 2 transmits distance information to the camera ECU 5, and the camera ECU 5 executes the same processing as step S24 to detect an object. You may In this case, the camera ECU 5 corresponds to the estimation unit. In such a configuration, transmission of image information having a large amount of data between the ECUs is not performed, so the time required for communication between the ECUs can be shortened.

  The sensor system may include an ECU other than the sensor ECU 2 and the camera ECU 5 as an estimation unit, and the ECU may receive distance information and image information from the sensor ECU 2 and the camera ECU 5 to detect an object.

  In addition, the present invention may be applied to a sensor system that estimates a situation using an image captured by a camera and information acquired by a sensor other than an ultrasonic sensor.

1 Ultrasonic sensor 2 Sensor ECU
3 light emitting unit 4 camera 5 camera ECU

Claims (4)

A sensor system that estimates a situation using an image captured by a camera (4) and information acquired by a sensor (1),
A light emitting unit (3),
A sensor control unit (2) for controlling the sensor and the light emitting unit;
An estimation unit (2, 5) configured to estimate a situation using an image captured by the camera and information acquired by the sensor;
The sensor control unit issues an operation instruction to the sensor and simultaneously issues a light emission instruction to the light emitting unit.
When the light emission of the light emitting unit is detected from the image captured by the camera, the estimation unit uses the image selected based on the time when the light emission is detected and the information acquired by the sensor. Sensor system to estimate. The sensor control unit issues an operation instruction to the sensor a plurality of times, and causes the light emitting unit to emit light in a different pattern for each operation instruction.
When the pattern of the light emission instruction issued by the sensor control unit matches the light emission pattern detected from the image captured by the camera, the estimation unit matches the image captured by the camera and the information acquired by the sensor The sensor system according to claim 1, wherein the situation is estimated using.   The sensor system according to claim 1, wherein the sensor is an ultrasonic sensor that measures a distance to an object by transmitting and receiving ultrasonic waves. Mounted on the vehicle,
The sensor system according to claim 3, wherein an object outside the vehicle is detected using an image captured by the camera and a distance measured by the sensor.

Images