JP2020067754A - Object detection device - Google Patents

Abstract

To rapidly detect an object even when a camera to be used for detecting the object in the periphery of a vehicle is changed over.SOLUTION: An object detection device is used in a vehicle mounting multiple cameras. The object detection device changes over a changeover object camera being at least one of the multiple cameras between a use state where image information is used for detecting an object and a non-use state where image information is not used for detecting an object, in accordance with a travel state of a vehicle in S12, S13. When the changeover object camera is changed over from the non-use state into the use state, the object detection device detects an object with the use of pre-changeover image information obtained from the changeover object camera before performing changeover into the non-use state and with the use of post-changeover image information obtained from the changeover object camera after performing changeover into the use state in S17.SELECTED DRAWING: Figure 2

Description

  The present disclosure relates to an object detection device.

  Patent Document 1 discloses a vehicle provided with a front camera arranged in the front part of the vehicle, a side camera arranged on the side of the passenger seat side of the vehicle, and a rear camera arranged in the rear part of the vehicle. Discloses a driving support device that detects an object around the vehicle. In this driving support device, in order to reduce the load related to the process of detecting an object, one camera that images the direction in which the driver should perform safety confirmation is mounted on the vehicle according to the running state of the vehicle. Selected from multiple cameras. Specifically, based on the direction change amount, the front camera is selected when it is determined that the vehicle is moving forward, and the side camera is selected when it is determined that the vehicle is turning to the left. If it is determined that the vehicle is moving backward, the rear camera is selected. Then, the picked-up image is acquired by the selected one camera, and the objects around the vehicle are detected by comparing the data of the plurality of picked-up images continuous in time series.

JP, 2011-159186, A

  By the way, in the above-mentioned driving support device, in order to detect an object, a plurality of captured images that are continuous in time series are required. For this reason, when the selected camera is switched, the object cannot be detected until a plurality of captured images are acquired from that point, and there is a problem that it takes time before the object can be detected.

  An aspect of the present disclosure is to make it possible to detect an object more quickly even when a camera used for detecting an object around a vehicle is switched.

  One aspect of the present disclosure is an object detection device (6) used in a vehicle (1) equipped with a plurality of cameras (2a to 2d), the detection unit (S17) and a switching unit (S12, S13). And The detection unit performs a detection process of detecting an object using a plurality of pieces of image information obtained from a plurality of cameras and having different imaging times. The switching unit determines whether the switching target camera, which is at least one of the plurality of cameras, is used depending on the running state of the vehicle, in which the image information is used in the detection process and in which the image information is not used in the detection process. Switch to the used state. When the switching target camera is switched from the non-use state to the use state, the detection unit provides the pre-switching image information obtained from the switching target camera before the non-use state is switched and the switching target after being switched to the use state. Detection processing is performed using the switched image information obtained from the camera.

  According to such a configuration, by using the pre-switching image information in the detection process, it is possible to shorten the time until the object is in the detectable state, as compared with the case where the pre-switching image information is not used. Becomes Therefore, even when the camera used for detecting the object around the vehicle is switched, the object can be detected earlier.

It is a block diagram which shows the structure of an object detection apparatus. It is a flowchart of an object detection process. It is a flowchart of a correction process. It is explanatory drawing about the image information obtained from a switching object camera. It is a figure of the image information before correction | amendment and the image information after correction | amendment in the detection of the stationary object using a moving stereo.

Hereinafter, exemplary embodiments of the present disclosure will be described with reference to the drawings.
[1. Constitution]
The vehicle 1 shown in FIG. 1 includes a front camera 2a, a rear camera 2b, a left side camera 2c, a right side camera 2d, a vehicle speed sensor 3, a steering angle sensor 4, a shift lever position sensor 5, and object detection. The device 6 and the monitor 7 are provided.

  Each of the cameras 2a to 2d is an image pickup device that picks up an image of the periphery of the vehicle 1, and outputs a signal representing a picked-up image to the object detection device 6. The front camera 2a, the rear camera 2b, the left side camera 2c, and the right side camera 2d are arranged so as to image the front, rear, left side, and right side of the vehicle 1, respectively.

The vehicle speed sensor 3 is a sensor for detecting the traveling speed of the vehicle 1. The vehicle speed sensor 3 outputs a signal corresponding to the detected traveling speed to the object detection device 6.
The steering angle sensor 4 is a sensor for detecting the steering angle of the steering of the vehicle 1. The steering angle sensor 4 outputs a signal corresponding to the detected steering angle to the object detection device 6.

  The shift lever position sensor 5 is a sensor for detecting the position of the shift lever operated by the driver of the vehicle 1. The shift lever position sensor 5 outputs a signal corresponding to the detected position to the object detection device 6.

  The object detection device 6 is mainly composed of a well-known microcomputer having a CPU 61, a ROM 62, a RAM 63, a flash memory 64 and the like. The CPU 61 executes the program stored in the ROM 62, which is a non-transitional physical recording medium. By executing the program, a method corresponding to the program is executed. Specifically, the object detection device 6 executes an object detection process shown in FIG. 2 described later according to the program. The object detection device 6 may include one microcomputer or a plurality of microcomputers. Further, the method of realizing the function of each unit included in the object detection device 6 is not limited to software, and a part or all of the function may be realized by using one or a plurality of hardware. For example, when the above function is realized by an electronic circuit that is hardware, the electronic circuit may be realized by a digital circuit, an analog circuit, or a combination thereof.

The monitor 7 is a display for displaying an image, and is provided at a position where the driver of the vehicle 1 can visually recognize it.
[2. processing]
Next, the object detection process executed by the object detection device 6 will be described with reference to the flowchart of FIG. This object detection process is periodically executed while the ignition switch is on.

  First, in S11, the object detection device 6 determines whether or not the position of the shift lever is the drive “D” based on the signal from the shift lever position sensor 5. When the object detection device 6 determines in S11 that the position of the shift lever is not the drive, for example, when it is determined that the position of the shift lever is reverse "R" or parking "P", the process proceeds to S12. .

On the other hand, when the object detection device 6 determines in S11 that the position of the shift lever is the drive, the process proceeds to S13.
In S12 and S13, the object detection device 6 uses the front camera 2a and the rear camera 2b to detect the use state in which the image information is used to detect the object and the image information is used to detect the object according to the traveling state of the vehicle 1. Switch to either unused or unused state. The image information mentioned here is a captured image obtained from each of the cameras 2a to 2d. Further, the detection of the object is performed using image information of a camera in use among the plurality of cameras 2a to 2d, which will be described in detail later. As the image information of the camera in use, a plurality of pieces of image information at different image capturing times are used for detecting an object.

  That is, in the present embodiment, in order to reduce the processing load, the object is not detected using all of the image information obtained from the four cameras 2a to 2d, but is obtained from the three cameras in use. The object is detected using only the image information obtained. Specifically, among the four cameras 2a to 2d, the states of the front camera 2a and the rear camera 2b are switched according to the position of the shift lever such that one is in the used state and the other is not in the used state.

  When the shift lever is in the drive position, the front camera 2a is in use and the rear camera 2b is not in use. When the shift lever is not in the drive position, the front camera 2a is not in use and the rear camera 2b is in use. On the other hand, the left side camera 2c and the right side camera 2d are always in use, and the states are not switched. That is, in the present embodiment, the front camera 2a and the rear camera 2b are the switching target cameras whose states are to be switched.

  In S12, the object detection device 6 switches the rear camera 2b to the use state and the front camera 2a to the non-use state, and then shifts the processing to S14. If the rear camera 2b is in use and the front camera 2a is not in use, that state is maintained.

  In S13, the object detection device 6 switches the front camera 2a to the use state and the rear camera 2b to the non-use state, and then shifts the processing to S14. If the front camera 2a is in use and the rear camera 2b is not in use, that state is maintained.

  In S14, the object detection device 6 acquires the image information at the present time from the three cameras in use and stores it in the RAM 63. That is, the image information for one frame is stored for each of the three cameras in use in each cycle of the object detection process of FIG. In the present embodiment, the image capturing time when the image information is captured, the traveling speed of the vehicle 1 at the image capturing time, and the steering angle of the vehicle 1 at the image capturing time are also stored in the RAM 63 together with the image information. Specifically, when the rear camera 2b is in use and the front camera 2a is not in use, image information of the rear camera 2b, image information of the left side camera 2c, and image information of the right side camera 2d are stored, The image information of the front camera 2a is not stored. When the front camera 2a is in use and the rear camera 2b is not in use, the image information of the front camera 2a, the image information of the left side camera 2c and the image information of the right side camera 2d are stored, and the rear camera 2b is stored. The image information of is not stored. In order to reduce the amount of information stored in the RAM 63, the object detection device 6 stores, in the RAM 63, only the information necessary for the process of S17 described later, instead of the image information, for example. May be.

  As described above, the object detection process is periodically executed, so that the image information of the three cameras in use is periodically stored. Specifically, while the shift lever is not in the drive position, that is, while the rear camera 2b is in use and the front camera 2a is not in use, image information of the rear camera 2b, image information of the left side camera 2c, and right side The image information of the direction camera 2d is continuously stored in time series. Then, the state of the switching target camera is switched by changing the position of the shift lever to the drive, and while the front camera 2a is in the use state and the rear camera 2b is in the non-use state, the front camera is replaced with the front camera 2b. The image information of 2a is continuously stored in time series. The image information stored in the RAM 63 while the position of the shift lever is the drive is held in the RAM 63 in a mode that can be used for the process of S17 described later even when the position of the shift lever is not the drive. The image information stored in the RAM 63 while the shift lever is not in the drive position is held in the RAM 63 in a mode that can be used in the process of S17, which will be described later, even while the shift lever is in the drive position.

  Subsequently, in S15, the object detection device 6 determines whether or not the state of the switching target camera has been switched in the current cycle. Specifically, when the rear camera 2b is switched from the unused state to the used state and the front camera 2a is switched from the used state to the unused state in S12 described above, it is determined that the state of the switching target camera is switched. To be done. Also, in S13 described above, even when the front camera 2a is switched from the unused state to the used state and the rear camera 2b is switched from the used state to the unused state, it is determined that the state of the switching target camera is switched. . On the other hand, when the state of the switching target camera is maintained in S12 or S13, it is determined that the state of the switching target camera is not switched.

When the object detection device 6 determines in S15 that the state of the switching target camera has been switched, the process proceeds to S16, the correction process described below is executed, and then the process proceeds to S17.
On the other hand, when the object detection device 6 determines in S15 that the state of the switching target camera has not been switched, the correction process of S16 is skipped and the process proceeds to S17.

  Here, the correction processing executed by the object detection device 6 will be described with reference to the flowchart of FIG. In the correction process, when it is determined that the state of the switching target camera is switched in S15 described above, the change in the movement amount and the direction of the vehicle 1 in the latest non-use period, which is the period in which the switching target camera is in the non-use state. This is a process of correcting the pre-switching image information using the amount. The non-use period is, in other words, a period from when the switching target camera is switched from the use state to the non-use state to when the switching target camera is switched from the non-use state to the use state.

  As shown in FIG. 4, in the present embodiment, the image information obtained from the switching target camera is used as the pre-switching image in the period before the time when the state of the switching target camera is switched last, in other words, in the past period. This is called information. In addition, in the present embodiment, after switching the image information obtained from the switching target camera that is currently in use in the period after the state in which the switching target camera is last switched, in other words, in the current period. This is referred to as image information.

  Returning to FIG. 3, first, in S21, the object detection device 6 determines whether the image information stored in the RAM 63 has passed a predetermined time or more from the image capturing time. In this embodiment, the time from the time when the image information is captured to the current time is calculated. Then, in the image information stored in the RAM 63, if there is even one image information that has passed the predetermined time or more from the image capturing time, it is determined that the predetermined time or more has passed. On the other hand, when all of the image information stored in the RAM 63 has not passed the predetermined time or more from the image capturing time, it is determined that the predetermined time has not passed.

When the object detection device 6 determines in S21 that the predetermined time or more has elapsed, the process proceeds to S22.
In S22, the object detection device 6 deletes the image information stored in the RAM 63 that has passed a predetermined time or more from the image capturing time, and shifts the processing to S23.

On the other hand, when the object detection device 6 determines in S21 that the predetermined time or more has not elapsed, the process of S22 is skipped and the process proceeds to S23.
In S23, the object detection device 6 determines whether or not at least one of the amount of movement and the amount of change in the direction of the vehicle 1 during the most recent non-use period of the switching target camera that is currently in use is equal to or greater than a preset threshold value. To determine. In the present embodiment, based on the traveling speed and the steering angle of the vehicle 1 stored in the RAM 63 during the non-use period of the switching target camera which is currently in use, the movement amount and the change amount of the direction of the vehicle 1 during the non-use period. Is calculated. Next, it is determined whether or not the first condition that the calculated movement amount of the vehicle 1 is equal to or larger than the first threshold value is satisfied. Further, it is determined whether or not the second condition that the calculated change amount of the direction of the vehicle 1 is equal to or larger than the second threshold value is satisfied. Then, when at least one of the first condition and the second condition is satisfied, it is determined that at least one of the movement amount and the change amount of the direction of the vehicle 1 is equal to or more than a preset threshold value. On the other hand, when both the first condition and the second condition are not satisfied, it is determined that both the movement amount and the change amount of the direction of the vehicle 1 are not equal to or more than the preset threshold value.

When the object detection device 6 determines in S23 that at least one of the movement amount and the change amount of the direction of the vehicle 1 is equal to or greater than the preset threshold value, the process proceeds to S24.
In S24, the object detection device 6 deletes the pre-switching image information of the switching target camera currently in use from the image information stored in the RAM 63. Then, the correction process of FIG. 3 is completed.

On the other hand, when the object detection device 6 determines in S23 that both the movement amount and the change amount of the direction of the vehicle 1 are not equal to or more than the preset threshold value, the process proceeds to S25.
In step S25, the object detection device 6 determines, based on the movement amount and the change amount of the direction of the vehicle 1 calculated in step S23, of the image information stored in the RAM 63 before switching the switching target camera which is currently in use. Correct image information. In the example shown in FIG. 4, the pre-switching image information to be corrected is the image information before time t4.

  As will be described later, in the present embodiment, a moving object and a stationary object are detected using image information of 5 frames at different image capturing times. An optical flow method is used to detect a moving object, and a moving stereo method is used to detect a stationary object.

  The pre-switching image information after correction generated as described below is used for detecting the moving object. First, the bird's-eye view conversion is performed on the image information from time t1 to time t4. Then, using the movement amount and the change amount of the direction of the vehicle 1 calculated in S23, the relative position based on the vehicle 1 of the bird's-eye converted image information at time t1 to time t4 is corrected. That is, the image information from time t1 to time t4 is corrected as if it is continuous in time series with the switched image information at time tx of the front camera 2a.

  On the other hand, for detection of a stationary object, post-correction image information after correction generated as follows is used. That is, the relative position of the object included in the image information from time t1 to time t4 based on the vehicle 1 is corrected using the movement amount of the vehicle 1, the change amount of the direction, and the parameter of the camera calculated in S23. . The camera parameters include information such as the camera mounting position in the vehicle 1 and lens distortion. Specifically, as shown in FIG. 5, for example, when the vehicle 1 is moving backward, the image A including the object detected while the vehicle 1 is moving backward is stored. When the position of the shift lever is changed to drive and the vehicle 1 moves forward several meters, and the position of the shift lever is changed to reverse again and the vehicle 1 moves backward, the position and size of the object in the image A are The image B corrected based on the movement amount of 1 or the like is generated. Then, the correction process of FIG. 3 is completed.

  Returning to FIG. 2, in S17, the object detection device 6 detects the object by using the latest 5 frame image information of each camera currently in use among the image information stored in the RAM 63. In the image information of the switching target camera, when the latest switched image information for 5 frames is stored, the object is detected using only the switched image information. On the other hand, immediately after the state of the switching target camera is switched, and when the post-switching image information for 5 frames is not stored, the post-switching image information as well as the post-switching image information is used to detect the object. Is detected. For example, when the post-switching image information for only one frame is stored, the latest pre-switching image information for the four frames is used.

  Then, in S17, the object detection device 6 alerts the driver by, for example, displaying a captured image in which the detected object is surrounded by a frame on the monitor 7. Then, the object detection process of FIG. 2 is completed.

[3. effect]
According to the embodiment described in detail above, the following effects can be obtained.
(3a) In the present embodiment, the object can be detected after the state of the switching target camera is switched, as compared with the case where only the image information after switching is used for detecting the object, that is, the case where the image information before switching is not used. It is possible to shorten the time required to reach the state. Therefore, even when the state of the switching target camera is switched, it is possible to detect the object earlier.

  Here, this effect will be described in comparison with a comparative example in which the pre-switching image information is not used when the state of the switching target camera is switched, that is, the pre-switching image information is deleted from the RAM 63. Specifically, when the position of the shift lever is changed from drive to reverse while the vehicle 1 is moving forward and the vehicle 1 moves backward by several meters, the position of the shift lever is changed to drive again and the vehicle 1 moves forward. An example will be described.

  First, in the comparative example, when the position of the shift lever is changed from the drive to the reverse, the image information of the front camera stored in the RAM while the position of the shift lever is the drive is deleted from the RAM. Therefore, when the position of the shift lever is changed to the drive again, the object detection device needs to store the image information of the front cameras, the number of which is necessary for detecting the object, from the beginning. Therefore, even if an object is detected in front of the vehicle before the position of the shift lever is changed from drive to reverse, it takes a certain amount of time to re-detect the object. As a result, the amount of movement of the vehicle increases and the vehicle may collide with the object.

  On the other hand, in this embodiment, when the position of the shift lever is changed from drive to reverse, the image information of the front camera 2a stored in the RAM 63 while the position of the shift lever is the drive is directly stored in the RAM 63. Retained. Therefore, even if the position of the shift lever is changed to the drive again, the object detection device can correct the pre-switching image information of the front camera 2a held in the RAM 63 and use it for detecting the object. Therefore, according to the present embodiment, even when the state of the switching target camera used for detecting the object around the vehicle 1 is switched, the object can be detected earlier.

  (3b) In the present embodiment, of the image information stored in the RAM 63, the image information that has passed a predetermined time or more from the image capturing time is deleted. As a result, it is possible to prevent image information with low reliability that has passed too much time from being used for detecting an object. Therefore, erroneous detection of an object can be suppressed.

  (3c) In the present embodiment, when at least one of the movement amount and the change amount of the direction of the vehicle 1 in the latest non-use period of the switching target camera which is currently in use is equal to or greater than a preset threshold value, The pre-switching image information of the switching target camera is deleted. Thereby, for example, it is possible to make it difficult to use image information that is out of the imaging range of the camera for detecting an object. Therefore, erroneous detection of an object can be suppressed.

Note that S12 and S13 correspond to the processing as the switching unit, S25 corresponds to the processing as the correction processing unit, and S17 corresponds to the processing as the detection unit.
[4. Other Embodiments]
Although the embodiments of the present disclosure have been described above, it is needless to say that the present disclosure is not limited to the above embodiments and can take various forms.

  (4a) In the above embodiment, the correction process is executed on the pre-switching image information of the switching target camera which is currently in use, triggered by the switching of the state of the switching target camera. However, for example, as the vehicle 1 moves, the correction process may be executed such that the pre-switching image information of the switching target camera that is currently in the unused state is gradually corrected.

  (4b) The pre-switching image information may not be used to detect an object when the switching target camera that is currently in use is not in use for a predetermined period of time or longer. That is, in the above-described embodiment, the image information that has passed the predetermined time from the image capturing time is deleted from the RAM 63. However, for example, the pre-switching image information of the switching target camera may be deleted from the RAM 63 when the latest non-use period of the switching target camera which is currently in use is a predetermined time or more.

  (4c) In the image information stored in the RAM 63, when the amount of movement of the vehicle 1 from the time when the image is captured is equal to or greater than a preset third threshold value, the pre-switching image information is used for object detection. You may not. Further, if the change amount of the orientation of the vehicle 1 from the time when the image is captured is equal to or larger than a preset fourth threshold value among the image information stored in the RAM 63, the pre-switch image information is used to detect the object. You may not be allowed. That is, in the above-described embodiment, when at least one of the movement amount and the change amount of the direction of the vehicle 1 in the latest non-use period of the switching target camera which is in use at the present time is equal to or greater than a preset threshold value, the switching is performed. The pre-switching image information of the target camera is deleted from the RAM 63. However, for example, in the image information stored in the RAM 63, an image satisfying the condition that at least one of the movement amount and the change amount of the direction of the vehicle 1 from the time when the image is captured is equal to or greater than a preset threshold value. The information may be deleted from the RAM 63.

(4d) The number of cameras to be switched is not limited to two, and may be one or three or more.
(4e) The functions of one constituent element in the above-described embodiment may be distributed as a plurality of constituent elements, or the functions of a plurality of constituent elements may be integrated into one constituent element. Moreover, you may omit a part of structure of the said embodiment. Further, at least a part of the configuration of the above-described embodiment may be added to or replaced with the configuration of the other above-described embodiment.

  (4f) The present disclosure provides, in addition to the above-described object detection device 6, a system including the object detection device 6 as a component, a program for causing a computer to function as the object detection device 6, a medium in which the program is recorded, and an object. It can be realized in various forms such as a detection method.

  1 ... Vehicle, 2a ... Front camera, 2b ... Rear camera, 2c ... Left side camera, 2d ... Right side camera, 3 ... Vehicle speed sensor, 4 ... Steering angle sensor, 5 ... Shift lever position sensor, 6 ... Object detection device, 7 ... Monitor, 61 ... CPU, 62 ... ROM, 63 ... RAM, 64 ... Flash memory.

Claims (5)

An object detection device (6) used in a vehicle (1) equipped with a plurality of cameras (2a to 2d),
A detection unit (S17) that performs detection processing for detecting an object using the plurality of pieces of image information obtained from the plurality of cameras and having different image capturing times;
A switching target camera, which is at least one of the plurality of cameras, is used in a state of use in which the image information is used in the detection process and in which the image information is used in the detection process, according to a running state of the vehicle. A switching unit (S12, S13) for switching to a non-used state,
Equipped with
When the switching target camera is switched from the non-use state to the use state, the detection unit includes pre-switching image information obtained from the switching target camera before being switched to the non-use state, and the use state. An object detection device that performs the detection process using the switched image information obtained from the switching target camera after being switched to. The object detection device according to claim 1, wherein
Further comprising a correction processing unit (S25) that corrects the pre-switching image information by using the movement amount and the change amount of the direction of the vehicle during the period when the switching target camera is in the non-use state,
When the switching target camera is switched from the non-use state to the use state, the detection unit performs the detection process using the corrected pre-switching image information and the post-switching image information, Object detection device. The object detection device according to claim 1 or 2, wherein
The object detection device, wherein the detection unit does not use the pre-switching image information for which a predetermined time or more has elapsed from the time of being imaged, in the detection processing. The object detection device according to any one of claims 1 to 3,
The detection unit does not use the pre-switching image information in the detection process when the amount of movement of the vehicle during the period in which the switching target camera is in the unused state is equal to or greater than a preset first threshold value. , Object detection device. The object detection device according to any one of claims 1 to 4,
When the change amount of the orientation of the vehicle during the period when the switching target camera is in the non-use state is equal to or more than a second threshold value set in advance, the detection unit sets the pre-switching image information to the detection process. Object detection device not used.

Images