WO2021235001A1 - 画像処理装置 - Google Patents

画像処理装置 Download PDF

Info

Publication number
WO2021235001A1
WO2021235001A1 PCT/JP2021/002142 JP2021002142W WO2021235001A1 WO 2021235001 A1 WO2021235001 A1 WO 2021235001A1 JP 2021002142 W JP2021002142 W JP 2021002142W WO 2021235001 A1 WO2021235001 A1 WO 2021235001A1
Authority
WO
WIPO (PCT)
Prior art keywords
pitch angle
search range
image processing
reliability
unit
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2021/002142
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
圭汰 仲澤
哲也 山田
宏治 土井
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Astemo Ltd
Original Assignee
Hitachi Astemo Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hitachi Astemo Ltd filed Critical Hitachi Astemo Ltd
Priority to JP2022524877A priority Critical patent/JP7350168B2/ja
Publication of WO2021235001A1 publication Critical patent/WO2021235001A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R11/00—Arrangements for holding or mounting articles, not otherwise provided for
    • B60R11/02—Arrangements for holding or mounting articles, not otherwise provided for for radio sets, television sets, telephones, or the like; Arrangement of controls thereof
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/16—Anti-collision systems
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules

Definitions

  • the present invention relates to an in-vehicle control device installed in a vehicle such as a motorcycle, which acquires an outside world situation from a camera, recognizes the outside world situation, and generates a control signal, and particularly relates to an image processing device.
  • ACC Adaptive Cruise Control
  • the difference between a two-wheeled vehicle and a four-wheeled vehicle is that the amount of subduction in the front and rear of the vehicle body caused by braking, driving, road surface conditions, etc. is large.
  • ACC is realized by using a camera for image recognition
  • the tracking process of the camera cannot catch up and there is a high possibility that the tracking fails.
  • a camera for image recognition is used, not limited to the ACC, there is a problem corresponding to the vehicle behavior, particularly the inclination of the pitch angle, which occurs in the vehicle.
  • Patent Document 1 discloses a technique of changing the cutting range in consideration of the vehicle behavior when the image processing unit generates an image to be used for recognition from the captured image.
  • Patent Document 1 in a camera developed for other vehicles such as four wheels, when the image processing unit cuts out an image to be used in the recognition unit from the captured image, the pitch angle generated in the vehicle is obtained from the sensor information, and the pitch is obtained. It is described that a cropping method is used in which the influence of the pitch angle is removed at the stage of generating an image to be passed to the recognition unit by setting the cropping range of the captured image in the image processing unit according to the angle.
  • Patent Document 1 When the technique of Patent Document 1 is applied to tracking processing as it is, when the cutting position correction direction based on the pitch angle and the moving direction of the detection target predicted by the tracking processing are different, the image in the direction in which the detection target moves is in the image processing stage. If it is cut off with, it may not be possible to capture the detection target within the search range. Further, in the prior art, since the pitch angle of the own vehicle is estimated from the sensor information and the cutting range is changed, there is a possibility that the cutting range may be erroneous when the sensor information outputs an invalid value. Therefore, the cutting method, which is a conventional technique, cannot solve the problem of accuracy of tracking processing when a pitch angle occurs.
  • An object of the present invention is to provide an image processing apparatus capable of improving the accuracy of tracking processing.
  • the image processing device of the present invention uses a tracking processing unit that sets a search range for a target object for each of a plurality of frame images captured by an in-vehicle camera and tracks the target object in chronological order, and vehicle pitch angle information. It is characterized by having a search range calculation unit for changing the search range.
  • an image processing device capable of improving the accuracy of tracking processing. Further features relating to the present invention will be apparent from the description herein and the accompanying drawings. In addition, problems, configurations, and effects other than those described above will be clarified by the following description of the embodiments.
  • the schematic block diagram of the image processing apparatus in Example 1 of this invention An example of the tracking process supported by the present invention. An example showing a difference from the prior art in Example 1 of the present invention. An example in which the search range is set according to the reliability in the first embodiment of the present invention.
  • the flowchart of image processing in Example 1 of this invention An example of application to the roll angle in Example 1 of the present invention.
  • the schematic block diagram of the image processing apparatus in Example 2 of this invention. The flowchart of image processing in Example 2 of this invention.
  • the schematic block diagram of the image processing apparatus in Example 3 of this invention The flowchart of image processing in Example 3 of this invention.
  • the image processing device of the present invention is applied to a motorcycle will be described as an example, but the present technology is not limited to the motorcycle.
  • it can be applied to all vehicles in which pitching can occur, such as a tricycle such as a three-wheeled motorcycle and a four-wheeled vehicle such as a truck.
  • the detection target is detected from the past frame (frame n-1), and an identifier is added to the detection target.
  • the identifier A is added to the motorcycle and the identifier B is added to the preceding vehicle.
  • the positions and moving directions of the detection targets A and B are predicted using the Kalman filter from the tracking processing results in the past frames.
  • the search range of the detection targets A and B in the current frame (frame n) is set from the prediction result.
  • the search range is set by predicting that the motorcycle A moves in the direction approaching the own vehicle, that is, moving to the lower left in the image, and the preceding vehicle B is from the own vehicle.
  • the search range is set by predicting that the vehicle will move upward in the direction away from the image, that is, in the image.
  • search the search range currently set in the frame After associating the detection result of the object in the current frame with the prediction result, it is determined whether the detection target detected in the current frame and the detection targets A and B in the past frame are similar objects. If it is determined that the detection target of the current frame is an object similar to the detection targets A and B of the past frame, the same identifier is added, and if it is determined that the object is different from the past frame, a new identifier is added.
  • the above tracking process recognizes the past frame and the current frame in association with each other, but at that time, it is processed on the premise that the camera position is constant.
  • the camera itself is tilted and the position of the detection target in the frame moves. If the position of the detection target in the frame changes depending on the pitch angle, there is a problem that the detection target cannot be associated between the past frame and the current frame, and tracking fails. Therefore, in the tracking process, it is necessary to deal with the vehicle behavior that occurs in the vehicle, particularly the inclination of the pitch angle.
  • motorcycles and other two-wheeled vehicles have the characteristic that the amount of subduction in the front and rear of the vehicle body caused by braking, driving, road surface conditions, etc. is large. Therefore, for example, when an image processing device for a passenger car is mounted on a motorcycle as it is and recognition processing is performed using an image captured by an image recognition camera, it becomes impossible to capture the detection target within the search range. There is a possibility that tracking may fail due to the influence of the pitch angle generated on the body of the motorcycle in the tracking process.
  • the image processing unit does not cut out an image in consideration of the vehicle behavior as in the conventional case, but the recognition unit of the recognition camera receives information on the inclination of the pitch angle of an arbitrary vehicle. Execute the process to be used.
  • the problem (1) how to set the search range of the tracking process when the pitch angle changes, and the problem (2) how to track when the pitch angle information of the vehicle cannot be accurately acquired.
  • the purpose is to set the search range of the process or to solve these problems (1) and (2).
  • the search range set from the movement direction of the detection target predicted from the previous frame by the tracking process is searched by using the pitch angle information generated in the vehicle.
  • the range is updated, and for the above problem (2), the reliability for determining whether the acquired pitch angle information is accurate is set, and the search range of the tracking process can be changed according to the pitch angle information and the reliability.
  • the means for using the pitch angle sensor will be described using the first embodiment, and the second embodiment will be used for the means for using the output of the vehicle control unit instead of the pitch angle sensor.
  • the means for using the control driving force calculated inside the image processing device instead of the pitch angle sensor will be described with reference to Example 3.
  • the pitch angle information is acquired from the pitch angle sensor 2 for the pitch angle of the vehicle that changes from moment to moment, and the change in the position of the detection target on the recognition image when the pitch angle occurs is used as the deviation information 51. It is output, and whether the output of the deviation information 51 is accurate is output as the reliability 52. Then, a method of setting the search range of the detection target on the recognition image is provided from the deviation information 51, the reliability 52, and the prediction result of the detection target calculated by the tracking processing unit 10.
  • the conventional technology estimates the pitch angle generated in the own vehicle from the sensor information, and adopts a cutting method that changes the cutting position in the image processing unit in response to the change in the position of the detection target in the captured image due to the pitch angle.
  • We are trying to deal with it see FIG. 3 (a)).
  • the focus is on improving the recognition accuracy mainly in consideration of the matching process for detecting an object, and the tracking process is not specified.
  • the tracking process tracks the detected target by predicting the position and moving direction of the detected target, setting the search range, and associating the detected target with each other in a plurality of frames. In the tracking process, it is necessary to set the search range in consideration of the moving direction of the detection target in addition to the pitch angle information generated in the own vehicle.
  • the cutout position in the image is moved upward in order to cope with the change in the pitch angle. Therefore, in normal tracking processing, the search range is set in consideration of the moving direction of the detection target of the identifiers A and B. Therefore, if the movement direction of the detection target of the identifier A is downward, the image processing unit moves. The image in the direction is cut off, and the search range for identifying the identifier A cannot be sufficiently set. Therefore, there is a possibility that the mapping of the identifier A fails and the detection target of the identifier A cannot be identified.
  • the problem of pitch angle change is to be addressed by correcting the cutting position when the image processing unit generates a recognition image.
  • the image processing unit does not perform cutting in consideration of the pitch angle information when generating the recognition image.
  • the recognition unit 6 uses the pitch angle information to variably set the search range of the detection target in the tracking process according to the pitch angle information (see FIG. 3C).
  • the image processing unit since the image processing unit does not perform cutting in consideration of the pitch angle information, the problem of the prior art that the moving direction of the detection target is cut in the tracking processing does not occur. Since the search range is set based on the pitch angle information, the detection target can be captured within the search range even when the pitch angle changes in the vehicle as long as the detection target exists within the range of the captured image. Therefore, even if the pitch angle changes, it is possible to associate the detection target among a plurality of frames.
  • the configuration diagram of this embodiment is shown in FIG.
  • the vehicle is provided with a camera 1 and a pitch angle sensor 2, which are connected using a CAN bus 3.
  • the pitch angle sensor 2 is not particularly limited, but for example, an inertial measurement unit IMU (Inertial Measurement Unit) can be used.
  • IMU Inertial Measurement Unit
  • a three-dimensional angle including a pitch angle, an angular velocity, and an acceleration are obtained. These data are transmitted as data such as CAN and Ethernet.
  • FIG. 1 shows the internal configuration of the camera 1 which is an image processing device.
  • the camera 1 includes an image pickup unit 4, an image processing unit 5, a recognition unit 6, a control generation unit 7, and a communication interface unit 8.
  • the image pickup unit 4 captures an image with an image pickup element such as a CMOS sensor.
  • the image processing unit 5 uses the image captured by the image pickup unit 4 to generate an image to be used in the recognition process.
  • the recognition unit 6 detects vehicles and lanes, and tracks the time series of detection targets.
  • the control generation unit 7 generates a control signal to an actuator such as a vehicle brake or an accelerator based on the recognition result of the recognition unit 6.
  • the communication interface unit 8 communicates the camera with an external ECU such as CAN.
  • the recognition unit 6 includes a storage unit 9, a tracking processing unit 10, and a change amount calculation unit 20.
  • the storage unit 9 stores past recognition results such as detection of other vehicles such as the preceding vehicle and lanes of the lane, road surface recognition, and tracking processing results.
  • the tracking processing unit 10 sets a search range from the prediction information that predicts the movement of the detection target by using the Kalman filter for the detection target to which the identifier is added, and performs time-series tracking of the detection target.
  • the change amount calculation unit 20 outputs the vertical change amount of the target object in the frame image as the deviation information 51 from the pitch angle information.
  • the change amount calculation unit 20 outputs the change in the position of the detection target on the recognition image when the pitch angle is generated from the pitch angle information 50 acquired from the pitch angle sensor 2 as the deviation information 51, and also outputs the deviation information. Is output as the reliability 52.
  • the pitch angle information 50 includes a pitch angle generated in the own vehicle, a pitch angle change amount per unit time, and a pitch angle change direction.
  • the tracking method of the detection target using the Kalman filter is described as an example, but the tracking processing method is not limited to this example.
  • the tracking processing unit 10 has a search range calculation unit 11 that changes the search range from the output of the change amount calculation unit 20 and the prediction information of the detection target with respect to the search range set by the prediction information of the detection target.
  • the tracking processing unit 10 updates the search range to the search range changed by the search range calculation unit 11.
  • the tracking processing unit 10 determines whether or not the detection target exists in the updated search range by performing a full search in the search range. As a result of the search, if there is a detection target to which tracking is performed and an identifier is added in the past, the same identifier is added to track the detection target. When a new detection target is discovered, a new identifier is added.
  • the deviation information 51 may include a change amount in the vertical direction and a change amount in the left-right direction, and may output the change amount of the detection target on the recognition image as a pixel amount.
  • the reliability 52 is calculated from the amount of change in the pitch angle per unit time, and when the amount of change in the pitch angle per unit time is larger than the threshold value, it is output as low reliability and the amount of change in the pitch angle per unit time is When it is smaller than the threshold value, it is output as high reliability.
  • the reliability 52 is treated as a numerical value, but a level, an electric signal, or the like may be used as the reliability.
  • the reliability 52 is not only calculated from the amount of change in the pitch angle per unit time, but it is also possible to set the reliability according to the road surface condition during traveling based on the road surface recognition information stored in the storage unit 9. be. For example, when it is recognized that the road surface condition is uneven or a steep slope, it is output as low reliability, and conversely, when it is determined that the road surface is flat with few irregularities, it is output as high reliability.
  • the reliability 52 is not limited to high reliability and low reliability, and may be used in multiple stages.
  • the search range calculation unit 11 changes the search range according to the reliability, as shown in FIG.
  • the search range in the tracking process is reduced as compared with the case where the reliability 52 is the threshold value. This makes it possible to reduce unnecessary CPU load.
  • the search range in the tracking process is expanded as compared with the case where the reliability 52 is the threshold value. This makes it possible to take a large margin and respond to momentary changes in the pitch angle.
  • FIG. 5 shows a flowchart for setting the search range of the recognition image in the tracking process using the pitch angle information acquired from the pitch angle sensor 2.
  • the detection target is detected from the recognition image, and an identifier is added to the detection target (S1). Then, the past tracking processing result is acquired from the storage unit 9 (S2). The movement direction of the detection target is predicted from the past tracking information, and the search range within the frame is set (S3).
  • the pitch angle information 50 of the pitch angle sensor 2 is acquired through the communication interface 8 (S4).
  • the change amount calculation unit 20 calculates the deviation information 51 of the detection target in the frame and its reliability 52 from the pitch angle information 50 (S5). Subsequently, it is determined whether the reliability 52 is high or low (S6). When the reliability 52 is larger than the threshold value, it is determined to be high, and when it is smaller than the threshold value, it is determined to be low.
  • the vertical direction of the search range set in the frame is expanded beyond the preset reference search range (S7).
  • the vertical direction of the search range is reduced from the preset reference search range (S8).
  • the position of the search range is updated according to the deviation information 51 (S9).
  • the tracking processing unit 10 searches within the updated search range, and more specifically, searches the entire search range to see if the detection target to which the identifier is added according to the updated search range exists in the search range. Then, when the detection target exists within the updated search range, the same identifier is added and confirmed (S10). Then, the result of the tracking process performed by the above flow is stored in the storage unit (S11).
  • the camera 1 of the present embodiment uses a tracking processing unit 10 that sets a search range of a target object for each of a plurality of frame images captured by the camera 1 and tracks the target object in chronological order, and pitch angle information of the vehicle. It also has a search range calculation unit 11 that changes the search range of the tracking processing unit 10.
  • the camera 1 is generated by an image pickup unit 4 that captures an image, an image processing unit 5 that generates an image, a plurality of pitch angles output from the pitch angle sensor 2, and an image processing unit 5. It is provided with a recognition unit 6 that performs image recognition from an image. Then, the recognition unit 6 includes a tracking processing unit 10 that performs tracking processing for tracking an object in the frame in time series, a storage unit 9 that stores past recognition results, and deviation information 51 of the object in the frame from the pitch angle. It has a change amount calculation unit 20 for calculating.
  • the tracking processing unit 10 includes a search range calculation unit 11 that sets a search range in the tracking processing from the input from the change amount calculation unit 20 and the past recognition result. Then, the change amount calculation unit 20 calculates whether the deviation information 51 is accurate in addition to the deviation information 51, and outputs it as the reliability 52.
  • the search range calculation unit 11 sets the search range in the recognized image by using the deviation information 51, the reliability 52, and the past tracking processing result.
  • the search range is set from the movement direction of the detection target predicted from the previous frame by the tracking process, and the pitch angle generated in the motorcycle is used for the set search range. , Change the search range of the tracking process. Then, a reliability 52 for determining whether the acquired pitch angle information is accurate is set, and the search range of the tracking process is made variable by the reliability 52.
  • the search range calculation unit 11 makes the search range variable according to the reliability 52. For example, if it is determined that the reliability 52 is low, the search range is expanded, and if it is determined that the reliability 52 is high, the search range is reduced.
  • the change amount calculation unit 20 calculates the deviation information (vertical change amount) 51 of the object in the recognition image from the pitch angle information 50 acquired from the pitch angle sensor 2 as the pixel amount, and calculates the calculation accuracy of the search range calculation unit 11. Improve.
  • the pitch angle which is the vehicle behavior that occurs in the front-rear direction of the vehicle, has been focused on, but even when the roll angle occurs in the left-right direction of the vehicle as shown in FIG.
  • the search range can be expanded or contracted as shown in FIG. 6B by the same method based on the information 51 and the reliability 52, and the present invention is applied even when the pitch angle and the roll angle occur at the same time. It is possible.
  • Example 2 In the second embodiment, instead of acquiring the pitch angle information 50 from the pitch angle sensor 2, a method of acquiring the pitch angle information 50 from the output of the control device that controls the braking force and the driving force of the vehicle will be described. It is possible to calculate the pitch angle generated in the vehicle in real time from the output of the control device that controls the control driving force of the vehicle, and the present proposal can be applied to the vehicle that does not have the pitch angle sensor. The method is added to the first embodiment.
  • Example 1 The configuration diagram of this embodiment is shown in FIG. The differences from Example 1 are as follows. Instead of the pitch angle sensor 2, a driving force control device 40 and a braking force control device 41 are used. Then, the recognition unit 6 acquires the control driving force control amount 53 of the driving force control device 40 and the braking force control device 41 via the communication interface 8. In this embodiment, the pitch angle information calculation unit 30 for calculating the pitch angle information 50 generated in the vehicle is added to the first embodiment.
  • the detection target is detected from the recognition image, and an identifier is added to the detection target (S1).
  • the past tracking processing result is acquired from the storage unit 9 (S2). Then, the moving direction of the detection target is predicted from the acquired past tracking information, and the search range within the frame is set (S3).
  • the control driving force control amount 53 is acquired from the driving force control device 40 and the braking force control device 41, and the pitch angle information calculation unit 30 calculates the pitch angle information 50 (S20).
  • the calculation method of the pitch angle information 50 is performed by using a known method.
  • the change amount calculation unit 20 calculates the deviation information 51 of the detection target in the frame and its reliability 52 from the pitch angle information 50 (S5). Subsequently, it is determined whether the reliability 52 is high or low (S6), and if it is determined that the reliability 52 is low, the search range is expanded in the vertical direction (S7). If it is determined that the reliability 52 is high, the vertical direction of the search range is reduced (S8). After that, the position of the search range is updated according to the deviation information 51 (S9).
  • the tracking processing unit 10 searches within the updated search range, and searches the entire search range to see if the detection target to which the identifier is added exists within the search range. Then, when the detection target exists, the same identifier is added and confirmed (S10). Then, the result of the tracking process performed by the above flow is stored in the storage unit 9 (S11). As described above, the process of S20 is added instead of the process of S4 of the first embodiment.
  • the image processing device of the second embodiment has a communication interface 8 for communicating with the control device of the vehicle, and the recognition unit 6 has a driving force control amount 53 from the driving force control device 40 and the braking force control device 41.
  • the pitch angle information calculation unit 30 that calculates the pitch angle information 50 from the control driving force control amount 53 is added to the configuration of the first embodiment, so that the pitch angle sensor 2 is not provided. It is possible to set the search range for the vehicle in consideration of the vehicle behavior, and it is possible to improve the tracking accuracy.
  • Example 3 In the third embodiment, instead of acquiring the pitch angle information 50 from the pitch angle sensor 2 of the first embodiment, it is output from the control generation unit 7 that generates a control signal to an actuator such as a brake or an accelerator based on the recognition result. It is characterized in that the pitch angle information 50 is acquired from the target control driving force control amount 54.
  • the third embodiment is configured to be completed within the camera 1, and does not require the pitch angle sensor of the first embodiment or the external control driving force control amount 53 as in the second embodiment. Therefore, it can be expected to minimize the work of conforming to each vehicle type. The method is added to the first embodiment.
  • Example 3 The block diagram of Example 3 is shown in FIG. The differences from Example 1 are as follows. Instead of the pitch angle sensor 2 of the first embodiment, a pitch angle information calculation unit 60 that calculates the pitch angle information 50 by inputting the target control driving force control amount 54 output from the control generation unit 7 is added.
  • the target driving force control amount 54 is a control amount calculated inside the camera, the change in the control amount due to the override when the driver operates the brake or the accelerator is not included.
  • a difference occurs between the pitch angle information 50 calculated by the target driving force control amount 54 and the pitch angle actually generated in the vehicle.
  • a driver operation detection unit 70 that acquires driver operation information 55 indicating whether or not the driver has operated the brake or accelerator via the communication interface 8 and outputs override information 56. ..
  • the detection target is detected from the recognition image, and an identifier is added to the detection target (S1).
  • the past tracking processing result is acquired from the storage unit 9 (S2).
  • the movement direction of the detection target is predicted from the past tracking information, and the search range within the frame is set (S3).
  • the pitch angle information calculation unit 60 calculates the pitch angle information 50 from the target driving force control amount 54 calculated by the control generation unit 7 in the camera from the recognition result of the camera (S30). Next, the change amount calculation unit 20 calculates the deviation information 51 of the detection target in the frame and its reliability 52 from the pitch angle information 50 (S5).
  • the driver operation information 55 is acquired via the communication interface unit 8 (S31). With the driver operation information 55 as an input, it is determined whether or not the override is detected by the driver operation detection unit 70 (S32). When the driver's operation detection unit 70 detects the driver's override, the search range in the tracking process is expanded to the maximum (S33).
  • the search range is expanded in the vertical direction (S7). If it is determined that the reliability is high, the vertical direction of the search range is reduced (S8). After that, the position of the search range is updated according to the deviation information 51 (S9).
  • the tracking processing unit 10 searches within the updated search range, and searches the entire search range to see if the detection target to which the identifier is added exists within the search range. Then, when the detection target exists, the same identifier is added and confirmed (S10). Then, the result of the tracking process performed by the above flow is stored in the storage unit 9 (S11).
  • the processing of S30 is added instead of S4 in Example 1, and the processing of S31, S32, and S33 is further added.
  • the method of calculating the pitch angle information 50 from the target driving force control amount 54 is different, but a method of setting the search range of the tracking process using the deviation information 51 of the detection target and its reliability 52. Is the same as that of the first embodiment.
  • the pitch angle information 50 calculated from the target driving force control amount 54 has a difference from the pitch angle generated in the vehicle, so that the search range is accurate. Will disappear. Therefore, when an override is detected, it is determined that the reliability is the lowest, and the search range is expanded to the maximum.
  • the camera 1 has a control generation unit 7 that calculates a control amount calculated internally for controlling the vehicle from the recognition result of the recognition unit 6, and the pitch angle information calculation unit 60 has a pitch angle information calculation unit 60.
  • the pitch angle information 50 is calculated from the output of the control generation unit 7, and the deviation information 51 of the object in the frame is calculated from the pitch angle information 50 output from the pitch angle information calculation unit 60 by the change amount calculation unit 20.
  • the recognition unit 6 has a driver operation detection unit 70 that detects the driver's operation, and when an override is detected, the change amount calculation unit 20 outputs that the reliability is low.
  • the technology introduced in all the examples can be applied to both stereo cameras and monocular cameras. This proposal is also applicable to LiDAR, which outputs point cloud information. Further, although the pitch angle information is obtained by three methods in all the embodiments, it is not particularly limited as long as it is a means for calculating the pitch angle information in real time.
  • the pitch angle can be set in the vehicle equipped with the recognition camera by adding a device corresponding to the pitch angle for the two-wheeled vehicle to the recognition camera developed for the four-wheeled vehicle. Even if it occurs, by setting the search range using the pitch angle information during the tracking process, it is possible to track the detection target in time series as long as the detection target exists in the captured image.
  • the search range is expanded when sensor information such as the pitch angle sensor outputs an invalid value, and by increasing the margin, the detection target is out of the search range for tracking. It is possible to solve the problem that fails. If the reliability is high, the search range can be reduced and unnecessary calculation load can be eliminated. In the past, when the calculation load was high, processing such as thinning out the search range was performed, but this invention eliminates the need for processing, which contributes to improvement in recognition accuracy. In addition, effects such as suppressing the temperature rise of the hardware can be expected.
  • the search range can be set in consideration of the vehicle behavior even for a vehicle not provided with the pitch angle sensor, and according to the third embodiment, the present invention can be realized inside the camera. It can be expected to minimize the work of adapting to each vehicle type. Further, in the first to third embodiments, the technique has been described focusing on the pitch angle generated in the vehicle, but the technique can be applied not only to the pitch angle but also to the case where the pitch angle and the roll angle are generated at the same time.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Theoretical Computer Science (AREA)
  • Signal Processing (AREA)
  • Mechanical Engineering (AREA)
  • Traffic Control Systems (AREA)
  • Image Analysis (AREA)
  • Fittings On The Vehicle Exterior For Carrying Loads, And Devices For Holding Or Mounting Articles (AREA)
  • Studio Devices (AREA)
PCT/JP2021/002142 2020-05-22 2021-01-22 画像処理装置 Ceased WO2021235001A1 (ja)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2022524877A JP7350168B2 (ja) 2020-05-22 2021-01-22 画像処理装置

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2020090109 2020-05-22
JP2020-090109 2020-05-22

Publications (1)

Publication Number Publication Date
WO2021235001A1 true WO2021235001A1 (ja) 2021-11-25

Family

ID=78707916

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2021/002142 Ceased WO2021235001A1 (ja) 2020-05-22 2021-01-22 画像処理装置

Country Status (2)

Country Link
JP (1) JP7350168B2 (https=)
WO (1) WO2021235001A1 (https=)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2002117392A (ja) * 2000-10-06 2002-04-19 Nissan Motor Co Ltd 車間距離推定装置
WO2011040119A1 (ja) * 2009-09-30 2011-04-07 日立オートモティブシステムズ株式会社 車両制御装置
JP2015210538A (ja) * 2014-04-23 2015-11-24 本田技研工業株式会社 光源検出装置、前照灯制御システム、および光源検出方法
WO2015177985A1 (ja) * 2014-05-22 2015-11-26 ヤマハ発動機株式会社 ピッチ角制御システム、ピッチ角制御方法および車両
WO2017138286A1 (ja) * 2016-02-12 2017-08-17 日立オートモティブシステムズ株式会社 移動体の周囲環境認識装置
WO2020090320A1 (ja) * 2018-10-31 2020-05-07 ソニーセミコンダクタソリューションズ株式会社 情報処理装置、情報処理方法及び情報処理プログラム

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2002117392A (ja) * 2000-10-06 2002-04-19 Nissan Motor Co Ltd 車間距離推定装置
WO2011040119A1 (ja) * 2009-09-30 2011-04-07 日立オートモティブシステムズ株式会社 車両制御装置
JP2015210538A (ja) * 2014-04-23 2015-11-24 本田技研工業株式会社 光源検出装置、前照灯制御システム、および光源検出方法
WO2015177985A1 (ja) * 2014-05-22 2015-11-26 ヤマハ発動機株式会社 ピッチ角制御システム、ピッチ角制御方法および車両
WO2017138286A1 (ja) * 2016-02-12 2017-08-17 日立オートモティブシステムズ株式会社 移動体の周囲環境認識装置
WO2020090320A1 (ja) * 2018-10-31 2020-05-07 ソニーセミコンダクタソリューションズ株式会社 情報処理装置、情報処理方法及び情報処理プログラム

Also Published As

Publication number Publication date
JP7350168B2 (ja) 2023-09-25
JPWO2021235001A1 (https=) 2021-11-25

Similar Documents

Publication Publication Date Title
US11186279B2 (en) Control device for vehicle travelling
JP6996353B2 (ja) 物体認識装置及び車両走行制御システム
JP6323473B2 (ja) 走行制御装置
US10755421B2 (en) Tracking device
CN110023164A (zh) 车辆运动控制装置、车辆运动控制方法和车辆运动控制系统
US12139137B2 (en) Simulator for evaluating vehicular lane centering system
CN116034359A (zh) 用于以至少两个互不依赖的成像环境检测传感器来进行环境检测的方法、用于执行该方法的设备、车辆以及对应设计的计算机程序
JP6936098B2 (ja) 対象物推定装置
JP2019067115A (ja) 路面検出装置
CN111989541A (zh) 立体摄像机装置
CN112955356A (zh) 停车辅助装置及停车辅助方法
CN112714718B (zh) 车辆控制方法及车辆控制装置
JP6481627B2 (ja) 車両用走行制御装置
US12377838B2 (en) Driving assistance device, driving assistance method, and storage medium
US12391239B2 (en) Driving assistance device, driving assistance method, and storage medium
JP7236849B2 (ja) 外界認識装置
JP7350168B2 (ja) 画像処理装置
JP2017052412A (ja) 運転支援装置、及び運転支援方法
JP2006298254A (ja) 走行支援装置
US20230260293A1 (en) Travel environment recognition apparatus
JP7181956B2 (ja) 移動体の制御装置及び制御方法並びに車両
CN120716398B (zh) 一种车辆悬架预瞄控制的方法和装置
JP6423190B2 (ja) 車両の挙動制御装置及び車両の挙動制御方法
JP6082293B2 (ja) 車両用白線認識装置
JP7788008B2 (ja) 画像処理装置及び画像処理方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21809460

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2022524877

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21809460

Country of ref document: EP

Kind code of ref document: A1