JP2014104840A - On-vehicle image processing system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To initialize a recognized result at a proper timing in each image recognition processing.SOLUTION: An on-vehicle image processing system 100 comprises: an image acquisition part 1 for acquiring a photographed image from each of a front camera 200 and a rear camera 201; a travel direction detection part 2 for detecting a direction of travel of an own vehicle; an image selection part 3 for selecting an image photographed by a camera for photographing the direction of travel of the own vehicle detected by the travel direction detection part 2; an environment recognition part 6 for recognizing an environment of the own vehicle on the basis of the image selected by the image selection part 3; a foreign matter recognition part 7 for recognizing a foreign matter adhering to a camera lens on the basis of the image selected by the image selection part 3; a direction change detection part 5 for detecting a change of the direction of travel detected by the travel direction detection part 2; and a behavior detection part 8a for detecting a behavior of the own vehicle by the direction change detection part 5. The processing system 100 is further equipped with an initialization part 8 which initializes the result recognized by the environment recognition part 6 when a first behavior is detected by the behavior detection part 8a, and initializes the result recognized by the foreign matter recognition part 7 when a second behavior is detected.

Description

  The present invention relates to an in-vehicle image processing apparatus mounted on a vehicle.

  There is known an in-vehicle image processing apparatus that automatically selects videos of a plurality of cameras installed in a vehicle according to driving conditions and performs image processing according to the driving conditions on the videos (Patent Literature). 1).

JP 2007-22176 A

  In an in-vehicle image processing device that performs multiple image recognition processes using video from multiple cameras, when initializing the recognition results of each image recognition process according to the behavior of the vehicle, it is initialized at an appropriate timing for each image recognition process. Could not be converted.

  An in-vehicle image processing apparatus according to the present invention includes an image acquisition unit that acquires captured images from a front camera that captures the front of the vehicle and a rear camera that captures the rear of the vehicle, and a traveling direction detection unit that detects the traveling direction of the vehicle. And an image selection unit that selects a captured image of the first camera that captures the traveling direction of the vehicle detected by the traveling direction detection unit from the front camera and the rear camera, and a captured image of the first camera, An external recognition unit that recognizes the external environment of the vehicle, a foreign matter recognition unit that recognizes a foreign matter attached to the lens of the first camera based on a captured image of the first camera, and a change in the traveling direction detected by the traveling direction detection unit. It has a direction change detection unit for detecting and a behavior detection unit for detecting the behavior of the vehicle. When the first behavior is detected by the behavior detection unit, the recognition result of the external recognition unit is initialized, and the behavior detection unit First Behavior, characterized in that it comprises an initialization unit for initializing the recognition result of the foreign object recognition unit when detecting.

  According to the present invention, when the recognition result of each image recognition process is initialized according to the behavior of the vehicle, the recognition result can be initialized at an appropriate timing for each image recognition process.

It is a block diagram which shows the structure of the vehicle-mounted image processing apparatus by the 1st Embodiment of this invention. It is a flowchart regarding operation | movement of the vehicle-mounted image processing apparatus by the 1st and 2nd embodiment of this invention. It is a flowchart regarding operation | movement of the direction switching detection part in the vehicle-mounted image processing apparatus by the 1st and 2nd embodiment of this invention. It is a flowchart regarding operation | movement of the initialization part in the vehicle-mounted image processing apparatus by the 1st Embodiment of this invention. It is a flowchart regarding operation | movement of the external field recognition part in the vehicle-mounted image processing apparatus by the 1st and 2nd embodiment of this invention. It is a flowchart regarding operation | movement of the foreign material recognition part in the vehicle-mounted image processing apparatus by the 1st and 2nd embodiment of this invention. It is a flowchart regarding operation | movement of the initialization part in the vehicle-mounted image processing apparatus by the 2nd Embodiment of this invention. It is a flowchart regarding operation | movement of the initialization part in the vehicle-mounted image processing apparatus by the 2nd Embodiment of this invention.

-First embodiment-
FIG. 1 is a block diagram showing a configuration of an in-vehicle image processing apparatus 100 according to the first embodiment of the present invention. An in-vehicle image processing apparatus 100 illustrated in FIG. 1 is a control device mounted on a vehicle, and includes an image acquisition unit 1, a traveling direction detection unit 2, an image selection unit 3, an image processing unit 4, and direction switching detection. Unit 5, external recognition unit 6, foreign object recognition unit 7, initialization unit 8, and information storage unit 9. Hereinafter, a vehicle on which the in-vehicle image processing apparatus 100 is mounted is referred to as a host vehicle.

  The image acquisition unit 1 acquires captured images from the front camera 200 and the rear camera 201 provided in the host vehicle. The front camera 200 and the rear camera 201 are electronic cameras, and photograph the surrounding environment (for example, road surface) in front of or behind the vehicle. Hereinafter, a captured image captured by the front camera 200 is referred to as a front image, and a captured image captured by the rear camera 201 is referred to as a rear image. The image acquisition unit 1 outputs the acquired front image and rear image to the image selection unit 3.

  The traveling direction detection unit 2 detects information regarding the shift position of the host vehicle as information regarding the traveling direction of the host vehicle. Information about the shift position of the host vehicle is obtained from an ECU (Engine Control Unit) (not shown), for example. The shift position includes, for example, an R position (reverse, reverse), a D position (forward, drive), an N position (neutral), a P position (parking, parking), a first speed position (low), and the like.

  When the traveling direction detection unit 2 detects information on the shift position of the host vehicle, the traveling direction detection unit 2 stores the information in a ring buffer provided in the information storage unit 9. Information relating to at least two shift positions is stored in the ring buffer. Information regarding the shift position stored in the ring buffer is referred to by the image selection unit 3 and the direction switching detection unit 5.

  The image selection unit 3 selects one of the front image and the rear image input from the image acquisition unit 1 according to the latest information on the shift position of the host vehicle stored in the ring buffer of the information storage unit 9. And output to the image processing unit 4. The image selection unit 3 selects a front image and outputs it to the image processing unit 4 when the latest shift position of the host vehicle is a shift position related to the forward movement of the host vehicle such as the D position or the first speed position. On the other hand, when the latest shift position of the host vehicle is the R position, the image selection unit 3 selects a rear image and outputs it to the image processing unit 4.

  The image processing unit 4 performs image processing for performing viewpoint conversion on the front image or the rear image input from the image selection unit 3, and generates a bird's-eye view image looking down from above the host vehicle. The image processing unit 4 outputs this overhead image to the external world recognition unit 6 and the foreign object recognition unit 7.

  The direction switching detection unit 5 detects switching of the traveling direction of the host vehicle, that is, switching of the shift position of the host vehicle, based on the information regarding the shift positions of the plurality of host vehicles stored in the ring buffer of the information storage unit 9. . The direction switching detection unit 5 determines whether, for example, the shift position of the host vehicle has been switched to the R position, or whether the shift position of the host vehicle has been switched from the R position to another shift position (for example, the D position). To detect.

  The external environment recognition unit 6 performs image recognition processing on the bird's-eye view image input from the image processing unit 4 and recognizes the external environment around the host vehicle. In the present embodiment, the external environment recognition unit 6 recognizes a parking frame provided on the road surface in front of or behind the host vehicle by using a white line recognition method described later, and coordinates of four points forming the parking frame. , Upper left (TLX [p], TLY [p]), upper right (TRX [p], TRY [p]), lower left (BLX [p], BLY [p]), lower right (BRX [p], BRY [ p]) and the number p are stored as external recognition results. Here, p is an ID given to distinguish each when there are a plurality of small regions in the overhead image.

  The foreign matter recognition unit 7 recognizes foreign matter attached to the lens of the front camera 200 or the rear camera 201 based on the overhead image input from the image processing unit 4 using a foreign matter detection method described later. The foreign matter is, for example, mud, water droplets, water droplet traces, snow melting agents and the like. In the present embodiment, the overhead image is divided into small areas, and the start point x coordinate DSX [d], start point y coordinate DSY [d], end point x coordinate DEX [d], and end point y coordinate of the small area where the foreign matter exists. DEY [d] and the number d of the small areas are stored in the information storage unit 9 as the recognition result of the foreign object recognition unit 7. Here, “d” is an ID given to distinguish each of a plurality of small areas in the overhead image.

  The recognition result of the external recognition unit 6 is stored in the information storage unit 9 and is input to a higher-level vehicle control device (not shown) than the in-vehicle image processing device 100. In the present embodiment, in-vehicle image processing apparatus 100 recognizes a parking frame line, and outputs parking frame information for parking frames around the vehicle based on the recognition result of the parking frame. For example, the in-vehicle image processing apparatus 100 outputs information indicating the direction of the parking frame relative to the vehicle and the distance to the parking frame as parking frame information. The parking frame information is output to a host vehicle control device (not shown) connected to the in-vehicle image processing device 100, and is used for vehicle parking assistance, travel control, and the like. For example, it is possible to automatically recognize that there is a parking lot around the vehicle. For example, in a parking lot environment, it is possible to automatically switch and display a bird's-eye view around the vehicle on the monitor. . As a result, it is possible to suppress a situation in which it is erroneously detected as a parking lot on a public road, and to switch the video presented to the user at an appropriate timing.

  Moreover, the vehicle-mounted image processing apparatus 100 further outputs relative position information from the own vehicle regarding the parking frame as the parking frame information based on the recognition result of the parking frame. For example, the in-vehicle image processing apparatus 100 outputs the position information of the parking frame in the actual environment, such as the end point coordinates of the left and right frame lines of the parking frame, the angle of the parking frame line, and the intercept. The parking frame information is output to a host vehicle control device (not shown) connected to the in-vehicle image processing device 100, and is used for vehicle parking assistance, travel control, and the like. For example, by calculating the travel route to the parking frame based on the relative position / posture from the vehicle to the parking frame, and notifying the driver of the timing of brake and shift position change and the amount of steering angle operation Parking assistance can be performed. As a result, a parking operation can be completed in a short time even for a driver who is unfamiliar with driving operations such as garage entry.

  Furthermore, the relative position / posture from the vehicle to the parking frame calculates the travel route to the parking frame, automatically calculates the amount of forward / backward / turning of the vehicle, and the vehicle motions according to the calculation result It may be used for automatically controlling. As a result, a parking operation can be completed safely and accurately even for a driver unfamiliar with driving operations such as garage entry.

  The recognition result of the foreign object recognition unit 7 is classified for each camera, stored in the information storage unit 9, and input to a vehicle control device (not shown) higher than the in-vehicle image processing device 100. The host vehicle control device (not shown) uses the recognition result of the foreign object recognition unit 7 for the removal control of the foreign object attached to the lens of the front camera 200 or the rear camera 201.

The initialization unit 8 includes a behavior detection unit 8a that detects the behavior of the host vehicle. Based on the behavior of the host vehicle detected by the behavior detection unit 8a, the external recognition unit 6 and the foreign object recognition stored in the information storage unit 9 The recognition result of the unit 7 is initialized. Specifically, the coordinates of the four points forming the parking frame, which is the output of the external recognition unit 6, upper left (TLX [p], TLY [p]), upper right (TRX [p], TRY [p]), lower left ( BLX [p], BLY [p]), lower right (BRX [p], BRY [p]), and the number p are set to zero. The start point x-coordinate DSX [d], start point y-coordinate DSY [d], end point x-coordinate DEX [d], end point y-coordinate DEY [d], and The number d is set to zero. Note that the initialization of the external recognition unit 6 and the foreign matter recognition unit 7 is to set the output to zero, and all internal variables used in the middle of the calculation may be cleared to zero here, or partly May be held. Behavior detecting unit 8a in the first embodiment, as the behavior of the vehicle is detected (timing) of the elapsed time T _C from the time the traveling direction of the switching of the vehicle is detected by the direction switching detection unit 5 .

  The information storage unit 9 includes a RAM, a ROM, and the like, and includes a control program for the in-vehicle image processing apparatus 100, an image acquisition unit 1, a traveling direction detection unit 2, an image selection unit 3, an image processing unit 4, and a direction switching detection unit 5. Various information generated by the external recognition unit 6, the foreign matter recognition unit 7, the initialization unit 8, and the like are stored.

  FIG. 2 is a flowchart regarding the operation of the in-vehicle image processing apparatus 100. In step S <b> 10, the in-vehicle image processing apparatus 100 performs processing of the image acquisition unit 1 and acquires a front image and a rear image from the front camera 200 and the rear camera 201. In step S20, the in-vehicle image processing apparatus 100 performs the process of the traveling direction detection unit 2 and detects the shift position of the host vehicle. Note that the processing order of step S10 and step S20 can be interchanged and may be processed in parallel.

  In step S <b> 30, the in-vehicle image processing apparatus 100 performs the process of the direction switching detection unit 5. In step S <b> 40, the in-vehicle image processing apparatus 100 performs the process of the initialization unit 8. Details of steps S30 and S40 will be described later.

  In step S <b> 50, the in-vehicle image processing apparatus 100 performs the process of the image selection unit 3 and inputs either the front image or the rear image to the image processing unit 4. In step S <b> 60, the in-vehicle image processing apparatus 100 performs the processing of the image processing unit 4, generates an overhead image based on the front image or the rear image, and inputs the overhead image to the external world recognition unit 6 and the foreign object recognition unit 7. .

  In step S <b> 70, the in-vehicle image processing apparatus 100 performs processing of the external environment recognition unit 6. In step S <b> 80, the in-vehicle image processing apparatus 100 performs the foreign object recognition unit 7. Note that the processing order of step S70 and step S80 can be interchanged, and may be processed in parallel.

  The vehicle-mounted image processing apparatus 100 returns to the process of step S10 when the processes of the external recognition unit 6 and the foreign object recognition unit 7 are completed. Thereafter, the in-vehicle image processing apparatus 100 repeats the processing from step S10 to step S80.

  The processing of the direction switching detection unit 5 and the initialization unit 8 executed in step S30 and step S40 in FIG. 2 will be described. Between the direction switching detection unit 5 and the initialization unit 8, two flags, a rear initial initialization flag and a front initial initialization flag, are shared. The rear initial initialization flag and the front initial initialization flag are set to either ON (TRUE) or OFF (FALSE), respectively.

  The rear initial initialization flag is set to ON by the direction switching detection unit 5 when the shift position is switched from the shift position other than the R position to the R position. On the other hand, the front initial initialization flag is set to ON by the direction switching detection unit 5 when switching from the R position to another shift position.

  When the rear initial initialization flag or the front initial initialization flag is set to ON by the direction switching detection unit 5, the initialization unit 8 turns off two new flags, that is, an external recognition initialization flag and a foreign object recognition initialization flag. Then, the rear initial initialization flag or the front initial initialization flag that was set to ON by the direction switching detection unit 5 is set to OFF.

The external world recognition initialization flag is used for initialization control of the external world recognition unit 6. The external world recognition initialization flag is on in the initial state. When the external world recognition initialization flag is off and the behavior detection unit 8a detects the predetermined first behavior of the host vehicle, the external world recognition unit 6 is initialized and the external world recognition initialization flag is set to on. The The first behavior is that the first time T ₁ is elapsed from when, for example, by the direction switching detection unit 5 is the traveling direction of the switching of the vehicle is detected. First time T ₁ may, for example, the vehicle when the shift position of the vehicle is switched is the time that continues to move by inertia in the traveling direction corresponding to the shift position before the switching.

The foreign object recognition initialization flag is used for initialization control of the foreign object recognition unit 7. The foreign object recognition initialization flag is on in the initial state. When the foreign object recognition initialization flag is off and the behavior detection unit 8a detects a predetermined second behavior of the host vehicle, the foreign object recognition unit 7 is initialized and the foreign object recognition initialization flag is set to on. The The second behavior is that the second time T ₂ has elapsed since the example by the direction switching detection unit 5 is the traveling direction of the switching of the vehicle is detected.

  FIG. 3 is a flowchart relating to the processing of the direction switching detection unit 5 in step S30 of FIG. In step S <b> 301, the in-vehicle image processing apparatus 100 acquires information regarding the shift position stored in the ring buffer of the information storage unit 9. In step S302, the in-vehicle image processing apparatus 100 detects shift position switching based on the information regarding the shift position acquired in step S301.

  In step S303, the in-vehicle image processing apparatus 100 determines whether or not the shift position of the host vehicle has been switched from the shift position other than the R position to the R position. The determination in step S303 is affirmative when, for example, the position is switched from the D position, P position, N position, 1st gear position, or the like to the R position. Then, when the shift position of the host vehicle is switched from the R position to another shift position, when the shift position is switched between shift positions other than the R position, or when the shift position is not switched, a negative determination is made in step S303. The The in-vehicle image processing apparatus 100 proceeds with the process to step S304 when the negative determination is made in step S303, and proceeds with the process to step S305 when the positive determination is made in step S303.

  In step S304, the in-vehicle image processing apparatus 100 determines whether or not the shift position of the host vehicle has been switched from the R position to a shift position other than the R position. The determination in step S304 is negative when the shift position is switched between shift positions other than the R position or when the shift position is not switched, and the shift position of the host vehicle is switched from the R position to another shift position. An affirmative determination is made when The in-vehicle image processing apparatus 100 proceeds with the process to step S307 when a negative determination is made at step S304, and proceeds with the process to step S306 when an affirmative determination is made at step S304.

  In step S305, the in-vehicle image processing apparatus 100 sets the rear initial initialization flag to ON. That is, the in-vehicle image processing apparatus 100 sets the rear initial initialization flag to ON when the shift position of the host vehicle is switched from the shift position other than the R position to the R position.

  In step S306, the in-vehicle image processing apparatus 100 sets the front initial initialization flag to ON. That is, the in-vehicle image processing apparatus 100 sets the front initial initialization flag to ON when the shift position of the host vehicle is switched from the R position to a shift position other than the R position.

  In step S307, the in-vehicle image processing apparatus 100 sets the rear initial initialization flag and the front initial initialization flag to off. That is, the in-vehicle image processing apparatus 100 turns off the rear initial initialization flag and the front initial initialization flag when the shift position is switched between shift positions other than the R position or when the shift position is not switched. Set.

  The in-vehicle image processing apparatus 100 sets the rear initial initialization flag and the front initial initialization flag in step S305, step S306, or step S307, and then ends the processing in FIG. 3 and performs step S40 in FIG. Proceed to the process.

  FIG. 4 is a flowchart relating to the processing of the initialization unit 8 in step S40 of FIG. In step S401, the vehicle-mounted image processing apparatus 100 determines whether or not the rear initial initialization flag is set to ON. The in-vehicle image processing apparatus 100 proceeds with the process to step S402 when the rear initial initialization flag is set to ON, and proceeds with the process to step S404 when the rear initial initialization flag is set to OFF.

  In step S <b> 402, the in-vehicle image processing apparatus 100 holds the recognition result of the foreign object recognition unit 7 for the front camera 200 in the information storage unit 9 as it is. In step S403, the in-vehicle image processing apparatus 100 sets the rear initial initialization flag to OFF.

  In step S404, the in-vehicle image processing apparatus 100 determines whether or not the front initial initialization flag is set to ON. The in-vehicle image processing apparatus 100 proceeds to step S405 when the front initial initialization flag is set to ON, and proceeds to step S409 when the front initial initialization flag is set to OFF.

  In step S <b> 405, the in-vehicle image processing apparatus 100 initializes the recognition result of the foreign object recognition unit 7 for the rear camera 201. In step S406, the in-vehicle image processing apparatus 100 sets the front initial initialization flag to OFF.

At step S407, vehicle image processing apparatus 100, to the behavior detecting unit 8a, resets the counting of the elapsed time _{T C.} In step S408, the in-vehicle image processing apparatus 100 sets the external recognition recognition flag and the foreign object recognition initialization flag to OFF. That is, the in-vehicle image processing apparatus 100 sets the external environment recognition unit 6 and the foreign object recognition unit 7 in a state where the initialization can be started when the behavior detection unit 8a detects a predetermined behavior of the host vehicle.

At step S409, the in-vehicle image processing apparatus 100, the behavior detecting unit 8a acquires the elapsed time _{T C} being clocked. At step S410, vehicle image processing apparatus 100 is an environment recognizing initialization flag is turned off, and the elapsed time T _C obtained in step S409 determines whether larger or not than the first time T _1. The vehicle-mounted image processing apparatus 100 proceeds with the process to step S411 when the determination in step S410 is affirmative, and proceeds with the process to step S413 when the determination in step S410 is negative.

  In step S411, the in-vehicle image processing apparatus 100 initializes the recognition result of the external recognition unit 6. In step S <b> 412, the in-vehicle image processing apparatus 100 sets the external world recognition initialization flag to ON. That is, after the recognition result of the external recognition unit 6 is initialized, it is set so that the recognition result of the external recognition unit 6 is not initialized until step S401 or step S404 is affirmed again.

At step S413, vehicle image processing apparatus 100, the foreign matter recognition initialization flag is turned off, and the elapsed time T _C obtained in step S409 determines whether larger or not than the second time T _2. The in-vehicle image processing apparatus 100 proceeds with the process to step S414 when the determination in step S413 is affirmative, and terminates the process in FIG. 4 and proceeds to the process in step S50 in FIG. 2 when the determination in step S413 is negative.

  The operation of the direction switching detection unit 5 and the initialization unit 8 shown in FIGS. 3 and 4 will be described using a specific example. As a first specific example, a case will be described in which the shift position of the host vehicle is switched from the D position to the R position and then the shift position of the host vehicle is maintained at the R position.

  When the shift position of the host vehicle is switched from the D position to the R position, information relating to the D position and information relating to the R position are stored in the ring buffer of the information storage unit 9. When the in-vehicle image processing apparatus 100 starts the process of the direction switching detection unit 5, the vehicle-mounted image processing apparatus 100 acquires information on these shift positions stored in the ring buffer (step S301 in FIG. 3), and acquires the information on the acquired shift positions. The shift position change is detected by comparison (step S302).

  Since the shift position of the host vehicle is switched from the D position, which is a shift position other than the R position, to the R position, the in-vehicle image processing apparatus 100 makes an affirmative determination in step S303 and sets the rear initial initialization flag to on. (Step S305). And the vehicle-mounted image processing apparatus 100 complete | finishes the process of the direction switching detection part 5, and starts the process of the initialization part 8 shown in FIG.

  When the in-vehicle image processing apparatus 100 starts the processing of the initialization unit 8, the rear initial initialization flag is set to ON, so that an affirmative determination is made in step S 401 and the foreign object detection information of the front camera 200 is held. Then, the rear initial initialization flag is set to OFF (steps S402 and S403).

Then, the vehicle-mounted image processor 100, as well to start counting the elapsed time T _C to the behavior detecting unit 8a, set off an environment recognizing initialization flag and foreign matter recognition initialization flag (step S407 and step S408). Vehicle image processing apparatus 100, in step S409, the acquired elapsed time _{T C} from the behavior detecting unit 8a, the process proceeds to step S410. Step S407 immediately after the start of counting of the elapsed time _{T C} In order elapsed time _{T C} is the first time _{T 1} following a predetermined, in step S410 is negative, the process proceeds to step S413. Similarly, immediately after the start of counting of the elapsed time T _C in step S407, since the elapsed time T _C is the second time T ₂ following a predetermined, step S413 is negative, the processing in step S50 in FIG. 2 move on. At this time, the external environment recognition initialization flag and the foreign object recognition initialization flag remain set to off.

  Thereafter, the in-vehicle image processing apparatus 100 performs the processing from step S10 again after performing the processing from step S50 to step S80 in FIG. In step S20 in the second round, the traveling direction detection unit 2 detects information related to the R position again. Then, the process of FIG. 3 is started again.

  Since the shift position of the host vehicle remains the R position, switching of the shift position of the host vehicle is not detected in step S302 of the second round, and both step S303 and step S304 are negatively determined. As a result, the in-vehicle image processing apparatus 100 sets the rear initial initialization flag and the front initial initialization flag to OFF (step S307), and starts the process of FIG.

Since the rear first initialization flag and the front first initialization flag are both set to OFF, the vehicle-mounted image processor 100 is to both a negative decision in the step S401 and step S404, acquires the elapsed time T _C ( Step S409).

Elapsed time T _C is still first hour T ₁ and if not also reach the one of the second time T ₂ are, step S410 and step S413 and is negative, again step S50 and subsequent steps in FIG. 2 Start.

Repeat 2 lap processing similar, when N ₁ lap elapsed time T _C exceeds the first hour T _1, set off environment recognizing initialization flag is in the process of the first round of step S408 Since this is done, an affirmative determination is made in step S410. The vehicle-mounted image processing apparatus 100 initializes the recognition result of the external recognition unit 6 (step S411), and sets the external recognition recognition flag to ON (step S412). After the outside world recognition initialization flag is set to ON in step S412, the determination in step S410 is negative until the outside world recognition initialization flag is set to OFF again in step S408.

When the shift position is switched, the host vehicle may move by inertia in a direction different from the traveling direction corresponding to the changed shift position. When the recognition result of the external environment recognition unit 6 is initialized immediately in response to the shift position change even though the own vehicle is moving by inertia, the movement direction of the own vehicle and the processing target area of the external environment recognition unit 6 There is a possibility that the alignment is not achieved, which may cause a malfunction of the in-vehicle image processing apparatus 100 or the like. After the shift position is switched to the first hour T ₁ is passed to the recognition result of the external world recognizing unit 6 By not initialized, not the moving direction and the processing target area of the external world recognizing unit 6 of the vehicle Malfunction due to matching can be prevented.

Also, when the N ₂ lap elapsed time T _C exceeds a second time T _2, because the foreign matter recognition initialization flag remains set to OFF in the process of the first round of step S408, the step S413 A positive determination is made. Since the vehicle-mounted image processing apparatus 100 is in the R position, it makes a positive determination in step S414, initializes the foreign object recognition result of the front camera (step S415), and sets the foreign object recognition initialization flag to ON (step S416). After the foreign object recognition initialization flag is set on in step S416, the determination in step S413 is negative until the foreign object recognition initialization flag is set off again in step S408.

When the camera that captures the captured image that is the basis of the overhead image is switched according to the traveling direction of the host vehicle, and the captured image of the camera is not used for processing for a long time, the foreign object recognition unit 7 for the camera The recognition result may have no memorized meaning. Therefore, to initialize recognition result of the foreign object recognition unit 7 when from the shift position is switched second time T ₂ has elapsed.

  Next, as a second specific example, a case where the shift position of the host vehicle is switched from the R position to the D position and then the shift position of the host vehicle is maintained at the D position will be described.

  When the shift position of the host vehicle is switched from the R position to the D position, information related to the R position and information related to the D position are stored in the ring buffer of the information storage unit 9. When the in-vehicle image processing apparatus 100 starts the process of the direction switching detection unit 5, the vehicle-mounted image processing apparatus 100 acquires information on these shift positions stored in the ring buffer (step S301 in FIG. 3), and acquires the information on the acquired shift positions. The shift position change is detected by comparison (step S302).

  The in-vehicle image processing apparatus 100 makes a negative determination in step S303, makes an affirmative determination in step S304, and sets the front initial initialization flag to on (step S306). Then, the in-vehicle image processing apparatus 100 proceeds from the process of the direction switching detection unit 5 to the process of the initialization unit 8 illustrated in FIG.

  When the processing of the initialization unit 8 is started, the vehicle-mounted image processing apparatus 100 makes a negative determination in step S401 because the rear initial initialization flag is set to off. Since the front initial initialization flag is set to ON, the in-vehicle image processing apparatus 100 makes a positive determination in step S404 to initialize the foreign object detection information of the rear camera 201 and set the front initial initialization flag. Set to OFF (steps S405 and S406). The subsequent processing is the same as that of the first specific example except that the determination in step S414 is negative.

  FIG. 5 is a flowchart relating to an example of the external recognition unit 6 in which the process is performed in step S70 of FIG. The flowchart illustrated in FIG. 5 relates to white line recognition processing. In step S <b> 501, the in-vehicle image processing apparatus 100 applies a lateral edge filter to all lines of the overhead image output from the image processing unit 4. In step S502, the in-vehicle image processing apparatus 100 extracts rising and falling peak points from the overhead image after application of the edge filter.

  In step S503, the in-vehicle image processing apparatus 100 extracts a peak point that forms a pair such that the interval between the rising and falling peak points extracted in step S502 is a predetermined distance. In step S504, the in-vehicle image processing apparatus 100 performs grouping processing on the rising peak points extracted in step S503 to perform grouping and extract rising edges, and to the falling peak points extracted in step S503. On the other hand, grouping processing is performed to group and extract rising edges. In step S505, the in-vehicle image processing apparatus 100 filters the rising edge and the falling edge extracted in step S504 with the length of the edge.

  In step S506, the vehicle-mounted image processing apparatus 100 extracts the coordinates of both end points of the rising edge and the falling edge left by the filtering in step S505. The vehicle-mounted image processing apparatus 100 extracts the white line by combining the rising edge and the falling edge left by the filtering in step S505 based on the pair in step S503.

  In step S507, the in-vehicle image processing apparatus 100 selects two white lines from a plurality of white lines. In step S508, the in-vehicle image processing apparatus 100 determines whether or not the angle formed by the two white lines selected in step S507 is equal to or smaller than a predetermined angle. In other words, the in-vehicle image processing apparatus 100 determines whether or not the two white lines selected in step S507 are substantially parallel. The in-vehicle image processing apparatus 100 proceeds to step S509 when the determination in step S508 is affirmative, and proceeds to step S512 when the determination in step S508 is negative.

  In step S509, the in-vehicle image processing apparatus 100 determines whether the interval between the two white lines selected in step S507 is within a predetermined range. The predetermined range is 2 m to 3 m, which is about the width of a lane or a parking section. The in-vehicle image processing apparatus 100 proceeds to step S510 when step S509 is affirmed, and proceeds to step S512 when step S509 is negative.

  In step S510, the in-vehicle image processing apparatus 100 determines whether or not the deviation between the lower ends of the two white lines selected in step S507 is within a predetermined range. In the parking section provided diagonally, the lengths of the left and right white lines representing the parking section may be different. In step S510, it is determined whether there is no deviation between the lower ends of the two white lines selected in step S507, or whether the amount of deviation is in an obliquely provided parking section. The in-vehicle image processing apparatus 100 proceeds to step S511 when step S510 is affirmed, and proceeds to step S512 when step S508 is negative.

  In step S511, the in-vehicle image processing apparatus 100 determines the coordinates of the four points forming the parking frame formed by the two white lines selected in step S507, upper left (TLX [p], TLY [p]), upper right (TRX [P], TRY [p]), lower left (BLX [p], BLY [p]), lower right (BRX [p], BRY [p]), and the number p are stored as external recognition results. In step S512, the in-vehicle image processing apparatus 100 determines whether or not the determinations in steps S508 to S510 have been made for all combinations of white lines. The in-vehicle image processing apparatus 100 ends the process of FIG. 5 when step S512 is determined to be affirmative, and returns to step S507 when step S512 is determined to be negative and configures a combination that has not yet been confirmed. Select two white lines.

  FIG. 6 is a flowchart relating to an example of the foreign object recognition unit 7 in which the process is performed in step S80 of FIG. In step S601, the in-vehicle image processing apparatus 100 divides the overhead image output from the image processing unit 4 into a plurality of small regions. In step S602, the in-vehicle image processing apparatus 100 creates a histogram in each small region. In step S603, the in-vehicle image processing apparatus 100 compares the histogram created in step S602 with the histogram created when the process of FIG. 6 was executed last time. In step S604, the in-vehicle image processing apparatus 100 evaluates each small region based on the comparison performed in step S603. That is, a calculation for increasing the score is performed for an area where there is no change in the histogram. In step S605, the in-vehicle image processing apparatus 100 detects a small area whose score exceeds a predetermined threshold, and the start point x coordinate DSX [d], start point y coordinate DSY [d], and end point x coordinate DEX of the small area. [D], the end point y coordinate DEY [d], and the number d thereof are stored in the information storage unit 9 as the recognition result of the foreign object recognition unit 7.

-Second embodiment-
A second embodiment of the present invention will be described. The second embodiment of the present invention differs from the first embodiment in the processing of the initialization unit 8 and the behavior detection unit 8a. In the second embodiment, the behavior detecting unit 8a, together with the shift position detecting an elapsed time T _C from being switched to obtain a vehicle speed V _sp and the yaw rate V _yr of the vehicle from the ECU of the vehicle .

  7 and 8 are flowcharts relating to the processing of the initialization unit 8 in the second embodiment. In the second embodiment, when the process of the initialization unit 8 is started, the process is started from step S401 in FIG. Among the processes shown in FIG. 7 and FIG. 8, the same processes as those in the flowchart shown in FIG. 4 in the first embodiment are denoted by the same reference numerals, and the description thereof is omitted.

  In step S701 in FIG. 7, the in-vehicle image processing apparatus 100 holds the foreign object recognition result of the rear camera 201 as in step S402.

In step S702 of FIG. 7, the in-vehicle image processing device 100, the elapsed time _{T C,} the vehicle speed _{V sp} of the vehicle acquires a yaw rate _{V yr} of the vehicle from the behavior detecting unit 8a, step S710 of FIG. 8 Proceed with the process.

  In step S710 in FIG. 8, the in-vehicle image processing apparatus 100 determines whether or not the external world recognition initialization flag is set to OFF. The in-vehicle image processing apparatus 100 proceeds to step S711 when step S710 is affirmed, and proceeds to step S712 when step S710 is negative.

In step S711 of FIG. 8, the in-vehicle image processing apparatus 100 determines whether or not the elapsed time _{T C} exceeds the first hour T1, or, whether the vehicle speed _{V sp} of the vehicle acquired in step S709 is zero It is determined whether or not is established. The in-vehicle image processing apparatus 100 proceeds to step S411 in FIG. 8 when step S711 is affirmed, and proceeds to step S712 when step S711 is negative.

  In step S712 in FIG. 8, the in-vehicle image processing apparatus 100 determines whether or not the foreign object recognition initialization flag is set to off. If the determination in step S712 is affirmative, the in-vehicle image processing apparatus 100 ends the processing in FIG. 8 and proceeds to the processing in step S50 in FIG. 2, and proceeds to processing in step S713 if the determination in step S712 is negative.

In step S713 of FIG. 8, the in-vehicle image processing apparatus 100 determines whether or not the elapsed time _{T C} exceeds the first hour T1, or the vehicle speed _{V sp} of the vehicle acquired in step S709 exceeds a predetermined speed _{V 1} not Whether or not the yaw rate V _{yr of the host} vehicle acquired in step S709 exceeds a predetermined yaw rate V ₂ is determined. If the determination in step S713 is affirmative, the in-vehicle image processing apparatus 100 proceeds to step S414 in FIG. 8, and if the determination in step S713 is negative, the in-vehicle image processing apparatus 100 ends the processing in FIG. 8 and performs the processing in step S50 in FIG. Proceed to

  When the vehicle speed or yaw rate of the host vehicle is high, foreign matter adhering to the lens of the camera of the host vehicle may be removed by wind pressure or the like. If a foreign object is removed due to wind pressure or the like while traveling, holding a recognition result when the foreign object is attached may cause unnecessary processing (for example, a foreign object removing operation). In step S713, conditions based on the vehicle speed and yaw rate of the host vehicle are included to prevent such unnecessary processing.

  In step S714 of FIG. 8, the foreign object recognition result of the rear camera is initialized.

According to the embodiment described above, the following operational effects can be obtained.
The in-vehicle image processing apparatus 100 includes an image acquisition unit 1, a traveling direction detection unit 2, an image selection unit 3, an external environment recognition unit 6, a foreign object recognition unit 7, and an initialization unit 8 having a behavior detection unit 8a. Prepare. The image acquisition unit 1 acquires captured images from a front camera 200 that captures the front of the host vehicle and a rear camera 201 that captures the rear of the host vehicle. The traveling direction detection unit 2 detects the shift position (traveling direction) of the host vehicle. The image selection unit 3 selects a captured image of the camera corresponding to the shift position of the host vehicle detected by the traveling direction detection unit 2 from the front camera 200 and the rear camera 201. The external environment recognition unit 6 recognizes the external environment of the host vehicle based on the overhead image corresponding to the captured image selected by the image selection unit 3. The foreign object recognition unit 7 recognizes a foreign object attached to the camera lens based on the overhead image corresponding to the captured image selected by the image selection unit 3. The direction switching detection unit 5 detects a shift position change detected by the traveling direction detection unit 2. Behavior detecting unit 8a, the elapsed time T _C, etc. after the change in the shift position is detected by the direction switching detection unit 5, detects the behavior of the host vehicle. Initialization unit 8, the behavior detecting unit together with the first hour T ₁ is greater than the elapsed time T _C initializes a recognition result of the external world recognizing unit 6 when detected by 8a, the second time by the behavior detecting unit 8a T ₂ Is detected, the recognition result of the foreign object recognition unit 7 is initialized.
Since it did in this way, when the image | video selected from the image | video of several cameras switches, initialization can be performed at an appropriate timing for every process using an image | video.

The embodiment described above can be implemented with the following modifications.
The traveling direction detection unit 2 acquires information on the shift position of the host vehicle from the ECU as the traveling direction of the host vehicle. However, information on the traveling direction of the host vehicle may be detected by other methods. For example, a GPS receiver may be provided, and the position of the host vehicle may be detected based on the received GPS signal, and the traveling direction of the host vehicle may be detected based on a change in the position. Moreover, you may decide to acquire the information regarding the advancing direction of the own vehicle from the navigation apparatus connected to the own vehicle, the information terminal provided with the GPS receiver, etc.

  In FIG. 2, the process of step S10 is always performed after the process of step S80. However, the process returns to the process of step S10 only when the speed of the host vehicle is equal to or lower than a predetermined speed (for example, 40 km / h or less). May be.

  The cameras connected to the in-vehicle image processing apparatus 100 are not limited to the front camera 200 and the rear camera 201. You may decide to further provide the camera which image | photographs the left side and the right side of the own vehicle.

Behavior detecting unit 8a may be to detect the pulse interval of the vehicle speed pulse output from the ECU of the vehicle instead of the elapsed time T _C. The pulse interval of the vehicle speed pulse increases as the host vehicle decelerates regardless of the traveling direction. The initialization unit 8 may perform peak detection for the pulse interval, and may initialize the recognition result of the external recognition unit 6 when the pulse interval decreases to a predetermined value or less after the peak is detected.

Behavior detecting unit 8a detects the elapsed time T _C from a shift position is switched, it may be detected screen display switches that are displayed to the driver. For example, when a screen operation (navigation operation, change of camera display format) occurs by the driver or when an obstacle is detected by sonar, a display to inform the driver of the screen display to be displayed to the driver is displayed. For example, when it appears on the screen. The initialization unit 8 may initialize the recognition result of the external recognition unit 6 and / or the recognition result of the foreign object recognition unit 7 when the switching of the screen display displayed on the driver is detected.

  Furthermore, in the external environment recognition unit 6, when using the own vehicle behavior estimation in which the traveling direction is estimated using the shift position of the own vehicle and the movement amount is estimated using the wheel speed, immediately after the shift position of the own vehicle is switched. In spite of the wheels rotating due to inertia, the traveling direction estimated by the shift position is the reverse direction, and thus the behavior is different from the actual one. Therefore, in addition to the first initialization that initializes the recognition result of the external recognition unit 6 by the processing based on the above-described embodiment, a second initialization that initializes only the contents related to the vehicle behavior is provided. May be.

Specifically, the host vehicle speed _Vsp is acquired, and at the timing when the value becomes zero, the movement of the host vehicle estimated by dead reckoning is initialized to zero as the second initialization.

  The embodiments and modifications described above are merely examples, and the present invention is not limited to these contents as long as the features of the invention are not impaired. Further, the embodiments and modifications described above may be combined and executed as long as the features of the invention are not impaired.

DESCRIPTION OF SYMBOLS 1 Image acquisition part 2 Progression direction detection part 3 Image selection part 4 Image processing part 5 Direction switching detection part 6 External field recognition part 7 Foreign substance recognition part 8 Initialization part 8a Behavior detection part 9 Information storage part 100 Car-mounted image processing apparatus

Claims (8)

An image acquisition unit that acquires captured images from a front camera that captures the front of the vehicle and a rear camera that captures the rear of the vehicle;
A traveling direction detector for detecting a traveling direction of the vehicle;
An image selection unit that selects a captured image of a first camera that captures the traveling direction of the vehicle detected by the traveling direction detection unit from the front camera and the rear camera;
An external recognition unit that recognizes the external environment of the vehicle based on a captured image of the first camera;
A foreign substance recognizing unit for recognizing a foreign substance attached to the lens of the first camera based on a photographed image of the first camera;
A direction change detection unit that detects a change in the travel direction detected by the travel direction detection unit;
A behavior detection unit for detecting the behavior of the vehicle by the direction change detection unit; when a first behavior is detected by the behavior detection unit, the recognition result of the external recognition unit is initialized and the behavior detection is performed An initialization unit that initializes the recognition result of the foreign object recognition unit when the second behavior is detected by the unit;
An in-vehicle image processing apparatus comprising: The in-vehicle image processing apparatus according to claim 1,
The behavior detection unit detects an elapsed time from when the change in the traveling direction is detected by the direction change detection unit,
The initialization unit initializes the recognition result of the external field recognition unit when the first elapsed time is detected by the behavior detection unit, and the second detection time is detected by the behavior detection unit. An in-vehicle image processing apparatus characterized by initializing a recognition result of a foreign object recognition unit. The in-vehicle image processing device according to claim 1 or 2,
The behavior detection unit further detects the speed of the vehicle,
The initialization unit initializes the recognition result of the external field recognition unit when the first speed is detected by the behavior detection unit, and recognizes the foreign object when the second speed is detected by the behavior detection unit. An in-vehicle image processing apparatus characterized by initializing a recognition result of a unit. The in-vehicle image processing device according to claim 3,
The behavior detection unit further detects the yaw rate of the vehicle,
The initialization unit initializes a recognition result of the foreign object recognition unit when a second speed is detected by the behavior detection unit or when the predetermined yaw rate is detected. Image processing device. The in-vehicle image processing apparatus according to claim 1,
The traveling direction detection unit detects the position of the shift lever of the vehicle,
The shift lever has a plurality of positions including at least an R position where the vehicle travels backward,
The in-vehicle image processing apparatus, wherein the direction change detection unit detects that the shift lever is changed to the R position and the shift lever is changed from the R position to another position. . The in-vehicle image processing apparatus according to claim 1,
The in-vehicle image processing apparatus, wherein the external environment recognition unit detects a white line provided on a road surface around the vehicle. The in-vehicle image processing apparatus according to claim 1,
The behavior detection unit detects a display change of the in-vehicle monitor,
The initialization unit initializes the recognition result of the external recognition unit and / or the recognition result of the foreign object recognition unit when the display change of the in-vehicle monitor detected by the behavior detection unit is a predetermined change. An in-vehicle image processing apparatus characterized by the above. The in-vehicle image processing apparatus according to claim 1,
The initialization unit performs a first initialization for initializing a recognition result of the external recognition unit when the behavior detection unit detects the first behavior, and the behavior detection unit performs the second initialization. An in-vehicle image processing apparatus that performs a second initialization for initializing a portion related to the vehicle behavior of the recognition result of the external recognition unit when a behavior is detected.

Images