JP7218237B2 - Moving object detector - Google Patents

Description

The present invention relates to a moving object detection device for detecting peripheral moving objects moving around a moving object.

2. Description of the Related Art In recent years, development of a moving object detection device that is mounted on a vehicle such as an automobile and detects a moving object that moves around the vehicle has been progressing. A moving object is, for example, a pedestrian. When a pedestrian is detected, the moving object detection device notifies the presence of the pedestrian to the driver of the vehicle. As a result, the driver can drive the vehicle while being aware of the presence of pedestrians.

For example, Patent Literature 1 discloses a pedestrian recognition device that estimates the state of a person present in front of a vehicle. The pedestrian recognition device detects a person from image data generated by an imaging unit that captures an image of the area ahead of the vehicle, and sets an area in which the person is detected as a processing area. The pedestrian recognition device generates a brightness gradient image in the processing area from the image data, and determines whether or not the detected person is walking based on the generated brightness gradient image.

JP 2013-214258 A

A front camera mounted in front of a vehicle cannot photograph a pedestrian depending on the positional relationship between the front camera and the pedestrian. If the front camera cannot capture a pedestrian, the pedestrian recognition device according to Patent Literature 1 cannot detect the pedestrian.

When the own vehicle turns right, the moving object detection device can photograph a pedestrian walking on the right side of the own vehicle by using the front camera and the right side camera that photographs the right side of the vehicle. In this case, the pedestrian recognition device must detect pedestrians from both the image data generated by the front camera and the image data generated by the right side camera.

However, when detecting a pedestrian from two image data, the load on the pedestrian recognition device increases. As a result, it becomes difficult for the pedestrian recognition device to detect pedestrians in real time from each of the two image data.

SUMMARY OF THE INVENTION An object of the present invention is to provide a moving object detection apparatus capable of reducing the detection load of a moving object moving around a vehicle.

In order to solve the above problems, a first invention is a moving object detection device comprising an image acquisition section, a first detection section, a priority area setting section, an image quality adjustment section, and a second detection section. . The image acquisition unit acquires a first captured image generated by a first camera mounted on the vehicle, and acquires a second captured image generated by a second camera mounted on the vehicle. The first detection unit detects a first moving object moving around the vehicle from the first captured image acquired by the image acquisition unit. The priority area setting unit determines the priority area based on the position of the first moving object detected by the first detection unit in the first captured image and the moving direction of the detected first moving object in the first captured image. A non-priority area is set in the second captured image acquired by the image acquiring unit. The image quality adjustment unit adjusts the image quality of the image of the non-priority area by reducing the number of pixels of the second captured image. The second detection section detects a second moving object moving around the vehicle from each of the image of the priority detection area and the image of the non-priority detection area whose image quality is adjusted by the image quality adjustment section.

According to the first invention, the image quality adjusting section adjusts the image quality of the image of the non-priority area by reducing the number of pixels of the second captured image. A second detection unit detects a moving object from the image of the priority area and the image of the non-priority area whose image quality is adjusted in the second captured image. That is, the second detection section detects the moving object from the second captured image having a smaller number of pixels than the second captured image acquired by the image acquisition section. The first invention can reduce the detection load of a moving object when detecting a moving object from each of two captured images in parallel.

A second invention is the first invention, wherein when the position of the first moving object in the first captured image is within a cooperation reference region preset in the first captured image, the priority region setting unit includes: Set a priority area and a non-priority area. The image of the cooperation reference region in the first captured image includes an image in which the overlapping range in which the capturing range of the second camera and the capturing range of the first camera overlap is captured.

According to the second invention, the cooperation reference area includes an image of an overlapping range in which the imaging range of the first camera and the imaging range of the second camera overlap in the first captured image. When the first moving object is positioned within the cooperation reference area, the priority area setting unit sets the priority area and the non-priority area in the second captured image. Accordingly, the second invention performs processing for detecting a moving object from each of the first captured image and the second captured image when there is a high possibility that both the first camera and the second camera can capture the moving object. can start. Therefore, the second invention can shorten the period during which the moving object detection load increases.

A third invention is the second invention, wherein the priority region setting unit determines that the first moving object is moving faster than the vehicle and that the first moving object It is determined whether the moving object and the vehicle are traveling in the same direction. The priority area setting unit sets the priority area and the non-priority area in the second captured image when determining that the first moving object is faster than the vehicle and that the first moving object and the vehicle are traveling in the same direction. do.

According to the third invention, when the first moving object is faster than the vehicle and the first moving object and the vehicle are traveling in the same direction, the priority area and the non-priority area are set in the second captured image. be. Thereby, when the vehicle turns right or left, the moving object can be detected using the first camera and the second camera, so that the detection accuracy of the moving object can be improved.

A fourth invention is the second or third invention, wherein when the position of the first moving object in the first captured image is within a cooperation reference region preset in the first captured image, priority region setting is performed. The unit sets the priority area and the non-priority area in the acquired first captured image. The image quality adjustment unit adjusts the image quality of the image of the non-priority area set in the first captured image by reducing the number of pixels of the first captured image. The first detection unit detects a moving object from the image of the priority area set as the first captured image, and detects the moving object from the image of the non-priority area set as the first captured image and whose image quality has been adjusted. do.

According to the fourth invention, when the first moving object is positioned within the cooperation reference area in the first captured image, the priority area setting section sets the priority area and the non-priority area in the first captured image. The image quality adjustment unit reduces the number of pixels of the first captured image. The first detection unit detects the first moving object from the image of the non-priority area in the first captured image and the image of the priority area set in the first captured image and whose image quality has been adjusted. detection load can be further reduced.

A fifth invention is the first to fourth inventions, further comprising a movement status acquisition unit. The movement status acquisition unit acquires the movement status of the vehicle. When the movement status acquired by the movement status acquisition section satisfies a predetermined condition, the first detection section starts detection of the first moving object.

According to the fifth invention, detection of a moving object is started in accordance with the moving state of the vehicle, so that a constant increase in the detection load of the moving object can be suppressed.

In a sixth invention according to any one of the fifth inventions, the movement status acquisition section includes an entry determination section and a turn determination section. The entry determination unit determines whether or not the vehicle will enter the intersection. The turning determination unit determines whether or not the vehicle will turn. The priority area setting unit sets the priority area and the non-priority area when the entry determination unit determines that the vehicle enters the intersection and the turn determination unit determines that the vehicle turns.

According to the sixth invention, when a vehicle turns right or left at an intersection, a moving object can be detected from both the first captured image and the second captured image. Therefore, it is possible to prevent the vehicle from colliding with a moving object when the vehicle turns right or left at an intersection.

A seventh aspect of the invention is the sixth aspect of the invention, wherein the entry determination unit detects a pedestrian crossing from an image captured by a front camera that captures an image in front of the vehicle. The turning determination unit determines whether or not a turning direction of the vehicle has been indicated by a direction indicating switch mounted on the vehicle. The first detection unit starts detecting the first moving object when the pedestrian crossing is detected by the entry determination unit and the turning direction is indicated by the direction switch.

According to the seventh invention, when a pedestrian crossing is detected and the direction switch is operated, the first detector starts detecting the moving object. This makes it possible to determine whether to turn right, turn left, or go straight at an intersection based on the intention of the driver of the vehicle.

An eighth invention is any one of the first to fourth inventions, wherein the vehicle is equipped with a plurality of cameras. The image acquisition section includes an image selection section and an image output section. The image selection unit selects each of the plurality of captured images generated by the plurality of cameras according to a predetermined order. The image output section outputs the captured image selected by the image selection section to the first detection section as the first captured image. When the first detection unit detects the first moving object, the priority area setting unit sets the photographed image in which the first moving object is detected, the position of the first moving object in the first photographed image, and the first moving object. The image selecting unit is instructed to determine a second captured image based on the direction of movement in the first captured image and to output the determined second captured image to the image quality adjustment unit.

According to the eighth invention, the second captured image is determined based on the captured image in which the first moving object is detected, the position of the first moving object, and the moving direction of the first moving object. The eighth invention can flexibly change the captured image used for detecting the moving object based on the relative position of the moving object with respect to the vehicle.

A ninth invention is the eighth invention, further comprising a movement status acquisition unit. The movement status acquisition unit acquires the movement status of the vehicle. When the movement status acquired by the movement status acquisition section indicates that the vehicle is traveling in a parking lot, the image selection section starts selecting a plurality of captured images.

According to the ninth invention, when the vehicle runs in the parking lot, the image selection section starts selecting the captured image. As a result, moving objects can be quickly detected when the vehicle is traveling in a parking lot.

A tenth invention is a moving object detection method, comprising a) step, b) step, c) step, d) step, and e) step. The a) step obtains a first captured image generated by a first camera mounted on the vehicle, and obtains a second captured image generated by a second camera mounted on the vehicle. The b) step detects a first moving object moving around the vehicle from the acquired first captured image. The c) step acquires a priority area and a non-priority area based on the detected position of the first moving object in the first captured image and the moving direction of the detected first moving object in the first captured image. set to the selected second captured image. The d) step adjusts the image quality of the image of the non-priority area by reducing the number of pixels of the second captured image. The e) step detects a second moving object moving around the vehicle from each of the image of the priority detection area and the image of the non-priority detection area whose image quality has been adjusted.

A tenth invention is used for the first invention.

ADVANTAGE OF THE INVENTION According to this invention, the moving object detection apparatus which can reduce the detection load of the moving object which moves around a vehicle can be provided.

1 is a functional block diagram showing a configuration of a vehicle equipped with a moving object detection device according to a first embodiment of the invention; FIG. 2 is a diagram showing an example of the arrangement of cameras shown in FIG. 1; FIG. 2 is a functional block diagram showing the configuration of the moving object detection device shown in FIG. 1; FIG. 4 is a functional block diagram showing the configuration of an image acquisition unit shown in FIG. 3; FIG. 4 is a functional block diagram showing the configuration of a movement status acquisition unit shown in FIG. 3; FIG. 2 is a diagram for explaining an outline of the operation of the moving object detection device shown in FIG. 1; FIG. 2 is a flowchart showing the operation of the moving object detection device shown in FIG. 1; FIG. 8 is a flowchart of vehicle behavior identification processing shown in FIG. 7; FIG. FIG. 8 is a flowchart of a cooperation start determination process shown in FIG. 7; FIG. FIG. 7 is an enlarged view of the right photographed image shown in FIG. 6; 2 is a diagram schematically showing a shooting range of the camera shown in FIG. 1; FIG. FIG. 8 is a flowchart of a cooperation detection process shown in FIG. 7; FIG. FIG. 11 is a diagram showing an example of a priority area and a non-priority area set in the right shot image shown in FIG. 10; FIG. 7 is a diagram showing an example of a priority area and a non-priority area set in the front shot image shown in FIG. 6; FIG. 6 is a functional block diagram showing the configuration of a moving object detection device according to a second embodiment of the present invention; 16 is a flowchart showing the operation of the moving object detection device shown in FIG. 15; FIG. 17 is a flowchart of the cooperation start determination process shown in FIG. 16; FIG. 16 is a diagram showing an example of a start condition table used by the priority area setting unit shown in FIG. 15; FIG. FIG. 11 is a functional block diagram showing the configuration of a moving object detection device according to a third embodiment of the present invention; 20 is a diagram showing an example of priority areas set by a priority area setting unit shown in FIG. 19; FIG. 4 is a diagram showing an example of detection positions of a moving object detected by a first detection unit shown in FIG. 3; FIG. FIG. 2 is a diagram showing a CPU bus configuration; FIG.

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. The same reference numerals are given to the same or corresponding parts in the drawings, and the description thereof will not be repeated.

[1. Configuration of vehicle 100]
FIG. 1 is a functional block diagram showing the configuration of a vehicle 100 equipped with a moving object detection device 10 according to an embodiment of the invention. Referring to FIG. 1, vehicle 100 includes a vehicle body 1, a left front wheel 1FL, a right front wheel 1FR, a left rear wheel 1RL, a right rear wheel 1RR, and axles 2F and 2R.

The left front wheel 1FL is arranged on the front left side of the vehicle body 1. As shown in FIG. The right front wheel 1FR is arranged on the front right side of the vehicle body 1 . The front left wheel 1FL and the front right wheel 1FR rotate about an axle 2F arranged in front of the vehicle body 1 as a rotation axis. The axle 2F is a drive shaft of the vehicle 100, and transmits the output of the engine (not shown) to the front left wheel 1FL and the front right wheel 1FR. The front left wheel 1FL and the front right wheel 1FR are steering wheels of the vehicle 100. FIG. The steering angles of the front left wheel 1FL and the front right wheel 1FR change according to the rotation of a steering wheel (not shown).

The left rear wheel 1RL is arranged on the rear left side of the vehicle body 1 . The right rear wheel 1RR is arranged on the rear right side of the vehicle body 1 . The left rear wheel 1RL and the right rear wheel 1RR rotate about an axle 2R arranged at the rear of the vehicle body 1 as a rotation axis.

Vehicle 100 further includes direction indicator switch 4 , display device 5 , moving object detection device 10 , front camera 31 , left side camera 32 , rear camera 33 , and right side camera 34 . In the following description, the front camera 31, the left side camera 32, the rear camera 33, and the right side camera 34 are collectively referred to simply as "cameras".

Direction switch 4 is operated by the driver of vehicle 100 . The direction indicator switch 4 outputs a direction indicator signal 4S to the left winker 7L, the right winker 7R, and the moving object detection device 10 shown in FIG. 7 according to the driver's operation. The direction instruction signal 4S is a signal that instructs blinking of the left winker 7L or the right winker 7R.

The display device 5 is a thin panel such as a liquid crystal display device, and is arranged at a position easily visible to the driver. For example, the display device 5 is arranged near the speedometer of the vehicle 100 . The display device 5 receives the detection result information 47 from the moving object detection device 10 and displays the received detection result information 47 . The detection result information 47 will be described later.

The front camera 31 captures a scene in front of the vehicle 100 to generate a forward captured image 31A, and supplies the generated forward captured image 31A to the moving object detection device 10 . The left side camera 32 captures the scenery to the left of the vehicle 100 to generate a left captured image 32A, and supplies the generated left captured image 32A to the moving object detection device 10 . The rear camera 33 captures the scenery behind the vehicle 100 to generate a rearward captured image 33A, and supplies the generated rearward captured image 33A to the moving object detection device 10 . The right side camera 34 captures the scenery to the right of the vehicle 100 to generate a right captured image 34A and supplies the generated right captured image 34A to the moving object detection device 10 .

In the following description, each of the front shot image 31A, the left shot image 32A, the rear shot image 33A, and the right shot image 34A may be simply referred to as a "shot image". A captured image is a frame that constitutes a moving image. Each camera mounted on vehicle 100 generates a captured image at predetermined time intervals and outputs the generated captured image to moving object detection device 10 .

The moving object detection device 10 detects a moving object moving around the vehicle 100 using a captured image captured by a camera mounted on the vehicle 100 . A moving object is, for example, a pedestrian or a bicycle. When detecting a moving object, the moving object detection device 10 generates detection result information 47 and outputs the generated detection result information 47 to the display device 5 . The detection result information 47 is specifically an image indicating the presence of a moving object. This allows the driver to grasp the presence of moving objects moving around the vehicle 100 .

[2. camera placement]
FIG. 2 is a diagram showing an example of arrangement of the cameras shown in FIG. First, the world coordinate system will be explained. The origin Ow of the world coordinate system is the center of gravity of the vehicle 100 . The Y-axis extends in the straight-ahead direction of vehicle 100 . The positive direction of the Y-axis is the direction from the rear end surface of vehicle 100 toward the front end surface. The X-axis extends in a direction perpendicular to both the straight-ahead direction and the vertical direction of the vehicle 100 . The positive direction of the X-axis is the direction from the left side to the right side of vehicle 100 .

The front camera 31 is provided near the license plate attached to the front end of the vehicle 100, and its optical axis 31K is directed in the positive direction of the Y axis. The rear camera 33 is provided near the license plate attached to the rear end of the vehicle 100, and its optical axis 33K is directed in the negative direction of the Y axis. The mounting positions of the front camera 31 and the rear camera 33 are desirably on the X-axis, but they may be positioned slightly off the X-axis.

The left side camera 32 is provided on the left door mirror 6L of the vehicle 100, and its optical axis 32K is directed in the negative direction of the X axis. The right side camera 34 is provided on the right door mirror 6R of the vehicle 100, and its optical axis 34K is directed in the positive direction of the X axis.

The camera lens is a wide-angle lens with an angle of view of 180° or more. By using four cameras mounted on the vehicle 100, it is possible to photograph the entire surroundings of the vehicle 100. FIG.

Vehicle 100 further includes a left winker 7L and a right winker 7R. The left winker 7L and the right winker 7R blink according to the direction signal 4S output from the direction switch 4. FIG.

[3. Configuration of moving object detection device 10]
FIG. 3 is a functional block diagram showing the configuration of the moving object detection device 10 shown in FIG. Referring to FIG. 3, moving object detection apparatus 10 includes image acquisition unit 11, movement status acquisition unit 12, first detection unit 13, priority area setting unit 14, image quality adjustment unit 15, second detection A unit 16 and a detection result output unit 17 are provided.

The image acquisition section 11 receives the movement status signal 48 from the movement status acquisition section 12 . When the movement status signal 48 indicates that the vehicle 100 will turn right or left at an intersection, the image acquisition unit 11 selects two captured images from the captured images received from the camera mounted on the vehicle 100. - 特許庁One of the two captured images is the initial detection image 21A, and the other is the linked detection image 22A. The image acquisition unit 11 outputs image selection information 41 indicating the selected initial detection image 21A and cooperation detection image 22A to the priority area setting unit 14 .

When the movement status signal 48 indicates that the vehicle 100 turns right or left, the image acquisition unit 11 outputs the acquired initial detection image 21A to the first detection unit 13 each time it acquires the initial detection image 21A. . The image acquisition unit 11 does not output the initial detection image 21A and the cooperation detection image 22A to the image quality adjustment unit 15 until it receives cooperation start information 44 (to be described later) from the priority area setting unit 14 .

When the image acquisition unit 11 receives the cooperation start information 44 from the priority area setting unit 14, the image acquisition unit 11 changes the output destination of the captured image. Specifically, the image acquisition unit 11 outputs the acquired initial detection image 21A and the cooperation detection image 22A to the image quality adjustment unit 15 every time it acquires the initial detection image 21A and the cooperation detection image 22A. Image output to the first detection unit 13 is stopped.

In addition, the image acquisition unit 11 outputs the front captured image 31A to the movement state acquisition unit 12 regardless of the behavior of the vehicle 100 .

The movement status acquisition unit 12 receives the direction indication signal 4S from the direction indication switch 4 and receives the front shot image 31A from the image acquisition unit 11 . Based on the received direction instruction signal 4S and forward photographed image 31A, the movement status acquisition unit 12 determines whether the vehicle 100 is to turn right, turn left, or go straight at an intersection, and displays a movement status indicating the determination result. A signal 48 is output to the image acquisition unit 11 .

The first detection unit 13 receives the initial detection image 21A from the image acquisition unit 11 and detects a moving object from the received initial detection image 21A. When the first detection unit 13 receives a priority area image 21B and a non-priority area image 22C, which will be described later, from the image quality adjustment unit 15, the first detection unit 13 detects the moving object from the priority area image 21B and the non-priority area image 21C instead of the initial detection image 21A. To detect.

When detecting a moving object, the first detection unit 13 outputs detection position information 42 to the priority area setting unit 14 and the detection result output unit 17 . The detected position information 42 includes the position of the moving object in the initial detected image 21A. The first detection unit 13 outputs the moving direction information 43 to the priority area setting unit 14 . The moving direction information 43 indicates the moving direction of the moving object in the initial detection image 21A.

The priority area setting unit 14 receives detection position information 42 and moving direction information 43 from the first detection unit 13 . The priority area setting unit 14 uses the received detection position information 42 and movement direction information 43 to determine whether to start cooperative detection. Cooperation detection will be described later. When determining to start cooperation detection, the priority area setting unit 14 outputs cooperation start information 44 to the image acquisition unit 11 . Also, the priority area setting unit 14 sets both a priority area and a non-priority area, which will be described later, in each of the initial detection image 21A and the cooperation detection image 22A. The priority area setting section 14 outputs priority area information 45 indicating the set priority area and non-priority area to the image quality adjustment section 15 .

The image quality adjustment unit 15 receives the priority area information 45 from the priority area setting unit 14 . Based on the received priority area information 45, the image quality adjustment section 15 cuts out the priority area image 21B and the non-priority area image 21C from the initial detection image 21A received from the image acquisition section 11. FIG. The priority area image 21B is an image included in the priority area of the initial detection image 21A. The non-priority area image 21C is an image included in the non-priority area of the initial detection image 21A.

The image quality adjustment unit 15 adjusts the image quality of the cut-out non-priority area image 21C so that the quality of the cut-out non-priority area image 21C is lower than the image quality of the cut-out priority area image 21B. The image quality adjustment unit 15 outputs the clipped priority area image 21B and the non-priority area image 21C whose image quality is adjusted to the first detection unit 13 .

Based on the received priority area information 45, the image quality adjustment section 15 cuts out the priority area image 32B and the non-priority area image 32C from the cooperation detection image 22A received from the image acquisition section 11. FIG. The priority area image 22B is an image included in the priority area of the cooperation detection image 22A. The non-priority area image 22C is an image included in the non-priority area of the cooperation detection image 22A.

The image quality adjustment unit 15 adjusts the image quality of the clipped non-priority area image 22C so that the clipped non-priority area image 22C has a lower image quality than the clipped priority area image 22B. The image quality adjustment unit 15 outputs the clipped priority area image 22B and the non-priority area 22C whose image quality is adjusted to the second detection unit 16 .

The second detection unit 16 receives the priority area image 22B and the image quality-adjusted non-priority area image 22C from the image quality adjustment unit 15 . The second detection unit 16 detects a moving object from each of the received priority area image 22B and non-priority area image 22C. That is, the second detection unit 16 detects a moving object from the cooperation detection image 22A. The second detection section 16 outputs the detection position information 46 to the detection result output section 17 . The detection position information 46 indicates the result of detecting a moving object from the priority area image 22B and the non-priority area image 22C.

The detection result output unit 17 receives detection position information 42 from the first detection unit 13 and receives detection position information 46 from the second detection unit 16 . The detection result output unit 17 generates detection result information 47 indicating the presence of a moving object moving around the vehicle 100 based on the received detection position information 42 and 44, and displays the generated detection result information 47. Output to device 5 . The detection result information 47 is, for example, the initial detection image 21A or the linked detection image 22A in which the detected moving object is emphasized.

[4. Configuration of Image Acquisition Unit 11]
FIG. 4 is a functional block diagram showing the configuration of the image acquisition unit 11 shown in FIG. 3. As shown in FIG. Referring to FIG. 4 , image acquisition portion 11 includes an image selection portion 111 and an image output portion 112 .

Based on the movement status signal 48 received from the movement status acquisition section 12, the image selection section 111 selects the initial detection image 21A and the cooperative detection image 22A from the captured images 31A-34. The image selection unit 111 outputs image selection information 41 indicating the selected initial detection image 21A and cooperation detection image 22A to the priority area setting unit 14 and the image output unit 112 .

The image output unit 112 receives the captured images 31A to 34A from the camera, and receives the image selection information 41 from the image selection unit 111 . The image output unit 112 outputs the initial detection image 21A among the received photographed images 31A to 34A to the first detection unit 13 based on the received image selection information 41 .

When receiving the cooperation start information 44 from the priority area setting unit 14 , the image output unit 112 stops outputting the initial detection image 21</b>A to the first detection unit 13 . The image output unit 112 starts outputting the initial detection image 21A and the cooperation detection image 22A to the image quality adjustment unit 15 .

Further, the image output unit 112 outputs the forward photographed image 31A to the movement state acquisition unit 12 regardless of the movement state signal 48 received from the movement state acquisition unit 12 .

[5. Configuration of Movement Status Acquisition Unit 12]
FIG. 5 is a functional block diagram showing the configuration of the movement status acquisition unit 12 shown in FIG. 3. As shown in FIG. With reference to FIG. 5 , movement status acquisition unit 12 includes an entry determination unit 121 and a turn determination unit 122 .

The entry determination unit 121 receives the forward captured image 31A from the image output unit 112 . The entry determination unit 121 determines whether or not the vehicle 100 will enter the intersection using the received forward photographed image 31A, and outputs the entry information 49 indicating the determination result to the turn determination unit 122 . Specifically, the entry determination unit 121 determines that the vehicle 100 will enter the intersection when a pedestrian crossing is detected from the forward captured image 31A.

The turn determination unit 122 receives the direction indication signal 4S from the direction indication switch 4 and the entry information 49 from the entry determination unit 121 . When the received entry information 49 indicates entry into an intersection, the turn determination unit 122 determines whether or not the received direction instruction signal 4S instructs blinking of the winkers. When the direction instruction signal 4S instructs the blinking of the right winker 7R, the turn determination unit 122 determines that the vehicle 100 will turn right at the intersection, and outputs the movement status signal 48 indicating that the vehicle 100 is turning right to the image acquisition unit 11. do. When the direction instruction signal 4S instructs the blinking of the left winker 7L, the turn determination unit 122 determines that the vehicle 100 will turn left at the intersection, and outputs the movement status signal 48 indicating that the vehicle 100 is turning left to the image acquisition unit 11. do.

[6. Operation outline]
FIG. 6 is a diagram for explaining an outline of the operation of the moving object detection device 10 shown in FIG. An outline of the operation of the moving object detection device 10 will be described with reference to FIG. 6, taking as an example the case where the vehicle 100 turns right. In FIG. 6, the road R1 extends in the north-south direction, and the road R2 extends in the east-west direction. A road R1 and a road R2 intersect at an intersection P.

The vehicle 100 travels on the road R1 toward the intersection P and turns right at the intersection P. When the ignition switch of the vehicle 100 is turned on, the moving object detection device 10 repeatedly executes the process of detecting the pedestrian crossing from the front captured image 31A. Like the vehicle 100, the bicycle 25 travels along the road R1 toward the intersection P and crosses the crosswalk C2.

At time t0, moving object detection device 10 detects crosswalk C1 at intersection P, and receives direction indication signal 4S from direction indication switch 4 indicating blinking of right winker 7R. As a result, the moving object detection device 10 determines that the vehicle 100 will turn right (step S1).

At time t0, the moving object detection device 10 selects the right shot image 34A as the initial detection image, and selects the front shot image 31A as the linked detection image. The moving object detection device 10 starts processing to detect a moving object from the right captured image 34A. The front shot image 31A is not used for moving object detection until cooperative detection is started.

The moving object detection device 10 detects the bicycle 25 from the right captured image 34A at time t1 after time t0 (step S2). As a result of step S2, the position 25P and vector 25V of the bicycle 25 in the right captured image 34A are specified. A vector 25V indicates the direction and amount of movement of the bicycle 25 in the right captured image 34A.

When the position 25P and the vector 25V of the bicycle 25 satisfy the conditions for starting cooperative detection, the moving object detection device 10 determines to start cooperative detection (step S3). Details will be described later.

When it is decided to start cooperative detection, the moving object detection device 10 sets the priority area 54B and the non-priority area 54C in the right captured image 34A. The priority area 54B is the left half of the right captured image 34A. The non-priority area 54C is the right half of the right captured image 34A. The moving object detection device 10 sets the priority area 51B and the non-priority area 51C in the front shot image 31A. The priority area 51B is the right half of the front shot image 31A, and the non-priority area 51C is the left half of the front shot image 31A.

The moving object detection device 10 generates a priority area image 34B and a non-priority area image 34C by clipping the priority area 54B and the non-priority area 54C from the right captured image 34A. The moving object detection device 10 generates a priority area image 31B and a non-priority area image 31C by cutting out the priority area 51B and the non-priority area 51C from the front shot image 31A.

The moving object detection device 10 thins out the pixels of the generated non-priority area image 34C to adjust the image quality of the non-priority area image 34C. The size of the non-priority area image 34C whose image quality has been adjusted is smaller than the non-priority area image 34C clipped from the right captured image 34A. The image quality of the non-priority area image 31C is similarly adjusted. The image quality of the priority area images 31B and 34B is not adjusted.

The moving object detection device 10 detects a moving object from the priority area images 31B and 34B. The moving object detection device 10 detects a moving object from the non-priority area images 31C and 34C whose image quality has been adjusted.

In cooperation detection, both a priority area and a non-priority area are set in each of the initial detection image 21A and the cooperation detection image 22A. The moving object detection device 10 adjusts the image quality of the non-priority area images 21C and 22C by thinning out the pixels included in the non-priority area images 21C and 22C. The non-priority area images 21C and 24C whose image quality is adjusted have fewer pixels than the non-priority area images 21C and 22C whose image quality is not adjusted. Therefore, the processing load of detecting a moving object from the non-priority area images 21C and 22C whose image quality has been adjusted is smaller than the processing load of detecting a moving object from the clipped non-priority area images 21C and 22C. Therefore, the moving object detection apparatus 10 can suppress the detection delay of the moving object even when the moving object is detected simultaneously from the front captured image 31A and the right captured image 34A.

[7. Operation details]
[7.1. Overall operation]
FIG. 7 is a flow chart showing the operation of the moving object detection device 10. As shown in FIG. Hereinafter, the overall operation of the moving object detection device 10 will be described with reference to FIG. The moving object detection device 10 starts the processing shown in FIG. 7 when the ignition switch of the vehicle 100 is turned on.

The movement status acquisition unit 12 identifies the behavior of the vehicle 100 (step S10). Although the details will be described later, the movement status acquisition unit 12 determines whether the vehicle 100 makes a right turn, a left turn, or goes straight ahead at an intersection. Step S10 corresponds to step S1 shown in FIG.

If the movement status acquisition unit 12 determines that the vehicle 100 is going straight (No in step S20), the moving object detection device 10 proceeds to step S70. If the movement status acquisition unit 12 determines that the vehicle 100 will turn right or left (Yes in step S20), the moving object detection device 10 proceeds to step S30.

In step S30, the moving object detection device 10 determines whether or not to start cooperative detection. Step S30 corresponds to steps S2 to S3 shown in FIG. Details of step S30 will be described later.

When moving object detection apparatus 10 determines not to start cooperative detection (No in step S40), the process proceeds to step S70. When moving object detection apparatus 10 determines to start cooperative detection (Yes in step S40), moving object detection apparatus 10 executes cooperative detection (step S50). Details of step S50 will be described later.

When moving object detection apparatus 10 determines not to end cooperative detection (No in step S60), it repeats cooperative detection (step S50). When moving object detection apparatus 10 determines to end cooperative detection (Yes in step S60), the process proceeds to step S70. The termination condition of cooperation detection will be described later.

In step S70, the moving object detection device 10 determines whether the ignition signal has changed from on to off. If the ignition signal continues to be on (No in step S70), moving object detection device 10 returns to step S10. If the ignition signal has changed from on to off (Yes in step S70), the moving object detection device 10 ends the processing shown in FIG.

[7.2. Vehicle behavior identification (step S10)]
FIG. 8 is a flowchart of the vehicle behavior identification process (step S10) shown in FIG. Each time the image acquisition unit 11 acquires the front shot image 31A from the front camera 31, the movement status acquisition unit 12 executes the processing shown in FIG. 8 to determine whether the vehicle 100 will go straight at the intersection.

Specifically, the entry determination unit 121 acquires the front shot image 31A from the image acquisition unit 11 (step S101).

The entry determination unit 121 detects a pedestrian crossing from the front captured image 31A acquired in step S101 (step S102). Step S102 will be described by taking the detection of the crosswalk C1 shown in FIG. 6 as an example. The entry determination unit 121 detects a plurality of white lines each extending in the traveling direction of the vehicle 100 . Since the plurality of detected white lines are arranged in the lateral direction of the vehicle 100, the entry determination unit 121 determines that the pedestrian crossing C1 has been detected.

When the crosswalk C1 is detected, the entry determination unit 121 outputs the entry information 49 indicating that the vehicle 100 is entering the intersection to the turn determination unit 122 . When the crosswalk C1 cannot be detected, the entry determination unit 121 outputs the entry information 49 indicating that the vehicle 100 does not enter the intersection to the turn determination unit 122 .

Note that the entry determination unit 121 may determine to enter the intersection P when both the crosswalks C1 and C2 shown in FIG. 6 are detected. The algorithm used in step S102 is not particularly limited as long as the entry determination unit 121 can detect the pedestrian crossing from the front captured image 31A.

When turning determination unit 122 receives entry information 49 indicating that vehicle 100 does not enter the intersection (No in step S103), turn determination unit 122 outputs movement status signal 48 indicating that vehicle 100 is going straight to image acquisition unit 11. , the process shown in FIG. 8 ends.

When the turning determination unit 122 receives the entry information 49 indicating that the vehicle 100 is entering the intersection (Yes in step S103), the turn determination unit 122 determines whether or not the direction instruction signal 4S instructing the blinking of the right winker 7R is received. (Step S104).

When turning determination unit 122 receives direction instruction signal 4S instructing blinking of right winker 7R (Yes in step S104), turn determination unit 122 determines that vehicle 100 turns right at the intersection (step S105). The turn determination unit 122 outputs the movement status signal 48 indicating that the vehicle 100 is to turn right to the image acquisition unit 11, and ends the processing shown in FIG.

When the direction instruction signal 4S instructing blinking of the right winker 7R has not been received (No in step S104), the turn determination unit 122 determines whether or not the direction instruction signal 4S instructing blinking of the left winker 7L has been received. (Step S106).

When turning determination unit 122 receives direction instruction signal 4S instructing blinking of left winker 7L (Yes in step S106), turn determination unit 122 determines that vehicle 100 will turn left at the intersection (step S107). The turn determination unit 122 outputs the movement status signal 48 indicating that the vehicle 100 is to turn left to the image acquisition unit 11, and ends the processing shown in FIG. In this manner, the movement status acquisition unit 12 uses the direction instruction signal 4S to determine whether to turn right, turn left, or go straight at an intersection. Accordingly, the movement status acquisition unit 12 can predict the behavior of the vehicle 100 at the intersection based on the intention of the driver of the vehicle 100 . Further, when the moving object detection device 10 determines that the vehicle 100 will turn right or left, it starts detecting a moving object, which will be described later. Therefore, when the vehicle 100 turns right or left at an intersection, it is possible to prevent the vehicle from colliding with a moving object.

When turning determination unit 122 does not receive direction instruction signal 4S instructing blinking of left winker 7L (No in step S107), it determines that vehicle 100 will go straight at the intersection (step S108). The turn determination unit 122 outputs the movement status signal 48 indicating that the vehicle 100 is going straight to the image acquisition unit 11, and ends the processing shown in FIG.

[7.3. Cooperation start judgment (step S30)]
FIG. 9 is a flow chart of the cooperation start determination process (step S30) shown in FIG. When the moving object detection device 10 determines that the vehicle 100 will turn right or left at the intersection (Yes in step S20 shown in FIG. 7), the moving object detection device 10 starts the process shown in FIG. Hereinafter, the cooperation start determination process (step S30) will be described in detail with reference to FIG. 9, taking as an example the case where the vehicle 100 turns right at the intersection P shown in FIG.

(Selection of captured image)
When the movement status signal 48 indicating that the vehicle 100 is to turn right at the intersection is received from the movement status acquisition part 12, the image selection part 111 selects the initial detection image 21A and the cooperative detection image 22A (step S301). Since the vehicle 100 turns right, the image selection unit 111 selects the right captured image 34A as the initial detection image, and selects the front captured image 31A as the linked detection image. The image selection unit 111 outputs image selection information 41 indicating the selected initial detection image 21A and cooperation detection image 22A to the priority area setting unit 14 and the image output unit 112 .

(Detection of moving objects)
Based on the image selection information 41 received from the image selection unit 111, the image output unit 112 notifies that the right shot image 34A has been selected as the initial detection image 21A and the front shot image 31A has been selected as the linked detection image 22A. recognize. When image output unit 112 receives right side captured image 34 A selected as an initial detection image from right side camera 34 (Yes in step S<b>302 ), image output unit 112 outputs received right side captured image 34 A to first detection unit 13 . do. The first detection unit 13 detects a moving object from the right captured image 34A received from the image acquisition unit 11 (step S303).

FIG. 10 is an enlarged view of the right captured image 34A shown in FIG. Detection of a moving object (step S303) will be described in detail by taking the right shot image 34A shown in FIG. 10 as an example.

The first detection unit 13 generates an edge image (not shown) from the right captured image 34A, and extracts a region 25R surrounded by edges from the generated edge image. When the size of the extracted region 25R is larger than a preset reference size, the first detection unit 13 determines that the extracted region 25R is an object. In the example shown in FIG. 10, the object surrounded by the area 25R is the bicycle 25. In the example shown in FIG. For convenience of explanation, the region 25R is indicated by a rectangle in FIG. The first detection unit 13 determines the center of the extracted area 25R as the position 25P of the bicycle 25. FIG.

When a moving object is detected based on the size of the area surrounded by edges, the first detection unit 13 cannot identify the moving object as a pedestrian or a bicycle. In the following description, first detection unit 13 assumes that the object surrounded by region 25R is bicycle 25. As shown in FIG. Note that when pattern matching is used, the first detection unit 13 can identify bicycles, pedestrians, and the like.

The first detection unit 13 determines whether the bicycle 25 is a moving object based on the position 25P of the bicycle 25 and the area 25R. Hereinafter, in the description of step S303, the right captured image 34A shown in FIG. 10 is referred to as the "current captured image", and the right captured image 34A acquired immediately before the current captured image is referred to as the "past captured image". . In FIG. 10, the past area 26R is the area of the bicycle 25 detected from the past detection image. The past position 26P is the center of the past area 26R. In FIG. 10, the distance D from the position 25P to the past position 26P is exaggerated.

The position 25P and the past position 26P are described by the pixel coordinate system of the right captured image 34A. The origin of the pixel coordinate system is the upper left vertex of the captured image. In the pixel coordinate system, the Xa axis extends rightward from the origin, and the Ya axis extends downward from the origin.

The first detection unit 13 determines that the bicycle 25 detected from the currently captured image is a moving object when the following two conditions are satisfied.

The first condition is that the ratio between the size of the region 25R and the size of the past region 26R is within a predetermined range. The difference between the generation time of the currently captured image and the generation time of the previously captured image corresponds to the frame interval of the moving image generated by the right side camera 34 . When the bicycle 25 is detected from both the currently captured image and the previously captured image, it is assumed that the size of the region 25R does not change significantly from the size of the previous region 26R. Therefore, the ratio between the size of the area 25R and the size of the past area 26R is used as one of the moving object detection conditions.

A second condition is that the distance D is equal to or less than a predetermined distance. If the distance D is extremely large, it is unlikely that the region 25R and the past region 26R show the same object even if the speed of the vehicle 100 and the frame interval of the moving image are considered. Therefore, the distance D is used as one of the moving object detection conditions.

When the first detection unit 13 determines that the bicycle 25 (moving object) has been detected from the currently captured image, the first detection unit 13 outputs detection position information 42 including the position 25P to the priority area setting unit 14 and detection result output unit 17 . The first detection unit 13 generates the vector 25V and outputs movement direction information 43 including the generated vector 25V to the priority area setting unit 14 . The vector 25V starts at a past position 26P and ends at a position 25P.

In the present embodiment, the first detection unit 13 may transform the position of the object specified in the captured image into the world coordinate system. In this case, the first detection unit 13 may determine whether the second condition is satisfied, for example, based on the position of the object in the world coordinate system.

(Decision to start cooperation detection)
Referring again to FIG. 9, priority area setting unit 14 executes steps S304 to S306 to determine whether or not to start cooperation detection.

The priority area setting unit 14 determines whether or not a moving object has been detected from the initial detection image 21A based on the detection position information 42 received from the first detection unit 13 (step S304). If no moving object is detected (No in step S304), the priority area setting unit 14 determines not to start cooperative detection (step S309), and ends the processing shown in FIG. If a moving object is detected (Yes in step S304), the priority area setting unit 14 proceeds to step S305.

Based on the image selection information 41 received from the image acquisition unit 11, the priority area setting unit 14 specifies the initial detection image 21A and the cooperation detection image 22A. Based on the detection position information 42 received from the first detection unit 13, the priority area setting unit 14 determines whether or not the moving object is positioned within the cooperation reference area preset in the initial detection image 21A (step S305). The reason for executing step S305 will be described later.

When the vehicle 100 turns right, the cooperation reference area is the left half area of the right-side photographed image 34A that is divided into two equal parts. In the example shown in FIG. 10, the bicycle 25 is positioned within the left half area of the right captured image 34A (Yes in step S305). In this case, the priority area setting unit 14 proceeds to step S306, which will be described later.

When the bicycle 25 is positioned within the right half area of the right side captured image 34A shown in FIG. 10 (No in step S305), the priority area setting unit 14 determines not to start cooperation detection (step S309). ), the process shown in FIG. 9 ends.

The reason for executing step S305 will be described. Step S305 is executed to determine whether or not the moving object detected from the initial detection image 21A is detected from the linked detection image 22A. A detailed description will be given below.

FIG. 11 is a diagram schematically showing the photographing range of the camera shown in FIG. FIG. 11 shows the imaging range of each camera in a fan shape. The camera can photograph a subject positioned within its photographing range. The front camera 31 can photograph a subject within the photographing range 31H. The right side camera 32 can photograph a subject within the photographing range 34H. The rear camera 33 can photograph a subject within the photographing range 33H. The left side camera 34 can photograph a subject within a photographing range 34H. In practice, each camera can shoot farther than the arc indicating the shooting range.

The imaging range 34H is divided into two based on the optical axis 34K of the right side camera 34. FIG. The cooperation reference area set in the right shot image 34A corresponds to the left partial shooting range 341 with respect to the optical axis 34K.

The partial shooting range 341 includes the overlapping shooting range 35 shot by both the front camera 31 and the right side camera 34 . That is, when a moving object is located within the cooperation reference area set in the right captured image 34A, there is a high possibility that this moving object will be captured not only by the right side camera 34 but also by the front camera 31 . Moving object detection apparatus 10 determines that a moving object can be continuously detected while vehicle 100 is turning right by detecting a moving object using not only right captured image 34A but also front captured image 31A. Therefore, the priority area setting unit 14 uses the position of the moving object within the cooperation start area of the initially detected image as one of the conditions for starting cooperation detection.

That is, when there is a high possibility that both the front camera 31 and the right side camera 34 can capture moving objects, the priority area setting unit 14 starts cooperative detection using the front captured image 31A and the right captured image 34A. As a result, it is possible to shorten the time for executing the process of detecting a moving object in parallel from each of the two captured images, thereby suppressing an increase in the load of detecting a moving object.

Referring to FIG. 9 again, priority area setting unit 14 determines whether the moving object moves in the same direction as the traveling direction of vehicle 100 at a speed equal to or higher than the speed of vehicle 100, based on moving direction information 43 received from first detecting unit 13. (step S306). The reason for executing step S306 will be described later.

When the speed of the bicycle 25 is slower than the speed of the vehicle 100, or when the bicycle 25 is traveling in the direction opposite to the traveling direction of the vehicle 100 (No in step S306), the detection result output unit 17 outputs the initial detection image 21A. The detection result information 47 is output using the result of detecting the moving object from (step S308). For example, when a moving object is detected from the right captured image 34A, the detection result output unit 17 detects that the moving object exists on the right side of the vehicle 100 based on the detected position information 42 received from the first detection unit 13. A message indicating that is generated, and the generated message is output to the display device 5 . Since the cooperative detection has not started, the result of detecting the moving object from the cooperative detection image 22A is not used in step S308. After step S308, the priority area setting unit 14 determines not to start cooperation detection (step S309), and the priority area setting unit 14 ends the processing shown in FIG.

If the moving object is traveling in the same direction as the traveling direction of vehicle 100 at a speed equal to or higher than the speed of vehicle 100 (Yes in step S306), priority area setting unit 14 determines to start cooperative detection (step S307). The priority area setting unit 14 outputs cooperation start information 44 indicating the start of cooperation detection to the image acquisition unit 11 . The image acquisition unit 11 stops outputting the initial detection image 21A to the first detection unit 13 and starts outputting both the initial detection image 21A and the linked detection image 22A to the image quality adjustment unit 15 . The image acquisition unit 11 outputs the acquired two captured images to the image quality adjustment unit 15 every time it acquires the two captured images of the initial detection image 21A and the cooperation detection image 22A.

Step S306 is executed to determine whether or not there is a possibility that the vehicle 100 will collide with a pedestrian or the like crossing the crosswalk C2. The reason for executing step S306 will be described in detail with reference to the right captured image 34A shown in FIG.

Assume that the bicycle 25 travels in the same direction as the vehicle 100 at a speed slower than that of the vehicle 100 . Whether bicycle 25 is slower than vehicle 100 is determined based on the orientation of vector 25V. If the bicycle 25 is slower than the vehicle 100, the vector 25V points in the positive direction of the Xa axis. This is because the bicycle apparently moves backward with respect to the vehicle 100 . In this case, since the bicycle 25 reaches the crosswalk C2 later than the vehicle 100, the possibility of the vehicle 100 coming into contact with the bicycle 25 at the crosswalk C2 is small. In this case, the priority area setting unit 14 does not start cooperation detection.

Assume that the bicycle 25 travels in the same direction as the vehicle 100 at a speed faster than that of the vehicle 100 . Bicycle 25 is faster than vehicle 100 when vector 25V points in the negative direction of the Xa axis. This is because the bicycle 25 apparently moves forward with respect to the vehicle 100 . In this case, the bicycle 25 reaches the crosswalk C2 more than the vehicle 100, so there is a high possibility that the vehicle 100 will enter the intersection P while the bicycle 25 is passing through the crosswalk C2. The priority area setting unit 14 determines that the vehicle 100 may collide with the bicycle 25 and decides to start cooperative detection in order to increase the detection accuracy of the bicycle 25 . By starting cooperative detection, the detection accuracy of the bicycle 25 can be improved.

Assume that the bicycle 25 travels at the same speed as the vehicle 100 and in the same direction as the vehicle 100 travels. In this assumption, vector 25V is a zero vector because bicycle 25 is apparently stopped relative to vehicle 100 . In this case, there is a high possibility that the vehicle 100 and the bicycle 25 will reach the crosswalk C2 at the same time. Therefore, the priority area setting unit 14 determines that the vehicle 100 may collide with the bicycle 25, and decides to start cooperative detection. By starting cooperative detection, the detection accuracy of the bicycle 25 can be improved.

When the vehicle 100 turns left, the left shot image 32A is selected as the initial detection image 21A, and the front shot image 31A is detected as the cooperative detection image 22A. The cooperation reference area set in the left shot image 32A is the right half area of the left shot image 32A.

[7.4. Coordination Detection Processing (Step S50)]
FIG. 12 is a flow chart of the cooperation detection process (step S50) shown in FIG. Referring to FIG. 12, priority area setting unit 14 determines whether a priority area and a non-priority area are set for each of the two captured images selected in step S301 shown in FIG. 9 (step S501). ).

If the priority area and the non-priority area have already been set (No in step S501), the priority area setting unit 14 proceeds to step S503.

If the priority area and the non-priority area are not set (Yes in step S501), the priority area setting unit 14 sets both the priority area and the non-priority area to each of the initial detection image 21A and the cooperation detection image 22A ( step S502). The priority area setting unit 14 outputs to the image quality adjustment unit 15 priority area information 45 indicating the priority area and the non-priority area set in step S502.

Step S502 will be described in detail by taking as an example a case where the right shot image 34A is selected as the initial detection image 21A and the front shot image 31A is selected as the linked detection image 22A.

FIG. 13 is a diagram showing an example of priority areas and non-priority areas set in the right captured image 34A shown in FIG. Referring to FIG. 13, priority region setting unit 14 divides right captured image 34A into two equal regions, left and right. The priority area setting unit 14 sets the left area of the bisected area as the priority area 54B and sets the right area as the non-priority area 54C. The priority area 54B matches the cooperation reference area set in the right captured image 34A. This is because when the image acquisition unit 11 newly acquires the right-side photographed image 34A, there is a high possibility that the moving object will be detected from the cooperation reference region of the newly acquired right-side photographed image 34A.

FIG. 14 is a diagram showing an example of priority areas and non-priority areas set in the front shot image 31A shown in FIG. Referring to FIG. 14, priority region setting unit 14 divides front shot image 31A into two equal regions, left and right. When the right captured image 34A is the initial detection image 21A, the priority area setting unit 14 sets the right area as the priority area 51B and the left area as the non-priority area 51C among the divided areas.

When the left shot image 32A is the initial detected image 21A, the priority area setting unit 14 sets the right area of the left shot image 32A as the priority area 51B and sets the left area as the non-priority area 51C. The priority area setting unit 14 sets the left area of the front shot image 31A as the priority area 51B, and sets the right area as the non-priority area 51C.

Referring to FIG. 12 again, image quality adjustment unit 15 cuts out priority areas from each of initial detection image 21A and cooperation detection image 22A based on priority area information 45 received from priority area setting unit 14 (step S503). ). Thereby, priority area images 21B and 22B are generated. The image quality adjustment unit 15 outputs the priority area image 21B cut out from the initial detection image 21A to the first detection unit 13 . The image quality adjustment unit 15 outputs the priority area image 22B cut out from the cooperation detection image 22A to the second detection unit 16 .

For example, when the vehicle 100 turns right, the priority area image 34B shown in FIG. 6 is generated from the right captured image 34A. A priority area image 31B shown in FIG. 6 is generated from the front captured image 31A.

The image quality adjustment unit 15 cuts out non-priority regions from each of the initial detection image 21A and the cooperation detection region 22A based on the priority region information 45 received from the priority region setting unit 14 (step S504). As a result, non-priority area images 21C and 22C are generated.

For example, when the vehicle 100 turns right, the non-priority area image 34C shown in FIG. 6 is generated from the right captured image 34A. A non-priority area image 31C shown in FIG. 6 is generated from the front captured image 31A.

The image quality adjustment unit 15 adjusts the image quality of the non-priority area images 21C and 22C generated in step S504 (step S505). The image quality adjustment unit 15 outputs the non-priority area image 21C whose image quality has been adjusted to the first detection unit 13 . The image quality adjustment unit 15 outputs the non-priority area image 22C whose image quality has been adjusted to the second detection unit 16 .

For example, when the vehicle 100 turns right, the image quality adjustment unit 15 adjusts the image quality of the non-priority area image 34C so that the image quality of the non-priority area image 34C is lower than the image quality of the priority area image 34B. The image quality adjustment unit 15 adjusts the image quality of the non-priority area image 31C so that the image quality of the non-priority area image 31C is lower than the image quality of the priority area image 31B.

The image quality of the non-priority area image is adjusted by thinning out the pixels of the non-priority area image. Alternatively, the image quality adjustment unit 15 adjusts the image quality of the non-priority area image by making the resolution of the non-priority area image lower than the resolution of the captured image. That is, the image quality adjustment unit 15 reduces the image quality of the non-priority area image by reducing the non-priority area image.

The moving object detection apparatus 10 detects a moving object from the priority area images 21B and 22B and the non-priority area images 21C and 22C (step S506). Specifically, the first detection unit 13 detects a moving object from each of the priority area image 21B and the non-priority area image 21C generated from the initial detection image 21A. The second detection unit 16 detects a moving object from each of the priority area image 22B and the non-priority area image 22C generated from the cooperation detection image 22A. The detection method used in step S506 is the same as the detection method used in step S303 shown in FIG.

However, when detecting a moving object from the non-priority area image, the first detection unit 13 and the second detection unit 16 change the reference size used for comparison with the size of the area surrounded by edges. This is because the non-priority area image is reduced by the image quality adjustment unit 15 . Specifically, the reference size is adjusted according to the ratio between the size of the non-priority area image before reduction and the size of the reduced non-priority area image.

When using the priority area image 34B and the non-priority area image 34C for detecting a moving object, the load on the first detection unit 13 is reduced more than when using the right captured image 34A received from the right side camera 34 for detecting a moving object. can be made smaller. This is because the number of pixels of the non-prioritized area image has decreased due to the reduction of the non-prioritized area image 34C. Similarly, by using the priority area image 31B and the non-priority area image 31C, the second detection unit 16 can reduce the detection load of moving objects.

The detection result output unit 17 outputs the detection result information 47 based on the detection position information 42 received from the first detection unit 13 and the detection position information 46 received from the second detection unit 16 to the display device 5 (step S507). ).

Step S507 will be described in detail. The detected position information 42 indicates the position of the first moving object detected by the first detector 13 . The detected position information 46 indicates the position of the second moving object detected by the second detector 16 . In each of the detected position information 42 and 46, the position of the moving object is described in a pixel coordinate system that specifies the pixels of the captured image. The detection result output unit 17 transforms the position of the first moving object recorded in the detected position information 42 into a position in the world coordinate system.
The detection result output unit 17 transforms the position of the first moving object recorded in the detected position information 46 into a position in the world coordinate system.

Based on the position of the first moving object in the world coordinate system and the position of the second moving object in the world coordinate system, the detection result output unit 17 calculates the distance from the first moving object to the second moving object in the world coordinate system. Calculate The detection result output unit 17 determines that the first moving object and the second moving object are the same object when the calculated distance is equal to or less than a preset reference distance. In this case, the detection result output unit 17 outputs the initial detection image 21A or the cooperative detection image 22A in which the detected second moving object is emphasized to the display device 5 as the detection result information 47 .

When the detection result output unit 17 determines that the first moving object and the second moving object are not the same object, the detected second moving object is emphasized, and a front shot image including a moving object detection message, which will be described later. 31A is output as the detection result information 47 . The moving object detection message specifically indicates that the first moving object exists to the right of the right end of the front shot image 31A.

[7.5. End judgment of cooperation detection (step S50)]
Referring to FIG. 7 again, moving object detection apparatus 10 determines whether or not to end cooperative detection after cooperative detection processing (step S50) (step S60). If the moving object detection device 10 ends the cooperative detection (Yes in step S60), the process proceeds to step S70. If the moving object detection device 10 does not end the coordinated detection (No in step S60), the coordinated detection process (step S50) is executed again. That is, the moving object detection device 10 uses the new initial detection image 21A and the new cooperative detection image 22A acquired by the image acquisition unit 11 to perform the processing shown in FIG.

The determination in step S60 will be described with reference to FIG. 6 as an example. When the blinking of the right winker 7R ends, the moving object detection device 10 determines that the right turn of the vehicle 100 has ended, and determines the end of cooperative detection (step S60). The cooperation detection ends when the image acquisition unit 11 stops outputting the initial detection image 21A and the cooperation detection image 22A. When the vehicle 100 turns left, the moving object detection device 10 may determine whether or not the left blinker 7L has finished blinking.

The moving object detection apparatus 10 may determine that the vehicle 100 has finished turning right or left based on the moving distance from the current position of the vehicle 100 to the position of the vehicle 100 at the cooperative detection start time. For example, the moving object detection device 10 periodically acquires the position of the vehicle 100 from a GPS (Global Positioning System) device mounted on the vehicle 100, and uses the acquired position to calculate the movement distance. The moving object detection device 10 determines that the vehicle 100 has finished turning right or left when the calculated distance is greater than a preset distance threshold.

Alternatively, the moving object detection apparatus 10 may determine that the vehicle 100 has finished turning right or left when a predetermined period of time has elapsed from the start time of cooperative detection.

As described above, when a moving object is detected from the initial detection image 21A, the moving object detection apparatus 10 starts cooperative detection based on the position of the moving object and the moving direction of the moving object in the initial detection image 21A. Decide whether or not When determining to start cooperative detection, the moving object detection apparatus 10 sets both the priority area and the non-priority area in the initial detection image 21A and the cooperation detection image 22A. The moving object detection device 10 reduces the non-priority area images 21C and 22C clipped from the initial detection image 21A and the linked detection image 22A. The moving object detection apparatus 10 detects a moving object from the priority area images 21B and 22B clipped from the initial detection image 21A and the linked detection image 22A, and the reduced non-priority area images 21C and 22C. As a result, the moving object detection apparatus 10 can reduce the number of pixels in the initial detection image 21A and the joint detection image 22A, thereby reducing the load of detecting a moving object from the initial detection image 21A and the joint detection image 22A.

The priority area set in each of the initial detection image 21A and the joint detection image 22A is set so as to include the overlap shooting range in the initial detection image 21A and the joint detection image 22A. Therefore, the moving object detection device 10 can suppress a decrease in moving object detection accuracy.

Also, the priority area setting unit 14 sets priority areas and non-priority areas not only in the cooperation detection image 22A but also in the initial detection image 21A. As a result, the moving object detection apparatus 10 can reduce the load of detecting a moving object from the initial detection image 21A when cooperative detection is performed.

[Second embodiment]
[1. composition]
FIG. 15 is a functional block diagram showing the configuration of a moving object detection device 10A according to the second embodiment of the invention. 10 A of moving object detection apparatuses are mounted in the vehicle 100. FIG. Moving object detection device 10A detects a moving object when vehicle 100 runs in a parking lot. The moving object detection device 10A will be described below, focusing on the differences from the moving object detection device 10 shown in FIG.

Referring to FIG. 15, moving object detection apparatus 10A includes image acquisition unit 11A, movement condition acquisition unit 12A, and image acquisition unit 11A, movement condition acquisition unit 12A, and priority area setting unit 14 instead of image acquisition unit 11, movement condition acquisition unit 12, and priority area setting unit 14. and a priority area setting unit 14A.

Movement status acquisition unit 12A acquires front shot image 31A from image acquisition unit 11, and determines whether vehicle 100 is traveling in a parking lot using the acquired front shot image 31A. Movement status acquisition unit 12A outputs travel information 61 indicating whether vehicle 100 is traveling in a parking lot to image acquisition unit 11A.

When the first detection unit 13 detects a moving object, the priority area setting unit 14A starts coordinated detection based on the captured image in which the moving object is detected and the position and moving direction of the detected moving object. or not. The position and moving direction of the moving object are specified in the captured image in which the moving object is detected. When determining to start cooperative detection, the priority area setting unit 14A determines the captured image in which the moving object is detected as the initial detection image, and determines the coordinated detection image based on the captured image in which the moving object is detected. The priority area setting unit 14A outputs image designation information 62 designating the determined initial detection image and cooperation detection image to the image acquisition unit 11 .

The image acquisition unit 11A receives the travel information 61 from the movement status acquisition unit 12A. When the travel information 61 indicates travel in a parking lot, the image acquisition unit 11 outputs each of the captured images 31A to 34A to the first detection unit 13 according to a preset output order. That is, in this embodiment, the initial detection image 21A changes according to the output order of the captured images 31A to 34A. The output order will be described later.

Further, when receiving the image designation information 62 from the priority area setting unit 14A, the image acquisition unit 11A outputs the initial detection image and the cooperative detection image designated by the received image designation information 62 to the image quality adjustment unit 15 .

[2. motion]
FIG. 16 is a flow chart showing the operation of the moving object detection device 10A shown in FIG. The moving object detection device 10A starts the processing shown in FIG. 16 when the ignition switch of the vehicle 100 is turned on.

Referring to FIG. 16, movement status acquisition unit 12A identifies the behavior of vehicle 100 using front captured image 31A received from image acquisition unit 11 (step S11). Specifically, when detecting a preset pattern of white lines in a parking lot from the front captured image 31A, the movement status acquiring unit 12A determines that the vehicle 100 is traveling in the parking lot.

Note that the movement status acquisition unit 12A may determine that the vehicle 100 is traveling in a parking lot when a plurality of vehicles other than the vehicle 100 are parked side by side. Alternatively, movement status acquisition unit 12A determines whether vehicle 100 is traveling in a parking lot based on map information in which the location of the parking lot is set and the current location acquired from a GPS device (not shown). may

When the movement status acquiring unit 12A determines that the vehicle 100 is not traveling in the parking lot (No in step S21), the moving object detection device 10A proceeds to step S71. When the movement status acquisition unit 12A determines that the vehicle 100 is traveling in the parking lot (Yes in step S21), the moving object detection device 10A performs cooperation start determination processing for determining whether or not to start cooperation detection. (Step S31) is executed. The details of the cooperation start determination process (step S31) will be described later.

If the moving object detection device 10A does not decide to start cooperative detection (No in step S41), the process proceeds to step S71.

When the moving object detection device 10A determines to start cooperative detection (Yes in step S41), it executes cooperative detection processing (step S51). Since the cooperation detection process (step S51) shown in FIG. 16 is the same as the cooperation detection process (step S51) shown in FIG. 7, the description thereof will be omitted. The cooperation detection process (step S51) is repeated until the cooperation detection end condition is satisfied (Yes in step S61).

In the present embodiment, when vehicle 100 leaves the parking lot, moving object detection apparatus 10A determines that the end condition of cooperative detection is satisfied. Alternatively, the moving object detection apparatus 10A may determine that the condition for ending cooperative detection is satisfied when the moving object is no longer detected in both the initial detection image 21A and the cooperative detection image 22A. The moving object detection device 10A may determine that the condition for ending cooperative detection is satisfied when the speed of the vehicle 100 reaches or exceeds a predetermined speed. The predetermined speed is, for example, 30 km/h. This is because when the vehicle 100 is traveling at a speed equal to or higher than the predetermined speed, it is considered that the vehicle 100 is traveling on the road instead of the parking lot. When the vehicle 100 stops, the moving object detection device 10A may determine that the conditions for ending cooperative detection are satisfied.

When the end condition of cooperative detection is satisfied (Yes in step S61), the moving object detection device 10A determines whether or not the ignition signal has turned off (step S71). If the ignition signal is on (No in step S71), the moving object detection device 10A returns to step S11. If the ignition signal has turned off (Yes in step S71), the moving object detection device 10A ends the processing shown in FIG.

In this manner, when the vehicle 100 is traveling in the parking lot, the moving object detection device 10A determines the captured images 31A to 34A as the initial detection image 21A according to a predetermined order. When a moving object is detected from the initial detection image 21A, a cooperative detection image 22A is determined based on the initial detection image 21A, the detected position of the moving object, and the moving direction of the moving object. The moving object detection device 10A can flexibly change the initial detection image 21A and the cooperative detection image 22A used for detecting the moving object according to the position of the moving object around the vehicle. Further, the moving object detection device 10A starts determining an initial detection image when determining that the vehicle 100 is traveling in a parking lot. Thereby, a moving object moving in a parking lot can be quickly detected.

[3. Cooperation start judgment (step S31)]
FIG. 17 is a flowchart of the cooperation start judgment (step S31) shown in FIG. The image acquisition unit 11A selects one of the acquired photographed images 31A to 34A as the initial detection image 21A according to a predetermined order (step S311). The image acquisition unit 11A outputs the selected captured image to the first detection unit 13 as an initial detection image 21A. The image acquisition unit 11 outputs image selection information 41 indicating the selected captured image to the priority area setting unit 14A.

Each time the image acquisition unit 11A executes step S311, the initial detection image 21A changes according to the output order. For example, the output order is set in order of the front shot image 31A, the left shot image 32A, the rear shot image 33A, and the right shot image 34A. In this case, the image acquisition unit 11A first selects the forward captured image 31A as the initial detection image 21A. Next, the left shot image 32A is selected as the initial detection image 21A. Finally, the right captured image 34A is selected as the initial detected image 21A. After that, the front shot image 31A is again selected as the initial detection image 21A.

The first detection unit 13 detects a moving object from the initial detection image 21A received from the image acquisition unit 11A (step S312). Since step S312 is the same processing as step S303 shown in FIG. 9, its description is omitted. The first detection unit 13 outputs the detected position information 42 and the moving direction information 43 to the priority area setting unit 14 . The first detection section 13 outputs the detection position information 42 to the detection result output section 17 .

If the position of the moving object is not recorded in the detection position information 42 received from the first detection unit 13, the priority area setting unit 14A determines that the moving object has not been detected from the initial detection image 21A (No in step S313). ). The priority area setting unit 14 determines not to start cooperation detection (step S317), and ends the processing shown in FIG.

If the position of the moving object is recorded in the detection position information 42 received from the first detection unit 13, the priority area setting unit 14A determines that the moving object has been detected from the initial detection image 21A (Yes in step S313). . The priority area setting unit 14A determines whether or not the position of the moving object in the initially detected image 21A and the vector of the moving object in the initially detected image satisfy conditions for starting cooperative detection (step S314).

FIG. 18 is a diagram showing an example of the start condition table 55 used by the priority area setting section 14A. The priority area setting unit 14A uses the start condition table 55 shown in FIG. 18 to determine whether to start cooperation detection.

The priority area setting unit 14A identifies the captured image selected as the initial detection image 21A based on the image selection information 41 received from the image acquisition unit 11A. The priority area setting section 14A specifies the position of the moving object in the initial detection image 21A based on the detection position information 42 received from the first detection section 13 . The priority area setting unit 14A identifies the movement direction of the moving object in the initial detection image 21A based on the movement direction information 43 received from the first detection unit.

If the combination of the specified captured image, position, and vector direction is recorded in the start condition table 55, the start condition for cooperation detection is satisfied (Yes in step S314). In this case, the priority area setting unit 14A determines to start cooperation detection (step S315). The priority area setting unit 14A determines the cooperation detection image 22A based on the specified combination of the captured image, position, and vector direction (step S316).

The priority area setting unit 14A sets both the priority area and the non-priority area in each of the initial detection image 21A and the cooperation detection image 22A based on the start condition table 55 (step S317). Therefore, the moving object detection device 10A omits steps S501 and S502 shown in FIG. 12 when executing the cooperative detection process (step S51). After that, the priority area setting unit 14 ends the processing shown in FIG. 17 .

If the combination of the specified captured image, position, and vector direction is not recorded in the start condition table 55, the priority area setting unit 14A determines that the condition for starting cooperation detection is not satisfied (No in step S314). ). The priority area setting unit 14A determines not to start cooperation detection (step S317), and ends the processing shown in FIG.

A combination of the specified photographed image, position of the moving object, and moving direction of the moving object will be described with reference to FIG. 18 . When the moving object is a bicycle 25, the moving direction of the moving object is the direction of vector 25V.

(When the right captured image 34A is the initial detection image 21A)
When the initial detection image 21A is the right captured image 34A, the position of the moving object is the left area of the right captured image 34A, and the moving direction of the moving object is left, the priority area setting unit 14A sets the start condition table 55 to decide to start cooperative detection. In this case, the cooperation detection image 22A is determined to be the forward captured image 31A based on the start condition table 55. FIG. This is because there is a high possibility that the moving object will be detected in both the front shot image 31A and the right shot image 34A. This combination corresponds to the case where vehicle 100 in the first embodiment turns right.

In this case, the priority area of the initial detection image 21A is determined to be the left area where the moving object is detected. The non-priority area of the initial detection image 21A is set to the right area opposite to the left area. The priority area and non-priority area of the cooperation detection image 22A are set to the right area and left area recorded in the start condition table 55, respectively.

When the initial detection image 21A is the right captured image 34A, the position of the moving object is the right region of the right captured image 34A, and the moving direction of the moving object is right, the priority region setting unit 14A sets the start condition table 55 to initiate a joint detection decision. In this case, the cooperation detection image 22A is determined to be the rear captured image 33A based on the start condition table 55 . This is because there is a high possibility that the moving object will be detected in both the rear shot image 33A and the right shot image 34A.

In this case, the priority area of the initial detection image 21A is determined to be the right area where the moving object is detected. The non-priority area of the initial detection image 21A is set to the left area opposite to the right area. The priority area and non-priority area of the cooperation detection image 22A are set to the left area and right area recorded in the start condition table 55, respectively.

(When the left shot image 32A is the initial detection image 21A)
When the initial detection image 21A is the left shot image 32A, the position of the moving object is the right area of the left shot image 32A, and the moving direction of the moving object is right, the priority area setting unit 14A sets the start condition table 55 to initiate a joint detection decision. In this case, the cooperation detection image 22A is determined to be the forward captured image 31A based on the start condition table 55. FIG. This is because there is a high possibility that the moving object will be detected in both the front shot image 31A and the right shot image 34A. This combination corresponds to the case where vehicle 100 in the first embodiment makes a left turn.

In this case, the priority area of the initial detection image 21A is determined to be the right area where the moving object is detected. The non-priority area of the initial detection image 21A is set to the left area opposite to the right area. The priority area and non-priority area of the cooperation detection image 22A are set to the left area and right area recorded in the start condition table 55, respectively.

When the initial detection image 21A is the left shot image 32A, the position of the moving object is the left area of the left shot image 32A, and the moving direction of the moving object is left, the priority area setting unit 14A sets the start condition table 55 to initiate a joint detection decision. In this case, the cooperation detection image 22A is determined to be the rear captured image 33A based on the start condition table 55 . This is because there is a high possibility that the moving object will be detected in both the rear shot image 33A and the right shot image 34A.

In this case, the priority area of the initial detection image 21A is determined to be the left area where the moving object is detected. The non-priority area of the initial detection image 21A is set to the right area opposite to the left area. The priority area and non-priority area of the cooperation detection image 22A are set to the right area and left area recorded in the start condition table 55, respectively.

(When the front shot image 31A is the initial detection image 21A)
When the initial detection image 21A is the front shot image 31A, the position of the moving object is the left area of the front shot image 31A, and the moving direction of the moving object is left, the priority area setting unit 14A adds to initiate federation detection decisions based on In this case, the left shot image 32A is determined as the cooperation detection image 22A based on the start condition table 55 . This is because there is a high possibility that the moving object will be detected in both the front shot image 31A and the left shot image 32A.

In this case, the priority area of the initial detection image 21A is determined to be the left area where the moving object is detected. The non-priority area of the initial detection image 21A is set to the right area opposite to the left area. The priority area and non-priority area of the cooperation detection image 22A are set to the right area and left area recorded in the start condition table 55, respectively.

When the initial detection image 21A is the front shot image 31A, the position of the moving object is the right area of the front shot image 31A, and the moving direction of the moving object is right, the priority area setting unit 14A adds to initiate federation detection decisions based on In this case, the cooperation detection image 22A is determined to be the right captured image 34A based on the start condition table 55 . This is because there is a high possibility that the moving object will be detected in both the front shot image 31A and the right shot image 34A.

In this case, the priority area of the initial detection image 21A is determined to be the right area where the moving object is detected. The non-priority area of the initial detection image 21A is set to the left area opposite to the right area. The priority area and non-priority area of the cooperation detection image 22A are set to the left area and right area recorded in the start condition table 55, respectively.

(When the rear captured image 33A is the initial detection image 21A)
When the initial detection image 21A is the rear photographed image 33A, the position of the moving object is the right region of the rearward photographed image 33A, and the moving direction of the moving object is right, the priority region setting unit 14A adds to initiate federation detection decisions based on In this case, the left shot image 32A is determined as the cooperation detection image 22A based on the start condition table 55 . This is because there is a high possibility that the moving object will be detected in both the left shot image 32A and the rear shot image 33A.

In this case, the priority area of the initial detection image 21A is determined to be the right area where the moving object is detected. The non-priority area of the initial detection image 21A is set to the left area opposite to the right area. The priority area and non-priority area of the cooperation detection image 22A are set to the left area and right area recorded in the start condition table 55, respectively.

When the initial detection image 21A is the rear shot image 33A, the position of the moving object is the left area of the rear shot image 33A, and the moving direction of the moving object is left, the priority area setting unit 14A sets to initiate federation detection decisions based on In this case, the cooperation detection image 22A is determined to be the right captured image 34A based on the start condition table 55 . This is because there is a high possibility that the moving object will be detected in both the rear shot image 33A and the right shot image 34A.

In this case, the priority area of the initial detection image 21A is determined to be the left area where the moving object is detected. The non-priority area of the initial detection image 21A is set to the right area opposite to the left area. The priority area and non-priority area of the cooperation detection image 22A are set to the right area and left area recorded in the start condition table 55, respectively.

As described above, the moving object detection apparatus 10A according to the second embodiment selects each of the captured images 31A to 34A in a predetermined order as the initial detection image 21A when the vehicle 100 travels in the parking lot. . When a moving object is detected from the initial detection image 21A, the moving object detection device 10A determines a cooperative detection image 22A from the captured images 31A to 34A based on the detection result of the moving object, and detects both the priority area and the non-priority area. are set in each of the initial detection image 21A and the linked detection image 22A. As a result, even when the position of a moving object that is likely to come into contact with the vehicle 100 cannot be assumed, the moving object detection device 10 can detect the moving object through cooperative detection. As a result, moving objects can be continuously detected with high accuracy.

[Third embodiment]
[1. composition]
FIG. 19 is a functional block diagram showing the configuration of a moving object detection device 10B according to the third embodiment of the invention. Referring to FIG. 19 , moving object detection apparatus 10B includes priority area setting section 14B instead of priority area setting section 14 . The priority area setting unit 14B changes the priority area and non-priority area set in the cooperation detection image 22A according to the position or movement direction of the moving object detected from the initial detection image 21A.

The moving object detection device 10B has the same configuration as the moving object detection device 10 except that the priority region setting unit 14B is replaced with the priority region setting unit 14B. Therefore, in the description of this embodiment, the description common to the first embodiment will be omitted.

[2. motion]
As described above, the priority area setting unit 14B changes the priority area and non-priority area set in the cooperation detection image 22A according to the position or moving direction of the moving object detected from the initial detection image 21A. The setting of the priority area by the priority area setting unit 14B will be described below by dividing the change of the priority area based on the detected position of the moving object and the change of the priority area based on the amount of movement of the moving object.

(Priority area setting based on movement amount)
Setting of the priority area by the priority area setting unit 14B will be described below by taking as an example the case where the bicycle 25 shown in FIG. 6 is detected from the right-side photographed image 34A selected as the initially detected image 21A. Referring to FIG. 6, vector 25V indicates the moving direction and amount of movement of bicycle 25 in right captured image 34A. The vector 25V is described by the pixel coordinate system in the right captured image 34A and points in the negative direction of the Xa axis.

FIG. 20 is a diagram showing an example of changes in priority areas set by the priority area setting unit 14B. The front shot image 31A shown in FIG. 20 is the same as the front shot image 31A shown in FIG. The priority area setting unit 14B changes the width of the priority area 51B set in the front shot image 31A based on the size of the vector 25V.

For example, when the size of the vector 25V is 0 (pixels) or more and less than 100 (pixels), the priority area setting unit 14B sets the priority area 51Bb in the front shot image 31A. When the size of the vector 25V is equal to or greater than 100 (pixels) and less than 150 (pixels), the priority area setting unit 14B sets the priority area 51Ba to the front shot image 31A. When the size of the vector 25V is 150 (pixels) or more, the priority area setting unit 14B sets the priority area 51B in the front shot image 31A. The right sides of the priority areas 51B, 51Ba, and 51Bb match the right side of the front shot image 31A.

In FIG. 20, in order to clearly show the priority areas 51Ba and 51Bb, the vertical size of the priority areas 51Ba and 51Bb is shown to be smaller than the vertical size of the forward shot image 31A. In reality, the vertical size of the priority areas 51Ba and 51Bb matches the vertical size of the front shot image 31A.

In the example shown in FIG. 20, the horizontal size W2 of the priority area 51Ba is two thirds of the horizontal size W3 of the priority area 51B. The horizontal size W1 of the priority area 51Bb is one third of the horizontal size W3 of the priority area 51B. That is, the priority area setting unit 14B increases the horizontal size of the priority area set in the cooperation detection image 22A as the size of the vector 25V increases.

It is considered that the position of the bicycle 25 detected from the front shot image 31A moves to the left of the front shot image 31A as the magnitude of the vector 25V increases. Based on this idea, the priority area setting unit 14B changes the horizontal size of the priority area set in the cooperation detection image 22A according to the size of the vector 25V.

Note that when the vector 25V is oriented in an oblique direction, the priority area setting unit 14B may determine the width of the priority area based on the magnitude of the Xa-axis component of the vector 25V. When the left captured image 32A is the initial detection image 21A, the left sides of the priority areas 51B, 51Ba, and 51Bb match the left sides of the front captured image 31A.

(Priority area setting based on detection position)
FIG. 21 is a diagram showing an example of detection positions of the bicycle 25 detected by the first detector 13. As shown in FIG. The front shot image 31A shown in FIG. 21 is the same as the front shot image 31A shown in FIG.

An example of setting the priority area based on the position P of the bicycle 25 detected from the right captured image 34A will be described below with reference to FIG.

The priority area setting unit 14B acquires the distance R from the left side of the right captured image 34A to the position 25P of the bicycle 25. FIG. Since the position 25P is described by the pixel coordinate system of the right shot image 34A, the distance R is the Xa coordinate value of the position 25P. Based on the obtained distance R, the priority area setting unit 14B sets any one of the priority areas 51B, 51Ba, and 51Bb shown in FIG. 20 in the front shot image 31A.

Specifically, when the obtained distance R is 0 (pixel) or more and less than R1 (pixel), the priority area setting unit 14B sets the priority area 51B in the front shot image 31A. The priority area setting unit 14B sets the priority area 51Ba in the front shot image 31A when the acquired distance R is equal to or greater than R1 (pixels) and less than R2 (pixels). The priority area setting unit 14B sets the priority area 51Bb in the front shot image 31A when the acquired distance R is equal to or greater than R2 (pixels) and equal to or less than R3 (pixels).

That is, the priority area setting unit 14B increases the horizontal size of the priority area set in the front shot image 31A as the distance R from the position 25P to the left side of the right shot image 34A decreases. The shorter the distance R, the higher the possibility that the bicycle 25 will be photographed by the front camera 31 . The shorter the distance R, the closer the bicycle 25 is to the left-right center of the front shot image 31A. Therefore, the priority area setting unit 14B increases the size of the horizontal difference of the priority areas.

When the left shot image 32A is the initial detection image 21A, the priority area setting unit 14B may obtain the distance R with reference to the right side of the left shot image 32A.

The priority area setting unit 14B merges the priority area set based on the movement amount and the priority area set based on the detection position to determine the final priority area. For example, the priority area setting unit 14B selects an area included in at least one of the priority area set based on the movement amount and the priority area set based on the detection position in the cooperation detection image as the final priority area. Determined as Alternatively, the priority area setting unit 14B determines an area included in both the priority area set based on the movement amount and the priority area set based on the detection position in the cooperation detection image as the final priority area. do. That is, the priority area setting unit 14B may determine the final priority area based on at least one of the priority area set based on the movement amount and the priority area set based on the detection position. After that, the priority area setting unit 14B sets areas other than the priority area set in the front shot image 31A as non-priority areas.

In this manner, the priority area setting unit 14B changes the priority area set in the cooperation detection image 22A based on the detected position and movement amount of the moving object. As a result, it is possible to dynamically set a region in which a moving object is highly unlikely to be detected in the cooperative detection image as a non-priority region, thereby further reducing the load of detecting a moving object.

Note that the priority area setting unit 14B may increase the horizontal size of the priority area set in the cooperation detection image over time. This is because the moving object is more likely to be detected from the non-priority area set in the cooperative detection image as time elapses. By enlarging the priority area over time, the priority area setting unit 14B can prevent a decrease in moving object detection accuracy.

[Modification]
In the above-described embodiment, an example has been described in which detection of a moving object is started when vehicle 100 turns right or left, or when vehicle 100 travels in a parking lot. However, the moving object detection device may start detecting the moving object when the movement state of the vehicle 100 satisfies a predetermined condition. The predetermined condition is that the speed of vehicle 100 is slower than a predetermined detection start speed, in addition to the above embodiment. The predetermined detection start speed is, for example, 20 km/h. In this way, by starting the detection of moving objects according to the moving state of the vehicle, it is possible to prevent the detection load of moving objects from constantly increasing.

In the above-described embodiment, an example has been described in which the priority area setting section sets the priority area and the non-priority area in the initial detection image 21A. However, the priority area setting unit does not have to set the priority area and the non-priority area in the initial detection image 21A. In this case, the first detection unit 13 detects a moving object from the initial detection image 21A received from the image acquisition unit 11 even when cooperative detection is executed.

In the above-described embodiment, an example was described in which the movement status acquisition unit 12 determines that the vehicle 100 is to turn right or left at an intersection based on the detection of a crosswalk and the blinking instruction of the blinker, but the present invention is not limited to this. For example, the movement status acquisition unit 12 may determine that the vehicle 100 turns right or left based on the travel route of the vehicle 100 acquired from a car navigation device mounted on the vehicle 100 . The travel route includes information specifying an intersection at which the vehicle 100 turns right or left. The movement status acquisition unit 12 calculates the distance from the current position of the vehicle 100 to the intersection designated to turn right or left based on the current position of the vehicle 100 and the acquired travel route. If the calculated distance is equal to or less than the predetermined distance, the movement status acquisition unit 12 may determine that the vehicle 100 will turn right or left.

Alternatively, the movement status acquisition unit 12 identifies the lane in which the vehicle 100 travels using the front captured image 31A, and determines whether the vehicle 100 enters an intersection where the vehicle 100 turns right or left based on the identified lane. may The movement status acquisition unit 12 determines that the vehicle 100 is to turn right when a right-turn-only lane is detected from the front captured image 31A. The movement status acquisition unit 12 determines that the vehicle 100 is to turn left when a left turn-only lane is detected from the forward captured image 31A.

Alternatively, the movement status acquisition unit 12 may start the process of detecting a moving object from the initial detection image 21A when either the left winker 7L or the right winker 7R blinks. When the left winker 7L is blinking, the left shot image 32A is selected as the initial detection image 21A. When the right winker 7R is blinking, the right shot image 34A is selected as the initial detection image 21A.

In the above embodiment, an example in which the moving object detection device includes the movement status acquisition unit 12 or 12A has been described, but the present invention is not limited to this. The moving object detection device does not have to include the movement status acquisition unit. The moving object detection device may use a preset captured image as the initial detection image. For example, the moving object detection device can use the front captured image 31A as the initial detection image regardless of the movement status of the vehicle 100 . In this case, the moving object detection apparatus may determine the cooperative detection image based on the start condition in the start condition table 55 shown in FIG. 18 when the forward captured image 31A is the initial detection image.

In the above embodiment, the image quality adjustment unit 15 maintains the image quality of the priority area image. However, the image quality adjustment unit 15 may lower the image quality of the priority area image than the image quality of the captured image.

In the above embodiment, an example has been described in which the image quality adjustment unit 15 adjusts the image quality of the image of the non-priority area by reducing the number of pixels of the image of the non-priority area set in the captured image. However, the image quality adjusting unit 15 may adjust the image quality of the captured image by reducing the number of pixels of the captured image, and cut out the image of the non-priority area from the captured image whose image quality has been adjusted. In this case, the image quality adjustment unit 15 changes the size of the non-priority area according to the decrease in the number of pixels. In other words, if the image quality adjustment unit 15 adjusts the image quality of the image of the non-priority area by reducing the number of pixels of the captured image, the order of adjusting the image quality and cutting out the non-priority area is not particularly limited.

Further, in the moving object detection apparatus described in the above embodiments, each functional block may be individually integrated into one chip by a semiconductor device such as an LSI, or may be integrated into one chip so as to include part or all of them. Also good.

Although LSI is used here, it may also be called IC, system LSI, super LSI, or ultra LSI depending on the degree of integration.

Further, the method of circuit integration is not limited to LSI, and may be implemented by a dedicated circuit or a general-purpose processor. An FPGA (Field Programmable Gate Array) that can be programmed after the LSI is manufactured, or a reconfigurable processor that can reconfigure connections and settings of circuit cells inside the LSI may be used.

Also, part or all of the processing of each functional block in each of the above embodiments may be implemented by a program. Part or all of the processing of each functional block in each of the above embodiments is performed by a central processing unit (CPU) in a computer. A program for performing each process is stored in a storage device such as a hard disk or ROM, and is read from the ROM or RAM and executed.

Further, each process of the above embodiment may be realized by hardware, or may be realized by software (including cases where it is realized together with an OS (operating system), middleware, or a predetermined library). . Furthermore, it may be realized by mixed processing of software and hardware.

For example, when the functional blocks of the above embodiments (including modifications) are implemented by software, the hardware configuration shown in FIG. Each functional unit may be realized by software processing using a hardware configuration connected by

Also, the execution order of the processing methods in the above embodiment is not necessarily limited to the description of the above embodiment, and the execution order can be changed without departing from the gist of the invention.

A computer program that causes a computer to execute the method described above and a computer-readable recording medium that records the program are included in the scope of the present invention. Examples of computer-readable recording media include flexible disks, hard disks, CD-ROMs, MOs, DVDs, DVD-ROMs, DVD-RAMs, large-capacity DVDs, next-generation DVDs, and semiconductor memories. .

The computer program is not limited to being recorded on the recording medium, and may be transmitted via an electric communication line, a wireless or wired communication line, a network represented by the Internet, or the like.

Although the embodiments of the present invention have been described above, the above-described embodiments are merely examples for carrying out the present invention. Therefore, the present invention is not limited to the above-described embodiment, and can be implemented by appropriately modifying the above-described embodiment without departing from the spirit of the present invention.

100 Vehicles 10, 10A Moving object detection devices 11, 11A Image acquisition units 12, 12A Movement status acquisition unit 13 First detection units 14, 14A, 14B Priority region setting unit 15 Image quality adjustment unit 16 Second detection unit 17 Detection result output unit 31,32,32,34 camera

Claims (10)

an image acquisition unit that acquires a first captured image generated by a first camera mounted on a vehicle and acquires a second captured image generated by a second camera mounted on the vehicle;
a first detection unit that detects a first moving object moving around the vehicle from the first captured image acquired by the image acquisition unit;
a priority area and a non-priority area based on the position of the first moving object detected by the first detection unit in the first captured image and the moving direction of the detected first moving object in the first captured image; a priority area setting unit that sets the second captured image acquired by the image acquisition unit;
an image quality adjustment unit that adjusts the image quality of the image of the non-priority area by reducing the number of pixels of the second captured image;
a moving object detection unit that detects a second moving object moving around the vehicle from each of the image of the priority area and the image of the non-priority area whose image quality is adjusted by the image quality adjustment unit. Device. The moving object detection device according to claim 1,
When the position of the first moving object in the first captured image is within a cooperation reference area preset in the first captured image, the priority area setting unit sets the priority area and the non-priority area. ,
The moving object detection device, wherein the image of the cooperation reference area in the first captured image includes an image of an overlapping range in which the capturing range of the second camera and the capturing range of the first camera overlap. The moving object detection device according to claim 2,
The priority area setting unit determines, based on a moving direction of the first moving object in the first photographed image, that the first moving object is moving faster than the vehicle and that the first moving object and the vehicle move in the same direction. determine whether it is progressing to
The priority area setting unit determines that the first moving object is faster than the vehicle and that the first moving object and the vehicle are traveling in the same direction, the priority area and the non-priority area. to the second captured image. The moving object detection device according to claim 2 or 3,
When the position of the first moving object in the first captured image is within a cooperation reference region preset in the first captured image, the priority region setting unit acquires the priority region and the non-priority region. set it to the first shot image that was captured,
The image quality adjustment unit adjusts the image quality of the image of the non-priority area set in the first captured image by reducing the number of pixels of the first captured image,
The first detection unit detects a moving object from the image of the priority area set in the first captured image, and moves from the image of the non-priority area set in the first captured image and whose image quality is adjusted. A moving object detection device that detects objects. The moving object detection device according to any one of claims 1 to 4, further comprising:
a movement status acquisition unit that acquires the movement status of the vehicle;
The moving object detection device, wherein the first detection unit starts detecting the first moving object when the movement status acquired by the movement status acquisition unit satisfies a predetermined condition. The moving object detection device according to claim 5,
The movement status acquisition unit
an entry determination unit that determines whether the vehicle enters an intersection;
a turn determination unit that determines whether the vehicle turns,
The priority area setting unit sets a priority area and a non-priority area when the entry determination unit determines that the vehicle enters an intersection and the turn determination unit determines that the vehicle turns. detection device. The moving object detection device according to claim 6,
The entry determination unit detects a pedestrian crossing from a captured image captured by a front camera that captures an area in front of the vehicle,
The turning determination unit determines whether or not a turning direction of the vehicle is indicated by a direction indicator switch mounted on the vehicle,
The moving object detection device, wherein the priority area setting unit sets a priority area and a non-priority area when the pedestrian crossing is detected by the entry determination unit and the turning direction is indicated by the direction switch. The moving object detection device according to any one of claims 1 to 4,
The vehicle is equipped with a plurality of cameras,
The image acquisition unit is
an image selection unit that selects each of the plurality of captured images generated by the plurality of cameras according to a predetermined order;
an image output unit that outputs the captured image selected by the image selection unit as the first captured image to the first detection unit;
When the first detection unit detects the first moving object, the priority area setting unit detects the captured image in which the first moving object is detected, the position of the first moving object in the first captured image, and the first moving object. determining the second captured image based on the movement direction of the first moving object in the first captured image, and instructing the image selection unit to output the determined second captured image to the image quality adjustment unit. detection device. The moving object detection device according to claim 8, further comprising:
a movement status acquisition unit that acquires the movement status of the vehicle;
The moving object detection device, wherein when the movement status acquired by the movement status acquisition section indicates that the vehicle is traveling in a parking lot, the image selection section starts selecting the plurality of captured images. obtaining a first captured image generated by a first camera mounted on a vehicle and obtaining a second captured image generated by a second camera mounted on the vehicle;
detecting a first moving object moving around the vehicle from the acquired first captured image;
The priority area and the non-priority area are obtained based on the detected position of the first moving object in the first captured image and the moving direction of the detected first moving object in the first captured image. a step of setting a second captured image;
adjusting the image quality of the image of the non-priority area by reducing the number of pixels of the second captured image;
and detecting a second moving object moving around the vehicle from each of the image of the priority area and the image of the non-priority area whose image quality has been adjusted.

Images