JP2012109789A - Detection device and detection method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide technique for easily detecting the position deviation of a camera, based on only vehicle exterior images captured by the camera.SOLUTION: A detection device easily determines the stop of a vehicle, based on only vehicle exterior images since the stop of another vehicle is determined based on the images of the other vehicle, which are included in respective multiple vehicle exterior images captured by a camera and temporally continuing. When the stop of the other vehicle is determined, the detection device easily detects position deviation in the optical axis of the camera, based on only the vehicle exterior images, since the position deviation is detected based on a comparison result between the derived stop position of the image of the other vehicle in the vehicle exterior images and a reference stop position.

Description

  The present invention relates to a technique for detecting a positional shift of an optical axis of a camera mounted on a vehicle.

  2. Description of the Related Art In recent years, an image display device has been developed that displays an external image taken by a camera mounted on a vehicle on a display mounted on the vehicle. According to such an image display device, the user can drive the vehicle with reference to the outside image displayed on the display.

  In order for the image display apparatus to properly perform its function, the optical axis of the camera mounted on the vehicle must be set at a predetermined angle.

  Therefore, when the optical axis of the camera set at a predetermined angle has shifted with the passage of time, the image display device detects the shift and informs the user, and schedules the optical axis of the camera to the user. It is necessary to reset the angle.

  The following is known as an image processing apparatus for detecting the positional deviation. That is, there is known an image processing apparatus that detects a positional shift of the optical axis of a camera when a part of a vehicle shown in a reference image outside the vehicle is shifted from an initial position. For this reason, the image processing apparatus can easily detect the positional deviation based only on the outside image. Such a technique is described in Patent Document 1, for example.

  Note that such an image display device can be said to be a detection device because it has a function of detecting a positional shift of the optical axis of the camera. Accordingly, the following description will be given as a detection device.

JP 2004-173037 A

  However, since such a detection apparatus detects the positional deviation based on a part of the vehicle shown in the vehicle outside image, it is necessary to mount a camera on the vehicle so that a part of the vehicle appears in the vehicle outside image. That is, there is a problem that the mounting position of the camera with respect to the vehicle is restricted.

  The present invention has been made in view of the above problems, and provides a technique capable of easily detecting a positional shift of a camera based on only an image outside the vehicle captured by the camera without limiting the mounting position of the camera with respect to the vehicle. The purpose is to do.

  In order to solve the above-mentioned problem, the invention of claim 1 is a detection device for detecting a positional deviation of an optical axis of a camera mounted either on the front side or the rear side of a vehicle, wherein the camera captures the time. Based on an image of another vehicle included in each of a plurality of consecutive outside images, a determination unit that determines whether the other vehicle is stopped, and when it is determined that the other vehicle is stopped, Derivation means for deriving a stop position of the image of the other vehicle, and detection means for detecting the positional deviation based on a result of comparing the derived stop position with a reference stop position. And

  According to a second aspect of the present invention, in the detection apparatus according to the first aspect, the determination unit is configured to detect the other vehicle based on a change in the image of the other vehicle included in each of the plurality of outside-vehicle images. It is characterized by determining stop.

  According to a third aspect of the present invention, in the detection apparatus according to the second aspect, the determination unit determines the change based on a feature point of an image of the other vehicle.

  According to a fourth aspect of the present invention, in the detection device according to the second or third aspect, the determination unit determines that the other vehicle is stopped when the change is lost, and the derivation unit includes: The position of the image of the other vehicle when the change disappears is derived as the stop position.

  According to a fifth aspect of the present invention, in the detection device according to the second or third aspect, the determination unit determines that the vehicle immediately before the change is a stop of the other vehicle, and the derivation unit. Is characterized in that the position of the image of the other vehicle immediately before the start of the change is derived as the stop position.

  The invention according to claim 6 is the detection device according to any one of claims 1 to 5, further comprising validation means for validating the derivation means when the vehicle is stopped. To do.

  The invention according to claim 7 is the detection device according to any one of claims 1 to 6, wherein the reference stop position is from when the camera is mounted at an appropriate position of the vehicle until a predetermined period elapses. Further, it is set based on statistical data of the stop position derived by the deriving means a plurality of times.

  The invention according to claim 8 is the detection apparatus according to any one of claims 1 to 7, wherein the detection means is the stop position derived by the derivation means a plurality of times within a predetermined period in the past from the present. A value based on the statistical data is used for comparison with the reference stop position.

  The invention according to claim 9 is a detection method for detecting a positional deviation of an optical axis of a camera mounted either on the front side or the rear side of a vehicle, and (a) continuous in time taken by the camera. A step of determining stopping of the other vehicle based on an image of another vehicle included in each of the plurality of images outside the vehicle; and (b) when the stop of the other vehicle is determined, Deriving a stop position of the image of the other vehicle, and (c) detecting the positional deviation based on a result of comparing the derived stop position with a reference stop position. Features.

  According to the first to ninth aspects of the present invention, the stop of the other vehicle is determined based on the image of the other vehicle included in each of a plurality of temporally continuous images taken by the camera. Can be easily determined from only the outside image.

  Further, when it is determined that the other vehicle is stopped, the positional deviation of the optical axis of the camera is detected based on the result of comparing the stop position of the image of the other vehicle in the derived outside image and the reference stop position. The positional deviation can be easily detected based only on the outside image.

  According to the second aspect of the present invention, since the stop of the other vehicle is determined based on the change in the image of the other vehicle included in each of the plurality of outside images taken by the camera, the stop is performed only for the outside image. It can be easily determined based on the above.

  According to the invention of claim 3, since the change in the image of the other vehicle is determined based on the feature point of the other vehicle, the processing load to be determined can be reduced.

  According to the invention of claim 4, when there is no change in the image of the other vehicle, the stop of the other vehicle is determined, and the image of the other vehicle in the outside image when the change is lost. Since the position is set as the stop position of the image of another vehicle, the stop position can be derived with higher accuracy.

  According to the invention of claim 5, it is determined that the stop of the other vehicle is immediately before the change of the image of the other vehicle starts, and the position of the image of the other vehicle in the outside image immediately before the start of the change is determined. Since it is set as the stop position of the image of another vehicle, the stop position can be derived with higher accuracy.

  According to the sixth aspect of the present invention, since the stop position of the image of the other vehicle is derived when the vehicle is stopped, the stop position of the image of the other vehicle can be derived with higher accuracy.

  According to the invention of claim 7, the reference stop position is based on statistical data of the stop position derived a plurality of times from when the camera is mounted at an appropriate position of the vehicle until a predetermined period elapses. Therefore, the positional deviation can be detected with higher accuracy.

  Further, according to the invention of claim 8, the detection means uses a value based on statistical data of the stop position derived a plurality of times within a predetermined period in the past from the present for comparison with the reference stop position. The displacement can be detected with high accuracy.

FIG. 1 is a diagram illustrating a system block of the detection apparatus. FIG. 2 is a diagram illustrating the vehicle. FIG. 3 is a diagram illustrating the vehicle. FIG. 4 is a diagram illustrating a vehicle and an outside image. FIG. 5 is a diagram for explaining a composite image. FIG. 6 is a diagram for explaining extraction of an image of another vehicle from the outside image. FIG. 7 is a diagram for explaining extraction of an image of another vehicle from the outside image. FIG. 8 is a diagram for explaining a method for determining stop of an image of another vehicle. FIG. 9 is a diagram showing a comparison between a stop position of an image of another vehicle and a reference stop position. FIG. 10 is a diagram illustrating a control flow executed by the detection apparatus. FIG. 11 is a diagram illustrating a control flow executed by the detection apparatus. FIG. 12 is a diagram for explaining a method for determining stop of an image of another vehicle. FIG. 13 is a diagram for explaining extraction of an image of another vehicle from the outside image. FIG. 14 is a diagram showing a comparison between a stop position of an image of another vehicle and a reference stop position.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

<First Embodiment>
FIG. 1 is a system block diagram of the detection apparatus according to the first embodiment. The detection device 2 executes a process of detecting a positional deviation of the optical axis of the front camera 3 mounted in front of the vehicle 1. And the detection apparatus 2 performs the process which notifies that, when the deviation is detected.

  In addition, the detection apparatus 2 performs the process which produces | generates a synthesized image based on the vehicle exterior image image | photographed with the camera mounted in the side of the vehicle 1, the front, and back. Therefore, it can be said that the detection device 2 is an image processing device.

  In order to describe these processes executed by the detection device 2, first, the configuration of the detection device 2 will be described based on FIG.

(Configuration of detection device)
The detection device 2 includes a front camera 3, a left side camera 4, a right side camera 5, a rear camera 6 (hereinafter referred to as cameras 3 to 6 when all cameras are described as a single unit), a vehicle speed sensor 7, and a shift position sensor 8. The display 9 and the sound output unit 10 are electrically connected by a cable.

  As shown in the system block diagram of FIG. 1, the detection apparatus 2 includes an image processing unit 21, a control unit 27, an input unit 20, an output unit 29, and the like.

  The image processing unit 21 is a hardware circuit such as an ASIC, for example. The image processing unit 21 includes a composite image generation unit 22, a determination unit 23, a derivation unit 24, a detection unit 25, a storage unit 26, and the like.

  The composite image generation unit 22 executes a process of generating a composite image based on the outside images taken by the cameras 3 to 6 (hereinafter, this process executed by the composite image generation unit 22 is referred to as a composite image generation process). Details of the composite image generation processing will be described later.

  The determination unit 23 executes processing for determining stop of another vehicle based on images of other vehicles included in a plurality of temporally continuous images taken by the front camera 3 (hereinafter, the determination unit 23). This process executed by is called determination process). Details of the determination process will be described later.

  The deriving unit 24 performs a process of deriving a stop position of an image of another vehicle in the outside image when it is determined that the other vehicle is stopped by the determination unit 23 executing the determination process (hereinafter, This process executed by the derivation unit 24 is referred to as a derivation process). Details of the derivation process will be described later.

  The detection unit 25 executes a process of detecting a positional deviation of the optical axis of the front camera 3 (hereinafter, this process executed by the detection unit 25 is referred to as a detection process). Details of the detection process will be described later.

  The storage unit 26 stores data such as a reference stop position used when the detection unit 25 executes the detection process.

  The control unit 27 includes, for example, a microcomputer. The microcomputer includes, for example, a CPU, a ROM, a RAM, and the like. The CPU realizes various functions by performing arithmetic processing according to various programs stored in the ROM. The control unit 27 includes a notification unit 28 and the like.

  When the detection unit 25 detects the positional deviation of the optical axis of the front camera 3, the notification unit 28 displays the fact on the display 9 and controls the sound output unit 10 to output a sound to the user. A process for informing that effect is executed (hereinafter, this process executed by the notification unit 28 is referred to as a notification process). Details of the notification process will be described later.

  The input unit 20 is an interface that is electrically connected to an external device, for example. The input unit 20 executes a process of causing the outside image taken by the cameras 3 to 6 to be input to the image processing unit 21 and the control unit 27.

  The output unit 29 is an interface that is electrically connected to an external device, for example. The output unit 29 executes a process of outputting and displaying an image outside the vehicle processed by the image processing unit 21 on the display 9.

  The display 9 is provided in the upper part of the center console between the driver's seat and the passenger seat of the vehicle 1.

  As shown in FIG. 2, the front camera 3 is provided in the vicinity of the license plate mounting position at the front end of the vehicle 1, and its optical axis is directed in the forward direction of the vehicle 1.

  As shown in FIG. 2, the left side camera 4 is provided on the left side mirror 30, and its optical axis is directed to the outside along the left direction with reference to the forward direction of the vehicle 1.

  As shown in FIG. 2, the right side camera 5 is provided on the right side mirror 31, and its optical axis is directed to the outside along the right direction with reference to the forward direction of the vehicle 1.

  As shown in FIG. 2, the rear camera 6 is provided in the vicinity of the license plate mounting position at the rear end of the vehicle 1, and its optical axis is directed in the direction opposite to the forward direction of the vehicle 1.

  Note that the attachment position of the front camera 3 and the rear camera 6 is preferably approximately the center in the left-right direction, but may be a position slightly shifted in the left-right direction from the center in the left-right direction.

  A fisheye lens is employed as the lens of the cameras 3 to 6, and the fisheye lens has an angle of view of 180 degrees or more. For this reason, it is possible to capture the entire periphery of the vehicle 1 by using the cameras 3 to 6.

  The vehicle speed sensor 7 outputs the speed at which the wheel rotates when the vehicle 1 moves to the detection device 2 as a vehicle speed signal. The vehicle speed sensor 7 is installed in the engine room of the vehicle 1.

  The shift position sensor 8 can be operated by a user to shift to change the transmission gear provided in the vehicle 1, detects this shift position, and outputs a signal indicating the shift position to the detection device 2. The shift position sensor 8 is set in the vicinity of a shift lever provided in the center portion of the center console of the vehicle 1.

  Next, the “composite image generation process”, “determination process”, “derivation process”, and “detection process” executed by each unit included in the image processing unit 21 and the notification unit 28 included in the control unit 27 execute. Details of the “notification process” will be described.

(Composite image generation processing)
First, the composite image generation process executed by the composite image generation unit 22 will be described.

  The composite image generation unit 22 acquires the outside-vehicle images G1 to G4 taken by the cameras 3 to 6 as shown in FIG. The composite image generation unit 22 outputs the acquired outside-vehicle images G <b> 1 to G <b> 4 to the detection device 2.

  The detection device 2 stores the input outside-vehicle images G1 to G4 in the storage unit 26.

  The composite image generation unit 22 included in the detection device 2 projects the outside-vehicle images G1 to G4 stored in the storage unit 26 onto a three-dimensional curved surface P in a virtual three-dimensional space as illustrated in FIG. The three-dimensional curved surface P has, for example, a substantially hemispherical shape (a bowl shape), and a central portion (a bottom portion of the bowl) is determined as a position where the vehicle 1 exists. A correspondence relationship is determined in advance between the position of each pixel included in the outside-vehicle images G1 to G4 and the position of each pixel of the solid curved surface P. For this reason, the value of each pixel of the three-dimensional curved surface P is determined based on this correspondence and the value of each pixel included in the outside-vehicle images G1 to G4.

  The correspondence between the positions of the pixels in the images G1 to G4 outside the vehicle and the positions of the pixels on the three-dimensional curved surface P is determined by the arrangement of the cameras 3 to 6 that are the four on-vehicle cameras in the vehicle 1 (the distance between each other, the height above the ground, the light Depending on the shaft angle. Therefore, the table data indicating this correspondence is included in the vehicle type data stored in the storage unit 26.

  Further, polygon data indicating the shape and size of the vehicle body included in the vehicle type data is used, and an image of the vehicle 1 that is a polygon model indicating the three-dimensional shape of the vehicle 1 is virtually configured. The configured image of the vehicle 1 is arranged in a substantially hemispherical center portion defined as the position of the vehicle 1 in the three-dimensional space in which the three-dimensional curved surface P is set.

  Furthermore, the virtual viewpoint AGL is set by the composite image generation unit 22 for the three-dimensional space where the three-dimensional curved surface P exists. The virtual viewpoint AGL is defined by the viewpoint position and the viewing direction, and is set at an arbitrary viewpoint position corresponding to the periphery of the vehicle 1 in this three-dimensional space toward an arbitrary viewing direction.

  Then, according to the set virtual viewpoint AGL, a necessary area on the three-dimensional curved surface P is cut out as an image. The relationship between the virtual viewpoint AGL and the necessary area on the three-dimensional curved surface P is determined in advance, and is stored in advance in the storage unit 26 as table data. On the other hand, rendering is performed with respect to a polygon model in accordance with the set virtual viewpoint AGL, and the resulting image of the two-dimensional vehicle 1 is superimposed on the clipped image. As a result, a composite image showing a state in which the vehicle 1 and the periphery of the vehicle 1 are viewed from an arbitrary virtual viewpoint AGL is generated.

  For example, when the virtual viewpoint AGL1 in which the viewpoint position is just above the center of the position of the vehicle 1 and the visual field direction is directly below is set, the vehicle 1 and the vehicle 1 so that the vehicle 1 is looked down from almost directly above the vehicle 1. A composite image G5 showing a peripheral area of the image is generated. Further, as shown in the figure, when the virtual viewpoint AGL2 in which the viewpoint position is the left rear of the position of the vehicle 1 and the visual field direction is a substantially forward direction in the vehicle 1 is set, the entire periphery from the left rear of the vehicle 1 is set. A composite image G6 showing the vehicle 1 and the area around the vehicle 1 is generated so as to overlook the vehicle.

  In the case of actually generating a composite image, it is not necessary to determine the values of all the pixels of the three-dimensional curved surface P, and only the values of the pixels in the area necessary corresponding to the set virtual viewpoint AGL are outside the vehicle. By determining based on the images G1 to G4, the processing speed can be improved.

  In the detection device 2, by using such a function of the composite image generation unit 22, a composite image viewed from an arbitrary viewpoint around the vehicle 1 is generated, and the generated composite image is output and displayed on the display 9. Can be made.

(Judgment process)
Next, determination processing executed by the determination unit 23 will be described.

  The determination process is a process of determining the stop of another vehicle based on images of the other vehicle 1 included in a plurality of temporally continuous images taken by the front camera 3.

  The determination process will be described more specifically. First, the determination unit 23 extracts images of other vehicles included in each of the plurality of outside-vehicle images. FIG. 6 is a diagram illustrating a method in which the determination unit 23 extracts another vehicle C included in one vehicle outside image among a plurality of vehicle outside images.

  For convenience of explanation, in the figure, the left-right direction of the vehicle exterior image is the X-axis direction, and the front-rear direction of the vehicle exterior image is the Y-axis direction.

  The determination unit 23 inputs the outside-vehicle image G7a photographed by the front camera 3. Then, for example, processing such as a Sobel filter is performed on the input outside-vehicle image G7a to extract the edge of the object included in the outside-vehicle image G7a.

  Next, the determination unit 23 extracts the edge of the object image included in the vicinity of the center of the left and right sides of the vehicle outside image G7b. Specifically, as indicated by a plurality of arrow lines included in the vehicle exterior image G7b shown in FIG. 6, the difference in luminance in the lateral direction of the vehicle exterior image G7b (the + direction of the X axis shown in FIG. 6) is analyzed. The determination unit 23 repeatedly executes the process of extracting the edge in the vertical direction a plurality of times.

  Next, the determination unit 23 extracts the edge of the image of the object included in the vicinity of the center of the left and right of the vehicle outside image G7c. Specifically, as indicated by a plurality of arrow lines included in the outside image G7c shown in FIG. 6, the difference in luminance in the vertical direction (the negative direction of the Y axis shown in FIG. 6) of the outside image G7c is analyzed. The determination unit 23 repeatedly executes the process of extracting the edge in the horizontal direction a plurality of times.

  Next, the point where the edge of the object image included in the vicinity of the left and right center of the vehicle outside image extracted in the vehicle outside image G7b and the edge of the object image included in the vicinity of the center of the vehicle outside image extracted in the vehicle outside image G7c overlap. Is extracted by the determination unit 23. Thereby, the determination part 23 can extract the corner point of the image of the object contained in the vicinity of the center of the left and right of the outside image G7d. It can be said that the corner point of the image of the object is a feature point of the object.

  Next, the determination unit 23 extracts an area determined by connecting the corner points of the object image included in the vicinity of the center of the left and right sides of the vehicle outside image G7d with a line.

  The reason for extracting the region based on the image of the object included in the vicinity of the left and right center of the vehicle outside image is to extract only the region based on the other vehicle C, and travels ahead of the vehicle 1 in the vehicle outside image G7d. This is because the position where the image of the other vehicle C is well reflected is near the center of the left and right sides of the outside image G7d.

  In addition, in the outside image G7d, other objects such as a building image are omitted so as to easily represent the other vehicle C.

  A method for determining a region existing in the vicinity of the left and right center of the outside-vehicle image G7d will be described more specifically with reference to FIG.

  First, as shown in SS1 of FIG. 7, the determination unit 23 extracts all corner points included in the image of the other vehicle C.

  Then, as shown in SS2 of FIG. 7, the lateral image of the outside image forming a rectangle including the most end point in the horizontal direction (the X direction + direction and − direction shown in FIG. 7) among all corner points. The determination unit 23 extracts a line L1 parallel to the direction (X axis shown in FIG. 7).

  Then, as shown at SS2 in FIG. 7, the vertical direction of the outside image that forms a rectangle including the most end point in the vertical direction (the + direction and the − direction of the Y axis shown in FIG. 7) among all the corner points. The determination unit 23 extracts a line L2 parallel to the direction (Y axis shown in FIG. 7).

  Then, as shown by SS3 in FIG. 7, the determination unit 23 extracts a rectangular area S composed of the lines L1 and L2.

  The above-described extraction process is performed by the determination unit 23 for all of the plurality of images outside the vehicle G7 to G10 that are temporally continuous and photographed by the front camera 3 shown in FIG. That is, the determination part 23 extracts the rectangular area | regions S1-S4 based on the image of the other vehicle C contained in each of the outside images G7-G10.

  In addition, other objects, such as an image of a building, are abbreviate | omitted so that it may be easy to represent the rectangular area | regions S1-S4 from the vehicle outside images G7-G10.

  And the derivation | leading-out part 24 carries out the change of the magnitude | size of a rectangular area | region based on several rectangular area | region S1-S4 extracted based on the image of the other vehicle C contained in each of several external image G7-G10. To analyze.

  As shown in FIG. 8, the vehicle 1 is analyzed based on the signal received from the vehicle speed sensor 7 when it is analyzed that the rectangular area S changes from the area S <b> 1 to the area S <b> 4 and changes. When it is determined that this image is stopped, the determination unit 23 determines that the other vehicle C is stopped.

  In addition, the case where it changes from area | region S1 to S4 and it changes and the change is lose | eliminated is a case where the user moves the vehicle 1 to the back of the other vehicle C which has stopped ahead, and stops.

  In addition, in the determination by the determination unit 23, the reason that the condition that the vehicle 1 is stopped is necessary when the vehicle 1 and another vehicle 1 in front of the vehicle 1 are traveling at a constant speed. This is because there is a possibility that the determination may be made in this case only with the image outside the vehicle.

  Since the rectangular region S extracted based on the other vehicle C included in a plurality of temporally continuous images taken by the front camera 3 in this process is simple in shape, the determination unit No. 23 can reduce the processing load for analyzing the change in the size of the rectangular region S. Moreover, the determination part 23 can improve the precision about determination of the stop of the other vehicle C based on the change of the magnitude | size of the rectangular area | region S. FIG.

  Further, the reason for determining the stop of the other vehicle C based on the change of the other vehicle C included in the vicinity of the left and right center of the outside image is that the image of the other vehicle C moves in the vicinity of the left and right center of the outside image. This is because if the stop of the other vehicle C is determined by the transition from the stopped state to the stopped state, it can be accurately estimated that the object ahead is the other vehicle C and the stop. .

(Derivation process)
Next, the derivation process executed by the derivation unit 24 will be described.

  The derivation process is a process of deriving the stop position of the image of the other vehicle C in the outside image when the stop of the other vehicle C is determined by the derivation process executed by the determination unit 23.

  The derivation process will be described more specifically. The stop position in the image outside the vehicle of the image of the other vehicle 1 included in the image outside the vehicle when the stop of the other vehicle C is determined by the determination process executed by the determination unit 23 is derived. That is, the rectangular area S shown in FIG. 8 is changed from the area S1 to S4, and the rectangular area S in which the change has disappeared, that is, the rectangular area S4 in the outside image G10 including the area S4, is changed. The derivation unit 24 derives the position.

  That is, the deriving unit 24 derives two corner points P1 and P2 below (Y-axis-direction shown in FIG. 8) the rectangular region S4 extracted based on the other vehicle C in the outside-vehicle image G10.

  The reason why the derivation unit 24 derives the two corner points P1 and P2 below the rectangular region S4 is that the two corner points P1 and P2 have substantially the same positional relationship with the road surface even if the vehicle types are different. This is because it is in the vicinity of bumpers and tires on the vehicle. More specifically, if the position of the optical axis of the front camera 3 is set appropriately, the positions where the two corner points P1, P2P1, and P2 appear in the outside image appear at substantially the same position every time. This is because, based on the positions of the two corner points, it is possible to detect the positional deviation of the optical axis of the front camera 3 detected by the detection process described later.

(Detection process)
Next, detection processing executed by the detection unit 25 will be described.

  The detection process refers to the optical axis of the front camera 3 based on the result of comparing the stop position of the image of the other vehicle C in the outside image derived by the derivation unit 24 executing the derivation process with the reference stop position. This is a process for detecting the positional deviation.

  The detection process will be described more specifically. First, the reference stop position will be specifically described. When the position of the optical axis of the front camera 3 is set to an appropriate position of the vehicle 1 so that the function of the detection device 2 can be exhibited, the image of the other vehicle C in the outside image captured by the front camera 3 is displayed. The detection unit 25 records the stop position over a predetermined period (for example, two months). And the detection part 25 sets a reference | standard stop position based on the statistical data of the recorded stop position.

  FIG. 9 shows a histogram SD which is statistical data indicating the relationship between the stop position and the number of occurrences, that is, the frequency. For example, as illustrated in FIG. 9, the detection unit 25 sets a stop position in a range of a relatively high frequency in the histogram SD as a reference stop position BSP.

  The reason why the reference stop position BSP is set based on such statistical data is that the inter-vehicle distance between the stopped vehicle 1 and the other vehicle C that stops ahead differs depending on the driving user, and the reference stop position BSP also depends on the user. This is because it is necessary to set a reference stop position BSP according to the user because they are different.

  The reference stop position BSP set in this way and the stop position of the image of the other vehicle C in the derived outside image when it is determined that the other vehicle C has stopped after a predetermined period has elapsed. The detection part 25 compares. When the comparison result that the stop position does not coincide with any of the reference stop positions BSP is greater than a predetermined number (for example, 10 times), the detection unit 25 detects the positional deviation of the optical axis of the front camera 3. To do.

  The reason why the positional deviation is detected when the comparison result that there is no coincidence is greater than the predetermined number of times is that the vehicle 1 is kept constant with the inter-vehicle distance from the other vehicle C stopping ahead even if the same user. It is because it does not necessarily stop.

  Such a process, that is, a comparison that any one of the reference stop positions BSP and the stop position of the image of the other vehicle C in the derived outside image does not match when the stop of the other vehicle C is determined. When the result is more than the predetermined number of times, since the detection process for detecting the positional deviation of the optical axis of the front camera 3 is executed, the detection unit 25 can improve the detection accuracy of the positional deviation.

(Notification process)
Next, the notification process performed by the notification unit 28 will be described. When the positional deviation of the optical axis of the front camera 3 is detected by executing the detection process, the detection unit 25 transmits a signal indicating that to the notification unit 28.

  When a signal indicating that is received, the notification unit 28 controls either the sound output unit or the display 9 to notify the user that the position of the optical axis of the front camera 3 is shifted.

  By executing the determination process, the derivation process, the detection process, and the notification process, which are described above, by each unit included in the detection apparatus 2, the detection apparatus 2 can easily detect the positional deviation based only on the outside image. Can do.

(Control flow)
Next, a control flow executed by the detection device 2 will be described.
The detection device 2 executes the first flow shown in FIG. 11 in a predetermined cycle (hereinafter referred to as the first cycle) when the user turns on the power and operates the start button for starting the function. . In this case, the detection device 2 executes the second flow shown in FIG. 12 in a cycle longer than the first cycle (hereinafter referred to as the second cycle).

  The first cycle is, for example, 10 ms, and the second cycle is, for example, 20 ms.

(First flow)
The detection device 2 executes the process of step SA1 of the first flow shown in FIG. 11 for each first period.

  The control unit 27 determines whether or not a signal indicating that the shift position is “R” has been received from the shift position sensor 8 (step SA1).

  When the signal indicating that the signal indicating that the shift position is “R” has been received from the control unit 27, the composite image generation unit 22 executes the above-described composite image generation processing. (Step SA2). Thereby, the composite image generation unit 22 generates a composite image in which the vehicle 1 is virtually viewed.

  Next, the composite image generation unit 22 executes processing for displaying the composite image on the display 9 (step SA3).

  That is, when the user reverses the vehicle 1 with the shift position set to “R”, the detection device 2 displays a composite image to the user. For this reason, the user can recognize obstacles around the vehicle 1 that are difficult to recognize when the vehicle 1 is moved backward.

  Next, the detection apparatus 2 ends the process based on the second flow shown in FIG.

  Note that when the signal indicating that the signal indicating that the shift position is “R” has not been received from the control unit 27, the detection device 2 ends the processing based on the second flow illustrated in FIG.

(Second flow)
The detection device 2 executes the process of step SB1 of the second flow shown in FIG. 12 every second period.

  First, the image processing unit 21 inputs a plurality of temporally continuous images taken by the front camera 3 via the input unit 20 (step SB1). The storage unit 26 temporarily stores a plurality of outside-vehicle images.

  Next, based on the plurality of outside images stored in the storage unit 26, the determination unit 23 determines whether the other vehicle C is stopped by executing the above-described determination process (step SB2).

  Next, the control unit 27 determines whether the vehicle 1 has stopped based on a signal received from the vehicle speed sensor (step SB3). For example, the control unit 27 determines whether or not the vehicle speed is 0 km / h based on a signal received from the vehicle speed sensor. Furthermore, the control unit 27 may determine whether or not the vehicle 1 has stopped based on a signal received from the shift position sensor 8. For example, the control unit 27 determines whether the shift position is “N” or “P” based on a signal received from the shift position sensor 8.

  When it is determined by the control unit 27 that the vehicle 1 is stopped (YES in step SB4), the above-described derivation process is executed to determine the stop position of the image of the other vehicle C in the outside image. The deriving unit 24 derives (step SB4). That is, when it is determined that the vehicle 1 is stopped, the control unit 27 enables the derivation process by the derivation unit 24.

  Next, based on the derived stop position and reference stop position, the detection unit 25 executes the detection process described above (step SB5).

  When the stop position does not coincide with the reference stop position based on the detection process executed by the detection unit (NO in step SB5), the control unit 27 counts up the counter CT (step SB6). The counter CT is set to 0 when a start button provided in the detection device 2 is operated.

  When the counter CT is greater than a predetermined number of times XT (for example, 10 times) (in the case of YES in step SB7), the notification unit 28 executes the above-described notification process, and the user indicates that the optical axis of the front camera 3 has shifted. (Step SB8).

  The reason why the counter CT is counted up when the stop position and the reference stop position do not coincide with each other and when the counter CT exceeds the predetermined number of times XT is notified to the user that the positional deviation has been detected is This is to prevent the user from being notified when the vehicle is parked with the distance between the other vehicles C being larger than usual.

  If it is not determined in step SB3 that the vehicle 1 is stopped (NO in step SB4), if the stop position matches the reference stop position in step SB5 (YES in step SB5), and step If the counter CT is not greater than the predetermined number XT in SB7 (NO in step SB7), the detection device 2 ends the process.

  By executing the processing as described above, the detection device 2 determines the stop of the other vehicle C based on the image of the other vehicle C included in each of the plurality of outside images taken by the front camera 3. Therefore, the stop can be easily determined based only on the outside image.

  Further, when the detection device 2 determines that the other vehicle C has stopped, the detection device 2 compares the stop position of the image of the other vehicle C in the derived outside vehicle image with the reference stop position, based on the result of the comparison of the front camera 3. Since the positional deviation of the optical axis is detected, the positional deviation can be easily detected based only on the outside image.

<Second Embodiment>
Next, a second embodiment will be described. In the following description, differences between the first embodiment and the second embodiment will be mainly described.

  In the first embodiment, the determination process is as follows: “A rectangular region S shown in FIG. 8 changes from the region S1 to S4 and changes, and the change is lost and the other vehicle C stops. The determination unit 23 determines that the change from the region S1 to the region S4 increases and the change disappears when the user moves the vehicle 1 to the back of the other vehicle C parked forward. This is the case where the vehicle is stopped. "

  In the determination process according to the second embodiment, the rectangular area S shown in FIG. 12 is changed from the area S1X to the area S4X, and the vehicle speed sensor 7 is analyzed. When it is determined that the vehicle 1 is stopped based on the signal received from, the determination unit 23 determines that the other vehicle C is stopped.

  In addition, the case where it changes small from area | region S1X to S4X, and the change is lose | eliminated is the case where the other vehicle C stopped ahead of the vehicle 1 which the user has stopped starts and goes away. is there.

  In such a case, the deriving unit 24 calculates a stop position in the outside image of the image of the other vehicle C included in the outside image captured by the front camera 3 immediately before the start of the change analyzed based on the plurality of outside images. To derive.

  When the determination unit 23 and the derivation unit 24 execute the above processing, the front camera 3 is started when another vehicle C stopped in front of the vehicle 1 stopped by the user starts and moves away. The deviation of the optical axis of the front camera 3 can be detected based on the image outside the vehicle taken by.

<Third Embodiment>
Next, a third embodiment will be described. In the following description, differences between the first embodiment and the third embodiment will be mainly described.

  In the determination process according to the first embodiment, it has been described that “the determination unit 23 determines whether the other vehicle C is stopped based on a plurality of temporally continuous images taken by the front camera 3”.

  In the third embodiment, the determination unit 23 determines whether the other vehicle C is stopped based on a plurality of temporally continuous images taken by the rear camera 6.

  The determination unit 23 inputs the outside-vehicle image G7aX taken by the rear camera 6. Then, for example, processing such as a Sobel filter is performed on the input outside-vehicle image G7aX to extract the edge of the object included in the outside-vehicle image G7aX.

  Next, the determination unit 23 extracts the edge of the object image included in the vicinity of the center of the left and right sides of the vehicle outside image G7bX. Specifically, as indicated by a plurality of arrow lines included in the outside image G7bX shown in FIG. 13, the difference in luminance in the lateral direction (the + direction of the X axis shown in FIG. 13) of the outside image G7bX is analyzed and The determination unit 23 repeatedly executes the process of extracting the edge in the vertical direction a plurality of times.

  Next, the determination unit 23 extracts the edge of the object image included in the vicinity of the center of the left and right sides of the vehicle outside image G7cX. Specifically, as indicated by a plurality of arrow lines included in the outside image G7cX shown in FIG. 13, the difference in luminance in the vertical direction (the negative direction of the Y axis shown in FIG. 13) of the outside image G7cX is analyzed. The determination unit 23 repeatedly executes the process of extracting the edge in the horizontal direction a plurality of times.

  Next, the point where the edge of the image of the object included in the vicinity of the left and right center of the vehicle outside image extracted from the vehicle outside image G7bX overlaps the edge of the image of the object included in the vicinity of the center of the left and right of the vehicle outside image extracted from the vehicle outside image G7cX. Is extracted by the determination unit 23. Accordingly, the determination unit 23 can extract the corner point of the image of the object included in the vicinity of the center of the left and right sides of the vehicle outside image G7dX. It can be said that the corner point of the image of the object is a feature point of the object.

  Next, the determination unit 23 extracts a region determined by connecting the corner points of the image of the object included in the vicinity of the center of the left and right sides of the outside image G7dX with a line.

  The reason why the region based on the image of the object included in the vicinity of the center of the left and right sides of the outside image is extracted is to extract only the region based on the other vehicle C, and travels behind the vehicle 1 in the outside image G7dX. This is because the position where the image of the other vehicle C is well reflected is near the center of the left and right sides of the outside image G7dX.

  In addition, in the outside image G7dX, other objects such as a building image are omitted so as to easily represent the other vehicle C.

  When the determination unit 23 executes the processing as described above, it is possible to detect the deviation of the optical axis of the rear camera 6 based only on the outside image captured by the rear camera 6.

<Fourth embodiment>
Next, a fourth embodiment will be described. In the following description, differences between the first embodiment and the fourth embodiment will be mainly described.

  Unlike the detection process in the first embodiment, the deriving unit 24 derives the detection process in the fourth embodiment based on the statistical data of the stop position derived multiple times within a predetermined period from the present to the present. The measured value is compared with the reference stop position.

  First, the stop position will be specifically described. The stop position of the image of the other vehicle C derived by the deriving unit 24 from a predetermined period (for example, two months) before the day when the detection process is performed to the day when the detection process is performed is detected by the detection unit 25. It is recorded. Then, based on the recorded statistical data of those stop positions, the deriving unit 24 derives a value (stop position) used by the detection unit 25 for comparison.

  The reason for deriving the stop position for comparison based on such statistical data is that the inter-vehicle distance between the stopped vehicle 1 and the other vehicle C ahead varies every time even the same user drives. This is because it is necessary to derive a stop position corresponding to the inter-vehicle distance that the user frequently takes.

  Specifically, as shown in FIG. 14, the detection unit 25 compares the peak DP1 of the set reference stop position histogram D1 with the peak DP2 of the histogram D2 that is statistical data of the derived stop position. As a result of comparing these peaks, when the difference between them is larger than a predetermined value, the detection unit 25 detects the positional deviation of the optical axis of the front camera 3.

  In such a process, that is, when the difference between the value of the peak DP1 of the set reference stop position histogram D1 and the value of the derived stop position histogram D2 of the peak DP2 is larger than a predetermined value, the front camera 3 Since the detection unit 25 detects the positional deviation of the optical axis, the detection unit 25 can improve the detection accuracy of the positional deviation.

<Modification>
The first embodiment of the present invention has been described above. However, the present invention is not limited to the first embodiment, and various modifications can be made. Below, the modification which is other embodiment is demonstrated. Of course, you may combine the form demonstrated below suitably.

<Modification 1>
Although the detection apparatus 2 according to the first embodiment has been described as “the lenses included in the cameras 3 to 6 are fisheye lenses”, the cameras 3 to 6 may include lenses that are narrower than the angle of view included in the fisheye lenses. . For example, the cameras 3 to 6 may be provided with standard lenses having an angle of view of 45 degrees, and the cameras 3 to 6 may be provided with wide angle lenses having an angle of view larger than 45 degrees and narrower than 180 degrees.

<Modification 2>
In the first embodiment, “the reference stop position is an image outside the vehicle derived by the deriving unit 24 a plurality of times from when the front camera 3 is mounted at an appropriate position of the vehicle 1 until a predetermined period elapses. It is set based on the statistical data of the stop position in the outside image of the image of the other vehicle included in the above ", but the statistical data of the stop position may be averaged data.

DESCRIPTION OF SYMBOLS 2 Detection apparatus 21 Image processing part 23 Judgment part 24 Derivation part 25 Detection part 27 Control part 28 Notification part 3 Front camera 4 Left side camera 5 Right side camera 6 Rear camera

Claims (9)

A detection device that detects a positional shift of an optical axis of a camera mounted on either the front or rear of a vehicle,
A determination means for determining stopping of the other vehicle based on an image of the other vehicle included in each of a plurality of images outside the vehicle taken in time taken by the camera;
Deriving means for deriving a stop position of an image of the other vehicle in the outside image when it is determined that the other vehicle is stopped;
Detecting means for detecting the positional deviation based on a result of comparing the derived stop position and a reference stop position;
A detection device comprising: The detection device according to claim 1,
The determination device determines whether the other vehicle is stopped based on a change in an image of the other vehicle included in each of the plurality of images outside the vehicle. The detection device according to claim 2,
The determination unit determines the change based on a feature point of an image of the other vehicle. In the detection apparatus in any one of Claim 2 or 3,
The determination means determines the stop of the other vehicle when the change is lost,
The deriving means derives the position of the image of the other vehicle when the change is lost as the stop position. In the detection apparatus in any one of Claim 2 or 3,
The determining means determines that the other vehicle is stopped immediately before the start of the change,
The deriving means derives the position of the image of the other vehicle immediately before the change starts as the stop position. In the detection apparatus in any one of Claims 1 thru | or 5,
The detection apparatus further comprising an enabling means for enabling the derivation means when the vehicle is stopped. The detection device according to any one of claims 1 to 6,
The reference stop position is set based on statistical data of the stop position derived by the deriving means a plurality of times from when the camera is mounted at an appropriate position of the vehicle until a predetermined period elapses. A detection device characterized by that. The detection device according to any one of claims 1 to 7,
The detection unit uses a value based on statistical data of the stop position derived by the deriving unit a plurality of times within a predetermined period in the past from the present for comparison with the reference stop position. . A detection method for detecting a positional shift of an optical axis of a camera mounted on either the front or rear of a vehicle,
(A) determining a stop of the other vehicle based on an image of the other vehicle included in each of a plurality of temporally continuous images taken by the camera;
(B) deriving a stop position of an image of the other vehicle in the outside image when it is determined that the other vehicle is stopped;
(C) detecting the positional deviation based on a result of comparing the derived stop position and a reference stop position;
A detection method comprising:

Images