JP2012178639A - Image processing device, parking control system, and image processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing technique capable of sufficiently exhibiting plural functions without increasing a processing load of an image processing device.SOLUTION: An image processing device causes an image with corrected distortion to be output to a display and displayed thereon, and meanwhile recognizes an image of a target based on an image with uncorrected distortion. Thus, the image processing device can prevent a processing load from being increased, and can exhibit plural functions sufficiently.

Description

  The present invention relates to an image processing technique for processing an image captured by a camera.

  2. Description of the Related Art In recent years, an image processing apparatus that outputs various images to a display after various corrections are performed on an image captured by a camera is known. For example, an image shown in an image taken by a camera may be distorted due to the characteristics of a lens provided in the camera. There has been proposed a technique for correcting an image showing a distorted image into an image showing an image without distortion (hereinafter referred to as distortion correction) (see, for example, Patent Document 1).

JP 2006-29739 A

  By the way, there is an image processing apparatus having a function of outputting an image captured by a camera to a display and a function of recognizing a target such as a human based on the image captured by the camera. In such an image processing apparatus, there is a problem that the processing load increases when an image subjected to distortion correction is used every time the respective functions are performed.

  The present invention has been made in view of the above problems, and an object of the present invention is to provide an image processing technique capable of sufficiently exhibiting each function without increasing the processing load of the image processing apparatus.

  In order to solve the above-mentioned problem, the invention of claim 1 is an image processing apparatus for processing an image captured by a camera, an acquisition means for acquiring the image captured by the camera, and an image in the acquired image Correction means for correcting image distortion, recognition means for recognizing a target image based on the image before correction, and output means for outputting the corrected image to a display for display. And

  According to a second aspect of the present invention, in the image processing apparatus according to the first aspect, the recognizing unit recognizes the image of the target with respect to a target region set at a substantially center of the image before correction. It is characterized by that.

  According to a third aspect of the present invention, in the image processing apparatus according to the first aspect, the recognition unit recognizes the target image with respect to a target region set in the vicinity of an end portion of the image before correction. The direction of recognizing the target image is changed according to the degree of distortion of the image of the target area.

  According to a fourth aspect of the present invention, in the image processing apparatus according to any one of the first to third aspects, the camera is mounted on a vehicle and photographs the periphery of the vehicle, and the target is the vehicle. It is an obstacle on the traveling route, and when the recognition means recognizes the image of the obstacle, it further comprises notification means for notifying the user to that effect.

  Further, in the image processing apparatus according to claim 4, when the obstacle is a human and the recognition unit recognizes the human image, a notification for informing the user of the fact is provided. Means.

  According to a sixth aspect of the present invention, in the image processing apparatus according to any one of the first to fifth aspects, the lens of the camera is a fisheye lens.

  Further, the invention of claim 7 is a parking control system for controlling the vehicle to be parked, an acquisition means for acquiring an image around the vehicle taken by a camera provided in the vehicle, and an image in the acquired image Correction means for correcting image distortion, recognition means for recognizing an image of a parking frame displayed on the road surface based on the image before correction, and output means for outputting and displaying the corrected image on a display And a control means for controlling the behavior of the vehicle based on the image of the parking frame recognized by the recognition means to park the vehicle in the parking frame.

  The invention of claim 8 is an image processing method for processing an image taken by a camera, wherein (a) acquiring the image taken by the camera; and (b) in the acquired image. A step of correcting image distortion; (c) a step of recognizing an image of a target based on the image before correction; and (d) a step of displaying the image after correction on a display. And

  According to the first and eighth aspects of the invention, an image in which distortion of the image in the image is corrected is displayed on the display, while an image that does not correct the distortion is used for processing for recognizing the target image. The processing load on the image processing apparatus can be prevented from increasing.

  According to the second aspect of the present invention, since the target image is recognized with respect to the target region set at the approximate center of the image with a small degree of image distortion, the recognition accuracy can be increased.

  According to the third aspect of the present invention, when a target image is recognized for a target region set in the vicinity of an edge of an image with a large degree of image distortion, the image is determined according to the degree of distortion. Since the direction in which the target image is recognized is changed, the recognition accuracy can be increased.

  According to the fourth aspect of the present invention, an image whose distortion is not corrected is used for processing for recognizing an image of an obstacle, so that the processing load of the image processing apparatus can be prevented from increasing. As a result, it is possible to notify the user of an obstacle present on the route traveled by the vehicle without delay.

  According to the fifth aspect of the present invention, since an image whose distortion is not corrected is used for processing for recognizing a human image, the processing load of the image processing apparatus can be prevented from increasing. As a result, it is possible to notify the user who is present around the vehicle without delay.

  According to the invention of claim 6, since a fisheye lens is used as the lens of the camera, a wide range of images can be acquired.

  According to the invention of claim 7, since the image before correcting the distortion is used for the control of parking the vehicle, the processing load of the parking control system can be prevented from increasing. As a result, the control can be executed without delay.

FIG. 1 is a block diagram of the parking control system according to the first embodiment. FIG. 2 is a diagram illustrating a camera mounting position in the vehicle according to the first embodiment. FIG. 3 is a diagram illustrating a shooting range of the camera mounted on the vehicle according to the first embodiment. FIG. 4 is a diagram illustrating a first control flow according to the first embodiment. FIG. 5 is a diagram illustrating an example of an image captured by each of a plurality of cameras mounted on the vehicle according to the first embodiment. FIG. 6 is a diagram illustrating an example of an image showing a distorted image in a barrel shape according to the first embodiment. FIG. 7 is a diagram for explaining a distortion correction method according to the first embodiment. FIG. 8 is a diagram for explaining a distortion correction method according to the first embodiment. FIG. 9 is a diagram for explaining a distortion correction method according to the first embodiment. FIG. 10 is a diagram for explaining a distortion correction method according to the first embodiment. FIG. 11 is a diagram for explaining a distortion correction method according to the first embodiment. FIG. 12 is a diagram illustrating a second control flow according to the first embodiment. FIG. 13 is a diagram illustrating an example of an image captured by the rear camera according to the first embodiment. FIG. 14 is a diagram illustrating that a vertical edge is detected in a region set at the approximate center of the image according to the first embodiment. FIG. 15 is a diagram illustrating that a horizontal edge is detected in a region set at a substantially center of the image according to the first embodiment. FIG. 16 is a diagram for explaining “garage parking” according to the first embodiment. FIG. 17 is a diagram illustrating a third control flow according to the first embodiment. FIG. 18 is a diagram illustrating an example of an image captured by the rear camera according to the first embodiment. FIG. 19 is a diagram for explaining a method of deriving a relative distance between the parking frame and the vehicle according to the first embodiment. FIG. 20 is a diagram illustrating a fourth control flow according to the second embodiment. FIG. 21 is a diagram illustrating an example of an image captured by the left side camera according to the second embodiment. FIG. 22 is a diagram for explaining a method for detecting a vertical edge of an image appearing in an area set in the vicinity of an end portion of an image according to the second embodiment. FIG. 23 is a diagram illustrating a detection result of a vertical edge of an image shown in an area set in the vicinity of an end portion of an image according to the second embodiment. FIG. 24 is a diagram illustrating a detection result of a horizontal edge of an image shown in an area set in the vicinity of an end portion of an image according to the second embodiment.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

<First Embodiment>
<1. Configuration>
FIG. 1 is a diagram illustrating a configuration of an image processing apparatus 2 according to the first embodiment. The image processing apparatus 2 has a function (hereinafter referred to as a display function) for displaying an image around the vehicle 1 taken by a rear camera 6 mounted on the vehicle 1 on the display 10. Further, when the image processing apparatus 2 recognizes a target on a route traveled by the vehicle 1 based on an image around the vehicle 1 captured by the rear camera 6 or the like, the image processing apparatus 2 is directed to a user (typically a driver of the vehicle 1). A function for notifying that effect (hereinafter referred to as a notification function) is provided. Further, the image processing device 2 controls the vehicle 1 based on an image around the vehicle 1 taken by the rear camera 6 or the like, and parks the vehicle 1 in a parking space in the parking frame (hereinafter referred to as a parking function). Is provided.

  The image processing apparatus 2 is provided in a parking control system SY as shown in FIG. That is, the parking control system SY includes an image processing device 2, a front camera 3, a left side camera 4, a right side camera 5, a rear camera 6 (hereinafter referred to as cameras 3 to 6), a display 10, and a parking control device 12. Etc. The image processing device 2 is electrically connected to the cameras 3 to 6, the display 10, the speaker 11, the parking control device 12, and the like through cables. The parking control device 12 is electrically connected to the image processing device 2, the engine control device 7, the steering device 8, the brake device 9, and the like through cables.

  As shown in FIG. 2, the front camera 3 is provided in the vicinity of the license plate mounting position at the front end of the vehicle 1, and its optical axis is directed in the straight traveling direction of the vehicle 1. The left side camera 4 is provided on the left side mirror, and its optical axis is directed to the outside along the left direction based on the straight traveling direction of the vehicle 1. The right side camera 5 is provided on the right side mirror, and its optical axis is directed to the outside along the right direction based on the straight traveling direction of the vehicle 1. The rear camera 6 is provided in the vicinity of the license plate mounting position at the rear end of the vehicle 1, and its optical axis is directed in the direction opposite to the straight traveling direction of the vehicle 1. Note that the attachment position of the front camera 3 and the rear camera 6 is preferably approximately the center in the left-right direction, but may be a position slightly shifted in the left-right direction from the center in the left-right direction. Each of the cameras 3 to 6 includes a fisheye lens. The angle of view of the cameras 3 to 6 equipped with the fisheye lens is 180 degrees or more. For this reason, by using the cameras 3 to 6, as shown in FIG. 3, it is possible to capture the entire periphery H <b> 1 to H <b> 4 of the vehicle 1.

  The image processing apparatus 2 includes an acquisition unit 20, an image processing unit 21, a control unit 22, and an output unit 23. The acquisition unit 20 is an interface, for example, and acquires images taken by the cameras 3 to 6. The output unit 23 is an interface, for example, and outputs an image corrected by the image processing unit 21 to the outside of the display 10 or the like. The image processing unit 21 includes a recognition unit 30, a correction unit 31, and the like. The image processing unit 21 is, for example, a hardware circuit such as an ASIC (Application Specific Integrated Circuit). The recognition unit 30 executes a process for recognizing a human image as a target based on the image before correction (hereinafter referred to as a first recognition process). Furthermore, the recognition unit 30 executes a process for recognizing an image of the target parking frame based on the image before correction (hereinafter referred to as a second recognition process). Details of the first recognition process and the second recognition process executed by the recognition unit 30 will be described later. The correction unit 31 executes processing for correcting image distortion in the image acquired via the acquisition unit 20 (hereinafter referred to as correction processing). Details of the correction processing executed by the correction unit 31 will be described later.

  The control unit 22 is a microcomputer including, for example, a CPU (Central Processing Unit), a ROM (Read Only Memory), and a RAM (Random Access Memory). The CPU realizes various functions by performing arithmetic processing according to various programs stored in the ROM. The control unit 22 includes a nonvolatile storage unit 32, a derivation unit 33, a notification unit 34, and the like. The nonvolatile storage unit 32 is, for example, an EEPROM (Electrically Erasable Programmable Read-Only Memory) or the like. The nonvolatile storage unit 32 stores correction parameters when the correction unit 31 executes correction processing. The deriving unit 33 executes a process of deriving a relative distance between the parking frame and the vehicle 1 (hereinafter referred to as a derivation process) based on the image of the parking frame displayed on the road surface that is the recognized target. Details of the derivation process executed by the derivation unit 33 will be described later. When the recognizing unit 30 recognizes the target person by executing the first recognition process, the notifying unit 34 controls the speaker 11 to notify the user of the fact.

  The parking control device 12 includes, for example, an electronic board on which electronic components such as a microcomputer are mounted. The parking control device 12 includes a parking control unit 35 and the like. The parking control unit 35 controls the speed, direction, stop position, etc. of the vehicle 1 in cooperation with the engine control device 7, the steering device 8, the brake device 9, and the like to the parking space in the parking frame. Park 1 That is, the parking control unit 35 performs control (hereinafter referred to as parking control) for parking the vehicle 1 in the parking space in the parking frame based on the relative distance between the parking frame derived by the deriving unit 33 and the vehicle 1. . Details of the parking control executed by the parking control unit 35 will be described later.

<2. Control>
Next, the control executed by the image processing apparatus 2 to exhibit the display function, the notification function, and the parking function will be described. The image processing device 2 executes first control for realizing the display function, second control for realizing the notification function, and third control for realizing the parking function at predetermined timings. Note that when the control of each function is executed at the same time, the image processing apparatus 2 executes the control of each function in parallel.

<2.1. First control>
First, the first control for realizing the display function will be described. FIG. 4 is a diagram showing a processing flow of the first control (hereinafter referred to as a first control flow). When a button for executing “parking control” is operated by the user, the image processing apparatus 2 executes the first control flow shown in FIG. 4 at a predetermined cycle (hereinafter referred to as a first cycle). Note that the first period is, for example, 25 ms. That is, the image processing apparatus 2 executes the process of step SA1 of the first control flow every first cycle. At that time, the acquisition unit 20 acquires an image of the rear periphery of the vehicle 1 taken by the rear camera 6, that is, an image Z4 shown in FIG. 5 (step SA1).

  The image Z4 captured by the rear camera 6 is acquired when the vehicle 1 is moving backward due to parking control. Further, when the vehicle 1 moves backward while turning to the left by the parking control, images taken by the left side camera 4 and the rear camera 6, that is, images Z2 and Z4 shown in FIG. 5 are acquired. Further, when the vehicle 1 moves backward while turning right by parking control, images taken by the right side camera 5 and the rear camera 6, that is, images Z3 and Z4 shown in FIG. 5 are acquired. Moreover, when the vehicle 1 stops in the parking space in a parking frame by parking control, the image which the front camera 3 image | photographed, ie, the image Z1 shown in FIG. 5, is acquired.

  Next, the correction unit 31 corrects image distortion in the image by executing correction processing (step SA2).

  While the cameras 3 to 6 having fisheye lenses having an angle of view θ of 180 degrees or more can shoot a wide range around the vehicle 1, the image shown in the captured image is distorted into a barrel shape. Originally, it is ideal that the image of the object shown in the image is similar to the actual object.

  Therefore, the image processing apparatus 2 needs to correct an image in which an image distorted in a barrel shape is reflected into an image in which such an ideal image is reflected (hereinafter, this correction is referred to as distortion correction). Various known methods can be used as the distortion correction method, and an example thereof will be described. First, an image in which the correction unit 31 pulls the four corners of an image Z2 showing a barrel-distorted image as shown in FIG. 6 in a direction opposite to the center of the image Z2 on a diagonal line, that is, as shown in FIG. Such an image Z2X is generated. As a result, the image Z2 that reflects the barrel-distorted image can be corrected to such an ideal image Z2X.

  The generation method will be specifically described. As illustrated in FIG. 6, the correction unit 31 arranges a plurality of blocks BL along an image Z <b> 2 that shows an image distorted in a barrel shape. In other words, the correction unit 31 grasps the image Z2 on which the grids along the barrel distortion as shown in FIG. 6 are superimposed. Then, as illustrated in FIG. 7, the correction unit 31 arranges a plurality of blocks BLX along the distortion in a virtual image Z2X that is distorted in reverse to the barrel distortion. In other words, the correction unit 31 grasps the image Z2X in which the grids along the distortion opposite to the barrel distortion as shown in FIG. 7 are superimposed.

  Then, the correction unit 31 stores the coordinates of the plurality of pixels included in the plurality of blocks BL arranged in the image Z2 in the virtual image Z2X according to the correction parameters stored in the nonvolatile storage unit 32. The coordinates are converted into coordinates in a plurality of arranged blocks BLX. This correction parameter is a parameter that matches the characteristics of the fisheye lens. Specifically, the correction parameter indicates a correspondence relationship between the coordinates of the plurality of pixels included in the plurality of blocks BL arranged in the image Z2 and the coordinates in the block BLX arranged in the virtual image Z2X. ing.

  Then, the correcting unit 31 transfers the plurality of pixels of the image Z2 to the virtual image Z2X based on the converted coordinates. Here, many blocks BL of the virtual image Z2X have a larger area than the blocks BL of the image Z2. For this reason. If each pixel of the image Z2 is simply transferred to the coordinate position of the virtual image Z2X, an area where no pixel is transferred exists in the block of the ideal image Z2X. Therefore, the correction unit 31 complements this area based on the color information of surrounding pixels. That is, the correction unit 31 arranges a pixel having information about an intermediate color of surrounding pixels in the area.

  When the correction unit 31 executes such correction processing, for example, as shown in FIG. 8, a distorted image Z1 photographed by the front camera 3 can be converted into an undistorted image Z1X. Further, as shown in FIG. 9, the distorted image Z2 captured by the left side camera 4 can be converted into an undistorted image Z2X. As shown in FIG. 10, the distorted image Z3 captured by the right side camera 5 can be converted into an undistorted image Z3X. Moreover, as shown in FIG. 11, the distorted image Z4 photographed by the rear camera 6 can be converted into an undistorted image Z4X. On the other hand, if such a complicated correction process is frequently executed, the processing load of the image processing apparatus 2 is increased. As a result, there is a possibility that each function may not be sufficiently exhibited.

  When such correction processing is completed, the output unit 23 outputs and displays the corrected image on the display 10 (step SA3 in FIG. 4). As a result, an image whose image distortion has been corrected is displayed on the display 10.

  When the image processing apparatus 2 executes such a first control flow, the display function can be appropriately functioned. In addition, since the first control flow is executed in the first cycle, an image of the periphery of the vehicle 1 can be displayed on the display 10 in real time at the same time as the vehicle 1 is parked. As a result, the user can confirm the state of the periphery of the vehicle 1 when the vehicle 1 is parked by an image displayed on the display 11.

<2-2. Second control>
Next, the second control for realizing the notification function will be described. FIG. 12 is a diagram showing a processing flow of the second control (hereinafter referred to as a second control flow). When the user operates a button for executing “parking control”, the image processing apparatus 2 performs the second control flow shown in FIG. 12 in a cycle longer than the first cycle (hereinafter referred to as the second cycle). Execute. Note that the second period is, for example, 33 ms. First, the image processing apparatus 2 executes the process of step SB1 of the second control flow shown in FIG. 12 every second period. At that time, the acquisition unit 20 acquires an image of the rear periphery of the vehicle 1 taken by the rear camera 6, that is, an image Z4 shown in FIG. 5 (step SB1). Note that the image Z4 captured by the rear camera 6 is acquired when the vehicle 1 is moving backward due to parking control.

  Next, the recognition unit 30 executes a first recognition process to recognize a human image that is a target included in the image (step SB2). Here, the target refers to an object that each function should recognize based on an image in order to exhibit the function.

  In order to explain the first recognition process in an easy-to-understand manner, a description will be given on the assumption that the vehicle 1 is moving backward by parking control. In this case, when the recognition unit 30 recognizes the target human image based on the image captured by the rear camera 6, the user is informed so that contact between the vehicle 1 and the human is avoided.

  When the vehicle 1 moves backward by the parking control, the vehicle 1 moves backward at a relatively low speed (for example, 3 km / h). For this reason, even if the user is informed when a person existing within a narrow range (for example, within 1 m) behind the rear portion of the vehicle 1 is recognized, the user is informed before the vehicle 1 comes into contact with the person. The vehicle 1 can be stopped. As a result, the user can avoid contact between the vehicle 1 and a human.

  That is, in order to notify the user of the danger of contact between the vehicle 1 and a human and to avoid the contact, the recognition unit 30 only recognizes the presence of a human being within 1 m behind the rear of the vehicle 1. It is enough.

  An image Z4X shown in FIG. 13 is an image taken by the rear camera 6. In the vicinity of the center of the image Z4X, a human image Hm existing within 1 m behind the rear of the vehicle 1 is shown. As shown in this image Z4X, it is empirically recognized that a person existing within 1 m from the rear of the vehicle 1 shows the image Hm of the person in the area F1 set at the approximate center of the image Z4X.

  Accordingly, when the recognition unit 30 executes the first recognition process for recognizing the human image Hm only for the region F1 set in the image Z4X, the recognition unit 30 recognizes the human image Hm for the entire image Z4X. The processing load can be reduced compared with the case of executing.

  Further, as described above, the image shown in the image Z4X photographed by the rear camera 6 provided with the fisheye lens has a characteristic of being distorted into a barrel shape. The image Z4X that reflects the barrel-shaped image increases in degree of distortion from the center to the periphery of the image Z4X. Therefore, when the first recognition process is performed based on such an image Z4X, The recognition accuracy of the human image Hm located around is relatively low.

  In other words, the image Z4X showing a barrel-distorted image becomes smaller in the degree of distortion from the periphery of the image Z4X toward the center. Therefore, the first recognition process is performed based on the image Z4X. If it performs, the recognition accuracy of the human image Hm located in the center will become comparatively high.

  Therefore, if the first recognition process is executed on the area F1 set at the approximate center of the image Z4X taken by the rear camera 6, it is possible to detect a person existing within 1 m behind the rear of the vehicle 1 with sufficient accuracy. Therefore, the first recognition process can be executed based on the image Z4X that is not subjected to the above-described correction process, that is, the image Z4X that shows an image distorted in a barrel shape. In other words, it is possible to prevent the processing load of the image processing apparatus 2 from increasing by not executing correction processing with a relatively large processing load in the second control flow.

  This first recognition process will be described more specifically. First, the recognizing unit 30 performs an edge detection process on the region F1 set at the approximate center of the image Z4X. In the edge detection process, an edge in the vertical direction of the area F1 set in the image Z4X, that is, an edge in the Y-axis direction shown in FIG. 13 is detected. Further, the edge detection process detects an edge in the horizontal direction of the region F1 set in the image Z4X, that is, an X-axis direction edge shown in FIG. FIG. 14 is a diagram illustrating that a vertical edge (hereinafter referred to as a vertical edge) is detected for the region F1 set in the image Z4X. FIG. 15 is a diagram showing that a horizontal edge (hereinafter referred to as a horizontal edge) is detected with respect to the region F1 set in the image Z4X. When the relatively short horizontal edge Ed2 and the relatively long vertical edge Ed1 exist in the region F1, or in the state where they are close to each other, the recognition unit 30 recognizes that the lower body of the human is the region F1. Recognize human beings by judging that the probability of being present is high.

  When such first recognition processing is completed, the notification unit 34 determines whether or not the target person has been recognized by the recognition unit 30 (step SB3 in FIG. 12).

  When a human is recognized, the notification unit 34 outputs a control signal for outputting a sound to that effect to the speaker 11 via the output unit 23 (step SB4).

  On the other hand, if no human is recognized in step SB3, the notification process is not executed and the process is terminated as it is.

  When the image processing apparatus 2 executes such a second control flow, the notification function can be appropriately functioned. In addition, since the human image Hm is recognized based on the image Z4X in which the correction process is not performed on the image Z4X in the second control, that is, the image Z4X that shows an image distorted in a barrel shape, the processing load on the image processing device 2 is increased. You can avoid it. As a result, when the user parks the vehicle 1, the user can know a person existing around the vehicle 1 without delay.

<2-3. Third control>
Next, 3rd control which implement | achieves a parking function is demonstrated. First, a parking control parking method executed by the parking control system SY will be described. The parking method executed by the parking control system SY is so-called “garage parking”.

  FIG. 16 is a diagram for explaining “garage parking” of the vehicle 1. In the parking lot as shown in FIG. 16, when the vehicle 1 receives an instruction from the user to start parking at the initial position P1, the parking control system SY moves the vehicle 1 from the initial position P1 to the reverse start position P2. Move forward. Next, the parking control system SY temporarily stops the vehicle 1 at the reverse start position P2. Next, the parking control system SY moves the vehicle 1 backward from the reverse start position P2 and stops at the parking space PS in the parking frame.

  FIG. 17 is a diagram showing a third control process flow (hereinafter referred to as a third control flow). When the button for executing “parking control” is operated by the user and the vehicle 1 once stops at the reverse drive start position P2, the image processing apparatus 2 performs the third control flow shown in FIG. 17 from the second cycle. Are executed in a long cycle (hereinafter referred to as a third cycle). Note that the third period is, for example, 45 ms. First, the image processing apparatus 2 executes the process of step SC1 of the third control flow every third cycle. At that time, the acquisition unit 20 acquires an image photographed by the rear camera 6 (step SC1).

  Next, the recognition unit 30 executes a second recognition process for recognizing the parking space PS in the parking frame of the parking lot based on the image captured by the rear camera 6 acquired via the acquisition unit 20 (step SC2). ).

  When the parking control system SY stops the vehicle 1 at the reverse start position P2, the recognizing unit 30 recognizes the parking frame existing behind the vehicle 1 based on the image Z4Y shown in FIG. Recognize the image L.

  Specifically, the recognizing unit 30 performs the edge detection process on the region F2 set at the approximate center of the image Z4Y. Then, when there is a high probability that the object grasped by the detected edge is the image L of the parking frame, the recognition unit 30 recognizes the presence of the image L of the parking frame. The reason for recognizing the image L of the parking frame for the area F2 is that the parking frame is set in the area F2 set at the approximate center of the image Z4Y photographed by the rear camera 6 when the vehicle 1 is positioned at the reverse start position P2. This is because it is empirically recognized that the image L is reflected.

  Accordingly, the recognition unit 30 performs the second recognition process only on the region F2 set in the image Z4Y, thereby performing the process as compared with the case where the second recognition process is performed on the entire image Z4Y. The load can be reduced.

  Further, as described above, the image shown in the image Z4Y photographed by the rear camera 6 equipped with the fisheye lens has a characteristic of being distorted into a barrel shape. The image Z4Y that reflects the barrel-shaped image has a degree of distortion that increases from the center to the periphery of the image Z4Y. Therefore, when the second recognition process is executed based on the image Z4Y, The recognition accuracy of the image L of the parking frame located is relatively low.

  In other words, the image Z4Y that shows an image distorted in a barrel shape has a smaller degree of distortion as it goes from the periphery of the image Z4Y to the center, so the second recognition process is performed based on the image Z4Y. However, the recognition accuracy of the image L of the parking frame located approximately in the center is relatively high.

  Therefore, the parking frame can be sufficiently detected by executing the second recognition process on the area F2 set at the approximate center of the image Z4Y captured by the rear camera 6 when the vehicle 1 is positioned at the reverse start position P2.

  Therefore, the second recognition process can be executed based on the image Z4Y that is not subjected to the above-described correction process, that is, the image Z4Y that shows an image distorted in a barrel shape. In other words, it is possible to prevent the processing load of the image processing apparatus 2 from being increased by not executing the correction process having a relatively large processing load in the third control flow.

  When the second recognition process is completed, the derivation unit 33 performs a derivation process for deriving the relative distance between the parking frame and the vehicle 1 based on the image L of the parking frame displayed on the recognized road surface (FIG. 17). Step SC3).

  FIG. 19 is a diagram for explaining a method for deriving the relative distance between the parking frame and the vehicle 1. The deriving unit 33 derives the relative distance between the parking frame and the vehicle 1 based on the parking frame image L recognized by the recognition unit 30. Specifically, the deriving unit 33 derives the relative distance between the parking space PS that is grasped based on the image L of the parking frame and the reverse start position P2 that is the position of the vehicle 1. Then, the deriving unit 33 derives a route R on the virtual map for guiding the vehicle 1 to the parking space PS based on the derived relative distance. The route R on the virtual map is derived so that the vehicle 1 can be guided from the reverse start position P2 to the parking space PS in the shortest time. Further, the route R on the virtual map is derived so that the vehicle 1 does not come into contact with other parked vehicles when the vehicle 1 is guided. As shown in FIG. 19, the virtual map is defined by xy coordinates when the vehicle 1 is looked down from a virtual viewpoint. In this virtual map, the reverse start position P2 of the vehicle 1 is set to the coordinate origin (x = 0, y = 0). Appropriate coordinates are also set for the route R on the virtual map.

  When the derivation process is completed, the parking control unit 35 executes parking control for parking the vehicle 1 in the parking space PS in the parking frame based on the relative distance between the parking frame derived by the derivation unit 33 and the vehicle 1 ( Step SC4 in FIG.

  The parking control unit 35 parks the vehicle 1 based on the route R on the virtual map derived by the deriving unit 33. That is, the parking control unit 35 moves the vehicle 1 to the parking space PS by controlling the behavior of the vehicle 1 so that the route R on the virtual map matches the current position of the vehicle 1 on the virtual map.

  When the image processing apparatus 2 executes such a third control flow, the parking function can be appropriately functioned. In addition, the parking frame image L is recognized based on the image Z4Y that is not subjected to the correction process in the third control, that is, the image Z4Y in which a barrel-shaped image is displayed, and the parking control is performed based on the recognized parking frame image L. Therefore, the control can be executed without delay.

  As described above, the image processing device 2 that executes the first control, the second control, and the third control outputs an image with corrected distortion to the display 10 and displays the image without correcting distortion. Since the target image is recognized based on this, the processing load on the image processing apparatus 2 can be prevented from increasing.

<3. Second Embodiment>
Next, a second embodiment will be described. In the first embodiment, when the vehicle 1 is moving backward due to parking control, the recognition unit 30 recognizes the human image Hm existing behind the vehicle 1 based on the image captured by the rear camera 6. It was. On the other hand, in the second embodiment, the vehicle 1 is located behind the left side of the vehicle 1 based on the image taken by the left side camera 4 when the vehicle 1 moves backward while turning leftward by the parking control. The recognition unit 30 recognizes an image of another vehicle.

  In the following, the configuration and processing flow of the image processing apparatus 2 of the second embodiment are substantially the same as those of the first embodiment, and therefore, description will be made focusing on differences from the first embodiment.

  FIG. 20 is a diagram illustrating a fourth control process flow (hereinafter referred to as a fourth control flow) for realizing the notification function exhibited by the image processing apparatus 2 according to the second embodiment. This fourth control flow is executed by the image processing apparatus 2 in a cycle longer than the first cycle (hereinafter referred to as the second cycle) when a button for executing “parking control” is operated by the user. . The second cycle is 33 ms, for example.

  The processes in steps SD1, SD3, and SD4 shown in FIG. 20 are the same as the processes in steps SB1, SB3, and SB4 shown in FIG. That is, the acquisition unit 20 acquires an image around the side of the vehicle 1 taken by the left side camera 4, that is, the image Z2 shown in FIG. 5 (step SD1). Next, the recognition unit 30 performs a third recognition process to recognize an image of another vehicle that is a target included in the image Z2 (step SD2).

  FIG. 21 is a diagram illustrating an example of an image Z2Y captured by the left side camera 4. In the vicinity of the left end of the image Z2Y, an image CR of another vehicle existing within 1 m behind the lateral rear portion of the vehicle 1 is shown. As shown in this image Z2Y, the other vehicles existing within 1 m from the rear side of the vehicle 1 show images of some of the four corners of the other vehicle in the region F3 set near the left end of the image Z2Y. This is often empirically recognized.

  Therefore, the recognition unit 30 can increase the accuracy of recognition of the image CR of another vehicle by executing the third recognition process on the region F3 set in the vicinity of the image Z2Y, and the processing load Can be prevented from increasing. This third recognition process will be described in detail.

  First, the recognizing unit 30 performs an edge detection process on a region F3 set near the end of the image Z2Y shown in FIG. When the image shown in the image is less distorted, the recognition unit 30 can relatively easily recognize the image CR of the other vehicle by detecting the vertical edge and the horizontal edge of the image shown in the region F3. However, the image Z2Y used by the recognizing unit 30 has a characteristic in which a projected image is distorted, and further, the degree of distortion increases from the center of the image Z2Y toward the periphery. For this reason, the distortion of the image shown in the region F3 set near the end of the image Z2Y is relatively large. Therefore, even if the vertical edge and the horizontal edge are detected for the region F3, the recognition unit 30 may not correctly detect the vertical edge and the horizontal edge of the image CR of another vehicle. As a result, the recognition unit 30 decreases the accuracy of recognizing other vehicles.

  Since there is such a possibility, the recognition unit 30 needs to detect the horizontal edge and the vertical edge of the image CR of another vehicle in consideration of the distortion of the image shown in the region F3. That is, the recognition unit 30 changes the direction in which the image of another vehicle is recognized according to the degree of distortion of the image shown in the region F3.

  FIG. 22 is a diagram illustrating a method for detecting a vertical edge of an image shown in the region F3 set near the end of the image Z2Y. In FIG. 22, an image obtained by the left side camera 4 having a high degree of image distortion (hereinafter referred to as “distorted image”) Z2Y and an image having a low degree of image distortion (hereinafter referred to as “normal image”). .) Z2a is shown for comparison. A region F3 that is an image detection target is set at the same position in both the distorted image Z2Y and the normal image Z2a.

  In the normal image Z2a, the vertical line of the image CR of the other vehicle appears in the region F3 at an angle of 90 degrees with respect to the horizontal line Hz. On the other hand, in the distorted image Z2Y, the vertical line of the image CR of another vehicle appears at an angle of 120 degrees with respect to the horizontal line Hz in the region F3. The horizontal line Hz is a line along the left-right direction (X-axis direction) of each image.

  Therefore, when the direction of the image of the normal image Z2a is the reference direction, the direction of the image of the distorted image Z2Y is inclined by 30 degrees with respect to the reference direction. The recognizing unit 30 stores in advance such an angle of inclination of the actual image direction with respect to the reference direction as an offset value Ov. Such an offset value Ov corresponds to the degree of image distortion in the region F3. The degree of image distortion increases as the distance from the position corresponding to the optical axis in the image increases. Therefore, the offset value Ov is defined according to the position of the region F3 with respect to the position corresponding to the optical axis in the image.

  The recognition unit 30 applies the offset value Ov when recognizing the image CR of another vehicle based on the distorted image Z2Y, and changes the direction in which the image of the other vehicle is recognized according to the offset value Ov. . Thereby, the recognition accuracy of the image CR of the other vehicle is increased. Specifically, when detecting the vertical edge, the recognizing unit 30 uses a vertical line that is 90 degrees with respect to the horizontal line Hz in the region F3 of the image Z2Y as a reference, and the vertical line of the reference An assumed line in which the angle of the offset value Ov is inclined is derived. Then, by detecting an edge substantially parallel to the assumed line, the vertical edge of the image shown in the region F3 can be detected with high accuracy. On the other hand, when detecting the horizontal edge, the recognizing unit 30 uses the horizontal line Hz as a reference in the region F3 of the image Z2Y, and an assumed line in which the angle of the offset value Ov is inclined with respect to the reference horizontal line. To derive. Then, by detecting an edge substantially parallel to the assumed line, the horizontal edge of the image shown in the region F3 can be detected with high accuracy.

  FIG. 23 is a diagram showing the detection result of the vertical edge of the image shown in the region F3. As shown in FIG. 23, an assumed line derived in consideration of the offset value Ov, that is, a vertical edge substantially parallel to the assumed line inclined by 30 degrees with respect to the vertical line is detected. In this case, the detected vertical edge includes a vertical edge of an image other than the image CR of another vehicle. FIG. 24 is a diagram showing the detection result of the horizontal edge of the image shown in the region F3. The recognizing unit 30 detects a horizontal edge substantially parallel to an assumed line derived in consideration of the offset value Ov, that is, an assumed line inclined by 30 degrees with respect to the horizontal line.

  And the recognition part 30 recognizes the image of another vehicle based on the detected several vertical edge and horizontal edge. That is, in the region F3, when the horizontal edge and the vertical edge exist in a state where they intersect, or when these edges exist close to each other, the recognition unit 30 recognizes an image of another vehicle.

  When such third recognition processing is completed, the notification unit 34 determines whether another target vehicle has been recognized by the recognition unit 30 (step SD3 in FIG. 20).

  When another vehicle is recognized, the notification unit 34 outputs a control signal for outputting a sound to that effect to the speaker 11 via the output unit 23 (step SD4).

  In step SD3, if no other vehicle is recognized, the notification process is not executed and the process ends.

  As described above, when the image processing apparatus 2 executes the fourth control flow, the notification function can be appropriately functioned. In the third recognition process, the recognition unit 30 changes the direction of recognizing an image of another vehicle according to an offset value indicating the degree of distortion of the image in the region F3. That is, the recognizing unit 30 detects the edge of the image shown in the region F3 in consideration of the offset value Ov, and recognizes the image CR of another vehicle based on the detected edge. Compared with the case of recognizing an image CR of another vehicle after correction, the processing load can be reduced. As a result, the user can know other vehicles existing behind the vehicle 1 without delay when the vehicle 1 is parked.

<4. Modification>
Although the embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments, and various modifications are possible. Hereinafter, such a modification will be described. All forms including those described in the above embodiment and those described below can be combined as appropriate.

  In the first embodiment, the notification function is exhibited when the image processing device 2 exhibits the parking function. On the other hand, when the user sets the shift gear to “R” and drives and parks the vehicle 1, the notification function may be exhibited.

  In the first embodiment, the target is a person and a parking frame. In the second embodiment, the target is another vehicle. On the other hand, the target may be an obstacle on the route on which the vehicle 1 travels. For example, the obstacle is a corrugated cardboard, a triangular cone, a bicycle, a tricycle, or the like existing on a route along which the vehicle 1 travels.

  In the second embodiment, when the vehicle 1 is parked in the garage and the vehicle 1 moves backward while turning leftward, another vehicle is based on the image Z2 photographed by the left side camera 4. The recognition unit 30 recognizes the image CR. On the other hand, when the vehicle 1 moves backward while turning rightward, the recognition unit 30 may recognize the image CR of another vehicle based on the image Z3 photographed by the right side camera 5.

DESCRIPTION OF SYMBOLS 1 Vehicle 2 Image processing apparatus 12 Parking control apparatus 20 Acquisition part 21 Image processing part 22 Control part 23 Output part 30 Recognition part 33 Derivation part 34 Notification part 35 Parking control part

Claims (8)

An image processing apparatus that processes an image captured by a camera,
Obtaining means for obtaining the image taken by the camera;
Correction means for correcting distortion of the image in the acquired image;
Recognition means for recognizing the image of the target based on the image before correction;
Output means for outputting and displaying the corrected image on a display;
An image processing apparatus comprising: The image processing apparatus according to claim 1.
The image recognizing apparatus, wherein the recognizing unit recognizes an image of the target with respect to a target region set at a substantially center of the image before correction. The image processing apparatus according to claim 1.
The recognition unit recognizes the target image with respect to a target region set in the vicinity of the edge of the image before correction, and recognizes the target image according to the degree of distortion of the target region image. An image recognition apparatus characterized by changing a direction to perform. The image processing apparatus according to any one of claims 1 to 3,
The camera is mounted on a vehicle and photographs the periphery of the vehicle.
The target is an obstacle on the route traveled by the vehicle,
When the recognizing unit recognizes the image of the obstacle, a notification unit that notifies the user of the fact,
An image processing apparatus further comprising: The image processing apparatus according to claim 4.
The obstacle is a human,
When the recognition unit recognizes the human image, a notification unit that notifies the user of the fact,
An image processing apparatus further comprising: The image processing apparatus according to any one of claims 1 to 5,
An image processing apparatus, wherein the lens of the camera is a fisheye lens. A parking control system for controlling a vehicle to be parked,
Obtaining means for obtaining an image of the periphery of the vehicle taken by a camera provided on the vehicle;
Correction means for correcting distortion of the image in the acquired image;
Recognizing means for recognizing an image of a parking frame displayed on the road surface based on the image before correction;
Output means for outputting and displaying the corrected image on a display;
Control means for controlling the behavior of the vehicle based on the image of the parking frame recognized by the recognition means to park the vehicle in the parking frame;
A parking control system further comprising: An image processing method for processing an image taken by a camera,
(A) acquiring the image taken by the camera;
(B) correcting distortion of the image in the acquired image;
(C) recognizing a target image based on the image before correction;
(D) displaying the corrected image on a display;
An image processing method comprising:

Images