JP2008148059A - Vehicle-surroundings monitor apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a vehicle-surroundings monitor apparatus, which can acquire intuitively understandable images and has high convenience. <P>SOLUTION: The vehicle-surroundings monitor apparatus has cameras 11a to 11h, an operation device 21 (an operation interface), a display device 22 (a display means), an indicator 23 (a camera position display means), and a monitor ECU 30. The cameras 11a to 11h are arranged for photographing the entire vehicle 10 at a specified interval. An image synthesizing part 31 (an image synthesizing means) of the monitor ECU 30 creates virtual images of virtual cameras which are virtually arranged at intermediate positions between the cameras 11a to 11h based on images photographed by the cameras 11a to 11h, and then synthesizes the virtual images and the photographed images continuously. Based on the images synthesized by the image synthesizing part 31, a display image creation part 32 (a display image creation means) of the monitor ECU 30 creates display images corresponding to arbitrary camera positions, and allows the display device 22 to display them. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a vehicle periphery monitoring device, and more particularly to a vehicle periphery monitoring device capable of monitoring the periphery of a vehicle with a plurality of cameras provided in the vehicle circumferential direction.

As this type of vehicle periphery monitoring device, three cameras that are provided at different positions in the circumferential direction of the vehicle and photograph the periphery of the vehicle, and when the vehicle is stopped, the captured images of the cameras are linearly attached to each other. One viewpoint image viewed from one virtual viewpoint is displayed on the display means, and when the vehicle is moving, there is provided a control means for displaying a multi-viewpoint image composed of a plurality of photographed images on the display means. It is described in Patent Document 1.
JP 2005-236493 A

  However, in the vehicle periphery monitoring device described in Patent Document 1, the virtual viewpoint position is limited to one depending on the mounting positions of a plurality of cameras on the vehicle, so that the image display is limited to a narrow range. , Not enough convenience. In this case, since there are restrictions on the mounting position and mounting direction of the camera on the vehicle, it is difficult to set the image display in a wide range simply by changing the mounting position on the vehicle. In addition, since one viewpoint image is created by linearly joining the captured images of the cameras, there is also a problem that even one viewpoint image is difficult to understand intuitively.

  The present invention has been made to address the above-described problems, and an object of the present invention is to provide a vehicle periphery monitoring device that can obtain an intuitively easy-to-understand image and is highly convenient.

  In order to achieve the above object, the present invention provides a plurality of cameras that are installed at different positions in the vehicle circumferential direction and photograph the periphery of the vehicle, and an intermediate position between the plurality of cameras based on the captured images of the plurality of cameras. Based on the synthesized image synthesized by the image synthesizing unit, and an image synthesizing unit that synthesizes the virtual image and the photographed image continuously. An image creating means for creating a display image corresponding to the camera position and displaying it on the display means is provided.

  In this vehicle periphery monitoring device, a virtual image of a virtual camera virtually set at an intermediate position between the plurality of cameras is created based on the images captured by the plurality of cameras, and the virtual image and the photographed image are continuously displayed. Is synthesized. Further, a display image corresponding to an arbitrary camera position is created based on the synthesized composite image.

  According to this, since the virtual position of the camera is not limited to one, the display image can be continuously set in a wide range, an intuitively easy-to-understand image can be obtained, and the usability of the vehicle periphery monitoring device can be improved. It becomes good.

  In carrying out the present invention, the plurality of cameras are installed at intervals over the entire circumference of the vehicle so as to form a region that overlaps images captured by adjacent cameras. It is also possible to create an interpolated image as the virtual image by converting images captured by the plurality of cameras. In this case, as the image processing method, for example, a light space method or a morphing method may be used.

  According to this, even if the mounting heights of the plurality of cameras on the vehicle are different from each other, a continuous 360 ° composite image (panoramic image) can be obtained over the entire circumference of the vehicle. It is possible to increase the accuracy, and it is possible to simply configure the above-described vehicle usability monitoring device with good usability.

  In carrying out the present invention, the image creating means can continuously display the display image on the display means in accordance with the setting change of the arbitrary camera position. According to this, it is possible to easily reach the image of the part to be viewed while referring to the images continuously displayed on the display means.

  In carrying out the present invention, the image creating means creates a display image by image conversion of the composite image in accordance with a setting change of the camera angle of view at the arbitrary camera position, and displays the display image on the display means. Is also possible. Further, the image creation means can create a display image by image conversion of the composite image and display the display means on the display means in accordance with a change in setting of the camera line-of-sight direction at the arbitrary camera position. The image creating means may create a display image by image conversion of the composite image in accordance with a setting change of a camera viewpoint (camera lens position) at the arbitrary camera position and display the display image on the display means. Is possible. In this case, for example, the image creation means may display the display image continuously on the display means in accordance with the setting change of the camera angle of view, the camera viewing direction, or the camera viewpoint. According to this, it is possible to examine in more detail the image of the part desired to be viewed while referring to images continuously displayed on the display means.

  In carrying out the present invention, the image creating means displays a display image at a pre-designated camera position on the display means in preference to a display image based on the composite image according to the running state of the vehicle. It is also possible to make it. In this case, the traveling state of the vehicle may be, for example, before departure of the vehicle, at the time of back, or at the time of width adjustment. According to this, it becomes possible to call attention of a passenger | crew effectively.

  In carrying out the present invention, the image creating means may cause the display means to display the display image at the camera position including the obstacle prior to the display image based on the composite image when the obstacle is detected. Is possible. This also makes it possible to alert the passengers effectively.

  In implementing the present invention, the camera position display means for displaying the camera position corresponding to the display image, the arbitrary camera position, the camera angle of view, the camera viewing direction, and the camera viewpoint are changed. It is also possible to provide an operation interface. In this case, the camera position display means may be displayed on a schematic plan view of the vehicle, for example. According to this, it is possible to clearly grasp the camera position by referring to the camera position on the schematic plan view of the vehicle, and the camera position, the camera angle of view, the camera line-of-sight direction, and the camera by operating the operation interface. The viewpoint setting can be easily changed.

  Further, in the implementation of the present invention, in conjunction with the operation of the operation interface, the image creating means corresponds to the setting change of the arbitrary camera position, the camera angle of view, the camera viewing direction, and the camera viewpoint. It is also possible to display a display image on the display means. According to this, it is possible to improve the operability when searching for an image to be viewed.

  In this case, the camera position display means may be set so that the display position of the determined camera is rotated in a certain direction as the arbitrary camera position is determined by the operation interface. Is possible. According to this, it becomes possible to make the determined camera position and the direction of the display image coincide with each other, and it is possible to eliminate the uncomfortable feeling when operating the operation interface.

a. First Embodiment Hereinafter, a first embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a functional block diagram schematically illustrating the entire vehicle periphery monitoring apparatus according to the first embodiment. This vehicle periphery monitoring apparatus includes cameras 11a to 11h, an operating device 21, a display 22, an indicator 23, and A monitoring electronic control unit 30 (hereinafter simply referred to as a monitoring ECU 30) is provided.

  The cameras 11a to 11h (photographing means) are, for example, well-known CCD cameras, and are installed in the vehicle 10 at intervals over the entire circumference of the vehicle. Specifically, the cameras 11a to 11h are located between adjacent cameras at the vehicle front, vehicle right front, vehicle right side, vehicle right rear, vehicle rear, vehicle left rear, vehicle left side, and vehicle left front positions, respectively. The camera 10 is attached to the vehicle 10 so as to form an area overlapping with a camera-captured image (indicated by a one-dot chain line in the figure). The cameras 11a to 11h are connected to the monitoring ECU 30 and output a data signal representing a captured image to the monitoring ECU 30 (display image creation unit 32).

  The operation device 21 (operation interface) is obtained by assembling a joystick-type operation lever 21b to the base portion 21a, and is provided in the vicinity of the display of the navigation device. The operation lever 21b is used when the setting of the display image that the occupant wants to see, that is, the setting of the camera position, the camera angle of view, the camera viewing direction, and the camera viewpoint, is changed by using the confirmation switch SW1 and the return switch SW2 together. . The setting change of the camera position, the camera angle of view, the camera viewing direction, and the camera viewpoint using the operation lever 21b will be described later. The operation device 21 is connected to the monitoring ECU 30 and the display / rotation circuit 24, and outputs a signal corresponding to the operation of the operation lever 21b to the monitoring ECU 30 (display image creation unit 32) and the display / rotation circuit 24.

  The display 22 (display means) is, for example, a well-known color liquid crystal display, and in this embodiment is also used as a display of a navigation device. The indicator 23 (camera position display means) is constructed by incorporating a plurality of indicator lamps 23b over the entire circumference of the schematic plan view 23a of the vehicle, and is connected to the operating device 21 via the display / rotation circuit 24. Yes. The indicator 23 is provided in the vicinity of the operation device 21 and is provided so as to be rotatable in the plane by a motor (not shown).

  The indicator lamp 23b represents an arbitrary camera position, and the indicator lamp 23b corresponding to the set camera position is lit. The display / rotation circuit 24 changes the lighting position of the indicator lamp 23b according to the operation of the operation lever 21a, and rotates the indicator 23 so that the lighting position of the indicator lamp 23b becomes the upper position by the ON operation of the confirmation switch SW1. Let

  The monitoring ECU 30 includes a microcomputer including a CPU, ROM, RAM, A / D converter, flash memory, and the like as main components, and repeatedly executes the image processing program shown in FIG. 2 every predetermined time after the ignition switch is turned on. . This image processing program shows a series of flow of image processing, and the monitoring ECU 30 causes the display 22 to display a display image created by executing this image processing program.

  Next, the operation of the first embodiment configured as described above will be described. When the occupant operates the ignition key to turn on the ignition switch, the monitoring ECU 30 repeatedly executes the image processing program of FIG. 2 every predetermined short time.

  The image processing program starts to be executed in step S10, and thereafter, the processes in steps S11 to S15 are repeatedly executed. First, the process from step S11 to step S14 will be briefly described.

  In step S11, an image capturing process, that is, the image composition unit 31 acquires captured image data from the cameras 11a to 11h. In step S12, an image composition process, that is, a composite image over the entire circumference of the vehicle 10 is created using the captured image data acquired by the image composition unit 31 in step S11. In step S13, display image creation processing, that is, the display image creation unit 32 creates an image to be displayed on the display device 22 using the composite image created in step S12 (for example, extraction and synthesis of a predetermined portion of the composite image). The image is converted (see FIG. 3). In step S14, the display 22 displays an image display process, that is, the display image created in step S13.

  Next, the processing of steps S12 and S13 will be specifically described. In the image synthesizing process in step S12, an image is synthesized by using, for example, a well-known ray space method as an image synthesizing algorithm. Here, the ray space method is one of methods for synthesizing images from arbitrary viewpoints, and FIG. 5 schematically shows the principle thereof. The cameras C1 and C2 are installed so that an area overlapping with each other's imaging area is formed (so that the target object O has a common field of view). Assuming an arbitrary plane S, rays connecting the cameras C1 and C2 and the edge of the target object O intersect the plane S at coordinates S1 and S2, respectively. On the other hand, a ray connecting the virtual camera (camera at an arbitrary viewpoint position) C3 virtually installed at an intermediate position between the cameras C1 and C2 and the edge of the target object O intersects the plane S and the coordinate S3.

  Therefore, based on the installation positions of the cameras C1 to C3 and the positional information of the coordinates S1 to S3, the captured object image of the target object O viewed from the cameras C1 and C2 is interpolated to perform the target object O viewed from the virtual camera C3. Virtual images can be created. In addition to the ray space method, for example, a known image morphing method may be used to create a virtual image (interpolated image) using the corresponding points of the captured image as a clue.

  Specifically, in step S12, as shown in FIG. 6, a virtual image of a virtual camera virtually installed at an intermediate position between the cameras 11a to 11h is created based on the captured image data of the cameras 11a to 11h. Then, the virtual image and the captured image are synthesized. Thereby, a continuous 360 ° composite image (panoramic image) is obtained over the entire circumference of the vehicle 10.

  This composite image is cut into an image at a camera position arbitrarily set according to the operation of the operation device 21 or is converted according to a change in setting of the camera angle of view, the camera viewing direction, and the camera viewpoint. Before describing the display image creation processing in S13, the operation of the operation unit 21 and the operation of the indicator 23 in conjunction with this operation will be described using the indicator display / rotation program in FIG. 4 and the indicator operation explanatory diagrams in FIGS. I will explain.

  Before the operation lever 21b of the operation device 21 is operated, the camera position on the indicator 23 is initially set, for example, in front of the vehicle, and the indicator lamp 23b positioned in front of the vehicle is turned on in the schematic plan view 23a of the vehicle. ing. When the operation lever 21b is tilted to the left C in FIG. 7A from this state, the camera position continuously moves counterclockwise in the circumferential direction of the vehicle 10 from the front of the vehicle to the front left of the vehicle. On the contrary, when the operation lever 21b is tilted to the right D in FIG. 7A, the camera position continuously moves in the clockwise direction in the circumferential direction of the vehicle 10 from the vehicle front to the vehicle right front. With the movement of the camera position, the lighting position of the indicator lamp 23b moves (step S41 in FIG. 4). FIG. 7A shows a case where the camera position is set at, for example, a substantially intermediate position between the vehicle right front position and the vehicle right side position, and FIG. 7B shows the camera position at the vehicle rear position, for example. The case where it is set to is shown.

  At each camera position shown in FIGS. 7A and 7B, the camera angle of view is set to a predetermined standard angle of view corresponding to the magnification of the composite image (state indicated by a solid line in the figure). From this state, when the operation lever 21b is tilted to the front side A in the figure, the camera angle of view at the camera position changes to the telephoto side (a state indicated by a one-dot chain line in the figure). On the contrary, when the operation lever 21b is tilted to the rear side B in the figure, the camera angle of view at the camera position changes to the wide-angle side (a state indicated by a broken line in the figure).

  Then, for example, when the confirmation switch SW1 is turned on at the camera position shown in FIG. 7B, the set camera position and camera angle of view are confirmed. Thereafter, even if the operation lever 21a is operated, the camera position and camera image are displayed. The corner settings cannot be changed. Further, as shown in FIG. 8, the indicator 23 is rotationally driven by the ON operation of the confirmation switch SW <b> 1 so that the lighting indicator lamp 23 b indicating the camera position is on the upper side (“Yes” is determined in step S <b> 42). Step S43 is executed).

  Note that when the return switch SW2 is turned on from the ON state of the confirmation switch SW1, the confirmation state of the camera position and the camera angle of view is canceled, and thereafter the camera position and the camera angle of view are set according to the operation of the operation lever 21a. Can be changed. Also, the indicator 23 is driven to return to the original position with the front of the vehicle on the upper side by the ON operation of the return switch SW2 ("No" is determined in step S42, and the process of step S44 is executed).

  When the determination switch SW1 is in the ON state, as shown in FIGS. 10A and 10B, when the operation lever 21b is tilted to the left C in FIG. The camera viewpoint (the lens position of the camera) moves downward at the same time as it continuously changes to the right rearward direction (state indicated by a broken line C in FIG. 10A). On the contrary, when the operation lever 21b is tilted to the right D in the figure, the camera line-of-sight direction continuously changes from the vehicle rearward direction to the vehicle left rearward direction, and at the same time, the camera viewpoint moves upward (FIG. 10A). The state indicated by the broken line D in FIG.

  On the other hand, when the operation lever 21b is tilted to the front side A in the figure, the camera viewpoint changes continuously from the substantially horizontal direction to the vehicle upward direction (a state indicated by the one-dot chain line of A in FIG. 10A). On the contrary, when the operation lever 21b is tilted to the rear side B in the figure, the camera viewpoint continuously changes from the substantially horizontal direction to the vehicle downward direction (a state indicated by a one-dot chain line in B in FIG. 10A). ).

  Returning to the display image creation process in step S13 in FIG. 2, the display image creation program shown in FIG. 3 is executed in this process. The display image creation program is started in step S30, and a signal corresponding to the tilt position of the operation lever 21b is input from the operation device 21 in step S31.

  In step S32, an image corresponding to the camera position and the camera angle of view corresponding to the tilted position of the operation lever 21b is created based on the composite image obtained by the process of step S12 in FIG. Specifically, when the camera position is set to the vehicle rear position shown in FIG. 9A and the camera angle of view is set to the standard by the operation of the operation lever 21b, the composite image is displayed. The part represented by GO1 is cut out at 9 (b), and the display image GO1 as shown in FIG. 9C is displayed on the display 22 (processing of steps S32 and S35).

  From this state, when the operation lever 21b is tilted to the C side and the camera position is set to the right rear position of the vehicle, the part represented by GC1 in FIG. The display image GC1 is displayed on the display 22 as shown in FIG. When the operation lever 21b is tilted to the D side and the camera position is set to the left rear position of the vehicle, a part represented by GD1 in FIG. The display image GD1 is displayed on the display 22 as shown in c). In these cases, the images from the display image GO1 to the display image GC1 or GD1 are continuously displayed on the display 22 with the change of the camera position (the processes of steps S32 and S35 are repeatedly executed).

  On the other hand, when the operation lever 21b is tilted to the A side and the camera angle of view is set to telephoto, the composite image represented by GA1 in FIG. The display image GA1 is displayed on the display 22 as shown in FIG. When the operation lever 21b is tilted to the B side and the camera angle of view is set to a wide angle, the composite image represented by GB1 in FIG. 9B is converted into an image, and FIG. The display image GB1 is displayed on the display 22 as shown in FIG. In this case, the images from the display image GO1 to the display image GA1 or GB1 are continuously displayed on the display 22 with the change of the camera angle of view (the processes of steps S32 and S35 are repeatedly executed).

  When the camera position and the camera angle of view are confirmed by turning on the confirmation switch SW1, “Yes” is determined in step S33, and the process of step S34 is executed. In step S34, an image corresponding to the camera line-of-sight direction and the camera viewpoint according to the tilt position of the operation lever 21b is created based on the composite image obtained by the process of step S12 in FIG. Specifically, as shown in FIGS. 10A and 10B, when the operation lever 21 b is tilted to the C side and the camera viewing direction is set to the right side of the vehicle, in FIG. The composite image represented by GO1 is converted into an image, and the display image GC2 is displayed on the display 22. The indicator 23 is rotationally driven by the ON operation of the confirmation switch SW1, and the indicator lamp 23b is positioned on the upper side.

  When the operation lever 21b is tilted to the D side and the camera viewing direction is set to the left side of the vehicle, the composite image represented by GO1 in FIG. It is displayed on the display 22. In this case, the images from the display image GO1 to the display image GC2 or GD2 are continuously displayed on the display 22 in accordance with the left / right change in the camera line-of-sight direction (the processes in steps S34 and S35 are repeatedly executed).

  On the other hand, when the operation lever 21b is tilted to the A side, the camera line-of-sight direction is set upward and the camera viewpoint is set downward, the composite image represented by GO1 in FIG. 10 (c) is converted into an image. The display image GA2 is displayed on the display 22. When the operation lever 21b is tilted to the B side, the camera line-of-sight direction is set downward and the camera viewpoint is set upward, the composite image represented by GO1 in FIG. The display image GB2 is displayed on the display 22. In this case, the images from the display image GO1 to the display image GA2 or GB2 are continuously displayed on the display 22 as the camera gaze direction and the camera viewpoint change up and down (processing in steps S34 and S35).

  As is clear from the above description, in the first embodiment, based on the captured images of the plurality of cameras 11a to 11h, the intermediate position between the plurality of cameras 11a to 11h is obtained by the process of step S12 in FIG. A virtual image of the virtual camera set virtually is created, and the virtual image and the captured image are continuously synthesized. Further, a display image corresponding to an arbitrary camera position is created based on the synthesized image by the process of step S13 in FIG. 2 (execution of the display image creation program in FIG. 3).

  Thereby, since the virtual position of the camera is not limited to one, the display image can be continuously set in a wide range, an intuitively easy-to-understand image can be obtained, and the usability of the vehicle periphery monitoring device is good. Become.

  Further, in the first embodiment, the plurality of cameras 11a to 11h are installed at intervals over the entire circumference of the vehicle 10 so as to form a region that overlaps the images captured by the cameras at adjacent positions. 2, the captured images of the cameras 11a to 11h are image-converted to create an interpolation image as a virtual image.

  Thereby, even if the mounting heights of the cameras 11a to 11h on the vehicle 10 are different or the camera parameters (viewing angle, resolution, etc.) are different, 360 is continuously extended over the entire circumference of the vehicle 10. A synthesized image (panoramic image) can be obtained, and the accuracy of the image can be increased, so that a user-friendly vehicle periphery monitoring device can be easily configured.

  Moreover, in the said 1st Embodiment, a display image is continuously displayed on the indicator 22 with the setting change of arbitrary camera positions by the process of step S32, S33, S35 of FIG. Thereby, it is possible to easily reach an image of a part to be viewed while referring to images continuously displayed on the display 22.

  In the first embodiment, the processing in steps S32, S33, and S35 in FIG. 3 is performed according to the setting change of the camera angle of view, the camera viewing direction, and the camera viewpoint in any camera position. A display image is created by image conversion of the composite image synthesized by the processing and is continuously displayed on the display 22. Thereby, it becomes possible to examine in detail the image of the part to be seen while referring to the images continuously displayed on the display 22.

  In the first embodiment, an indicator 23 for displaying the camera position corresponding to the display image is displayed on the schematic plan view 23a of the vehicle, and the camera position, camera angle of view, camera viewing direction and The setting of the camera viewpoint is changed. Thereby, it becomes possible to clearly grasp the camera position by referring to the camera position on the schematic plan view 23a of the vehicle, and the camera position, the camera angle of view, the camera line-of-sight direction, and the camera can be determined by operating the operation unit 21. The viewpoint setting can be easily changed.

  Further, in the first embodiment, in response to the operation of the operation unit 21, it is possible to change the setting of an arbitrary camera position, camera angle of view, camera viewing direction, and camera viewpoint by the processing in steps S32 to S35 in FIG. The displayed image is displayed on the display 22. This makes it possible to improve operability when searching for an image to be viewed.

  In the first embodiment, as the camera position is determined by the operation lever 21b of the operation device 21, the indicator 23 is set to rotate so that the determined display position of the camera faces a certain direction. ing. As a result, the determined camera position and the direction of the display image can be made to coincide with each other, and the uncomfortable feeling when operating the operation lever 21b can be eliminated.

b. Second Embodiment Next, a second embodiment of the present invention will be described. In addition to the configuration of the first embodiment, the vehicle periphery monitoring device according to the second embodiment includes an ultrasonic sonar 41, a vehicle speed sensor 42, a shift position sensor 43, and a specific display as indicated by a broken line in FIG. An image creation command unit 44 is provided. The monitoring ECU 30 of the second embodiment repeatedly executes the display image creation program of FIG. 12 every predetermined short time instead of the display image creation program of FIG. Since the other configurations of the second embodiment are the same as those of the first embodiment, the same members and portions performing the same functions as those of the first embodiment are denoted by the same reference numerals and detailed. Description is omitted.

  The ultrasonic sonar 41 detects the distance to an obstacle or a person by transmitting an ultrasonic wave, and is provided at, for example, four corners of the vehicle 10. The vehicle speed sensor 42 detects the vehicle speed. The shift position sensor 43 detects a signal for switching the shift lever to the D range or R range. The specific display image creation command unit 44 is connected to the ultrasonic sonar 41, the vehicle speed sensor 42, and the shift position sensor 43, and based on the detection signals from the sensors 41 to 43, the specific display image creation command program of FIG. Are repeatedly executed every predetermined short time. Further, an alarm sound or a specific message is output to the speaker 46 via the voice synthesis circuit 45 according to the running state of the vehicle.

  Hereinafter, the operation of the second embodiment will be described. In response to the ON operation of the ignition switch, the specific display image creation command unit 44 repeatedly executes the specific display image creation command program of FIG. 11 every predetermined short time. The execution of the specific display image creation command program is started in step 100, and in step 101, it is determined whether there is an obstacle or a person. The presence or absence of an obstacle or a person is determined to be present when the distance from the obstacle or person becomes equal to or less than a predetermined distance based on a signal from the ultrasonic sonar 41.

  As shown in FIGS. 13A and 13B, when the vehicle 10 is parked or stopped, that is, when the shift position is in the P range and there are no obstacles or people around the vehicle 10, the steps In S101, “No” is determined, and in Step S103, it is determined whether or not the vehicle speed V is 0 and the shift position is in the D range. If the shift position is neither the D range nor the R range, “No” is determined in steps S103 and S105, and the flag FLG is set to “0” in step S107. In this case, since “Yes” is determined in step S151 of the display image creation program in FIG. 12, the processing in steps S31 to S36 is repeatedly executed in the subsequent processing, as in FIG.

  When the shift position is switched to the D range from this state, “No” is determined in step S101, “Yes” is determined in S103, and the flag FLG is set to “2” in step S104. In this case, in the display image creation program of FIG. 12, “No” is determined in steps S151 and S152, “Yes” is determined in step S154, and the process of step S155 is executed. Thereby, as shown in FIG.13 (d), the image below the vehicle front by the camera 11a is displayed on the indicator 22. FIG. When the specific display image creation command unit 44 determines “Yes” in S103, as shown in FIG. 13C, the indicator 23b of the vehicle front position is turned on via the display / rotation circuit 24. A predetermined message is output to the speaker 46 via the voice synthesis circuit 45.

  On the other hand, when the shift position is switched to the R range, “No” is determined in steps S101 and S103, “Yes” is determined in S105, and the flag FLG is set to “3” in step S106. In this case, in the display image creation program of FIG. 12, “No” is determined in steps S151, S152, and S154, and the process of step S156 is executed. Thereby, although illustration is abbreviate | omitted, the image of the vehicle back downward by the camera 11e is displayed on the indicator 22. FIG. Further, when the specific display image creation command unit 44 determines “Yes” in S105, the indicator 23b at the vehicle rear position is turned on via the display / rotation circuit 24, and an alarm sound is emitted via the voice synthesis circuit 45. Sound.

  Further, when the pedestrian approaches the left rear portion of the vehicle 10 while the vehicle is stopped and traveling, for example, as shown in FIGS. In step S102, the flag FLG is set to “1”. In this case, in the display image creation program of FIG. 12, “No” is determined in step S151 and “Yes” is determined in step S152, and the process of step S153 is executed.

  As a result, as shown in FIG. 14D, the camera position close to the pedestrian is calculated, an optimal image is synthesized, and the left rear image of the vehicle is displayed on the display 22. Further, when the specific display image creation command unit 44 determines “Yes” in S101, as shown in FIG. 14C, the indicator 23b at the vehicle rear left position is turned on via the display / rotation circuit 24. An alarm sound is generated through the voice synthesis circuit 45. In the second embodiment, as shown in FIG. 14D, the specific display image is preferentially displayed on the map screen by the navigation.

  Similarly, an optimal composite image is created according to the approaching part of the pedestrian to the vehicle 10, and the image is displayed on the display 22. In this case, the indicator 23b corresponding to the approach position of the pedestrian to the vehicle 10 is turned on and an alarm sound is sounded in all cases.

  Further, as shown in FIG. 15 (a), when the vehicle 10 has shifted to the left sidewalk while the vehicle is running, “Yes” is determined in step S101, and the flag FLG is set to “1”. Set to. In this case, in the display image creation program of FIG. 12, “No” is determined in step S151 and “Yes” is determined in step S152, and the process of step S153 is executed. Thereby, as shown in FIG.15 (c), the image of the vehicle left front by the camera 11h near a sidewalk is displayed on the indicator 22. FIG. Further, when the specific display image creation command unit 44 determines “Yes” in S101, the indicator 23b at the vehicle left front position is turned on via the display / rotation circuit 24 as shown in FIG. An alarm sound is generated through the voice synthesis circuit 45.

  Similarly, when the vehicle 10 has shifted to the right sidewalk, the camera position near the sidewalk is calculated, an optimal image is synthesized, and an image on the right front side of the vehicle is displayed on the display 22. . In this case, the indicator 23b at the right front position of the vehicle is turned on and an alarm sound is emitted.

  According to the second embodiment, the processing of steps S <b> 152 to S <b> 156 of FIG. 12 is performed before the departure of the vehicle by the processing of steps S <b> 101, S <b> 103, and S <b> 105 in FIG. Since the specific images corresponding to the respective states are displayed on the display device 22 in preference to the display of the composite image according to the first embodiment, it is possible to effectively alert the passengers.

  In the said 1st Embodiment and 2nd Embodiment, although the cameras 11a-11h were provided over the perimeter of the vehicle 10, it is not restricted only to this, For example, only the site | part crossed from the front of the vehicle to the side. You may provide, and you may provide only in the site | part over the side from the back of the vehicle.

  Moreover, in the said 1st Embodiment and 2nd Embodiment, although the indicator 22 was combined with the display of a navigation apparatus, you may make it provide a dedicated indicator.

  Moreover, in the said 1st Embodiment and 2nd Embodiment, although the joystick type thing was used as the operation device 21, it may replace with this and may use a touch panel type thing, for example. In addition to or instead of these mechanical operation devices, for example, the operation device is displayed on the display device, and the camera position, the camera angle of view, the camera viewing direction, and the camera viewpoint can be changed by touching the display device. It is also possible to change the setting. Similarly, an indicator can be displayed on the display.

  In the first and second embodiments, the camera viewpoint is set downward when the camera viewing direction is set upward, and the camera viewpoint is set upward when the camera viewing direction is set downward. However, the present invention is not limited to this. For example, the operation system for the camera gaze direction and the camera viewpoint is changed, and the setting for only one of the camera gaze direction and the camera viewpoint is changed. May be.

The functional block diagram which showed roughly the whole vehicle periphery monitoring apparatus which concerns on 1st and 2nd embodiment of this invention. The flowchart which shows the image processing program which concerns on 1st Embodiment of this invention and is performed by monitoring ECU of FIG. The flowchart which shows the display image preparation routine among the image processing programs of FIG. The flowchart which shows the indicator display and rotation program which are related with 1st and 2nd embodiment of this invention, and are performed by monitoring ECU of FIG. The schematic diagram which shows the principle of the light-space method which is an example of an image synthesis algorithm. Explanatory drawing which showed the synthesized image synthesize | combined by the synthetic | combination process of the image of FIG. (A) is explanatory drawing which showed on the indicator the case where a camera position was set to the substantially middle position of a vehicle right front position and a vehicle right side position. (B) is explanatory drawing which showed on the indicator the case where a camera position was set to the vehicle rear position. Explanatory drawing which shows the case where an indicator is rotated so that a camera position may become an upper side by ON operation of a confirmation switch. (A) is explanatory drawing which showed on the indicator the case where a camera position was set to the vehicle back position. (B) The upper part is explanatory drawing which shows the display image corresponding to the setting change of a camera position. (B) The lower part is explanatory drawing which shows the display image corresponding to the setting change of a camera view angle. (C) is explanatory drawing which shows the display image corresponding to the setting change of a camera position or a camera angle of view. (A) is explanatory drawing which showed the case where a camera position was set to the vehicle back position on the indicator with the camera position on the upper side. (B) is explanatory drawing which showed the setting change of the camera gaze direction and the camera viewpoint in association with the operation direction of the operation lever. (C) is explanatory drawing which shows the display image corresponding to the setting change of a camera gaze direction and a camera viewpoint. The flowchart which shows the specific display image preparation command program which concerns on 2nd Embodiment of this invention and is performed by monitoring ECU of FIG. The flowchart which shows the display image creation routine among the image processing programs of FIG. 2 concerning 2nd Embodiment of this invention. Explanatory drawing which concerns on 2nd Embodiment of this invention, (a), (b) shows the case where the shift position of a vehicle exists in P range. (C) is explanatory drawing which shows the case where the indicator lamp of a vehicle forward position lights up at the time of vehicle departure. (D) is explanatory drawing which shows the case where a front lower image is displayed on a display device at the time of vehicle departure. Explanatory drawing which concerns on 2nd Embodiment of this invention, (a), (b) shows the case where a pedestrian approaches a vehicle. (C) is explanatory drawing which shows the case where the indicator lamp corresponding to a vehicle left back position lights in the case of (a), (b). (D) is explanatory drawing which shows the case where the image corresponding to the vehicle left back position of (c) is displayed on a display. Explanatory drawing which concerns on 2nd Embodiment of this invention, and (a) shows the case where a vehicle approaches a sidewalk. (B) is explanatory drawing which shows the case where the indicator lamp corresponding to a vehicle left front position lights. (C) is explanatory drawing which shows the case where the image corresponding to the vehicle left front position is displayed on a display.

Explanation of symbols

10 Vehicles 11a to 11h Camera 21 Operation unit (operation interface)
21b Operation lever SW1 Determination switch SW2 Return switch 22 Display (display means)
23 Indicator (camera position display means)
23a Plan view of vehicle 23b Indicator lamp 24 Display / rotation circuit 30 Monitoring ECU
31 Image composition unit (image composition means)
32 Display image creation unit (display image creation means)
41 Ultrasonic Sonar 41
42 Vehicle speed sensor 43 Shift position sensor 44 Specific display image creation command section

Claims (13)

A plurality of cameras installed at different positions in the vehicle circumferential direction to shoot the periphery of the vehicle;
An image composition means for creating a virtual image of a virtual camera virtually installed at an intermediate position between the plurality of cameras based on the photographed images of the plurality of cameras and continuously synthesizing the virtual image and the photographed images When,
A vehicle periphery monitoring apparatus, comprising: an image creating unit that creates a display image corresponding to an arbitrary camera position based on the synthesized image synthesized by the image synthesizing unit and displays the display image on the display unit.   The plurality of cameras are installed at intervals over the entire circumference of the vehicle so as to form a region that overlaps images captured by cameras at adjacent positions. The vehicle periphery monitoring apparatus according to claim 1, wherein the captured image is converted into an interpolated image as the virtual image.   The vehicle periphery monitoring device according to claim 1, wherein the image creation unit causes the display unit to continuously display the display image in accordance with the setting change of the arbitrary camera position.   The said image creation means produces the display image by the image conversion of the said synthesized image according to the setting change of the camera angle of view in the said arbitrary camera positions, and displays it on the said display means. The vehicle periphery monitoring device according to claim 1.   The said image creation means produces a display image by image conversion of the said synthesized image according to the setting change of the camera gaze direction or camera viewpoint in the said arbitrary camera positions, and makes it display on the said display means. The vehicle periphery monitoring apparatus as described in any one of them.   The vehicle periphery monitoring device according to claim 5, wherein the image creating unit continuously displays the display image on the display unit in accordance with a change in setting of the camera angle of view, the camera viewing direction, or the camera viewpoint.   2. The vehicle according to claim 1, wherein the image creating unit causes the display unit to display a display image at a pre-designated camera position in preference to a display image based on the composite image in accordance with a traveling state of the vehicle. Perimeter monitoring device.   The vehicle periphery monitoring device according to claim 7, wherein the traveling state of the vehicle is before departure of the vehicle, at the time of back, or at the time of width adjustment.   2. The vehicle periphery monitoring device according to claim 1, wherein the image creation unit displays a display image at a camera position including an obstacle on the display unit in preference to the display image based on the composite image when the obstacle is detected. . Camera position display means for displaying a camera position corresponding to the display image;
The vehicle periphery according to any one of claims 1 to 9, wherein an operation interface for setting and changing the arbitrary camera position, the camera angle of view, the camera viewing direction, and the camera viewpoint is provided. Monitoring device.   The vehicle periphery monitoring device according to claim 10, wherein the camera position display means is displayed on a schematic plan view of the vehicle.   In conjunction with the operation of the operation interface, the image creating means displays the display image corresponding to the setting change of the arbitrary camera position, the camera angle of view, the camera viewing direction, and the camera viewpoint on the display means. The vehicle periphery monitoring device according to claim 10 or 11.   The camera position display means is set so that the determined display position of the camera is rotated in a certain direction as the arbitrary camera position is determined by the operation interface. The vehicle periphery monitoring device according to any one of the above.

Images