JP2010166196A - Vehicle periphery monitoring device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a vehicle periphery monitoring device which is capable of generating a continuous vehicle periphery video of a near view and a distant view while suppressing the occurrence of double imaging at a joint between adjacent videos. <P>SOLUTION: In an around view monitor system wherein a plurality of pieces of image data acquired by front, rear, left, and right cameras 1, 2, 3, and 4 are synthesized into one image and the one synthetic image is projected on a screen of a monitor 8, an all around camera 5 for photographing a distant view around the vehicle with a wide angle of field covering angles of field of the front, rear, left, and right cameras 1, 2, 3, and 4 for photographing a near view around the vehicle in a plurality of vertically divided areas is provided besides the front, rear, left, and right cameras 1, 2, 3, and 4, and an image processing controller 7 sets an area being a dead angle in a distant view camera image photographed by the all around camera 5, as a near view camera image area and synthesizes one distant view camera image and a plurality of near view camera images to generate one synthetic image. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a vehicle periphery monitoring device that combines a plurality of image data acquired by a plurality of in-vehicle cameras into a single image and displays the generated combined image on a monitor screen.

  Conventionally, it is intended to obtain effective driving support information when parking, etc., by displaying a blind spot area for the vehicle occupant on the monitor image, and the viewpoint from the four front and rear left and right cameras There is known a vehicle periphery monitoring device called an around view monitor system that converts a virtual camera position set to a position into a viewpoint viewed downward and synthesizes a bird's-eye view image of front, rear, left, and right to generate an all-around bird's-eye view composite image (For example, refer to Patent Document 1).

JP 2007-41791 A

  However, in the conventional vehicle periphery monitoring device, four front and rear left and right cameras at different positions are used to synthesize continuous front video, rear video, left video, and right video. Due to the parallax between the cameras, there is a problem that a contradiction that a joint between adjacent videos lacks continuity easily occurs.

  This discrepancy in the joints between the images to be combined is relatively small when only the near view of the vehicle is combined, but when using four front and rear left and right cameras to combine images including a distant view, Since the shift of the distance to the object varies from camera to camera, even if correction is performed to shift the entire image, a double image appears at the joint between adjacent images.

  The present invention has been made paying attention to the above-described problem, and it is possible to generate a continuous vehicle surrounding image of a foreground and a distant view while suppressing the occurrence of a double image at a joint between adjacent images. An object is to provide a monitoring device.

In order to achieve the above object, in the present invention, a plurality of in-vehicle cameras, an image processing controller that synthesizes a plurality of image data acquired by the in-vehicle camera into one image, and the image processing controller In a vehicle periphery monitoring device comprising: a monitor that displays one composite image on a screen;
As the plurality of in-vehicle cameras, a distant view around the vehicle with a wide angle of view including the angle of view of the plurality of close-up cameras is used as a plurality of close-up cameras that photograph a close-up view around the vehicle in a plurality of vertically divided areas. Add a distant camera to shoot
The image processing controller sets, as a foreground camera image area, an area that becomes a blind spot for a distant camera image captured by the distant camera, and combines one distant camera image and a plurality of foreground camera images into one composite image. It is characterized by generating.

Therefore, in the vehicle periphery monitoring device according to the present invention, as a plurality of in-vehicle cameras, the angle of view of the plurality of foreground cameras is changed to a plurality of foreground cameras that capture a near view around the vehicle in a plurality of vertically divided areas. A far-field camera for photographing a distant view around the vehicle with a wide angle of view including Then, in the image processing controller, an area that becomes a blind spot for the far-field camera image photographed by the far-field camera is set as the near-field camera image area, and one far-field camera image and a plurality of near-field camera images are combined to form one composite image. Generated.
Accordingly, a distant view video in which a contradiction (double shot) occurring at the joint between adjacent images is a problem is a video based on one far view camera image, and thus completely continuous. In addition, the continuity between adjacent foreground images may be lacking in continuity, but since this contradiction is a foreground image, it is smaller than in the case of a distant view image.
As a result, it is possible to generate a continuous vehicle surrounding image of a foreground and a distant view while suppressing the occurrence of a double image at the joint between adjacent images.

1 is an overall system diagram illustrating an around view monitor system (an example of a vehicle periphery monitoring device) according to a first embodiment. FIG. 3 is a control block diagram illustrating an image processing control system in the around view monitor system according to the first embodiment. 6 is a flowchart illustrating an example of image deformation processing executed by an image deformation processing unit of the image processing controller 7 in the around view monitor system according to the first exemplary embodiment. 6 is a flowchart illustrating an example of a composition / superimposition display process executed by an image composition processing unit and a superimposition processing unit of the image processing controller 7 in the around view monitor system according to the first embodiment. It is a perspective view which shows the example of four camera installations, a virtual camera position, and a gaze direction in an around view monitor system. It is a figure which shows the example of a monitor display image | video by the all-around bird's-eye view which synthesize | combined the front-and-back left-right bird's-eye image image from four cameras in an around view monitor system. It is a figure which shows the example of a monitor display image | video which ideally combined the right camera image including the distant view, the rear camera image, and the left camera image in the around view monitor system. It is a figure which shows the example of a monitor display image in which the person became a double image when the right camera image including the distant view, the rear camera image, and the left camera image are actually combined in the around view monitor system. FIG. 6 is a perspective view showing a far view camera position and a blind spot area of the far view camera in the around view monitor system of the first embodiment. It is a figure which shows the example of a monitor display image | video which actually synthesize | combined the far view camera image | video, the right camera image | video, the rear camera image | video, and the left camera image | video in the around view monitor system of Example 1. FIG. It is a figure which shows the example of a monitor display image which synthesize | combined the far view camera image | video, the left camera image | video, the front camera image | video, and the right camera image | video in the around view monitor system of Example 1. FIG. It is a figure which shows the monitor display image example which synthesize | combined the far view camera image | video, the rear camera image | video, the left camera image | video, and the front camera image | video in the around view monitor system of Example 1. FIG. It is a figure which shows the monitor display image example which synthesize | combined the far view camera image | video, the front camera image | video, the right camera image | video, and the back camera image | video in the around view monitor system of Example 1. FIG. It is a figure which shows the example of a monitor display image which synthesize | combined the all-around camera image and the all-around bird's-eye view image in the around view monitor system of Example 1. FIG.

  Hereinafter, the best mode for realizing a vehicle periphery monitoring device of the present invention will be described based on Example 1 shown in the drawings.

First, the configuration will be described.
FIG. 1 is an overall system diagram illustrating an around view monitor system (an example of a vehicle periphery monitoring device) according to the first embodiment. Hereinafter, each component of the around view monitor system will be described with reference to FIG.

  As shown in FIG. 1, the around view monitor system includes a front camera 1 (vehicle camera, foreground camera), a rear camera 2 (vehicle camera, foreground camera), a left camera 3 (vehicle camera, foreground camera), A right camera 4 (on-vehicle camera, near view camera), an all-around camera 5 (on-vehicle camera, distant camera), a display image selection switch 6 (monitor image selection means), an image processing controller 7, and a monitor 8. ing.

  The four front and rear left and right cameras 1, 2, 3, and 4 are a plurality of in-vehicle cameras that capture an image of the periphery of the vehicle, and a plurality of foreground cameras that capture a close view of the entire circumference of the vehicle in four vertically divided areas. Provided as an example. The front camera 1 is installed at a bumper position or the like at the front of the vehicle with the optical axis of the lens facing obliquely downward, and uses a near view in front of the vehicle as an imaging range. The rear camera 2 is installed at a bumper position or the like at the rear of the vehicle with the lens optical axis directed obliquely downward, and takes a near view behind the vehicle as an imaging range. The left camera 3 is installed at a door mirror position or the like on the left side of the vehicle with the lens optical axis facing diagonally downward, and uses a near view on the left side of the vehicle as an imaging range. The right camera 4 is installed at a position such as a door mirror position on the right side of the vehicle with the lens optical axis directed obliquely downward, and takes a near view on the right side of the vehicle as an imaging range.

  The omni-directional camera 5 is an in-vehicle camera that captures an image of the periphery of the vehicle and has a 360 ° angle of view including the angle of view of the front, rear, left, and right cameras 1, 2, 3, and 4 that are four foreground cameras. It is provided as an example of a far-field camera for photographing. One omnidirectional camera 5 is set at the top of the vehicle V. In addition, as the all-around camera 5 that captures the entire periphery of the distant view, a camera that captures the reflected image of the convex mirror is used.

The display video selection switch 6 is connected to the image processing controller 7 and is a means for selecting a video pattern of a composite image to be displayed on the monitor screen 81 of the monitor 8 from a plurality of video patterns set as video patterns of the composite image. is there.
As video pattern options in the first embodiment, four video patterns obtained by combining one distant view image and three close-up images, an all-around bird's-eye view image pattern, and an all-around view image and an all-around distant view image composite image pattern And 6 types of video patterns.

The image processing controller 7 combines the five image data acquired by the front, rear, left and right cameras 1, 2, 3, 4 and the omnidirectional camera 5 into one image.
Then, an area that becomes a blind spot for the omnidirectional camera image taken by the omnidirectional camera 5 is set as a foreground camera image area, and one of the omnidirectional camera image and the four front, rear, left, and right cameras 1, 2, 3, and 4 is set. One composite image is generated by combining the near-field camera images from the three cameras. By this image synthesis, four video patterns obtained by combining one distant view image and three close-up view images are obtained (see FIGS. 10 to 13).
In addition, the image processing controller 7 converts the viewpoints from the four front and rear left and right cameras 1, 2, 3, and 4 into a viewpoint viewed downward from the virtual camera position set at the center upper position of the vehicle. An all-around bird's-eye view synthesized image is generated by synthesizing the bird's-eye view image. By this image synthesis, an all-around overhead video pattern is obtained (see FIG. 1).
Further, by combining this all-around bird's-eye view image and the all-around distant view image from the all-around camera 5, a composite image pattern of the all-around bird's-eye view image and the distant view image can be obtained (see FIG. 14).

  The monitor 8 displays one composite image generated by the image processing controller 7 on the screen. Here, the monitor 8 is provided at the position of the instrument panel in the passenger compartment. For example, the monitor 8 is a navigation system that guides an appropriate vehicle travel route from a departure place to a destination using a road map or a vehicle mark. It may be shared with a monitor.

  FIG. 2 is a control block diagram illustrating an image processing control system in the around view monitor system according to the first embodiment. FIG. 3 is a flowchart illustrating an example of image deformation processing executed by the image deformation processing unit of the image processing controller 7 in the around view monitor system according to the first embodiment. FIG. 4 is a flowchart illustrating an example of the composition / superimposition display process executed by the image composition processing unit and the superimposition processing unit of the image processing controller 7 in the around view monitor system according to the first embodiment. Hereinafter, an image processing control system in the around view monitor system will be described with reference to FIGS.

  As shown in FIG. 2, the image processing controller 7 includes an image data conversion unit 71, a CPU 72, a map memory 73, an image transformation processing unit 74, an image composition processing unit 75, a superposition memory 76, and a superposition process. A unit 77 and an image data conversion unit 78.

  The image data conversion unit 71 includes four cameras 1, 2, 3, 4 on the front, rear, left and right sides and five image data conversion units 71a, 71b, 71c, 71d, 71e corresponding to the all-around camera 5. And in each image data conversion part 71a, 71b, 71c, 71d, 71e, the camera image data from each image pick-up element (CCD, CMOS etc.) of each camera 1, 2, 3, 4 and an image deformation | transformation process part The coordinate data from 74a, 74b, 74c, 74d, and 74e are respectively input, and the pixel data corresponding to the coordinate position is output to each of the image transformation processing units 74a, 74b, 74c, 74d, and 74e.

  The CPU 72 inputs an operation signal from the panel touch type display image selection switch 6 of the monitor 8 and a reverse gear position selection signal from the inhibitor switch 9 that detects the selection position of the shift lever, and when the reverse gear position is selected. In order to display the composite video corresponding to the display video selection operation on the monitor 8, map data such as a foreground map and a bird's-eye view map is output to the map memory 73, and superimposition data such as a vehicle illustration is output to the superimposition memory 76.

  The map memory 73 receives the map data from the CPU 72 and the coordinate data from the image transformation processing units 74a, 74b, 74c, 74d, and 74e, and the map data corresponding to the coordinate data among the stored map data. The image is output to each of the image transformation processing units 74a, 74b, 74c, 74d, and 74e.

The image deformation processing unit 74 is composed of four cameras 1, 2, 3, 4 on the front, rear, left and right sides and five image deformation processing units 74a, 74b, 74c, 74d, 74e corresponding to the all-around camera 5. In each of the image transformation processing units 74a, 74b, 74c, 74d, and 74e, the pixel data from each of the image data conversion units 71a, 71b, 71c, 71d, and 71e is converted into a foreground image using the map data from the map memory 23. Then, image deformation processing for obtaining a distant view image or a bird's-eye view image is performed, and pixel data after the image deformation processing is output to the image composition processing unit 75.
An example of this image deformation process will be described with reference to the flowchart shown in FIG. 3. Map data is acquired (step S31), corresponding peripheral pixel data is then acquired (step S32), and pixel data interpolation processing is then performed. (Step S33) Next, the process of outputting pixel data corresponding to coordinates (Step S34) is repeated by changing the coordinate position.

  The image composition processing unit 75 inputs pixel data after image deformation processing from each of the image deformation processing units 74a, 74b, 74c, 74d, and 74e, and sets an area that becomes a blind spot for a distant camera image as a foreground camera image area. Then, one composite image is generated by combining one distant view camera image and a plurality of foreground camera images, and the pixel data of the composite image is output to the superimposition processing unit 77.

  The superimposition memory 76 receives the superimposition data from the CPU 72 and outputs the superimposition pixel data to be superimposed on the composite image to the superimposition processing unit 77. Here, the superimposition data refers to a vehicle illustration or the like by bird's-eye drawing displayed on the monitor screen 81 (see FIGS. 1 and 14).

The superimposition processing unit 77 superimposes the superimposed pixel data from the superimposition memory 76 on the pixel data of the composite image from the image synthesis processing unit 75 (superimpose), and the pixel data after the superimposition processing is converted into an image data conversion unit 78. Output to.
Here, an example of the composition / superimposition display processing by the image composition processing unit 75 and the superimposition processing unit 77 will be described with reference to the flowchart shown in FIG. 4. Each modified pixel data is acquired (step S41), and then the image composition processing is performed. Is performed (step S42), the corresponding superimposed pixel data is acquired (step S43), the pixel data is superimposed (step S44), and the coordinate-corresponding pixel data is output (step S45). It is done by processing.

  The image data conversion unit 78 receives pixel data after the superimposition processing from the superimposition processing unit 77, aggregates the pixel data, converts the pixel data into image data, and outputs the image data to the monitor 3.

Next, the operation will be described.
First, the “problem of the image composition technology including the near view and the distant view” will be described, and then the operation in the around view monitor system of the first embodiment is divided into “rear view monitor operation” and “composite video pattern selection operation”. I will explain.

[Problems of image composition technology including near view and distant view]
FIG. 5 is a perspective view illustrating an example of installation of four cameras, a virtual camera position, and a line-of-sight direction in the around view monitor system. FIG. 6 is a diagram illustrating an example of a monitor display video by a bird's-eye view of the entire periphery obtained by synthesizing front-back, left-right, and bird's-eye video images from four cameras in the around view monitor system. FIG. 7 is a diagram illustrating an example of a monitor display image obtained by ideally combining a right camera image including a distant view, a rear camera image, and a left camera image in the around view monitor system. FIG. 8 is a diagram illustrating an example of a monitor display image in which a person is double-shot when the right camera image including the distant view, the rear camera image, and the left camera image are actually combined in the around view monitor system. Hereinafter, the problem of the image composition technique including the near view and the distant view will be described with reference to FIGS.

  Currently, as shown in FIG. 5, the viewpoints from the four front and rear left and right cameras are converted to the viewpoint viewed downward from the virtual camera position set at the center upper position of the vehicle, and the front and rear left and right overhead images are synthesized. Thus, as shown in FIG. 6, a vehicle periphery monitoring device called an around view monitor system that synthesizes and displays a front-and-rear and left-and-right bird's-eye view image and displays it as shown in FIG.

  However, as shown in FIG. 5, continuous front video, rear video, left video, and right video are synthesized using four front / rear left / right cameras installed at different positions such as front / rear / right / left positions of the vehicle. Therefore, the parallax between the cameras tends to cause a contradiction that a joint between adjacent images lacks continuity.

  Here, as shown in FIG. 6, if only the vehicle near view is combined, the contradiction that the joint between the images to be combined lacks continuity is relatively small. However, when synthesizing a perspective image including a distant view using four front and rear left and right cameras, the contradiction that the joint between the images to be composed is lacking in continuity is larger than the composition of only the near view. .

  This is shown in FIGS. 7 and 8, which are examples of synthesizing panoramic images using left and right camera images and rear camera images. What is expected is an ideal composition with continuous seams, as shown in FIG. However, in practice, as shown in FIG. 8, a double image appears at the joint. In this double image, since the shift of the distance to the person who is the object varies from camera to camera, even if correction is performed to shift the entire image, a double image is produced at the joint between adjacent images. . In principle, the distance to all objects must be measured, and in the case of a moving image, the correction amount must be calculated in units of pixels for each frame.

  However, it is practically impossible to measure the distance to all objects and calculate the correction amount, and in fact, as shown in FIG. Inevitable. And, for example, as in the case of FIG. 8, when both the front person and the rear vehicle are duplicated, the distance from the vehicle is larger than the deviation width of the double person of the front person near the vehicle. The shift width of the double image of the far rear vehicle is increased. Therefore, not only does the driver and the passenger see the monitor display image as shown in FIG. 8 feel uncomfortable, but the positional relationship between the person and the vehicle existing behind the vehicle cannot be accurately recognized.

[Rear view monitor action]
FIG. 9 is a perspective view illustrating the far-field camera position and the blind spot area of the far-field camera in the around view monitor system of the first embodiment. FIG. 10 is a diagram illustrating an example of a monitor display image obtained by actually combining a distant view camera image, a right camera image, a rear camera image, and a left camera image in the around view monitor system according to the first embodiment. Hereinafter, the rear view monitoring operation will be described with reference to FIGS. 9 and 10.

The around view monitor system of Example 1 is
Add an omnidirectional camera 5 as a distant camera to the front, rear, left and right cameras 1, 2, 3, and 4 as a foreground camera.
A region that becomes a blind spot for the omnidirectional camera image taken by the omnidirectional camera 5 is set as the foreground camera image region.
-The foreground camera images from three or more of the four front and rear left and right cameras 1, 2, 3, and 4 are adjacent to the foreground camera image area.
To generate one composite image by combining one omnidirectional camera image and a plurality of foreground camera images.
It is characterized by.

  That is, as shown in FIG. 9, when the omnidirectional camera 5 is installed at the distant camera position, the vehicle body itself is reflected in the foreground portion. Therefore, an area that is a blind spot for the captured omnidirectional camera image is set as a foreground camera image area as indicated by a hatched area in FIG.

  Then, as shown in FIG. 10, the right camera image, the rear camera image, and the left camera image are adjacent to the foreground camera image area, and the entire upper area of the monitor screen 81 is defined as one distant camera image area. Is divided into a right camera image, a rear camera image, and a left camera image as one composite image. In this case, a circular arc joint is formed between the distant camera image area and the foreground camera image area, and two vertical joints are formed in the foreground camera image area.

Therefore, the far-field camera image in which the contradiction (double image) that occurs at the joint between adjacent images is a problem is a video based on one far-field camera image, and thus is completely continuous. As for the joint between adjacent foreground images, although there is a slight double image lacking in continuity as shown in the foot portion of the person in FIG. 10, this contradiction is a foreground image. Smaller than
For this reason, it is possible to generate a continuous vehicle surrounding image of a foreground and a distant view while suppressing the occurrence of a double image at the joint between adjacent images.

  In the first embodiment, an omnidirectional camera 5 having an angle of view of 360 degrees set at the top of the vehicle V is used as a far view camera. With this all-around camera 5, since all directions can be seen with one camera, there is an advantage that the cost is reduced from the viewpoint of ease of adjustment and equipment. In addition, it is easy to synthesize images because it can acquire video from a single viewpoint.

  In the first embodiment, an area that becomes a blind spot for the omnidirectional camera image captured by the omnidirectional camera 5 is set as a near-field camera image area, and is positioned at both sides of the one far-field camera image and the rear camera image based on the surrounding background behind the vehicle. One composite image is generated by combining three foreground camera images obtained by combining the left and right camera images. Therefore, a wide panoramic image can be obtained as the vehicle rear image, and when traveling backward, a useful rear view monitor image can be acquired in order to check the presence or position of an obstacle behind the vehicle. .

[Selection of composite video pattern]
FIG. 11 is a diagram illustrating an example of a monitor display image obtained by synthesizing the distant view camera image, the left camera image, the previous camera image, and the right camera image in the around view monitor system according to the first embodiment. FIG. 12 is a diagram illustrating an example of a monitor display image obtained by synthesizing the far-field camera image, the rear camera image, the left camera image, and the front camera image in the around view monitor system of the first embodiment. FIG. 13 is a diagram illustrating an example of a monitor display image obtained by synthesizing a distant view camera image, a front camera image, a right camera image, and a rear camera image in the around view monitor system according to the first embodiment. FIG. 14 is a diagram illustrating an example of a monitor display image obtained by synthesizing the all-around camera image and the all-around overhead view image in the around view monitor system according to the first embodiment. Hereinafter, the composite video pattern selection operation will be described with reference to FIGS. 1 and 10 to 14.

  In the first embodiment, an area that becomes a blind spot for an omnidirectional camera image captured by the omnidirectional camera 5 is set as a foreground camera image area, and one omnidirectional camera image and four front and rear left and right cameras 1, 2, 3, By combining the foreground camera images from three of the four cameras, one composite image is generated.

For this reason, the composite image by the following four combination patterns is obtained, and the following merit can be obtained for each.
(a) Monitor display video that combines distant camera video, right camera video, rear camera video, and left camera video (Figure 10)
When traveling backwards, a useful rear view monitor image can be acquired in order to confirm the presence or position of an obstacle behind the vehicle.
(b) Monitor display image that combines distant view camera image, left camera image, front camera image and right camera image (Fig. 11)
When traveling forward on a narrow road, a useful front view monitor image can be acquired in order to confirm the presence and position of an obstacle ahead of the vehicle.
(c) Monitor display image that combines distant view camera image, rear camera image, left camera image and front camera image (Fig. 12)
When traveling forward on a narrow road having a gutter or a wall on the left side, a useful side view monitor image can be acquired in order to confirm the presence and position of an obstacle on the left side of the vehicle.
(d) Monitor display image that combines distant view camera image, front camera image, right camera image and rear camera image (Fig. 13)
When traveling forward on a narrow road with a gutter or wall on the right side, useful side view monitor images can be acquired in order to confirm the presence or position of an obstacle on the right side of the vehicle.

  In the first embodiment, the viewpoint for converting the near view oblique viewpoint from the front, rear, left, and right cameras 1, 2, 3, and 4 into the viewpoint when the near view is viewed downward from the virtual camera position set at the center upper position of the vehicle. A bird's-eye view image of a close-up view around the vehicle is generated by the conversion process.

For this reason, the composite image by the following two combination patterns is obtained, and the following merit can be obtained for each.
(e) Monitor display video with a panoramic view of the surroundings composed of the front video, rear video, left video, and right video (Figure 1)
When performing parallel parking between vehicles parked in front and back, it is possible to grasp the positional relationship between the own vehicle position and the position of other vehicles in the front and rear, and changes in the positional relationship, and acquire useful around view monitor images Can do.
(f) Monitor display image (Fig. 14) that combines the all-around camera image and the all-around overhead image (front image, rear image, left image, and right image)
When performing parallel parking between vehicles parked in the front and back, not only can you grasp the positional relationship between the vehicle position and the position of the other vehicle in the front and rear, and changes in the positional relationship, but it is a bit farther away due to the three-dimensional composite image You can check the presence and location of obstacles. That is, it is possible to acquire a useful around view monitor image from the all-around overhead view image.

  Since the display video selection switch 6 is connected to the image processing controller 7, a preferable composite video pattern is selected from these composite video patterns by a selection operation using the display video selection switch 6 in accordance with the driving situation. It is possible to select a composite video pattern with a high degree of freedom.

Next, the effect will be described.
In the around view monitor system of the first embodiment, the following effects can be obtained.

  (1) A plurality of in-vehicle cameras (front, back, left and right cameras 1, 2, 3, 4) and a plurality of image data acquired by the in-vehicle cameras (front, back, left and right cameras 1, 2, 3, 4) In the vehicle periphery monitoring device (around view monitor system), comprising: an image processing controller 7 that performs composition processing on an image; and a monitor 8 that displays one composite image generated by the image processing controller 7 on a screen. As a plurality of in-vehicle cameras, a plurality of foreground cameras (front, rear, left, and right cameras 1, 2, 3, and 4) that shoot a foreground around the vehicle in a plurality of vertically divided areas are used. A far-field camera (all-around camera 5) for photographing a distant view around the vehicle with a wide angle of view including the angles of view of the cameras 1, 2, 3, 4) is added, and the image processing controller 7 Surrounding An area that becomes a blind spot for the far-field camera image captured by the camera 5) is set as a near-field camera image area, and one far-field camera image and a plurality of near-field camera images are combined to generate one composite image. For this reason, it is possible to generate a continuous vehicle surrounding image of a foreground and a distant view while suppressing the occurrence of a double image at the joint between adjacent images.

  (2) The far view camera is the omnidirectional camera 5 having a field angle of 360 degrees set at the top of the vehicle. For this reason, all the azimuths can be seen with a single unit, so that it is possible to reduce costs in terms of ease of adjustment and equipment, and it is possible to acquire video from a single point of view, so image synthesis is easy. be able to.

  (3) As the plurality of foreground cameras, a rear camera 2 whose imaging range is the rear of the vehicle, a left camera 3 whose imaging range is the left side of the vehicle, and a right camera 4 whose imaging range is the right side of the vehicle, The image processing controller 7 sets an area that becomes a blind spot for the omnidirectional camera image captured by the omnidirectional camera 5 as a foreground camera image area, One composite image is generated by combining three foreground camera images obtained by combining the left and right camera images with both side positions of the camera image. For this reason, when traveling backward, a useful rear-view monitor image can be acquired in order to confirm the presence or position of an obstacle behind the vehicle.

  (4) As the plurality of foreground cameras, a front camera 1 whose imaging range is the front of the vehicle, a rear camera 2 whose imaging range is the rear of the vehicle, a left camera 3 whose imaging range is the left side of the vehicle, and the right side of the vehicle The image processing controller 7 sets an area that becomes a blind spot for the omnidirectional camera image captured by the omnidirectional camera 5 as a foreground camera image area. By combining the all-around camera image and the foreground camera images from three of the four front, rear, left, and right cameras, one composite image is generated. Therefore, it is necessary to acquire rear view monitor images that are useful when traveling backward, front view monitor images that are useful when traveling forward on narrow roads, and left and right side view monitor images that are useful when traveling forward on narrow roads. Can do.

  (5) The image processing controller converts the viewpoints from the four front / rear left / right cameras to a viewpoint viewed downward from the virtual camera position set at the center upper position of the vehicle, and synthesizes the front / rear / left / right overhead images. Thus, the all-around bird's-eye view synthesized image is generated, and the all-around bird's-eye view synthesized image and the all-around view image are synthesized. For this reason, when parallel parking is performed between vehicles parked in front and back, the around view monitor image is more useful than only the all-around bird's-eye view composite image, including the presence or absence of obstacles slightly away from the three-dimensional composite image. Can be obtained.

  (6) Connected to the image processing controller 7 is a monitor video selection means (display video selection switch 6) for selecting one composite video to be displayed on the monitor 8 from a plurality of composite video patterns. For this reason, it is possible to select a preferable composite video pattern from a plurality of composite video patterns with a high degree of freedom of selection.

  As mentioned above, although the vehicle periphery monitoring apparatus of this invention has been demonstrated based on Example 1, it is not restricted to this Example 1 about a concrete structure, The invention which concerns on each claim of a claim Design changes and additions are allowed without departing from the gist.

  In the first embodiment, an example in which four cameras (front, rear, left and right) are used as a plurality of foreground cameras. However, as long as it has at least 2 cameras which image | photograph the adjacent foreground as a several near view camera, it is included in this invention. For example, as a plurality of foreground cameras, a front camera with an imaging range in front of the vehicle, a rear camera with an imaging range in the rear of the vehicle, a left camera with an imaging range on the left side of the vehicle, and an imaging range on the right side of the vehicle You may make it have at least 2 camera among right cameras to perform.

  In the first embodiment, an example in which one omnidirectional camera is used as the far view camera has been described. However, as long as the far-field camera has a wide angle of view including the angle of view of at least two near-field cameras, a plurality of far-field cameras may be used.

  In the first embodiment, an area that becomes a blind spot for an omnidirectional camera image captured by an omnidirectional camera is set as a foreground camera image area based on an around view monitor system, and one omnidirectional camera image and four front and rear left and right images are set. An example in which one composite image is generated by combining foreground camera images from three cameras among the cameras has been shown. However, an area that becomes a blind spot for a far-field camera image captured by one far-field camera is set as a near-field camera image area, and one composite image is generated by combining one far-field camera image and at least two near-field camera images. If it does, it is included in this invention.

  In the first embodiment, an example in which one video pattern is selected from the video patterns of a plurality of composite images using the display video selection switch has been described. However, an example in which a vehicle state, a traveling state, a parking state, or the like is detected and an appropriate video pattern is automatically selected according to the detected vehicle environmental condition may be used.

  In the first embodiment, the application example to the around view monitor system is shown, but the present invention can also be applied to a vehicle periphery monitoring device such as a side view monitor system, a back view monitor system, and a front view monitor system.

1 Front camera (vehicle camera, foreground camera)
2 Rear camera (vehicle camera, foreground camera)
3 Left camera (vehicle camera, foreground camera)
4 Right camera (vehicle camera, foreground camera)
5 All-around camera (vehicle camera, distant camera)
6 Display video selection switch (Monitor video selection means)
7 Image processing controller 71 Image data converter 72 CPU
73 Map Memory 74 Image Deformation Processing Unit 75 Image Composition Processing Unit 76 Superimposition Memory 77 Superimposition Processing Unit 78 Image Data Conversion Unit 8 Monitor 81 Monitor Screen

Claims (6)

A plurality of in-vehicle cameras, an image processing controller that synthesizes a plurality of image data acquired by the in-vehicle camera into one image, and a monitor that displays on the screen one synthesized image generated by the image processing controller; In the vehicle periphery monitoring device provided with
As the plurality of in-vehicle cameras, a distant view around the vehicle with a wide angle of view including the angle of view of the plurality of close-up cameras is used as a plurality of close-up cameras that photograph a close-up view around the vehicle in a plurality of vertically divided areas. Add a distant camera to shoot
The image processing controller sets, as a foreground camera image area, an area that becomes a blind spot for a distant camera image captured by the distant camera, and combines one distant camera image and a plurality of foreground camera images into one composite image. A vehicle periphery monitoring device that generates the vehicle periphery. In the vehicle periphery monitoring device according to claim 1,
1. The vehicle periphery monitoring device according to claim 1, wherein the far-field camera is an omnidirectional camera having a field angle of 360 degrees set at the top of the vehicle. In the vehicle periphery monitoring device according to claim 2,
As the plurality of foreground cameras, a rear camera having an imaging range at the rear of the vehicle, a left camera having an imaging range on the left side of the vehicle, and a right camera having an imaging range on the right side of the vehicle,
The image processing controller sets, as a foreground camera image area, an area that becomes a blind spot for an omnidirectional camera image captured by the omnidirectional camera. A vehicle periphery monitoring device characterized by generating one composite image by combining three close-up camera images obtained by combining left and right camera images. In the vehicle periphery monitoring device according to claim 2,
As the plurality of foreground cameras, a front camera with an imaging range in front of the vehicle, a rear camera with an imaging range in the rear of the vehicle, a left camera with an imaging range on the left side of the vehicle, and an imaging range on the right side of the vehicle A right camera, and
The image processing controller sets, as a foreground camera image area, an area that becomes a blind spot for an omnidirectional camera image captured by the omnidirectional camera, and includes three omnidirectional camera images and three of the four front, rear, left, and right cameras. A vehicle periphery monitoring device that generates one composite image by combining near-field camera images from a camera. In the vehicle periphery monitoring device according to any one of claims 1 to 4,
The image processing controller converts the viewpoints from the four front / rear left / right cameras to a viewpoint viewed downward from the virtual camera position set at the center upper position of the vehicle, and synthesizes the front / rear / left / right overhead images. A vehicle periphery monitoring apparatus, characterized by generating a surrounding bird's-eye view synthesized image and generating a three-dimensional synthesized image obtained by synthesizing the all-around bird's-eye view synthesized image and the all-around bird's eye view image. In the vehicle periphery monitoring device according to any one of claims 1 to 5,
A vehicle periphery monitoring device, wherein monitor image selection means for selecting one composite image to be displayed on the monitor from a plurality of composite image patterns is connected to the image processing controller.

Images