KR20200047257A - Device and a method for displaying aerial view images of a vehicle - Google Patents

Abstract

Provided are a device and a method for displaying images around a vehicle, which display images around a vehicle without a blind spot in an optimal display range in synchronization with the advance of the vehicle. The device for displaying images around a vehicle according to the present invention comprises an imaging unit arranged on the vehicle to capture images of the surroundings of the vehicle, an image storage unit to preserve the captured images, an image processing unit to process the captured images, and an image display unit to display the processed images. The image processing unit generates a vehicle surrounding image by supplementing a blind spot of the current image captured by the imaging unit with a record image created from previously captured images, and the image display unit superimposes and displays a transmission image of a subject vehicle on the generated vehicle surrounding image.

Description

A video display system around a vehicle and a video display method around a vehicle {DEVICE AND A METHOD FOR DISPLAYING AERIAL VIEW IMAGES OF A VEHICLE}

The present invention relates to a vehicle surrounding image display system for displaying a surrounding image of a vehicle such as an automobile, and a display method thereof, and more specifically, based on an image obtained from a camera shooting around the vehicle, the subject vehicle and its surroundings. The present invention relates to an image display system around a vehicle and a method of displaying an image around a vehicle that can display surrounding images including a direct view of the subject vehicle without blind spots in accordance with the movement of the subject vehicle when synthesizing and displaying the overlooking image.

The range that can be directly seen by the driver driving the vehicle is limited. In particular, there are a number of places in the rear of the vehicle that become a blind spot that cannot be seen from the driver's seat even when using a normal room mirror or door mirror. For example, when looking at the rear of the vehicle, part of the driver's field of view is blocked by the trunk or rear pillars. Thus, conventionally, a device that assists the driver in checking the surroundings of a vehicle has been proposed by mounting a camera that photographs the surroundings of the vehicle on a vehicle and displaying the captured image on a display.

For example, in the technology disclosed in Patent Document 1, a photographed image acquired by a plurality of cameras photographing the periphery of a vehicle is processed, and the display surface of the display means disposed at a position visible by the driver through the rearview mirror is displayed around the vehicle. It is proposed to display a composite image including an image of a rectangular area. In addition, in the technique disclosed in Patent Document 2, when the parking partition line detected from the photographed image is incomplete, a technique of superposing and displaying the past image photographed in the past by the photographing means is disclosed.

However, in the technique of patent document 1, since the situation near the lower part or the outer peripheral part of the vehicle cannot be reflected, it was not possible to confirm the positional relationship between the tire and the obstacle, and the positional relationship between the tire and the groove, etc. through the image. In addition, the technique of patent document 2 was unable to reflect the current situation because the unclear part of the video was replaced with the video captured in the past.

(Patent Document 1) Japanese Patent Publication No. 2006-135797 (Patent Document 2) Japanese Patent Publication No. 2006-311299

The present invention has been made in view of the above-mentioned conventional problems, and an object of the present invention is to display an image around a vehicle in which a situation near a lower portion or an outer circumference of a vehicle becomes a blind spot by a magnetic vehicle, and a blind spot according to the movement of the vehicle. An object of the present invention is to provide a video display system around a vehicle that optimizes the display range of a video around a vehicle without this.

The vehicle surrounding image display system according to one aspect of the present invention made to achieve the above object is a vehicle surrounding image display system that displays a vehicle surrounding image synchronized with the progress of the vehicle, and the image display system is disposed on the vehicle and A photographing means for photographing the surroundings of the vehicle, image storage means for preserving the image photographed by the photographing means, image processing means for image processing the photographed image, and image display means for displaying the image processed image The image processing means generates an image around the vehicle that supplements a rectangular portion of the current image photographed by the photographing means with a history image generated from an image previously taken, and the image display means The transmitted image of the own vehicle stored in the image storage means is superimposed on the generated vehicle surrounding image. Characterized in that it displays.

The photographing means includes one or more wide-angle cameras capable of photographing the rear or front periphery of the subject vehicle, and transmits an image photographed by the wide-angle camera to the image processing means in units of frames, and the image processing means is received The converted image may be converted into a sub-frame image in a continuous frame unit, and the converted sub-frame image may be synthesized to generate a history image.

The image display system further includes vehicle position detection means for calculating a direction, a moving direction, and a moving distance of the subject vehicle based on the captured image, wherein the moving distance of the vehicle is a predetermined movement. Each time a distance is reached, based on the calculated direction, moving direction, and moving distance of the subject vehicle, a composite position of the captured current image and an image taken before the current is adjusted for each frame, The history image can be updated.

The image processing means combines the captured current image with the generated historical image in accordance with the progress of the vehicle, and when there is a portion different from the historical image in the captured current image, It is possible to generate an image around a vehicle in which the image is highlighted.

The image processing means, according to the progress of the vehicle, based on the direction, the moving direction, and the moving distance of the subject vehicle calculated by the own vehicle position estimation means, the captured current image to be synthesized and before the current shooting You can crop the image, or set the location for image synthesis.

A vehicle surrounding image display method according to one aspect of the present invention made to achieve the above object is a method of displaying a vehicle surrounding image synchronized with the progress of both sides, and is a photographing means arranged on a vehicle to photograph the surrounding of the vehicle In the image display system around the vehicle having an image storage means for preserving the image photographed by the photographing means, image processing means for image processing the photographed image, and image display means for displaying the image processed image A step in which the image processing means receives the image photographed by the photographing means, converts the received image into a sub-frame image in a continuous frame unit, and synthesizes the converted sub-image to generate a history image; A vehicle main that complements the rectangular portion of the current image photographed by the photographing means with the generated history image. And a step of generating the above image and superimposing and displaying the transmitted image of the own vehicle stored in the image storage means on the generated image around the vehicle.

The step of generating the history image may include a step of extracting a feature point from the successive frame-level subtraction image and adjusting a synthesis position to match the extracted feature point.

The image display method around the vehicle further includes a step of the image processing means calculating a direction, a moving direction, and a moving distance of the subject vehicle based on the photographed image, and the step of generating the history image is the calculating Whenever the moving distance of the vehicle reaches a predetermined moving distance, based on the calculated direction, moving direction, and moving distance of the subject vehicle, the captured current image and an image taken before the current It may include the step of adjusting the composite position of each frame, to update the history image.

The step of superimposing and displaying the transmitted image is a combination of the captured current image and the generated historical image in accordance with the progress of the vehicle, and when the captured current image has a different portion from the historical image And, it may include a step of highlighting and displaying the image of the other portion.

The step of generating an image around the vehicle is based on the calculated direction of the vehicle, the moving direction, and the moving distance, in accordance with the progress of the vehicle, the captured current image being synthesized and previously taken before the current It may include a step of cropping an image or setting an image synthesis location.

According to the present invention, it is possible to display an image around a vehicle in which the situation near the lower part of the vehicle or the outer periphery becomes a blind spot by the subject vehicle, and furthermore, an image of a vehicle without a blind spot in which the display range is optimized in synchronization with the progress of the vehicle Can be displayed.

In addition, the composite position of the captured current image and the previously captured image based on the direction, moving direction, and moving distance of the subject vehicle calculated whenever the moving distance of the vehicle reaches a predetermined moving distance. Is adjusted for each frame, and since the historical image is updated, the processing load is reduced and the processing load is reduced as compared to the case where a synthesized image is generated using all the acquired frames. Accumulation of errors, that is, accumulation errors can be reduced.

1 is a view showing the overall configuration of a video display system around a vehicle according to an embodiment of the present invention.
2 is a diagram showing an example of a camera shooting range of the surrounding video display system according to the present embodiment.
3 is a block diagram showing an example of a functional configuration of a video display system around a vehicle according to the present embodiment.
Fig. 4 is a flowchart showing an image processing procedure until both surrounding video display systems according to the present embodiment acquire and display surrounding video of the vehicle.
5 is a diagram showing an example of the processed video at each stage of video processing by the video display system around the vehicle of the present embodiment.
6 is a view showing an example of a vehicle surroundings image displayed by the vehicle surroundings video display system of the present embodiment.
7 is a diagram showing an example of an image generated by additional processing of image processing in the present embodiment.
8 is a diagram showing an example of an image generated by another additional process of image processing in the present embodiment.
9 is a conceptual diagram showing another example of image processing of the video display system around the vehicle according to the present embodiment.
10 is a conceptual diagram showing still another image processing example of the video display system around the vehicle according to the present embodiment.

Hereinafter, specific examples of the embodiments for carrying out the present invention will be described in detail with reference to the drawings.

1 is a view showing the overall configuration of a video display system around a vehicle according to an embodiment of the present invention. As shown in Fig. 1, the vehicle surrounding video display system 1 according to the present embodiment is a vehicle-mounted monitor device comprising a camera (shooting means 3) installed on the rear of the vehicle 2, a liquid crystal display disposed in the vehicle, and the like. (Image display means 5), image processing ECU (image processing means 6), vehicle information DB 7, driving image DB (image storage means 8), vehicle ECU 9, and the like. It is also possible to install a camera on the front part of the vehicle 2, and to take a picture of the front and rear surrounding images. In addition, it is also possible to install a camera (not shown) on the left and right sides of the vehicle 2, and perform display including a conventional around view image using four cameras. Hereinafter, an example provided on the rear side will be described to simplify the description.

2 is a diagram showing an example of a camera shooting range of the surrounding video display system according to the present embodiment. In the present embodiment, the camera 3 is a solid-state imaging element such as a CCD or CMOS, for example, and is mounted near the rear center of the vehicle 2, as shown in Fig. 2, and the imaging direction is horizontal. It is installed toward the lower part. Then, the rear of the vehicle is photographed according to the traveling direction (ie, the moving direction) of the vehicle 2, and the photographed image (hereinafter referred to as a photographed image) is output as a video signal. The video signal output from the camera is converted into video data by video processing means, which will be described later, and is then displayed on the in-vehicle monitor device 5 as an image around the vehicle, for example, an overlooked video, through predetermined video processing. Further, in the present embodiment, the camera 3 is provided with a wide-angle lens, and one capable of photographing a range of about 110 degrees to 140 degrees left and right is used, but is not limited to this.

The in-vehicle monitor device 5 may be provided on the center console or panel surface of the vehicle 2 indoors, and displays a composite image generated from an image captured by the camera 3. In addition, the vehicle-mounted monitor device 5 may be used in combination with that used for the navigation device.

The image processing ECU (Electronic Control Unit) 6 is an electronic control unit that performs various image processing on image data of an image captured by the camera 3. In the present embodiment, the first image processing performed by the image processing ECU 6 generates coordinated images by changing coordinates with respect to image data of a captured image, and stores it in the traveling image DB 8 in units of frames. . The second image processing detects a characteristic part on the road, including a main line or a division line on which the vehicle 2 moves, from the image data for each frame, and uses the detected division line or display to detect feature points. Synthesis processing of time-series images by extraction and search of coincidence points is performed. Moreover, as a third image processing, in synchronism with the progress (ie, movement) of the subject vehicle, a sublime image supplementing a blind spot by the subject vehicle is generated, and image data superimposed on the transmission image of the subject vehicle is generated. do. The detailed configuration of the image processing ECU 6 will be described later.

The vehicle information DB (database 7) is a storage means in which various parameters relating to the vehicle 2 and the camera 3 are stored. In the vehicle information DB 7, for example, information about the vehicle model such as the minimum turning radius, vehicle width, tread length, and tire radius of the vehicle 2, internal parameters of the camera 3, optical axis direction, angle of view, and installation Information about the design value of the camera 3, such as a position, is stored.

The driving image DB 8 is related to additional information such as the camera position of the camera 3, the shooting direction angle, the vehicle position, the direction of the vehicle, and the like when the image data of the image taken by the camera 3 is taken. It is a storage means that is preserved in units. Further, in the present embodiment, an SSD (solid state drive) is used as a storage device of the driving image DB 8, but a magnetic disk, memory card, DVD, or the like can also be used as an external storage device.

The vehicle ECU 9 is an electronic control unit of the vehicle 2 that controls the operation of the engine, transmission, brake, and the like. Various sensors such as a vehicle speed sensor 11, a gyro sensor 12, a steering angle sensor 13, and a shift position detection sensor 14 are connected to the vehicle ECU 9. The image processing ECU 6 can calculate the vehicle speed, acceleration, steering angle of the vehicle 2, direction of the vehicle, moving distance, and the like from the output values of these sensors.

Next, a functional configuration of the video display system 1 around the vehicle according to the present embodiment will be described with reference to FIG. 3. 3 is a block diagram showing an example of a functional configuration of a video display system around a vehicle according to the present embodiment.

As shown in Fig. 3, the vehicle surroundings image display system 1 according to the present embodiment is constituted based on functional means realized by a vehicle surroundings image generation program executed by the image processing ECU 6, and each A predetermined peripheral device is connected to the functional means.

The image processing ECU 6 includes an image input unit 61, a vehicle signal input unit 62, a calculation unit 63, and an image output unit 64.

The image input unit 61 is an input interface for acquiring image data of images around a vehicle photographed by the camera 3. The image input unit 61 includes an A / D converter, and converts an image signal output from the camera 3 into a digital signal to generate image data in units of frames. Further, in the case of a type in which the camera 3 outputs a digital video signal, the A / D converter can be omitted. The image input unit 61 can also acquire an image as a continuous image (movie), but in this embodiment, in response to the movement of the vehicle, for each moving distance corresponding to the shooting range of the camera 3, a still screen unit, that is, Images are acquired in frame units. In addition, the interval for acquiring the image is preferably equal to or less than the length of the image in the moving direction (the shooting range indicated by the symbol L in FIG. 2) so as to obtain a portion left for pasting when the adjacent frames are combined and pasted. Do.

The vehicle signal input unit 62 is an interface for acquiring data (hereinafter referred to as movement amount data) related to movement of the vehicle detected by various sensors. The vehicle signal input unit 62 acquires movement amount data from the vehicle ECU 9 via, for example, an in-vehicle LAN.

The calculation unit 63 is composed of a well-known microcontroller equipped with a CPU, RAM, ROM, and the like. In addition, the ROM stores a video processing program, which will be described later, and various programs necessary for control of the vehicle-mounted monitor device 5 and the like. Moreover, RAM is a memory which temporarily stores various data calculated by the CPU.

Whenever the image data is acquired from the image input unit 61 in frame units, the operation unit 63 functions each of the function units described later, and the history image based on the image data of the captured image acquired this time and the image acquired earlier than this time. The vehicle surrounding image data is generated by synthesizing the image data of and outputs the generated vehicle surrounding image data to the image output unit 64.

The image output unit 64 outputs the image data around the vehicle output from the operation unit 63 to the vehicle-mounted monitor 5 in order in accordance with the movement of the vehicle. Thereby, a crew member, such as a driver in the vehicle, can view the image around the vehicle displayed on the vehicle-mounted monitor device 5.

The calculation unit 63 is a functional unit realized by execution of an image generation program around the vehicle, an image conversion unit 63a, a feature point extraction unit 63b, a host vehicle position estimation unit 63c, and a history image accumulation unit 63d, Including the moving distance estimator 63e, the direction key angle estimator 63f, the vehicle movement amount calculating unit 63g, the image synthesizing unit 63h, and the output image generating unit 63i, the driving image DB 8 and the vehicle ECU Based on the various information acquired from each peripheral device via (9), it is controlled to display the photographed image (see Fig. 5) on the vehicle-mounted monitor device.

When the image conversion unit 63a receives frame-based image data from the image input unit 61, it corrects the image distortion caused by wide-angle imaging on the received image data. Thereafter, a coordinate change is performed on the image data subjected to distortion correction, and processing is performed to generate image data of the subtracted image. Then, the image conversion unit 63a stores the generated image data of the subtracted image in the driving image DB 8 in units of frames. As a result, image data of past photographed images is accumulated. Here, as a technique for converting to a subtractive image, for example, a conventional technique disclosed in Japanese Patent Application Laid-open No. Hei 10-211849 can be used.

The feature point extracting unit 63b is a frame-by-frame image (hereinafter referred to as a frame) for a current image converted into a sublime image or a past image captured in the driving image DB 8 (accumulated sub-image). Feature points that exist in the screen of an image) are extracted. The feature point is a point that can be followed when the frame screen is moved, and in general, a point having a difference in luminance in both the vertical and horizontal directions in each frame image is extracted as the feature point. That is, the feature points can be detected by image processing such as edge processing, and various methods (SIFT, A-KAZE, SURF, etc.) have been proposed as known processing methods as methods for detecting the feature points. Feature points are extracted using this processing method.

The host vehicle position estimator 63c combines the feature points of the current frame image converted to the sublime image and the feature points of the previous frame image (accumulated subtitle image) accumulated in the driving image DB8. , Estimate the location of the host vehicle. Specifically, the video data corresponding to the immediately preceding frame is read from the driving video DB 8, and the feature points are selected from the feature points in the immediately preceding frame image and the current frame image. Then, the position of the selected feature point in the frame image is estimated, and the selected feature point is searched in the image as the center of the estimated value of the position in the frame image. By this search, the difference between the position in the video assumed from the previous position and the position in the actually detected video is detected. Then, the own vehicle position estimator 63c calculates the own vehicle position by correcting the estimated value of the own vehicle position from the difference between the feature point position in the frame image and the feature point position detected from the position immediately before the own vehicle position. In this process, SLAM (Simultaneous Localization And Mapping) technology is used.

The historical image accumulation unit 63d uses the feature point and the host vehicle position estimation unit 63c obtained by the feature point detection unit 31 in units of adjacent frames based on the vehicle movement amount information, based on the vehicle movement amount information, based on the vehicle movement amount information. Information related to the estimated own vehicle position is associated and stored in the driving image DB 8 as historical image data.

When the vehicle signal input unit 62 acquires the movement amount data of the vehicle, the movement distance estimation unit 63e is based on the movement distance included in the acquired movement amount data, and then determines the timing at which the image input unit 61 acquires the captured image. Estimate. That is, it is determined whether the moving distance of the vehicle has reached the moving distance corresponding to the frame size (L in the shooting range shown in FIG. 2) photographed by the camera 3. When the moving distance estimating unit 63e determines that the moving distance of the vehicle has reached a predetermined value (for example, 90% of the length of L of the shooting range), the moving image estimating unit 61 is photographed at that time. A signal for acquiring a frame image (hereinafter referred to as an image acquisition signal) is transmitted. The image acquisition signal is also transmitted to the direction key angle estimating unit 63f.

When the vehicle signal input unit 62 acquires the movement amount data of the vehicle, the direction key angle estimating unit 63f determines the angle key angle data included in the acquired movement amount data, and receives the image acquisition signal from the movement distance estimating unit 63e. , At the time when the image input unit 61 acquires the captured image, the direction key angle of the vehicle is transmitted to the vehicle movement amount calculating unit 63g.

The vehicle movement amount calculation unit 63g is an image input unit 61 based on the movement distance and direction key angle data including a current obtained by the movement distance estimation unit 63e and the direction key angle estimation unit 63f, and the current. Calculates how the vehicle moved in a predetermined time period, including the current. Each calculated data is stored in the driving image DB 8 in association with the frame-based image data acquired by the image input unit 61, and is used when adjusting a range of cropping (or trimming) the frame image. do.

When the image input unit 61 acquires image data of the captured image, the image synthesis unit 63h uses the latest captured image as an actual image. In addition, the frame image data of the time series acquired in the predetermined time period before the present is read from the driving image DB 8 and arranged according to the movement of the vehicle. Based on the calculation result of the moving contents of the vehicle calculated by the vehicle movement amount calculating unit 63g, each frame records the location of the current vehicle's captured image and the location of the previous frame image read from the driving image DB8. By adjusting the composite position so that the feature points extracted from are matched, the historical image data is generated, and the image data of the historical image is synthesized with the image data of the actual image so that the square part of the actual image is supplemented with the historical image, and the vehicle Create image data of surrounding images.

The output image generating unit 63i superimposes the image data of the generated vehicle surrounding image and the transmission image data of the own vehicle, and sequentially outputs the image data to the image output unit 64. Moreover, the output image generation unit 63i performs preset emphasis processing when an object detected by another sensor or a difference between the historical image and the current captured image is detected, and supplements the image data to the image output unit 64 send.

The image output unit 64 displays the image data of the vehicle surrounding image received from the output image generation unit 63i on the in-vehicle monitor device 5.

Next, with reference to Figs. 4 and 5, the operation of the video display system around the vehicle according to an embodiment of the present invention will be described. 4 is a flow chart showing a video processing procedure until the vehicle surroundings video display system according to the present embodiment acquires and displays surroundings images. Fig. 5 is a diagram showing an example of the processed video in each step of the video processing by the video display system around the vehicle of the present embodiment. In this embodiment, a case in which the vehicle is retracted (back) will be described.

As shown in Fig. 4, when the present system is activated, the road surface and the surroundings of the rear of the vehicle are photographed by the camera 3, and the image processing ECU 6 controls each functional means, and a series of Perform image processing.

First, the image processing ECU 6 acquires the image data of the surrounding image of the rear of the vehicle photographed by the camera 3 in the frame unit synchronized with the movement of the vehicle via the image input unit 61, and the driving image DB (8) Save (step S101). At this time, the image data of the frame image stored in the driving image DB 8 is image data converted into a subtractive image in units of frames by the image converter 63a as described above.

Specifically, the original image (camera image) output from the camera 3 is, for example, as shown in the image (a) of FIG. 5, to the camera 3 (wide-angle camera) disposed on the rear of the vehicle. As a result, since the white line drawn on the road (compartment line or display) and the surrounding area are images, the white line or the like is displayed skewed than the original state in response to the distance between the location of the white line and the vehicle (ie, the camera 3). Then, as shown in the image (b) of FIG. 5 by the image conversion unit 63a, after image distortion correction due to wide-angle imaging is performed, the coordinates are changed and shown in the image (c) of FIG. Image data of the subtractive image is generated.

Next, in synchronism with the movement of the vehicle, the video data of the subtracted image acquired in units of sequential frames is used by the feature point extraction unit 63b to search for matching points between the video data of the immediately preceding frame. This is extracted. For example, as shown in the image (d) of FIG. 5, the lozenge-shaped mark detected in the image of the frame N acquired at time t and the frame (N + 1) acquired at the time (t + 1) immediately thereafter. The rhombic mark detected in the image of) is extracted as a feature point (step S102).

Thereafter, by the own vehicle position estimator 63c, as shown in the image (e) of FIG. 5, using the extracted feature points, a matching point between the two images of the frame N and the frame N + 1 is searched. . In addition, the host vehicle position estimator 63c detects the difference between the two images by combining the positions of the feature points selected by matching point search in each frame image. The own vehicle position estimation unit 63c calculates the direction of the own vehicle, the moving direction, and the moving distance from the difference in the feature point positions (step S103). In this process, SLAM technology is used as described above. Various algorithms have been announced for SLAM, and there are Mono-SLAM and PTAM.

Next, the image synthesizing unit 63h sets an image boundary with a small number of discontinuous points, as shown in the image (f) of FIG. 5, based on the location of the selected feature point, etc., and immediately before being stored in the driving image DB (8). The composite position is adjusted so that the feature points of the frame N of the image data and the frame (N + 1) of the image data taken around the current vehicle match, and the image is synthesized as shown in the image (g) of FIG. 5. (Step S104). In the same procedure, the video data of the previous frame N and the video data of the previous frame (N-1) are synthesized to generate video data of a historical image in which a plurality of frames are synthesized. Further, when synthesizing such an image, the image synthesizing unit 63h appropriates the frame range of the historical image, as shown in the image (h) in FIG. 5, based on the amount of movement calculated by the vehicle movement amount calculating unit 63g. The video to be synthesized may be shaped by performing processing such as cropping.

Thereafter, the output image generation unit 63i is an image in which a transmission image of the subject vehicle is superimposed on a vehicle surrounding image obtained by synthesizing a vehicle's surrounding image and a history image of the current vehicle, as shown in image (i) of FIG. 5. Is displayed on the in-vehicle monitor device 5 (step S105).

6 is a view showing an example of a vehicle surroundings image displayed by the vehicle surroundings video display system of the present embodiment. By the above-described operation, after the system is started, the vehicle-mounted monitor device 5 displays an image supplemented by a rectangular portion of the current photographed image by a history image generated in synchronization with the movement of the vehicle. That is, with respect to the image A when the vehicle has started to retreat at the time t in FIG. 6, the images B and C of the subsequent time t + i1 and time t + i2 are respectively transferred. In the historical image generated from the image captured in the image, the rectangular portion of the current image is complemented and displayed. In addition, the range of the image displayed on the in-vehicle monitor device 5 is adjusted such that, after activation, the current position of the host vehicle is arranged in the center of the display screen.

Hereinafter, another example of the video processing by the video display system around the vehicle of the present embodiment will be described.

7 is a diagram showing an example of an image generated by additional processing of image processing in the present embodiment. In the video example shown in FIG. 7, when synthesizing the current photographed image and the historical image, a predetermined functional unit (not shown) included in the calculation unit 63 combines the current photographed image and the historical image, If there is an upper portion of the current photographed image that does not exist in the history image, it is determined whether or not the detected upper portion is an obstacle by image recognition in a predetermined manner. Then, when an obstacle exists in the current captured image and there is no obstacle in the image captured in the past, the output image generating unit 63i generates image data highlighting the obstacle. As a result, an obstacle is highlighted on the vehicle-mounted monitor device 5, so that the driver's attention can be aroused and driving errors can be reduced.

8 is a diagram showing an example of an image generated by another additional process of image processing in the present embodiment. In the example of the image shown in FIG. 8, when a moving object approaching from another sensor provided in the vehicle is detected, the output image generation unit 63i based on the detected information, the history of supplementing the blind spot to the current captured image Along with the image, an image around the vehicle is generated in the form of superimposing and displaying a predicted path of a moving object calculated by a predetermined function unit (not shown) provided in the image processing ECU 6. This makes it possible to draw attention to the moving object to the driver.

In the above, in the video display system around the vehicle according to the embodiment of the present invention, the procedure of image processing has been described in the case where the vehicle is retracted (back). It is possible.

9 is a conceptual diagram showing another example of image processing of the video display system around the vehicle according to the present embodiment. In the image processing example shown in FIG. 9, a camera 3 설치 is also installed in the front part of the vehicle, and shows the process of generating a history image around the vehicle when the vehicle is advanced. In this case, according to the sequence of image processing shown in FIG. 4, according to the progress (ie, movement) of the vehicle, a historical image in units of frames is generated, and the current captured image around the vehicle is supplemented with the historic image while surrounding the vehicle. By generating an image, it becomes possible to display an image around a vehicle supplemented by a blind spot including a vehicle underside.

10 is a conceptual diagram showing still another image processing example of the video display system around the vehicle according to the present embodiment. The image processing example shown in FIG. 10 is an example of an image around a vehicle displayed on the vehicle-mounted monitor device when an image around the vehicle is generated by using images captured by cameras installed at the front and rear parts of the vehicle, from the start of operation. It is a figure showing according to the passage of time. Generation of the image around the vehicle is omitted because it is the same as the above-described procedure.

As described above, the image display system around the vehicle according to the present invention is synchronized with the progress of the vehicle, and the image of the blind spot is sequentially visualized and updated, and finally, the image around the vehicle without the blind spot can be displayed. Thereby, a normal vehicle surrounding image displaying a real-time image and a rectangular image visualized and updated (square images are sequentially generated and synthesized) can be confirmed without discomfort. That is, it is possible to check the image of the ground and the position of obstacles that are not normally visible under the vehicle with the image. In particular, an image without a blind spot is generated in synchronization with the vehicle moving direction (that is, the moving direction), so that a sense of discomfort does not occur.

In addition, by using camera images before and after the vehicle, it is possible to generate similar images around the vehicle without a blind spot. Moving forward or backward in any direction of travel, a rectangular image can be generated, and by combining the images before and after, it is possible to similarly generate images around the vehicle with two cameras.

Since the surrounding image is photographed wide and a display image is generated using a part of the captured image, even when the steering angle of the handle is largely manipulated, a square can be generated without fail, so convenience is high. Compared to the case where all screens are processed, processing is performed using the necessary partial images, so that the processing load can be reduced.

In the image display system around the vehicle according to the present invention, when the steering angle of the steering wheel is largely cut, a range of images required to generate a square image is widened, so a range that is slightly wider than a range required to synthesize a square using a history image is used. After cropping, a composite image is generated, and then a square image is generated. As a result, by greatly cutting the steering angle, even if the displayed range is partially wider than the previously displayed range, the video can be displayed without missing.

In a parking lot, etc., the image captured in the past and the current image are combined, and if there is another part in the square generated image, the place is highlighted. As a result, attention is distracted at the time of driving in a familiar place, but driving errors can be reduced by highlighting different parts from the past.

From the results of the detection of the body in the captured image (speed, direction, etc.), objects that are likely to fall within the blind spot indicate the predicted course. Since the square display portion uses the past image, when the moving object is squeezed into the square, the position of the moving object cannot be displayed in real time, but by displaying the predicted path, attention can be paid to the moving object.

The embodiments of the present invention have been described in detail with reference to the drawings, but the present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the technical scope of the present invention. Do.

1: Video display system around the vehicle 2: Vehicle
3, 3 ＇: Camera 5: Vehicle monitor device
6: Image processing ECU 7: Vehicle information DB
8: Driving image DB 9: Vehicle ECU
11: vehicle speed sensor 12: gyro sensor
13: steering angle sensor 14: shift position detection sensor
61: video input unit 62: vehicle signal input unit
63: operation unit 63a: image conversion unit
63b: Feature point extraction unit 63c: Own vehicle position estimation unit
63d: history image accumulation unit 63e: moving distance estimation unit
63f: Direction key angle estimation unit 63g: Vehicle movement amount calculation unit
63h: image synthesis unit 63i: output image generation unit
64: video output

Claims (10)

It is an image display system around the vehicle that displays the image around the vehicle synchronized with the progress of the vehicle,
The video display system around the vehicle,
A photographing means disposed on a vehicle to photograph the surroundings of the vehicle;
Image storage means for preserving the image captured by said photographing means;
Image processing means for image processing the photographed image;
And image display means for displaying the image processed image,
The image processing means generates an image around the vehicle that supplements a rectangular portion of the current image photographed by the photographing means with a history image generated from an image previously photographed, and
The image display means superimposes and displays the transmitted image of the own vehicle stored in the image storage means on the generated vehicle surrounding image.
According to claim 1,
The photographing means includes one or more cameras capable of photographing the rear or front periphery of the subject vehicle, and transmits an image photographed by the camera to the image processing means,
The image processing means converts the received image into a continuous sub-frame image, and synthesizes the converted sub-image to generate a history image.
According to claim 2,
The vehicle surrounding image display system further includes a host vehicle position estimation means for calculating a direction, a moving direction, and a moving distance of the subject vehicle based on the captured image,
The image processing means, whenever the moving distance of the vehicle reaches a predetermined moving distance, based on the calculated direction, moving direction, and moving distance of the subject vehicle, the captured current image and before the current The image display system around the vehicle, characterized in that by adjusting the composite position with the image captured in the image for each frame, the history image is updated.
The method of claim 3,
The image processing means combines the captured current image with the generated historical image in accordance with the progress of the vehicle, and when there is a portion different from the historical image in the captured current image, A vehicle surrounding image display system, characterized by generating an image around the vehicle with the image highlighted.
The method of claim 3,
The image processing means, according to the progress of the vehicle, based on the direction, movement direction, and movement distance of the subject vehicle calculated by the host vehicle position estimation means, the captured current image to be synthesized and before the current image taken A video display system around a vehicle, characterized in that the cropped image is cropped or an image synthesis position is set.
It is a method to display the image around the vehicle synchronized with the progress of the vehicle,
A photographing means arranged on a vehicle to photograph the periphery of the vehicle, image storage means for preserving the image captured by the photographing means, image processing means for image processing the captured image, and displaying the image processed image In the video display system around the vehicle having a video display means,
The image processing means,
Receiving an image photographed by the photographing means, converting the received image into a sub-frame image in a continuous frame, and synthesizing the converted sub-image to generate a history image;
A step of generating an image around the vehicle that supplements the rectangular portion of the current image photographed by the photographing means with the generated history image;
And superimposing and displaying the transmitted image of the own vehicle stored in the image storage means on the generated vehicle surrounding image.
The method of claim 6,
The step of generating the history image includes extracting a feature point from the sub-frame image of the continuous frame unit and adjusting a composite position to match the extracted feature point.
The method of claim 7,
The video display method around the vehicle,
The image processing means further includes a step of calculating a direction, a moving direction, and a moving distance of the subject vehicle based on the captured image,
The step of generating the history image is based on the calculated direction, moving direction, and moving distance of the subject vehicle whenever the calculated moving distance of the vehicle reaches a predetermined moving distance. And adjusting a composite position of an image and an image captured before the current for each frame to update the history image.
The method of claim 8,
The step of superimposing and displaying the transmitted image is a combination of the captured current image and the generated historical image in accordance with the progress of the vehicle, and when the captured current image has a different portion from the historical image And a step of emphasizing and displaying the image of the other part.
The method of claim 8,
The step of generating an image around the vehicle is based on the calculated direction of the vehicle, the moving direction, and the moving distance, in accordance with the progress of the vehicle, the captured current image being synthesized and previously taken before the current And cropping an image, or setting a location for synthesizing the image.

Images