WO2018020936A1

WO2018020936A1 - Vehicle periphery monitoring device, and method for monitoring periphery of vehicle

Info

Publication number: WO2018020936A1
Application number: PCT/JP2017/023559
Authority: WO
Inventors: 近藤　大輔; 恒山口
Original assignee: カルソニックカンセイ株式会社
Priority date: 2016-07-28
Filing date: 2017-06-27
Publication date: 2018-02-01
Also published as: JP2018019250A; DE112017003775T5; CN109479122A; JP6691012B2

Abstract

The principal objective of the present invention is to make it possible to alleviate flicker resulting from the scenery around a vehicle, for example. The present invention relates to a vehicle periphery monitoring device (1) provided with: an image capturing unit (4) capable of capturing an image (3) of the periphery of the vehicle (2); an image processing unit (5) which subjects the image (3) captured by the image capturing unit (4) to image processing; and a display unit (7) which displays an image (6) which has been subjected to image processing by the image processing unit (5) (control unit). The image processing unit (5) is provided with a flicker correcting unit (22) which is capable of carrying out correction if the image (3) captured by the image capturing unit (4) contains flicker, in such a way as to alleviate flicker in the image (6) displayed by the display unit (7). The flicker correcting unit (22) determines that a displacement relative to the vehicle (2) of regularly arranged objects (24) in the image (3) captured by the image capturing unit (4) is flicker.

Description

Vehicle periphery monitoring device and vehicle periphery monitoring method

This case relates to a vehicle periphery monitoring device and a vehicle periphery monitoring method.

In recent years, development of a vehicle periphery monitoring device in which an image around the host vehicle is taken with a camera mounted on the vehicle and displayed on a display unit has been underway (for example, see Patent Document 1). Such a vehicle periphery monitoring device includes, for example, a side mirror monitor.

In the vehicle periphery monitoring device described in Patent Document 1, the boundary information indicating the boundary between the host vehicle and the surrounding scenery is used to perform a process for clarifying the boundary.

JP 2014-116756 A

However, in the vehicle periphery monitoring device described in Patent Document 1, since the boundary between the own vehicle and the surrounding scenery is only clarified, for example, flicker due to the scenery around the own vehicle cannot be eliminated. It was.

Therefore, the main purpose of this case is to solve the above-mentioned problems.

In order to solve the above problem,
An imaging unit capable of capturing an image of the surroundings of the own vehicle;
An image processing unit that performs image processing on an image captured by the imaging unit;
A display unit for displaying an image processed by the image processing unit,
The image processing unit includes a flicker correction unit that can be corrected so that flicker of an image displayed on the display unit is reduced when the image captured by the imaging unit has flicker.
The flicker correction unit is characterized by a vehicle periphery monitoring device that determines that the relative displacement of the regularly arranged object with the host vehicle in the image captured by the imaging unit is flicker. Further, the present invention is characterized by a vehicle periphery monitoring method using the vehicle periphery monitoring device.

According to this case, flicker due to scenery around the host vehicle can be reduced by the above configuration.

1 is an overall plan view of a vehicle on which a vehicle periphery monitoring device according to an embodiment is mounted. It is a block diagram of the periphery monitoring apparatus for vehicles of FIG. It is a block diagram of the flicker correction | amendment part of FIG. It is a figure which shows an example of the image image | photographed with the imaging part. FIG. 5 is a diagram illustrating a state in which the image in FIG. 4 is converted so that an object can be detected by an object detection unit. It is a figure which shows the state which frequency-converted the data of FIG. 5 in the frequency conversion part. It is a figure which shows an example of the image from which flicker was reduced. It is a figure which shows the mode of the process (with respect to each square) in an object detection part. It is a figure which shows the state of the data processed by the object detection part. It is a figure which shows the process with respect to the waveform frequency-converted by the frequency conversion part, and this waveform. It is an original image in the specific example in case the frame of an iron bridge looks flickering. It is a figure which shows the mode of the detection of the object in the specific example in case the frame of an iron bridge looks flickering. It is an original image in a specific example in the case where a point mark appears to flicker. It is a figure which shows the mode of the detection of the object in the specific example in case a placemark looks flickering. This is an image in which flickering is reduced in a specific example in which the point mark appears to flicker. It is an original image in the specific example in the case where the illuminations arranged at almost equal intervals in the tunnel appear to flicker. It is a figure which shows the mode of the detection of the object in the specific example in case the illumination arranged in a tunnel at substantially equal intervals is seen flickering. It is an image in which flickering is reduced in a specific example in the case where illuminations arranged at almost equal intervals in a tunnel appear to flicker. It is a flowchart showing the process in a flicker correction | amendment part.

Hereinafter, the present embodiment will be described in detail with reference to the drawings.
1 to 14 are for explaining this embodiment.

<Configuration> The configuration of this embodiment will be described below.

• Provide a vehicle periphery monitoring device that captures an image of the surroundings of the vehicle with a camera mounted on the vehicle and displays the image on the display unit. This vehicle periphery monitoring device is configured as, for example, a side mirror monitor.

Specifically, as shown in FIGS. 1 and 2, the vehicle periphery monitoring device 1 includes an imaging unit 4 that can capture an image 3 around the host vehicle 2, and
An image processing unit 5 that performs image processing on the image 3 captured by the imaging unit 4;
It is assumed that the image processing unit 5 includes a display unit 7 that displays an image 6 processed by the image processing unit 5.

Here, the imaging unit 4 is a monitor camera or the like provided on the left and right of the host vehicle 2. One image pickup unit 4 is installed in the front edge portion of the front door 11 of the host vehicle 2 at a position near the boundary portion between the door body 12 and the front window glass 13 substantially toward the rear of the vehicle. . Therefore, the image 3 photographed by the imaging unit 4 is the one that reflects the rear of the vehicle from the left and right sides of the host vehicle 2. The image processing unit 5 is a (calculation) control unit such as a computer having an image processing function.

The display unit 7 is a display panel such as a monitor (liquid crystal panel or organic EL panel) provided in a pair of left and right according to a pair of left and right monitor cameras. The pair of left and right display units 7 are installed in a part of an instrument panel provided in front of the driver's seat (for example, a position in front of the driver's seat). For this reason, the image 6 behind the vehicle by the display unit 7 is viewed at a position close to the center in the field of view of the driver who is facing the front and is located outside the field of view. It is much easier to see than the mirror image of the existing side mirror, and it is likely to become an obstacle in some situations. However, the number of installed display units 7 and the installation positions are not limited to the above.

In contrast to the basic configuration as described above, this embodiment has the following configuration.

(1) As shown in FIG. 2 (FIG. 3), the image processing unit 5 reduces the flickering of the image 6 displayed on the display unit 7 when the image 3 captured by the imaging unit 4 has flickering. Thus, a flicker correcting unit 22 that can be corrected is provided.
For example, the flicker correcting unit 22 determines that the relative displacement of the regularly arranged object 24 with the host vehicle 2 in the image 3 (see FIG. 4) photographed by the imaging unit 4 is flickering.

Here, “flicker” refers to, for example, an image 3 captured by the imaging unit 4 that is reflected in a state where light and dark are alternately repeated. In this case, the main object is flickering by the object 24, in particular, repeated light and darkness by the regularly arranged objects 24. In addition, when the difference between light and dark is sufficiently small, even the regularly arranged object 24 may not feel flickering.

As the regularly arranged objects 24, for example, as shown in FIG. 4, a framework 24a of an iron bridge, implantations planted at almost equal intervals on both sides of an expressway, and the like can be considered. Other examples will be described later. The regularly arranged objects 24 tend to flicker more as the distance between the objects 24 is narrower.

The relative displacement includes, for example, a case where the object 24 is stationary and the own vehicle 2 is moving, a case where the object 24 is moving and the own vehicle 2 is stationary, and a case where both the object 24 and the own vehicle 2 are moving. And are included. The higher the relative displacement speed, the greater the flicker.

(2) The flicker correcting unit 22 blurs the

flicker portions

31 and 32 in the image 3 as shown in FIG. 4 (as shown in FIG. 7 through the processes shown in FIG. 5 and FIG. 6). The flickers displayed on the display unit 7 are reduced (blur units 35 and 36).

Here, the flickering

portions

31 and 32 are mainly directed to the direct object (flickering portion 31) by the object 24 in the image 3, but depending on the situation, the image of the object 24 reflected on the front window glass 13. Indirect (such as a flickering portion 32) due to the above, an image of the object 24 reflected on the door body 12, and the like can also be included. Whether to include indirect or not can be determined by the magnitude of the difference between light and dark.

For “blur”, for example, it is conceivable to apply an effect (blurring process) to the image 3 such as motion blur (process for making an afterimage of movement), transparency and low contrast. However, the blurring process is not limited to the above.

(3) As shown in FIG. 3, the flicker correction unit 22 includes a sampling unit 41 that samples the image 3 captured by the imaging unit 4;
An object detection unit 42 (see FIGS. 5, 8, and 9) that detects an object 24 that is relatively displaced with respect to the host vehicle 2 from the image 3 sampled by the sampling unit 41;
A frequency conversion unit 43 (see FIG. 10) that converts the frequency of the data 42a used to detect the object 24 that is relatively displaced by the object detection unit 42;
A flicker determination unit 45 that determines flicker when the frequency component that is uncomfortable in the data 43a frequency-converted by the frequency conversion unit 43 exceeds a preset threshold value 44 (see FIGS. 6 and 10). And.

Here, the sampling unit 41, the object detection unit 42, the frequency conversion unit 43, and the flicker determination unit 45 may be configured as functional blocks on software in the image processing unit 5. The sampling unit 41 samples the image 3 for about several seconds. For example, in the case of the imaging unit 4 capable of capturing 15 frames per second, 45 frames of the image 3 are sampled in 3 seconds. However, the number of frames taken by the imaging unit 4 and the sampling time and number of images 3 are not limited to the above.

周波数 Frequency components that are uncomfortable are mainly high frequency components. In FIG. 6, the white portion is a frequency component that makes the user feel uncomfortable, and the black portion is the other low-frequency component. Further, FIG. 10 shows a threshold value 44 for cutting the other low frequency side components to make only the frequency components that feel uncomfortable.

The threshold value 44 can be appropriately adjusted while referring to the final image 6 so that a sense of incongruity does not occur.

The flicker determination unit 45 determines that there is flicker in the data 43a in FIG. When it is determined that there is flicker, the flicker correction unit 22 performs a blurring process on the

flicker portions

31 and 32.

(4) More specifically, the object detection unit 42 detects the object 24 that is relatively displaced with respect to the host vehicle 2 by performing discrete cosine transformation on the image 3 sampled by the sampling unit 41.
Further, the frequency conversion unit 43 performs frequency conversion on the data 42a used for detection of the object 24 that is relatively displaced by the object detection unit 42 using Fourier transform.

Here, as shown in FIG. 5, the image 3 sampled by the sampling unit 41 is first meshed and divided into fine vertical and horizontal grids 3a for each frame. In this state, the discrete cosine transform is performed by the object detection unit 42 for each square 3a. The size of the grid 3 a can be arbitrarily set according to the processing capability of the image processing unit 5 and the resolution required for the display unit 7.

Discrete cosine transform is one of transform methods for transforming discrete signals into the frequency domain. The discrete cosine transform is widely used for data compression and the like because it can reduce the data capacity without losing most of the original signal. The discrete cosine transform is characterized in that the frequency components of the signal after conversion are concentrated in a low frequency region. In this case, the discrete cosine transform as described above is used to detect the object 24 that is displaced relative to the host vehicle 2.

More specifically, when discrete cosine transform is performed for each square 3a, a state as shown in FIG. 8 is obtained. This is black and white so that the vertical direction (vertical direction) is the vertical axis of image 3 and the horizontal direction (horizontal direction) is the horizontal axis of image 3, the upper left is a low frequency component and the lower right is a high frequency component. It will be as shown. Of these, the changing object 24 is represented as a white portion. And the white part becomes large, so that the change of the object 24 is large.

Then, when the cells 3a at the same position are arranged in time series and the area of the white portion of each cell 3a is obtained, a graph as shown in FIG. 9 is obtained. For example, by checking whether or not the data 42a of this graph shows a waveform change (state (1) and state repetition (2)) that expands and contracts up and down, an object arranged regularly Whether there is a relative displacement by 24 can be obtained. In the waveform of this data 42a, it can be seen that the regularly arranged objects 24 are shown by the periodic peaks appearing.

Next, the waveform data 42a thus obtained is Fourier-transformed by the frequency converter 43 to obtain waveform data 43a as shown in FIG. The Fourier transform is a conversion technique for converting a certain waveform so as to be represented by superposition of sine waves.

These processes are performed for each square 3a. Specifically, such processing is not limited to the case of the steel bridge framework 24a as described above (see FIGS. 11A and 11B), for example, the point mark 24b as shown in FIGS. 12A to 12C. This is also effective in the case of the illumination 24c arranged at almost equal intervals in the tunnel as shown in FIGS. 13A to 13C.

After that, the flicker determination unit 45 performs the blurring process.

(5) Hereinafter, a vehicle periphery monitoring method using the above-described vehicle periphery monitoring device will be described.
In this vehicle periphery monitoring method, the imaging unit 4 captures an image 3 around the host vehicle 2,
The image processing unit 5 (control unit) performs image processing on the image 3 captured by the imaging unit 4,
The display unit 7 displays the image 6 processed by the image processing unit 5.
Then, when the image 3 captured by the imaging unit 4 has flickering, the image processing unit 5 corrects the flickering of the image 6 displayed on the display unit 7 to be reduced.
At this time, the flicker correcting unit 22 determines that the relative displacement of the regularly arranged object 24 with the host vehicle 2 in the image 3 captured by the imaging unit 4 is flickering.

(6) The flicker correction unit 22 reduces the flicker displayed on the display unit 7 by blurring the

flicker portions

31 and 32 of the image 3.

(7) The flicker correction unit 22 samples the image 3 captured by the imaging unit 4 by the sampling unit 41,
The object detection unit 42 detects the object 24 that is displaced relative to the host vehicle 2 from the image 3 sampled by the sampling unit 41,
The frequency conversion unit 43 converts the frequency of the data used to detect the object 24 that is relatively displaced by the object detection unit 42, and
The flicker determination unit 45 determines that flickering occurs when the frequency component that is felt uncomfortable in the data frequency-converted by the frequency conversion unit 43 exceeds a preset threshold value 44.

Specifically, as shown in the flowchart of FIG. 14, in step S1, image data (image 3) of the camera (imaging unit 4) is acquired,
In step S2, the sampling unit 41 samples for several frames,
In step S3, the object detection unit 42 extracts an object (object 24) shown at regular intervals,
If there are no objects appearing at regular intervals in step S4, the screen is directly output in step S6.
If there are objects appearing at equal intervals, an effect (blurring process) is applied to the portions appearing at equal intervals in step S5.
In step S6, the screen is output, and the process returns to the beginning and the above processing is repeated.

(8) The object detection unit 42 detects the object 24 that is relatively displaced with respect to the host vehicle 2 by performing discrete cosine transform on the image 3 sampled by the sampling unit 41,
The frequency conversion unit 43 performs frequency conversion on the data used for detection of the object 24 that is relatively displaced by the object detection unit 42 using Fourier transform.

<Action and effect> According to this embodiment, the following action and effect can be obtained.

The vehicle periphery monitoring device captures an image around the host vehicle with a camera mounted on the vehicle and displays the image on the display unit. The vehicle periphery monitoring device is, for example, a side mirror monitor.

Specifically, the imaging unit 4 captures an image 3 around the host vehicle 2, the image processing unit 5 performs image processing on the image 3 captured by the imaging unit 4, and the display unit 7 performs image processing on the image processing unit 5. The image-processed image 6 is displayed.

(Operation Effect 1) The image processing unit 5 can reduce the flicker of the image 6 displayed on the display unit 7 by the flicker correction unit 22 when the image 3 photographed by the imaging unit 4 has flicker. it can. Thereby, the part which looks troublesome due to flickering can be partially eliminated (or reduced) from the display of the display unit 7, and for example, the display unit 7 is arranged at a position where it is easily noticeable within the visual field range. Even in such a case, the troublesomeness caused by flickering can be eliminated or reduced. You can concentrate on driving.

At this time, the flicker correcting unit 22 determines that the relative displacement of the regularly arranged external object 24 in the image 3 photographed by the image capturing unit 4 with the host vehicle 2 is flickering. As a result, while traveling, for example, the framework 24a of the iron bridge, the planting planted at approximately equal intervals on both sides of the highway, the landmark 24b, and the lighting 24c arranged at approximately equal intervals in the tunnel, etc. It is possible to correct flickering due to the object 24 constituting the scenery around the vehicle 2.

Further, for example, flickering by a train or a freight car passing near the own vehicle 2 while the vehicle is stopped can be corrected by the above.

(Operation Effect 2) The flicker correcting unit 22 causes the flickering

portions

31 and 32 of the image 3 to be blurred and displayed on the display unit 7. Thereby, the flicker displayed on the display part 7 can be reduced reliably. The image 3 can be blurred by motion blur (processing for making an afterimage of motion), transparency, low contrast, or the like. By performing these blurring processes, display discomfort can be reduced. In addition, it is possible to prevent car sickness caused by continuing to watch the display having flicker.

(Operation Effect 3) In the flicker correction unit 22, the sampling unit 41 samples the image 3 captured by the imaging unit 4, and the object detection unit 42 is relative to the host vehicle 2 from the image 3 sampled by the sampling unit 41. The object 24 that is displaced is detected, the frequency converter 43 converts the frequency of the data used to detect the object 24 that is relatively displaced by the object detector 42, and the flicker determination unit 45 is frequency-converted by the frequency converter 43. When the frequency component in the data that is felt uncomfortable exceeds a preset threshold value 44, it is determined that the flicker is present. As a result, the flicker of the image 3 captured by the imaging unit 4 can be specifically detected and blurred.

(Operation Effect 4) The object detection unit 42 performs the discrete cosine transform on the image 3 sampled by the sampling unit 41 to detect the object 24 that is relatively displaced with respect to the host vehicle 2, and the frequency conversion unit 43 The data used to detect the object 24 that is relatively displaced by the object detection unit 42 is frequency-converted using Fourier transform. Thereby, the flickering object 24 can be actually detected.

DESCRIPTION OF SYMBOLS 1 Vehicle periphery monitoring apparatus 2 Own vehicle 3 Image 4 Imaging part 5 Image processing part 7 Display part 22 Flicker correction part 24 Object regularly arranged 31 Flicker part 32 Flicker part 35 Blur part 36 Blur part 41 Sampling part 42 Object Detection unit 42a Data 43 Frequency conversion unit 43a Data 44 Threshold value 45 Flicker judgment unit

Cross-reference of related applications

This application claims priority based on Japanese Patent Application No. 2016-148058 filed with the Japan Patent Office on July 28, 2016, the entire disclosure of which is fully incorporated herein by reference. .

Claims

An imaging unit capable of capturing an image of the surroundings of the own vehicle;
An image processing unit that performs image processing on an image captured by the imaging unit;
A display unit for displaying an image processed by the image processing unit,
The image processing unit includes a flicker correction unit that can be corrected so that flicker of an image displayed on the display unit is reduced when the image captured by the imaging unit has flicker.
The vehicle flicker monitoring device, wherein the flicker correction unit determines that the relative displacement of the regularly arranged object with the host vehicle in the image captured by the imaging unit is flicker.
The vehicle periphery monitoring device according to claim 1,
The vehicle periphery monitoring device, wherein the flicker correction unit is configured to reduce the flicker displayed on the display unit by blurring a flicker portion of an image.
The vehicle periphery monitoring device according to claim 1,
The flicker correction unit is a sampling unit that samples an image captured by the imaging unit;
An object detection unit that detects an object that is relatively displaced with respect to the host vehicle from the image sampled by the sampling unit;
A frequency converter that converts the frequency of the data used to detect the object that is relatively displaced by the object detector;
A vehicle periphery monitoring device, comprising: a flicker determination unit that determines flicker when a frequency component that is uncomfortable in data frequency-converted by the frequency conversion unit exceeds a preset threshold value .
In the vehicle periphery monitoring device according to claim 3,
The object detection unit is configured to detect an object that is relatively displaced with respect to the host vehicle by performing discrete cosine transform on the image sampled by the sampling unit,
The vehicle periphery monitoring device, wherein the frequency converter is configured to frequency-convert data used for detection of an object that is relatively displaced by the object detector using Fourier transform.
Take an image of the area around the vehicle with the imaging unit,
The image processing unit performs image processing on the image captured by the imaging unit,
The display unit displays the image processed by the image processing unit,
The image processing unit corrects the flicker correction unit so that the flicker of the image displayed on the display unit is reduced when the image captured by the imaging unit has flicker.
The vehicle flicker correction method, wherein the flicker correction unit determines that the relative displacement of the regularly arranged object with the host vehicle in the image captured by the imaging unit is flicker.
The vehicle periphery monitoring method according to claim 5, wherein
The vehicle flicker monitoring method, wherein the flicker correction unit reduces flicker displayed on the display unit by blurring a flicker portion of an image.
The vehicle periphery monitoring method according to claim 5, wherein
The flicker correction unit samples the image taken by the imaging unit by the sampling unit,
The object detection unit detects an object that is displaced relative to the host vehicle from the image sampled by the sampling unit,
The frequency conversion unit converts the frequency of the data used to detect the object that is relatively displaced by the object detection unit,
A vehicle periphery monitoring method, wherein a flicker judgment unit judges flicker when a frequency component that is felt uncomfortable in data frequency-converted by the frequency conversion unit exceeds a preset threshold value.
The vehicle periphery monitoring method according to claim 7,
The object detection unit detects an object relatively displaced with respect to the host vehicle by performing discrete cosine transform on the image sampled by the sampling unit,
The method of monitoring a vehicle periphery, wherein the frequency conversion unit frequency-converts data used for detection of an object that is relatively displaced by the object detection unit using Fourier transform.