WO2016151978A1

WO2016151978A1 - Image processing device, driving assistance system, and image processing method

Info

Publication number: WO2016151978A1
Application number: PCT/JP2016/000141
Authority: WO
Inventors: 哲郎奥山
Original assignee: パナソニックＩｐマネジメント株式会社
Priority date: 2015-03-26
Filing date: 2016-01-13
Publication date: 2016-09-29

Abstract

Provided is an image processing device capable of assisting in the driving of a vehicle while having a simple configuration. This image processing device (30), which improves the image quality of a moving image obtained by imaging with an imaging device (20), is equipped with: an approach region detection unit (32) for analyzing a moving image and thereby detecting an approach region, which is a region showing an object in the moving image approaching the imaging device; and an image quality changing unit (34) for improving the image quality of the image of the approach region detected by the approach region detection unit (32).

Description

Image processing apparatus, driving support system, and image processing method

The present disclosure relates to an image processing device, a driving support system, and an image processing method, and more particularly, to image processing for adaptively changing the image quality of a moving image obtained by photographing with an imaging device.

Conventionally, various technologies that support driving of a vehicle using image processing technology have been proposed (see, for example, Patent Document 1).

In Patent Document 1, it is adapted based on the relative speed information of a vehicle obtained from a distance sensor, object information obtained from edge information in an image obtained from an imaging device (camera), and environmental information (weather information such as rain and snow). Image processing. This makes it possible to quickly analyze the situation and perform image recognition accurately even in bad weather or at night.

JP 2013-186872 A

However, in Patent Document 1, there is a problem that special hardware such as a distance sensor and a device for acquiring environmental information is required in addition to an image pickup device generally installed in a car.

Therefore, the present disclosure provides an image processing device that can support driving of a vehicle with a simple configuration, a driving support system including the image processing device, and an image processing method.

One form of the image processing device according to the present disclosure is an image processing device that improves the image quality of a moving image obtained by photographing with an imaging device, and the imaging device in the moving image is analyzed by analyzing the moving image. An approach area detecting unit that detects an approach area that is an area that indicates an object approaching the camera, and an image quality changing unit that improves the image quality of the image of the approach area detected by the approach area detecting unit.

In addition, an aspect of the driving support system according to the present disclosure includes an imaging device used as an in-vehicle rear view camera, the image processing device that improves a moving image obtained by photographing with the imaging device, and the image processing. And a display device for displaying a moving image with high image quality.

In addition, an aspect of the image processing method according to the present disclosure is an image processing method for changing a quality of a moving image obtained by photographing with an imaging device, and the moving image is analyzed to analyze the moving image in the moving image. An approach area detecting step for detecting an approach area that is an area indicating an object approaching the imaging apparatus; and an image quality changing step for changing an image quality of an image in the approach area detected in the approach area detecting step.

The image processing apparatus, the driving support system including the image processing apparatus, and the image processing method according to the present disclosure can support driving of a vehicle with a simple configuration.

FIG. 1 is a diagram illustrating an overview of a driving support system according to an embodiment. 2 is a block diagram showing a detailed configuration of the image processing apparatus shown in FIG. FIG. 3 is a diagram showing an example of the weighting coefficient table shown in FIG. FIG. 4 is a flowchart showing a basic operation of the driving support system in the embodiment. FIG. 5 is a flowchart showing a detailed operation of the driving support system in the embodiment. FIG. 6 is a diagram illustrating a vanishing point detection method by the vanishing point detection unit of the image processing apparatus according to the embodiment. FIG. 7 is a diagram illustrating a motion vector detection method by the motion vector detection unit of the image processing apparatus according to the embodiment. FIG. 8 is a diagram illustrating an example of an image captured by the imaging device and an image displayed on the display device according to the embodiment.

Hereinafter, embodiments will be described in detail with reference to the drawings as appropriate. However, more detailed description than necessary may be omitted. For example, detailed descriptions of already well-known matters and repeated descriptions for substantially the same configuration may be omitted. This is to avoid the following description from becoming unnecessarily redundant and to facilitate understanding by those skilled in the art.

In addition, the inventors provide the accompanying drawings and the following description in order for those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter described in the claims. Absent. That is, each of the embodiments described below shows a preferable specific example of the present disclosure. Numerical values, components, arrangement positions and connection forms of components, steps, order of steps, and the like shown in the following embodiments are merely examples, and are not intended to limit the technology in the present disclosure. In addition, among the constituent elements in the following embodiments, constituent elements that are not described in the independent claims indicating the highest concept of the present disclosure are described as optional constituent elements that constitute a more preferable embodiment.

(Embodiment)
[1. Constitution]
FIG. 1 is a diagram illustrating an overview of a driving support system 10 according to an embodiment. The driving support system 10 is an in-vehicle system that supports driving of a vehicle using image processing, and includes an imaging device 20, an image processing device 30, and a display device 40.

The imaging device 20 is used as an in-vehicle rear view camera and captures a moving image. The imaging device 20 is, for example, a camera that is attached to the ceiling of the car 12 and photographs the back of the car 12 through a rear window. Note that the imaging device 20 is not limited to being installed in the interior of the car 12, and may be installed in any place (for example, the rear body of the car, the rear bumper, etc.) as long as the rear of the car 12 can be photographed. Also good.

The image processing device 30 improves the quality of a moving image obtained by photographing with the imaging device 20. The image processing device 30 is realized by, for example, a microcomputer including a program, a memory, and a processor, and is attached to the rear surface of the rear view mirror. Here, the improvement in image quality is to improve the quality of an image.

The display device 40 displays the moving image whose image quality has been improved by the image processing device 30. The display device 40 is, for example, an LCD used as a rear view mirror.

FIG. 2 is a block diagram showing a detailed configuration of the image processing apparatus 30 shown in FIG. The image processing apparatus 30 includes an approach region detection unit 32, an image quality change unit 34, a weight calculation unit 35, and a storage unit 36.

The approaching area detection unit 32 analyzes a moving image obtained by photographing with the imaging device 20 to detect an approaching region that is an area indicating an object approaching the imaging device 20 in the moving image. The approach area detection unit 32 includes a vanishing point detection unit 32a, a motion vector detection unit 32b, and a relative speed detection unit 32c.

The vanishing point detection unit 32a moves with respect to the moving image obtained by the imaging device 20 as the object gradually appears smaller in the moving image when the object appearing in the moving image moves away from the imaging device 20. A vanishing point indicating the tip is detected.

The motion vector detection unit 32b detects a motion vector in block units from the moving image obtained by the imaging device 20. A block is a group of one or more pixels, for example, a group of 8 × 8 pixels. The motion vector detecting unit 32b calculates a motion vector indicating how much each block in the frame is moving in which direction by performing block matching between the frames constituting the moving image. In block matching, the current frame and the previous frame are evaluated using a distance function such as calculating an absolute value error or a square error between pixel values of pixels at the same relative position constituting the block. Search for the block that best matches.

Due to the vanishing point detection unit 32a and the motion vector detection unit 32b, the approach region detection unit 32 moves the direction in which the motion vector detected by the motion vector detection unit 32b moves away from the vanishing point detected by the vanishing point detection unit 32a. If it is facing, a block having the motion vector is detected as an approaching area.

In the present embodiment, the approach region detection unit 32 analyzes not only the approach region but also the moving image obtained by the imaging device 20, thereby indicating an object that moves away from the imaging device 20 in the moving image. The separation area which is is also detected. More specifically, the approach area detection unit 32 has a block having a motion vector when the motion vector detected by the motion vector detection unit 32b faces the vanishing point detected by the vanishing point detection unit 32a. Is detected as a separation region.

The relative velocity detection unit 32c determines whether the motion vector detected by the motion vector detection unit 32b is facing away from or approaching the vanishing point detected by the vanishing point detection unit 32a. Based on the determination result and the magnitude of the motion vector, the relative speed detection unit 32c detects an approaching relative speed that is a relative speed at which the object included in the block having the motion vector approaches the imaging device 20. In the present embodiment, the relative speed detection unit 32c faces not only the approaching relative speed but also the direction in which the motion vector detected by the motion vector detection unit 32b moves away from the vanishing point detected by the vanishing point detection unit 32a. Judging whether it is facing or approaching. Based on the determination result and the magnitude of the motion vector, the relative speed detection unit 32c also detects a separation relative speed that is a relative speed away from the imaging device 20 of the object included in the block having the motion vector.

The storage unit 36 holds in advance a weighting coefficient table 36a indicating weighting coefficients corresponding to spatial positions in a moving image obtained by the imaging device 20. The storage unit 36 is, for example, a nonvolatile memory. As shown in FIG. 3, the weighting factor constituting the weighting factor table 36a is set to a value having a larger weight as the weighting factor is located lower in the moving image space. FIG. 3 is a diagram illustrating an example of the weighting coefficient table 36a. Here, the space in the moving image is divided into three regions in the vertical direction, and the weighting factor is set so that the lower region has a larger weighting factor.

For the approach relative speed detected by the relative speed detector 32c, the weight calculator 35 refers to the weight coefficient table 36a stored in the storage unit 36 for the weight coefficient corresponding to the block having the approach relative speed. It reads out from the memory | storage part 36, and weights an approach relative speed with the read weighting coefficient.

The image quality changing unit 34 adaptively increases the image quality of the image of the approaching area detected by the approaching area detection unit 32. More specifically, the image quality changing unit 34 increases the degree of image quality improvement for blocks with a higher approaching relative speed detected by the relative speed detecting unit 32c. More specifically, the image quality changing unit 34 increases the degree of image quality improvement for blocks with a higher approach relative speed weighted by the weight calculation unit 35. High image quality includes, for example, at least one of image processing such as increasing image sharpness, increasing contrast, and reducing noise reduction.

In the present embodiment, the image quality changing unit 34 adaptively lowers the image quality of the separation region detected by the approaching region detection unit 32 as well as improving the image quality. Here, the reduction in image quality means lowering the image quality. More specifically, the image quality changing unit 34 increases the degree of lowering the image quality for blocks with a higher separation relative speed detected by the relative speed detection unit 32c. More specifically, the image quality changing unit 34 increases the degree of lowering the image quality as the separation relative speed weighted by the weight calculating unit 35 increases. The reduction in image quality includes, for example, at least one of image processing such as reducing the sharpness of the image, reducing the contrast, and increasing noise reduction.

The moving image whose image quality has been changed by the image quality changing unit 34 is sent to the display device 40 and displayed.

The approach area detection unit 32, the image quality change unit 34, and the weight calculation unit 35 constituting the image processing apparatus 30 may be realized by software by a program executed by a processor, or a dedicated electronic circuit such as an LSI. It may be realized by hardware.

[2. Operation]
Next, operation | movement of the driving assistance system 10 in this Embodiment comprised as mentioned above is demonstrated.

[2-1. basic action]
FIG. 4 is a flowchart showing a basic operation (mainly an image processing method by the image processing device 30) of the driving support system 10 in the present embodiment. FIG. 4 shows image processing for the approaching area in the moving image (FIG. 4A) and image processing for the separation area (FIG. 4B). The following processing is performed for each frame constituting a moving image obtained by photographing by the imaging device 20.

As shown in FIG. 4A, the approach region detection unit 32 analyzes a moving image obtained by photographing with the imaging device 20, thereby detecting an object approaching the imaging device 20 in the moving image. The indicated area is detected as an approach area (approach area detection step S10).

Then, the image quality changing unit 34 increases the image quality of the image of the approaching area detected by the approaching area detecting unit 32 (image quality changing (image quality improving) step S12). In high image quality, at least one of image processing such as increasing the sharpness of the image, increasing the contrast, and decreasing the noise reduction is performed on the approaching area.

Further, as shown in FIG. 4B, the approach region detection unit 32 analyzes the moving image obtained by the imaging device 20 as well as the approach region, thereby moving away from the imaging device 20 in the moving image. A region indicating a moving object is detected as a separation region (separation region detection step S14).

Then, the image quality changing unit 34 lowers the image quality of the image of the separation area detected by the approaching area detecting unit 32 (image quality changing (lowering image quality) step S16). In the reduction in image quality, at least one of image processing, such as weakening the sharpness of the image, weakening the contrast, and applying strong noise reduction, is performed on the separation region.

As described above, in the driving support system 10 according to the present embodiment, a moving image in which an approach region including an approaching object is adaptively improved in image quality and a high-risk image region is clarified by image processing. An image is obtained. Further, the image processing adaptively lowers the image quality of the separation region including the moving object, and a moving image in which the image region having a relatively high risk is sharpened is obtained. Therefore, driving of a vehicle can be supported with a simple configuration without using special hardware such as a distance sensor or a device for acquiring environmental information as in the prior art.

[2-2. Detailed operation]
FIG. 5 is a flowchart showing a detailed operation (mainly an image processing method by the image processing device 30) of the driving support system 10 in the present embodiment. Here, a detailed procedure of the operation shown in FIG. 4 is shown. The following processing is performed for each frame constituting a moving image obtained by photographing by the imaging device 20.

First, the vanishing point detection unit 32a detects the vanishing point from the moving image obtained by the imaging device 20 (S20). FIG. 6 is a diagram illustrating a vanishing point detection method by the vanishing point detection unit 32a. As shown in FIG. 6A, the vanishing point detection unit 32a detects a perspective projection line 42 in the frame by edge detection or the like with respect to the frame 41, and a point 43 where the perspective projection line 42 converges is taken as a vanishing point. To detect. This is based on the fact that a stationary object moves toward a certain point (disappearance point) in a moving image taken from behind a car. By such detection, as shown in FIG. 6B, the perspective projection straight line and the vanishing point are detected depending on the moving direction of the imaging device 20 (the traveling direction of the car). Therefore, as shown in FIG. 6B, the vanishing point 43 is detected not only when the vehicle is traveling straight, but also when traveling right or left, and subsequent processing is performed. Is done. Further, as shown in FIG. 6B, when the vehicle turns right or left, the vanishing point moves to infinity on the left side and the right side, respectively, and is not detected.

Subsequently, the motion vector detection unit 32b detects a motion vector for each block of the moving image obtained by the imaging device 20 (S21). FIG. 7 is a diagram for explaining a motion vector detection method by the motion vector detection unit 32b. The motion vector detection unit 32b performs a search based on block matching between frames constituting the moving image (between the previous frame 44 and the current frame 45), and how much each block in the frame moves in which direction. Is calculated.

Note that either the vanishing point detection by the vanishing point detection unit 32a (S20) or the motion vector detection by the motion vector detection unit 32b (S21) may be performed first or in parallel. .

Subsequently, when the motion vector detected by the motion vector detection unit 32b is facing away from the vanishing point detected by the vanishing point detection unit 32a, the approaching region detection unit 32 approaches the block having the motion vector. Detect as a region. In addition, when the motion vector detected by the motion vector detection unit 32b faces the vanishing point detected by the vanishing point detection unit 32a, the approaching region detection unit 32 moves the block having the motion vector to the separation region. (S22). Specifically, the approach area detection unit 32 calculates a component in a direction away from (and approaches) the vanishing point detected by the vanishing point detection unit 32a among the motion vectors detected by the motion vector detection unit 32b. When the component is larger than a certain threshold value, a block having the motion vector is detected as an approach region (and a separation region).

Then, the relative velocity detection unit 32c determines whether the motion vector detected by the motion vector detection unit 32b is directed toward or away from the vanishing point detected by the vanishing point detection unit 32a. The relative speed detector 32c detects a relative speed (approaching relative speed and separation relative speed) based on the determination result and the magnitude of the motion vector (S23). Specifically, the relative velocity detection unit 32c moves away from the vanishing point detected by the vanishing point detection unit 32a among the motion vectors detected by the motion vector detection unit 32b for the blocks belonging to the approach region and the separation region ( And a component in the approaching direction are calculated, and the magnitude of the component is detected as the approaching relative speed (and the separation relative speed).

Subsequently, for the blocks belonging to the approach area and the separation area, the weight calculation unit 35 stores, in the storage unit 36, the weighting coefficient corresponding to the block having the relative speed for the relative speed detected by the relative speed detection unit 32c. The relative speed is read out from the weighting coefficient table 36a and the relative speed is weighted with the read weighting coefficient (S24). Specifically, the weight calculation unit 35 multiplies the relative speed detected by the relative speed detection unit 32c by the weight coefficient read from the storage unit 36 for the blocks belonging to the approach area and the separation area.

Finally, the image quality changing unit 34 increases the image quality of blocks belonging to the approaching region more strongly as the approaching relative speed weighted by the weight calculating unit 35 is larger. Further, the image quality changing unit 34 lowers the image quality of blocks belonging to the separation region more strongly as the separation relative speed weighted by the weight calculation unit 35 is larger (S25). For example, the image quality changing unit 34 uses a relative speed (approaching relative) as a filter calculation type (degree of sharpening), a contrast enhancement parameter, or a 3DNR (3-Dimensional Noise Reduction) parameter for sharpening a frame. The image quality can be improved and the image quality can be reduced depending on the relative speed.

FIG. 8 shows an image (FIG. 8A) taken by the imaging device 20, and an image (FIG. 8B) that is processed by the image processing device 30 and displayed on the display device 40. It is a figure which shows an example. As shown in (b) of FIG. 8, the displayed image is a region including an object approaching the car 80 on which the driving support system 10 is mounted, and an image that is a place close to the car 80. In the lower area, the image quality is improved. On the other hand, in the displayed image, the image quality is reduced in an area including an object that is far from the car 80 and an upper area in the image that is a place far from the car 80.

Thereby, in the moving image, an approach region including an approaching object, that is, a high-risk image region is clarified, and appropriate warning is given to the driver who views the display device 40.

[3. Effect]
As described above, the image processing apparatus 30 according to the present embodiment is an image processing apparatus that improves the quality of a moving image obtained by photographing with the imaging device 20, and captures a moving image by analyzing the moving image. An approach area detection unit 32 that detects an approach area that is an area indicating an object approaching the apparatus 20, and an image quality change unit 34 that improves the image quality of the approach area detected by the approach area detection unit 32. .

As a result, the image processing can adaptively improve the image quality of the approaching area containing the approaching object, and a moving image with a sharpened image area can be obtained. And an image processing apparatus that can support driving of a vehicle with a simple configuration without using special hardware such as an apparatus for acquiring environmental information.

In addition, the approach region detection unit 32 indicates a vanishing point that indicates a destination where the object gradually decreases in the moving image and moves when the object appearing in the moving image moves away from the imaging device 20 with respect to the moving image. A vanishing point detecting unit 32a for detecting the motion vector, and a motion vector detecting unit 32b for detecting a motion vector for each moving image block. When the motion vector detected by the motion vector detection unit 32b is facing away from the vanishing point detected by the vanishing point detection unit 32a, the approaching region detection unit 32 detects that the block having the motion vector is the approaching region. Detect that there is.

Thereby, since the approach region is detected by the movement of the object with respect to the vanishing point in the moving image, the approach region including the object approaching the car is surely detected.

Further, the approaching area detection unit 32 further determines whether the motion vector detected by the motion vector detection unit 32b is facing away from or approaching the vanishing point detected by the vanishing point detection unit 32a. . Based on the determination result and the magnitude of the motion vector, the approach area detection unit 32 detects a relative speed detection that detects a relative speed of approaching the imaging device 20 of the object included in the block having the motion vector. It has a part 32c. The image quality changing unit 34 increases the degree of image quality improvement for blocks with a higher approaching relative speed detected by the relative speed detection unit 32c.

As a result, the higher the image quality is, the higher the relative speed of the block that is approaching the car, the higher the speed of the object, and the more dangerous objects are sharpened in the moving image, and the driver is alerted. .

Further, the approach area detection unit 32 further has an approach relative speed with respect to the approach relative speed detected by the storage unit 36 holding the weighting coefficient corresponding to the spatial position in the moving image and the relative speed detection unit 32c. A weight calculation unit 35 that reads the weighting coefficient corresponding to the block from the storage unit 36 and weights the approaching relative speed with the read weighting coefficient is provided. In addition, the image quality changing unit 34 increases the degree of image quality improvement for the block having a higher approach relative speed weighted by the weight calculating unit 35.

As a result, the degree of image quality improvement is adjusted based on the approaching relative speed that is weighted corresponding to the spatial position in the moving image, so that the driver is alerted depending on the spatial position of the moving image. Is adjusted.

Further, the weighting coefficient held in the storage unit 36 is set to a value having a larger weight as the weighting coefficient is located in the lower part of the moving image space.

As a result, the area corresponding to the lower part of the moving image space, that is, the area closer to the car is weighted more heavily toward the approaching relative speed and the image quality is strongly improved. Becomes clearer and alerts the driver.

Further, the approach area detection unit 32 further detects a separation area that is an area indicating an object moving away from the imaging device 20 in the moving image by analyzing the moving image, and the image quality changing unit 34 further performs the approaching. The image quality of the separation area detected by the area detection unit 32 is reduced.

As a result, the image processing can adaptively reduce the image quality of the separation area including the moving away object, and a relatively high-definition image area can be obtained. In addition, an image processing apparatus capable of supporting driving of a vehicle with a simple configuration without using special hardware such as a distance sensor or an apparatus for acquiring environmental information is realized.

In addition, the approaching area detection unit 32, when the motion vector detected by the motion vector detection unit 32b faces the vanishing point detected by the vanishing point detection unit 32a, the block having the motion vector is separated. It is detected as an area.

Thereby, since the separation area is detected by the movement of the object with respect to the vanishing point in the moving image, the approaching area including the object moving away from the vehicle is surely detected.

Further, the approach area detection unit 32 further determines whether the motion vector detected by the motion vector detection unit 32b is facing away from or approaching the vanishing point detected by the vanishing point detection unit 32a. Based on the determination result and the magnitude of the motion vector, a relative speed detection unit 32c that detects a separation relative speed that is a relative speed of the object included in the block having the motion vector away from the imaging device 20 is provided. The image quality changing unit 34 increases the degree of lowering the image quality as the block having a higher separation relative speed detected by the relative speed detecting unit 32c.

As a result, the lower the image quality, the lower the image quality of the blocks that contain objects with a high relative speed moving away from the car, so the more dangerous objects in the moving image become clearer and alert the driver. Is done.

Further, the approach area detection unit 32 further has a separation relative speed with respect to the separation relative speed detected by the storage unit 36 holding the weighting coefficient corresponding to the spatial position in the moving image and the relative speed detection unit 32c. A weight calculation unit 35 that reads the weighting coefficient corresponding to the block from the storage unit 36 and weights the separation relative speed with the read weighting coefficient is provided. The image quality changing unit 34 increases the degree of lowering the image quality as the separation relative speed weighted by the weight calculating unit 35 increases.

As a result, the degree of image quality reduction is adjusted based on the separation relative speed that is weighted corresponding to the spatial position in the moving image, so that the driver is alerted depending on the spatial position of the moving image. Is adjusted.

In addition, the driving support system 10 according to the present embodiment includes an imaging device 20 that is used as an in-vehicle rear view camera, an image processing device 30 that improves the quality of a moving image obtained by imaging by the imaging device 20, and an image processing device. And a display device 40 for displaying a moving image with a high image quality.

As a result, the approach area including the approaching object is adaptively improved in image quality by the image processing, and a moving image with a high-risk image area is clarified is displayed. An image processing method capable of supporting driving of a vehicle with a simple configuration without using special hardware such as a sensor or a device for acquiring environmental information is realized.

Further, the image processing method in the present embodiment is an image processing method for improving the quality of a moving image obtained by photographing with the imaging device 20, and by analyzing the moving image, the moving image is transmitted to the imaging device 20 in the moving image. An approach area detecting step for detecting an approach area, which is an area indicating an approaching object, and an image quality changing step for improving the image quality of the image of the approach area detected in the approach area detecting step.

(Other embodiments)
As described above, the embodiments have been described as examples of the technology in the present disclosure. For this purpose, the accompanying drawings and detailed description are provided. Accordingly, among the components described in the accompanying drawings and the detailed description, not only the components essential for solving the problem, but also the components not essential for solving the problem in order to illustrate the above technique. May also be included. Therefore, it should not be immediately recognized that these non-essential components are essential as those non-essential components are described in the accompanying drawings and detailed description.

In addition, since the above-described embodiment is for illustrating the technique in the present disclosure, various modifications, replacements, additions, omissions, and the like can be performed within the scope of the claims or an equivalent scope thereof. That is, the present disclosure also includes various modifications made by those skilled in the art that are conceivable by those skilled in the art and other forms constructed by combining some components in the embodiments without departing from the gist of the present disclosure. It is included in the range.

For example, in the above embodiment, the image processing apparatus 30 includes the weight calculation unit 35 and the storage unit 36, but these are not necessarily required. Even if these are not provided, the approach region detection unit 32 and the image quality changing unit 34 adaptively improve the image quality of the approach region including the approaching object by the image processing, so that an image region with high risk can be obtained. A sharpened moving image is obtained.

In the above-described embodiment, the image processing device 30 has changed the image quality (higher image quality and lower image quality) for each frame constituting the moving image obtained by the imaging device 20, but the image quality is reduced by thinning out the frames. It may be changed, only one of high image quality and low image quality may be performed, or the frame may be thinned and output to the display device 40.

In addition, the technology in the present disclosure can be realized not only as an image processing apparatus, a driving support system, and an image processing method, but also a program including the image processing method as steps, and a computer-readable recording medium such as a DVD that records the program It can also be realized as.

The present disclosure can be applied as an image processing apparatus that can support driving of a vehicle with a simple configuration, a driving support system including the image processing apparatus, and an image processing method. Specifically, the present disclosure is applicable to an in-vehicle driving support system including a rear view camera, a rear view mirror, and an image processing device.

DESCRIPTION OF SYMBOLS 10 Driving support system 12 Car 20 Imaging device 30 Image processing device 32 Approach area detection part 32a Vanishing point detection part 32b Motion vector detection part 32c Relative speed detection part 34 Image quality change part 35 Weight calculation part 36 Storage part 36a Weight coefficient table 40 Display Equipment 41 Frame 42 Perspective projection line 43 Vanishing point 44 Front frame 45 Current frame 80 Car

Claims

An image processing device for improving the image quality of a moving image obtained by photographing with an imaging device,
By analyzing the moving image, an approach area detecting unit that detects an approach area that is an area indicating an object approaching the imaging device in the moving image;
An image processing apparatus comprising: an image quality changing unit that improves an image quality of an approaching area detected by the approaching area detecting unit.
The approach area detection unit
An erasure inspection that detects a vanishing point indicating a destination where the object gradually decreases in the moving image when the object that appears in the moving image moves away from the imaging device with respect to the moving image. And outing,
A motion vector detection unit that detects a motion vector in block units for the moving image;
The block having the motion vector is detected as the approaching area when the motion vector detected by the motion vector detection unit is facing away from the vanishing point detected by the vanishing point detection unit. The image processing apparatus according to 1.
The approach area detection unit further determines whether the motion vector detected by the motion vector detection unit is facing away from or disappearing from the vanishing point detected by the vanishing point detection unit, and Based on the determination result and the magnitude of the motion vector, a relative speed detection unit that detects an approach relative speed, which is a relative speed of the object included in the block having the motion vector, approaching the imaging device,
The image processing apparatus according to claim 2, wherein the image quality changing unit increases the degree of image quality of a block having a higher approaching relative speed detected by the relative speed detecting unit.
The approach area detection unit further includes:
A storage unit holding a weighting factor corresponding to a spatial position in the moving image;
For the approaching relative speed detected by the relative speed detecting unit, a weighting coefficient corresponding to the block having the approaching relative speed is read from the storage unit, and a weight calculating unit weights the approaching relative speed with the read weighting coefficient. Have
The image processing apparatus according to claim 3, wherein the image quality changing unit increases the degree of image quality for the block having a higher approaching relative speed weighted by the weight calculation unit.
The image processing apparatus according to claim 4, wherein the weighting coefficient held in the storage unit is set to a value having a larger weight as the weighting coefficient is located lower in the moving image space.
The approach area detection unit further detects a separation area that is an area indicating an object moving away from the imaging device in the moving image by analyzing the moving image;
The image processing apparatus according to claim 1, wherein the image quality changing unit further reduces the image quality of the image of the separation region detected by the approach region detecting unit.
The approach area detection unit
An erasure inspection that detects a vanishing point indicating a destination where the object gradually decreases in the moving image when the object that appears in the moving image moves away from the imaging device with respect to the moving image. And outing,
A motion vector detection unit that detects a motion vector in block units for the moving image;
The block having the motion vector is detected as the separation region when the motion vector detected by the motion vector detection unit faces the vanishing point detected by the vanishing point detection unit. 6. The image processing apparatus according to 6.
The approach area detection unit further determines whether the motion vector detected by the motion vector detection unit is facing away from or disappearing from the vanishing point detected by the vanishing point detection unit, and Based on the determination result and the magnitude of the motion vector, a relative speed detection unit that detects a separation relative speed that is a relative speed away from the imaging device of an object included in the block having the motion vector,
The image processing apparatus according to claim 7, wherein the image quality changing unit increases the degree of image quality reduction for a block having a larger separation relative speed detected by the relative speed detection unit.
The approach area detection unit further includes:
A storage unit holding a weighting factor corresponding to a spatial position in the moving image;
With respect to the separation relative speed detected by the relative speed detection unit, a weighting coefficient corresponding to the block having the separation relative speed is read from the storage unit, and a weight calculation unit weights the separation relative speed with the read weighting coefficient. Have
The image processing apparatus according to claim 8, wherein the image quality changing unit increases the degree of image quality reduction for a block having a larger separation relative speed weighted by the weight calculation unit.
An imaging device used as an in-vehicle rear view camera;
The image processing apparatus according to any one of claims 1 to 9, wherein a moving image obtained by photographing by the imaging apparatus is improved in quality.
A driving support system comprising: a display device that displays a moving image with high image quality by the image processing device.
An image processing method for changing the image quality of a moving image obtained by photographing with an imaging device,
An approach area detecting step of detecting an approach area that is an area indicating an object approaching the imaging device in the moving image by analyzing the moving image;
An image processing method comprising: an image quality changing step of changing an image quality of an image of the approaching area detected in the approaching area detecting step.