JP2010287029A - Periphery display device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a periphery display device which enables display in consideration of presence of occlusion area when displaying a surrounding bird's-eye view image in a vehicle such as an automobile. <P>SOLUTION: An image processing apparatus 10 includes: an image acquisition section 11 which incorporates image data; a three-dimensional information measurement section 12 which measures three-dimensional information based on the image data; a visual point position change image generation section 14 which creates a visual point position change image which has changed a visual point position of image data; a critical region determination section 13 which determines an obstacle area presumed to be an area which includes an obstacle from three-dimensional information on the visual point position change image, and which determines around the area which cannot obtain three-dimensional information to be a critical region, based on three-dimensional information measured in a three-dimensional information measurement section 12; a superposition information image generation section 15 which carries out marking to a critical region on a visual point position change image etc. based on the determination result in the critical region determination section 13; and a visual point position change image composition section 16 which compounds a visual point position change image. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a peripheral display device, and more particularly, to a peripheral display device that displays a bird's-eye view of a surrounding state of a mobile object capable of self-running.

  Images of the surroundings of the vehicle are individually captured by a plurality of imaging means, and a plurality of bird's-eye images are generated by converting the viewpoints of the respective images and using the road surface as a projection surface. Thus, various techniques for obtaining an overhead view image around the host vehicle are disclosed.

  For example, Patent Document 1 discloses a technique for improving visibility by performing object detection processing on a bird's-eye view image and drawing a pseudo vehicle on the bird's-eye view image when the detected object is a vehicle. Yes.

  Further, in Patent Document 2, visibility is created by creating a bird's-eye view image including a pseudo image (an image compressed in the height direction) considering the height information of an object using three-dimensional information measured by a stereo camera. A technique for improving the above is disclosed.

JP 2003-309844 A JP 2006-333209 A

  In any of the above-mentioned patent documents, an object image can be brought closer to a more natural image, but even if an object exists in an invisible area (occlusion area) behind the object, Since it is only displayed, it has a problem that an object behind it cannot be recognized.

  The present invention has been made to solve the above-described problems, and when displaying a bird's-eye view of the surroundings of a mobile object such as a car, it is possible to display in consideration of the presence of an occlusion area. An object of the present invention is to provide a peripheral display device.

  A first aspect of the peripheral display device according to the present invention includes an image acquisition unit that acquires peripheral image data in a mobile object that can be self-propelled, and a three-dimensional information measurement unit that measures three-dimensional information based on the image data. A viewpoint position change image generating unit that generates a viewpoint position change image in which the viewpoint position of the image data is changed to a position overlooking the moving body, and the viewpoint position change image based on the three-dimensional information A hazard area determination unit that identifies an obstacle area that is estimated to be an obstacle area above, and that determines an area where the three-dimensional information cannot be acquired around the obstacle area as a danger area, A warning is given when the obstacle area and the dangerous area exist in response to the determination result of the dangerous area determination unit.

  In a second aspect of the peripheral display device according to the present invention, the image data is acquired by a stereo camera, and the image acquisition unit associates feature points of two images acquired by the stereo camera with each other. The 3D information is obtained by calculating the 3D position of each point on the image from the parallax obtained as a result, and the dangerous area determination unit stores in advance the parallax information included in the 3D information. The obstacle area is identified by comparing with a parallax table in which the parallax information on the road surface image is tabulated, and the parallax cannot be measured because the three-dimensional information behind the obstacle area cannot be obtained. A possible area is determined as the dangerous area.

  In a third aspect of the peripheral display device according to the present invention, the parallax table is updated as needed by updating the image of the road surface.

  In a fourth aspect of the peripheral display device according to the present invention, when the dangerous area determination unit maps measurement points on a three-dimensional space obtained based on the three-dimensional information to a plane assumed to be a road surface, An area where the measurement points are sparse is determined as the dangerous area, and an area in the vicinity of the dangerous area where the measurement points are dense is determined as the obstacle area.

  In a fifth aspect of the peripheral display device according to the present invention, the dangerous area determination unit determines the density of the measurement points by comparing with a distribution state of the measurement point mapping with respect to the road surface image acquired in advance. .

  A sixth aspect of the peripheral display device according to the present invention is an image acquisition unit that acquires peripheral image data in a self-propelled movable body, and a three-dimensional information measurement unit that measures three-dimensional information based on the image data. And a viewpoint position change image generation unit that creates a viewpoint position change image that changes the viewpoint position of the image data to a position that is viewed from above the moving body, and an obstacle based on the image data, A dangerous region determination unit that determines a dangerous region based on the position of the obstacle in the viewpoint position change image, and the obstacle region and the dangerous region exist according to a determination result in the dangerous region determination unit Give a warning if you want.

  In a seventh aspect of the peripheral display device according to the present invention, the dangerous area determination unit determines an occlusion area at the position where the obstacle exists in the viewpoint position change image as the obstacle area.

  In an eighth aspect of the peripheral display device according to the present invention, the occlusion area is determined based on the relationship between the angle of view of the camera that acquired the image data and the position of the obstacle.

  A ninth aspect of the peripheral display device according to the present invention includes a superimposed information image generation unit that superimposes a marking image on the dangerous region on the viewpoint position change image based on a determination result in the dangerous region determination unit. .

  According to the first aspect of the peripheral display device according to the present invention, when displaying a bird's-eye view of the surrounding area in a mobile body such as an automobile, a warning is made by determining a dangerous area such as an occlusion area, for example. Can be given, and effective danger avoidance becomes possible.

  According to the second aspect of the peripheral display device of the present invention, the obstacle area is specified using the parallax information included in the three-dimensional information, and the three-dimensional information behind the obstacle area cannot be obtained and the parallax is not obtained. Since the area that cannot be measured is determined as the dangerous area, the obstacle area and the dangerous area can be identified relatively easily.

  According to the third aspect of the peripheral display device of the present invention, the parallax table is updated as needed by updating the road surface image, so that it is possible to cope with the case where the road surface state changes.

  According to the fourth and fifth aspects of the peripheral display device according to the present invention, by using the coarse / dense state of the three-dimensional measurement points as a determination criterion, the obstacle region and Hazardous areas can be identified.

  According to the sixth to eighth aspects of the peripheral display device according to the present invention, an obstacle is specified based on image data when displaying a surrounding bird's-eye view image on a mobile object such as a car. If possible, it is possible to determine the dangerous area relatively easily. For example, it is possible to determine a dangerous area such as an occlusion area and give a warning, and effective danger avoidance is possible.

  According to the ninth aspect of the peripheral display device of the present invention, the dangerous area can be visualized.

It is a figure which shows the example of arrangement | positioning of a stereo camera. It is a block diagram which shows the structure of the peripheral display apparatus which concerns on this invention. It is a figure explaining the concept of corresponding point search. It is a figure which shows typically the state in which a stereo camera image | photographs the road surface without an obstruction. It is the figure which overlapped and showed the measurement point on the two-dimensional image of the road surface. It is a figure which shows the relationship between a parallax table and the measurement parallax of the road surface without an obstruction. It is a figure which shows typically the state in which a stereo camera image | photographs the road surface with an obstruction. It is a figure which shows the relationship between a parallax table and the measurement parallax of the road surface with an obstruction. It is a bird's-eye view seen from the upper part of vehicles. It is a figure showing the relationship between parallax and the number (frequency) of a measurement point with a histogram. It is a figure showing the relationship between parallax and the number (frequency) of a measurement point with a histogram. It is the figure which mapped the measurement point when there is no obstacle on a road surface on a bird's-eye view image. It is the figure which mapped the measurement point when there is an obstacle on a road surface on a bird's-eye view image. It is a figure which shows typically the state in which a stereo camera image | photographs the road surface with a hollow. It is the figure which mapped the measurement point when there is a dent on a road surface on a bird's-eye view image.

<Embodiment>
<Device configuration>
FIG. 1 is a diagram showing an arrangement example of stereo cameras in a vehicle VC equipped with a peripheral display device according to the present invention.

  In FIG. 1, a vehicle VC includes four stereo cameras that acquire front, left, and right images, and four stereos that acquire left oblique front, left oblique rear, right oblique front, and right oblique rear images. And a camera.

  That is, a stereo camera SC1 that acquires a front image, a stereo camera SC2 that acquires a left image relative to the front, a stereo camera SC3 that acquires a rear image, and a stereo camera SC4 that acquires a right image relative to the front A stereo camera SC5 that acquires a left diagonally forward image, a stereo camera SC6 that acquires a diagonally backward left image, a stereo camera SC7 that acquires a diagonally backward right image, and a stereo camera SC8 that acquires a diagonally forward left image ing. Further, in FIG. 1, the in-view angle regions in the stereo cameras SC1 to SC8 are indicated by regions R1, R2, R3, R4, R5, R6, R7, and R8, and the monitoring range is indicated by a solid line.

  Further, in FIG. 1, two cameras included in each of the stereo cameras SC1 to SC8 are represented as cameras 1a and 1b, for example, for the stereo camera SC1.

  By using such a system, it is possible to acquire image data around the host vehicle and change it to an overhead view seen from above the host vehicle by changing the viewpoint, but in the following description, A case where the front image acquired by the stereo camera SC1 is used will be described as an example.

  The actual arrangement position of each camera differs depending on the vehicle type. For example, the stereo camera SC1 is arranged at the position of the room mirror, and the stereo cameras SC2 and SC4 are arranged at the positions of the left and right side mirrors, respectively. SC4 is arranged at the position of the rear bumper, and the stereo cameras SC5 to SC8 are all arranged at bumper positions at the four corners of the vehicle VC. The above arrangement is an example, and the arrangement of the stereo camera is not limited to this.

  FIG. 2 is a block diagram showing the configuration of the peripheral display device 100 according to the present invention. As shown in FIG. 2, the peripheral display device 100 includes stereo cameras SC <b> 1 to SC <b> 8, an image processing device 10 that performs image processing of image data obtained by the stereo cameras SC <b> 1 to SC <b> 8, and processing output from the image processing device 10. And a display unit 20 for displaying completed image data.

  The image processing apparatus 10 includes an image acquisition unit 11 that captures image data obtained by the stereo cameras SC1 to SC8, a 3D information measurement unit 12 that measures 3D information based on the image data acquired by the image acquisition unit 11, and Based on the viewpoint position change image generation unit 14 that generates a viewpoint position change image obtained by changing the viewpoint position of the image data acquired by the image acquisition unit 11 and the 3D information measured by the 3D information measurement unit 12, A dangerous area determination unit 13 that determines an obstacle area that is estimated to be an obstacle area from the three-dimensional information on the position-changed image, and determines an area in which the three-dimensional information cannot be acquired as a dangerous area in the vicinity thereof; Based on the determination result in the dangerous area determination unit 13, the superimposed information image generation unit 15 for marking the dangerous area on the viewpoint position change image and the stereo cameras SC1 to SC8. By combining the viewpoint position change image obtained in correspondence with the field angle region R1~R8 of respectively, and a viewpoint position change image synthesizing unit 16 to obtain an overhead view image around the vehicle VC. Hereinafter, each part will be described.

<Image acquisition unit 1>
The stereo camera can simultaneously shoot from different viewpoints with two cameras, and the image acquisition unit 1 captures image data obtained by each camera in units of pixels.

<Three-dimensional information measuring unit 12>
The three-dimensional information measuring unit 12 associates feature points of two images obtained by two cameras of the stereo camera with each other. This is called a corresponding point search and is a conventional method.

  FIG. 3 is a diagram for explaining the concept of corresponding point search, in which one of images captured by two cameras of stereo cameras is represented as a standard image Ga and the other is represented as a reference image Gb. Then, a pixel designated on the reference image Ga for searching for a corresponding pixel (corresponding point) Wb on the reference image Gb is set as Wa.

  If the coordinates of the pixel Wa are represented by (x, y) and the coordinates of the pixel Wb are represented by (x ′, y ′), the parallax d of both pixels is calculated by x−x ′.

  It should be noted that, in the corresponding point search, a method of calculating from the images by a gradient-based method and a method called SAD (Sum of Absolute Difference) method are known. In this method, a window that includes this attention point is set for the attention point on the reference image, which is one of two images taken from different viewpoints as in a stereo camera, and the other Set multiple windows of the same size on the reference image that is the same image, calculate the correlation value between the window on the standard image and each window on the reference image, and the window on the reference image with the highest correlation value And the barycentric position of the window is obtained as the corresponding point of the point of interest.

  There is also a method using a phase only correlation method (POC method). In this method, the pixel level corresponding point on the reference image corresponding to the pixel level attention point on the base image is calculated, and then the sub-pixel level shift amount of the corresponding point with respect to the target point is obtained. Association is possible.

  Then, three-dimensional information is acquired by performing parallelization, distortion correction, and three-dimensional reconstruction processing on the calculated corresponding points.

<Dangerous area determination unit 13>
The dangerous area determination unit 13 determines the obstacle area and the dangerous area based on the three-dimensional information provided from the three-dimensional information measurement unit 12, and the dangerous area determination method includes the following two methods. There is a way.

<Dangerous area determination method 1>
The three-dimensional information provided from the three-dimensional information measuring unit 12 includes parallax information of an image captured by a stereo camera, and a table of the parallax information about a road surface image that is stored in advance. The obstacle region is first identified by comparing with the parallax table. Hereinafter, it demonstrates concretely using FIGS. 4-9.

  FIG. 4 schematically shows a state in which the stereo camera SC1 that captures the front in the traveling direction (z direction) of the vehicle VC captures the road surface. The corresponding point search is performed in the in-angle region R1 of the stereo camera SC1. The measurement point M1 to perform is typically shown.

  FIG. 5 is a diagram showing a measurement point M1 superimposed on a two-dimensional image of a road surface. In the figure, the x direction is a direction crossing the road surface, and the z direction is a direction in which the road surface extends. The number of measurement points M1 is greater on the near side of the figure and decreases as the distance is greater. FIG. 6 shows a state in which no obstacle exists on the road surface, and the parallax table and the parallax information (actual measurement parallax) about the image of the road surface match.

  FIG. 6 is a diagram showing the relationship between the parallax table and the actual measured parallax, and shows the measurement points M1 in the two-dimensional image of the road surface divided into layers for each level in the z direction. Here, in the parallax table, when the parallax at the lowest layer is the reference parallax α, the parallax at the next level is expressed as a value (α−a) that is smaller than the reference parallax α by the numerical value a. Further, the parallax at the next level is represented by a value (α−b) that is smaller than the reference parallax α by the numerical value b, and further, the parallax at the next level is the numerical value c more than the reference parallax α. It is represented by a value (α-c) that is reduced by a small amount. Here, the magnitude relationship between the numerical values a to c indicates that the numerical value a is the smallest, the numerical value c is the largest (c> b> a), and the disparity becomes smaller as the distance from the camera increases.

  Since the parallax table is obtained from an image of the road surface that is captured in a state where no obstacle exists on the road surface, when there is no obstacle on the road surface, the actual measured parallax d (i, j) and the parallax are obtained. It matches the parallax of the table, d (i, j) = α at the lowest layer, d (i, j) = α−a at the next level, and d (i, j) = α−b at the next level. In the next level, d (i, j) = α−c. Here, i represents a designated position in the image horizontal direction (x direction), and j represents a designated position in the image vertical direction (z direction).

  FIG. 7 schematically shows a state in which the stereo camera SC1 captures a road surface when there is an obstacle on the road surface. The measurement point M1 on the road surface and the measurement point M2 on the obstacle OB are It is shown schematically.

  As shown in FIG. 7, when the obstacle OB exists on the road surface, the measurement point M2 is set on the surface of the obstacle OB, while the measurement point M1 is on the road surface behind the obstacle OB. It will not be set.

  FIG. 8 is a diagram showing the relationship between the parallax table and the actual measured parallax when the obstacle OB exists on the road surface, and the obstacle OB is shown as a region surrounded by a frame line.

  When the obstacle OB is a cube as shown in FIG. 7, the measurement point M2 is set on the surface facing the vehicle VC, and the position of the surface exists at any one of the measurement point levels in the two-dimensional image of the road surface. Will be. Since the surface exists at the level of α-a in FIG. 8, the parallax at the measurement point M2 is α-a regardless of the value.

  Therefore, when the obstacle OB exists, an area where the actual measurement parallax d (i, j) and the parallax of the parallax table do not coincide with each other appears. Therefore, when an area where the actual measurement parallax d (i, j) and the parallax in the parallax table do not match is confirmed, the length in the x direction of the area is calculated from the coordinates (i, j) of each measurement point, and x The position in the direction is acquired, and the obstacle region is specified by applying the length and position information in the x direction to the level where the surface facing the vehicle VC exists, in this case, the level of the parallax α-a.

  Behind the obstacle area, there is an unmeasured area where parallax cannot be measured, which is determined by the angle of view of the stereo camera SC1, and the area is determined to be a dangerous area DR hidden behind the obstacle area OR. .

  FIG. 9 is an example of a three-dimensional overhead view seen from above the vehicle VC. The obstacle area OR obtained by the above-described method is shown in front of the vehicle VC, and behind the obstacle area OR. A dangerous area DR which is hidden behind and cannot be seen is shown.

  FIG. 9 is created by performing viewpoint conversion of the two-dimensional image of the road surface shown in FIG. 5. As a result of performing viewpoint conversion, the measurement point M1 becomes wider as the distance from the vehicle VC increases. The dangerous area DR is represented as an area extending in a trapezoidal shape.

  Thus, by comparing the actual measured parallax d (i, j) with the parallax of the parallax table, the obstacle area can be easily identified. After the obstacle area is identified, the stereo camera By specifying an unmeasured area where the parallax cannot be measured, which is determined by the angle of view, as the dangerous area, it is possible to display in consideration of the occlusion area.

  Here, the parallax table may use a parallax value obtained by photographing the road surface in advance as a fixed value, but the parallax table is obtained periodically by photographing the road surface to obtain the parallax value and updating it as needed. Also good. For example, the road surface condition changes between a flat road surface and a slope, which is effective in such a case.

  Also, as shown in FIG. 5, the number of measurement points M1 increases toward the near side of the figure, that is, closer to the camera, and decreases toward the far side. ) Is represented as a histogram as shown in FIG.

  That is, in FIG. 10, the disparity is small at a position far from the camera, and the histogram is such that the disparity increases stepwise as the camera is approached.

  However, when there is an obstacle, this step-like histogram cannot be obtained, and the frequency protrudes only for one of the parallaxes, or conversely, the frequency decreases. This state is shown in FIG.

  In FIG. 11, hatching is given to the portion where the frequency protrudes, but it can be seen that there is an obstacle at a level having this parallax. In addition, the frequency is decreasing before the parallax where the frequency protrudes, that is, at a position far from the level where the obstacle exists, and the dangerous area can be identified from the parallax range where the frequency is decreasing.

  In order to obtain 3D information in the 3D information measuring unit 12, camera parameters are used. By mapping 3D measurement points obtained based on the 3D information to a plane assumed to be a road surface, The obstacle area and the dangerous area can be specified from the density state of the measurement points of the mapping result.

  FIG. 12 is a diagram in which the measurement points M1 when there are no obstacles on the road surface are mapped on the bird's-eye view image, and the distribution of the measurement points is not dense.

  FIG. 13 is a diagram in which the measurement point M1 when there is an obstacle on the road surface is mapped on the bird's-eye view image. Although the measurement point M2 exists on the surface of the obstacle, the measurement is performed in the depth direction, that is, in the z direction. There are no points, and the distribution of measurement points is uneven. By knowing the surface position of the obstacle, the obstacle area can be specified, and the dangerous area can be specified from the area where the measurement point does not exist. The determination of the coarse density of the measurement point M1 when there is an obstacle is made by comparing with the mapping result of the measurement point M1 when there is no obstacle on the road surface acquired in advance.

  As described above, by using the density state of the three-dimensional measurement points as the determination criterion, it is possible to specify the obstacle region and the dangerous region without requiring an accurate object detection process.

  In addition, by using this method, not only obstacles protruding on the road surface but also obstacles such as depressions on the road surface can be specified.

  FIG. 14 schematically shows a state in which the stereo camera SC1 that captures the forward direction of the vehicle VC in the traveling direction (z direction) captures the road surface, and the depression CP on the road surface in the in-angle region R1 of the stereo camera SC1. The case where there exists is shown typically.

  In FIG. 14, in the in-view region R1, there is a measurement point M1 other than the depression CP. However, in the depression CP, the measurement point M1 does not exist, and one wall surface of the depression, that is, the wall surface on the side visible from the vehicle VC. Only the measurement point M2 exists.

  FIG. 15 is a diagram in which measurement points M1 and M2 in the case where there is a depression CP on the road surface are mapped on the bird's-eye view image, and measurement points are placed on the wall surface on the back side in the traveling direction (z direction) of the vehicle VC in the depression CP Although M2 exists, the near side is an area where neither measurement point M1 nor M2 exists. In this case, the dangerous area DR is specified by an area where neither the measurement point M1 nor M2 exists.

<Dangerous area determination method 2>
In the dangerous area determination method 1 described above, an obstacle is detected using disparity information. However, the obstacle detection method is not limited to this, and the obstacle is detected using a conventional method. It may be detected.

  For example, a template for an object that may be an obstacle is prepared in advance, the template is applied to the two-dimensional image obtained from one of the two cameras that make up the stereo camera, pattern matching is performed, and matching is performed. In such a case, a method of determining that there is an object corresponding to the template may be adopted.

  In addition, a technique may be adopted in which teacher data is applied to the feature amount calculated from the two-dimensional image, pattern matching is performed, and when matching is performed, it is determined that there is an object corresponding to the teacher data.

  In addition, the position in the real space is calculated based on the three-dimensional information acquired using the stereo camera, and the road surface height corresponding to the position in the horizontal direction and the parallax direction (distance direction) is calculated based on the road model. In contrast, when the position is above the road surface, a method of detecting an obstacle on the road surface by extracting the data of the position as data representing a three-dimensional object may be adopted. This method is disclosed in Japanese Patent Application Laid-Open No. 2006-134035.

  The dangerous area corresponding to the detected obstacle is determined based on the position of the obstacle in the overhead image after the viewpoint position change generated by the viewpoint position change image generation unit 14. In this determination, it is realistic that the occlusion area determined by the relationship between the angle of view of the stereo camera SC1 and the position of the obstacle is the obstacle area. In this case, in the configuration shown in FIG. 2, the overhead image data after the viewpoint position change generated by the viewpoint position change image generation unit 14 is taken into the dangerous area determination unit 13.

<Viewpoint position change image generation unit 14>
The viewpoint position change image generation unit 14 changes the viewpoint position (coordinate conversion) so that an image captured by the stereo camera is an overhead image viewed from above the host vehicle. This process may be performed using a known technique, and each point of an image that is created in advance based on photographing conditions such as the direction of the optical axis of the photographing lens and the characteristics of the photographing lens and photographed with a stereo camera, and each of the overhead images. A method of performing conversion using a conversion table that associates points is common.

<Superimposition information image generation unit 15>
The superimposition information image generation unit 15 superimposes the marking image on the obstacle region and the dangerous region on the overhead image generated by the viewpoint position change image generation unit 14.

  This is a process of performing coloration by superimposing a layer image on a target area on the bird's-eye view image, and is a process of marking so that the driver can easily understand by looking at the in-vehicle monitor.

  Instead of marking a dangerous area or the like, marking may be performed on an area where accurate three-dimensional information can be acquired and the vehicle can travel safely.

<Viewpoint position change image composition unit 16>
The viewpoint position change image composition unit 16 synthesizes the bird's-eye view image obtained by changing the viewpoint position of the image obtained by each stereo camera, including the marked overhead view image generated by the superimposition information image generation unit 15. Thus, an overhead image around the vehicle VC is acquired.

<Display unit 20>
The display unit 20 corresponds to an in-vehicle monitor and displays processed image data output from the image processing device 10, but if there are many dangerous areas, the driver's attention is alerted with a warning sound or voice guidance. A warning may be given in an easy way.

<Modification>
In the peripheral display device 100 described above, the front image of the host vehicle has been described as an example. However, the obstacle region and the dangerous region can be detected for any image captured by any of the stereo cameras SC1 to SC8. For example, in the case of garage entry or width adjustment, the same processing may be performed on images taken by the rear and side stereo cameras.

SC1 to SC8 Stereo camera OR Obstacle area DR Hazardous area OB Obstacle VC Vehicle

Claims (9)

An image acquisition unit for acquiring peripheral image data in a self-propelled movable body;
A three-dimensional information measuring unit that measures three-dimensional information based on the image data;
A viewpoint position change image generation unit that creates a viewpoint position change image in which the viewpoint position of the image data is changed to a position overlooking the moving body;
Based on the three-dimensional information, an obstacle area estimated as an obstacle area on the viewpoint position change image is specified, and an area where the three-dimensional information cannot be obtained around the obstacle area is dangerous. A dangerous area determination unit for determining as an area,
A peripheral display device that receives a determination result from the dangerous area determination unit and gives a warning when the obstacle area and the dangerous area exist. The image data is acquired by a stereo camera,
The image acquisition unit
3D information is obtained by associating the feature points of the two images obtained by the stereo camera with each other and calculating the 3D position of each point on the image from the parallax obtained as a result,
The dangerous area determination unit
The obstacle region is identified by comparing the disparity information included in the three-dimensional information with a disparity table in which the disparity information about the road surface image stored in advance is tabulated, and behind the obstacle region The peripheral display device according to claim 1, wherein an area where the three-dimensional information cannot be acquired and parallax cannot be measured is determined as the dangerous area.   The peripheral display device according to claim 2, wherein the parallax table is updated as needed by updating an image of the road surface. The dangerous area determination unit
When a measurement point on a three-dimensional space obtained based on the three-dimensional information is mapped to a plane assumed to be a road surface, an area in which the measurement point is sparse is determined as the danger area, and in the vicinity of the danger area The peripheral display device according to claim 1, wherein an area where the measurement points are dense is determined as the obstacle area. The dangerous area determination unit
The peripheral display device according to claim 4, wherein the density of the measurement points is determined by comparison with a distribution state of the mapping of the measurement points for the road surface image acquired in advance. An image acquisition unit for acquiring peripheral image data in a self-propelled movable body;
A three-dimensional information measuring unit that measures three-dimensional information based on the image data;
A viewpoint position change image generation unit that creates a viewpoint position change image in which the viewpoint position of the image data is changed to a position overlooking the moving body;
A hazard area determination unit that identifies an obstacle based on the image data and determines a danger area based on the position of the obstacle in the viewpoint position change image,
A peripheral display device that receives a determination result from the dangerous area determination unit and gives a warning when the obstacle area and the dangerous area exist. The dangerous area determination unit
The peripheral display device according to claim 6, wherein an occlusion area at a position where the obstacle exists in the viewpoint position change image is determined as the obstacle area.   The peripheral display device according to claim 7, wherein the occlusion area is determined based on a relationship between an angle of view of a camera that has acquired the image data and a position of the obstacle.   The peripheral display device according to claim 1, further comprising a superimposition information image generation unit that superimposes a marking image on the dangerous region on the viewpoint position change image based on a determination result in the dangerous region determination unit.

Images