JP2005167638A - Mobile surrounding surveillance apparatus, vehicle, and image transforming method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To obtain an image which gives a driver less uncomfortable feeling and facilitates the checking of surroundings around a vehicle to reduce a burden on the driver and raise safety. <P>SOLUTION: An imaging means 11b converts into image data an optical image obtained from a picture taken over a view field of 360° around through an optical system 11a capable of performing central projective transformation. An image processing means 12 converts the image data into panorama image data or see-through image data, and a display means 13 displays an image corresponding to the second image data. It converts the vicinity of the view field center into image data so that a road surface is looked down, and a portion with a distance from the view field center is projected on a partly cylindrical surface or a partly spherical surface having an axis passing the projective transformation center at a position with a fixed distance from the projective transformation center in a plane parallel to the road surface with a radius equal to the distance between the projective transformation center and the road surface, thereby converting the portion into image data. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a moving body surroundings monitoring device for monitoring the surroundings of a moving body, a moving body equipped with the moving body monitoring apparatus, and an image conversion method, and more particularly to a moving body including a vehicle such as an automobile or a train that transports people or cargo. The present invention relates to a moving object surroundings monitoring device for monitoring the surroundings, a moving object on which the moving object is mounted, and an image conversion method.

  The increase in traffic accidents in recent years has become a major social problem. In particular, at intersections such as four corners and three-way intersections, accidents due to jumping out of people, collision accidents at encounters between vehicles, rear-end collisions between vehicles, etc. frequently occur. The cause of traffic accidents at such intersections is that both the driver and pedestrian of the vehicle actually have a narrower field of view than that required to confirm safety, and as a result, pay sufficient attention. Otherwise, it may be due to a delay in the perception of danger. For this reason, improvement of the vehicle itself, driver's attention and improvement of the road environment are strongly desired.

  Conventionally, in order to improve the road environment, mirrors have been installed where the field of view is obstructed by four corners or three-way intersections, but the field of view is narrow only by installing mirrors, and the number of mirrors installed However, it is not enough, so it cannot be said that it is safe.

  In addition, for the purpose of vehicle safety, in particular for rearward confirmation, a monitoring camera is installed at the rear of the vehicle, and a system that displays the image of the monitoring camera on a monitor installed on the side of the driver's seat or on the front panel through a cable is a bus. It is popular in large vehicles and some passenger cars.

  However, even in this case, the safety confirmation on the side of the vehicle is largely based on the driver's vision, and the recognition of the danger is often delayed where the field of view is obstructed by four corners or a three-way road. In addition, this type of camera is generally narrow in vision and can confirm the presence of obstacles in one direction and the danger of colliding with other objects, but it can collide with a wide range of obstacles and other objects. In order to confirm the danger, an operation such as changing the angle of the camera is necessary.

  As described above, in the conventional vehicle periphery monitoring device, importance is attached only to monitoring one direction, and in order to confirm 360 degrees around the vehicle, a plurality of units (four or more in total, front and rear, right and left) A surveillance camera was needed.

Therefore, for example, in Patent Document 1, an image is obtained for a 360-degree field of view around the vehicle, and an optical system capable of central projective transformation on the image and an optical image obtained through the optical system are first described. There has been proposed a method of displaying an image (overhead image or bird's eye view) whose visual field direction is 90 degrees downward (vertical axis direction downward) using an omnidirectional visual sensor including an imaging means for converting to image data.
JP 2002-218451 A

  As described above, when using a vehicle, there are often situations in which it is necessary to confirm and respond to safety. For example, it is necessary to check not only the front of the vehicle but also the left and right and the rear of the vehicle when checking the surroundings at the time of departure of the vehicle, when turning left or right, or when entering or leaving the parking lot or garage. It is very important for the driver of the vehicle to check these, but because of the structure of the vehicle, it is difficult to check the safety of the blind spot from the driver's seat, which is a heavy burden on the driver. Become.

  Moreover, in order to confirm 360 degrees around the vehicle using a conventional vehicle surrounding monitoring apparatus, a plurality of cameras are required. When using these cameras, the driver of the vehicle switches the camera that outputs the image displayed on the monitor or changes the direction of the camera according to the situation at that time, and the safety of the vehicle. It is necessary to confirm. Such an operation places a heavy burden on the driver of the vehicle.

  In addition, according to the surrounding monitoring device disclosed in Patent Document 1, an image of 360 degrees around can be obtained with one camera, but the image displayed on the monitor is a projection surface (for example, a road surface) around a surrounding object. It is a bird's-eye view image or a bird's-eye view projected onto (projected). For this reason, the image of the place away from the center of the visual field direction becomes a largely distorted image, which gives a great sense of discomfort to the driver of the vehicle.

  The present invention solves the above-mentioned conventional problems, and an image with a little uncomfortable feeling for the driver can be obtained, the situation around the moving body can be easily confirmed, the burden on the driver can be reduced, and safety is improved. It is an object of the present invention to provide a moving object surrounding monitoring device that can be enhanced, a moving object on which the moving object is mounted, and an image conversion method thereof.

  The moving object surroundings monitoring apparatus according to the present invention provides an optical system capable of obtaining a 360-degree visual field region and capable of central projective transformation on the image, and an optical image obtained through the optical system as first image data. Corresponding to the second image data obtained by the image processing means, the image processing means for converting the first image data into at least panoramic image data or fluoroscopic image data, Display means for displaying an image to be displayed, wherein the image processing means converts the arrangement surface including at least a part of the moving body into an image data overlooking the vicinity of the center of the field of view, and is separated from the center of the field of view by a certain distance. The visual field portion other than the vicinity of the visual field center has image conversion means for converting image data when viewed from the side of the moving body in the lateral direction from the moving body side. It is made.

  Preferably, the image conversion means in the moving object surroundings monitoring device of the present invention is such that the projective conversion is performed in a visual field portion other than the vicinity of the visual field center and in a plane including the projective conversion center and parallel to the moving object arrangement surface. At least one of a partial cylindrical surface and a partial spherical surface having at least one of a cylindrical axis and a spherical center point at a certain distance from the center and having a radius as the distance between the projective transformation center and the moving object placement surface Convert the image data to the projected image.

  Further preferably, the image conversion means in the moving object surroundings monitoring device of the present invention is characterized in that the projective conversion is performed in a visual field portion other than the vicinity of the visual field center and in a plane including the projective conversion center and parallel to the arrangement surface of the moving object. Each partial cylindrical surface having at least one of a cylindrical axis and a spherical center point at each position separated by a certain distance in the opposite direction from the center, and having a radius between the projection transformation center and the arrangement surface of the moving body Each image data is projected onto at least one of the partial spherical surfaces.

  Further preferably, in the moving object surroundings monitoring device of the present invention, the projecting surfaces are orthogonal to each other at positions spaced apart from the projecting conversion center within a plane that includes the projecting conversion center and is parallel to the arrangement surface of the moving object. Two partial cylindrical surfaces each having a cylindrical axis, and the partial spherical surface has a radius that is the same as the radius of the partial cylindrical surface, centered on the intersection of the cylindrical axes of the two partial cylindrical surfaces. And a spherical surface connecting the two partial cylindrical surfaces.

  Further preferably, in the omnidirectional visual sensor in the moving object surrounding monitoring apparatus of the present invention, the optical system is a mirror surface on a convex surface on one curved surface of a two-leaf hyperboloid obtained by rotating a hyperbola around an axis. The imaging means has an optical axis that coincides with the rotational axis of the hyperbola, and the principal point of the lens is disposed at the other focal position of the two focal points of the hyperboloid mirror Has been.

  Furthermore, preferably, in the omnidirectional visual sensor in the moving object surrounding monitoring apparatus of the present invention, the optical system has a convex surface on one curved surface of a two-leaf hyperboloid obtained by rotating a hyperbola around an axis, A main hyperboloid mirror having a mirror surface except for the tip of the convex portion through which light can be transmitted, and a sub-hyperboloid mirror having a mirror surface formed on the convex surface of the other curved surface, and the imaging means includes the light The axis coincides with the rotation axis of the hyperbola, and the principal point of the lens is disposed at the other focal position of the two focal points of the sub-hyperbolic mirror.

  Further preferably, the display unit in the mobile object periphery monitoring device of the present invention also serves as a position display unit that displays a position image of the mobile object on a map screen, and the mobile object position detection that detects the position of the mobile object And display control means for controlling the display means so as to switch between the position image of the moving body and the surrounding image of the moving body based on the position information detected by the moving body position detecting means.

  The moving body of the present invention is equipped with the moving body surroundings monitoring device according to any one of claims 1 to 7, thereby achieving the above object.

  Preferably, the moving body surroundings monitoring device in the moving body of the present invention is installed on at least one of the front and rear bumpers.

  Further preferably, the moving body surroundings monitoring device in the moving body of the present invention is installed at one of the positions on the right end and the left end of the front bumper, and on the diagonal line with respect to the installation position. It is also installed on the rear bumper.

  According to the image conversion method of the present invention, the first image data obtained by imaging an optical image obtained through an optical system capable of central projective transformation with respect to an image of a viewing field area of 360 degrees around the second image data is obtained. As image data, near the center of the visual field, the arrangement surface including at least a part of the moving body is converted into image data for bird's-eye view, and the moving body is disposed in the visual field portion other than the vicinity of the visual field center that is a fixed distance away from the visual field center. The image data is converted into image data when viewed from above the surface in the lateral direction from the movable body side, thereby achieving the above object.

  Preferably, in the image conversion method of the present invention, when the first image data is converted to the second image data, the moving object including the projective conversion center is disposed in a visual field portion other than the vicinity of the visual field center. A plane parallel to the surface has at least one of a cylindrical axis and a spherical center point at a certain distance from the projective transformation center, and the distance between the projective transformation center and the plane on which the moving object is arranged is a radius. The image data is converted into the second image data projected onto at least one of a partial cylindrical surface and a partial spherical surface.

  Further preferably, in the image conversion method of the present invention, when the first image data is converted into the second image data, an arrangement of the moving body including the projective conversion center in a visual field portion other than the vicinity of the visual field center. A plane parallel to the surface and at least one of a cylindrical axis and a spherical center point at each position spaced apart from each other in a direction opposite to each other from the projective transformation center. Are respectively converted into the second image data projected onto at least one of each partial cylindrical surface and each partial spherical surface whose radius is the distance between the first image data and the second image data.

  Further preferably, in the image conversion method according to the present invention, the projection planes are orthogonal to each other at positions spaced apart from the projection conversion center by a certain distance in a plane including the projection conversion center and parallel to the arrangement plane of the moving object. Two partial cylindrical surfaces having a cylindrical axis, and the partial spherical surface is centered on an intersection of the cylindrical axes of the two partial cylindrical surfaces and has the same radius as the radius of the partial cylindrical surface. A spherical surface connecting the two partial cylindrical surfaces.

  The operation of the present invention will be described below with the above configuration.

  The moving object surroundings monitoring apparatus of the present invention obtains an optical image obtained through an optical system in which an image in a 360-degree field of view is obtained by one omnidirectional visual sensor, and center projection conversion can be performed on the image. The first image data is converted by the imaging means. The first image data is converted into panoramic image data or fluoroscopic image data (second image data) by the image processing means, and an image corresponding to the second image data is displayed by the display means. Note that the display size and the display direction of the image displayed on the display unit can be controlled by the display control unit.

  This image processing means can convert image data that overlooks the arrangement surface (for example, road surface) of the moving object in the vicinity of the center of the field of view, and a part that is a fixed distance away from the center of the field of view is viewed from the side. Image conversion means that can convert image data is provided.

  The image conversion means, for example, for a portion that is a certain distance away from the center of the field of view, has an axis at a position that is a certain distance away from the projective transformation center in a plane that passes through the projective transformation center and is parallel to the plane of placement of the moving object. Conversion to image data projected onto a partial cylindrical surface or partial spherical surface having a radius of the distance between the conversion center and the arrangement surface of the moving body is possible.

  Alternatively, the image conversion means, for a portion that is a certain distance from the center of the visual field, passes axes that are orthogonal to each other at a position that is a certain distance away from the projection transformation center in a plane that passes through the projection transformation center and is parallel to the placement surface of the moving object. With the center of the intersection of the axes of the two partial cylindrical surfaces or the two partial cylindrical surfaces whose radius is the distance between the projective transformation center and the placement surface of the moving object, and the same radius as the radius of the partial cylindrical surface It can be converted into image data projected onto a partial spherical surface connecting two partial cylindrical surfaces.

  As a result, a bird's-eye-view bird's-eye view image is obtained near the center of the field of view, and an image as seen from the side from the side of a moving object such as a vehicle is obtained in the field of view that is a predetermined distance away from the center of the field of view. An image with little discomfort for the driver of the moving body can be obtained.

  Further, a position detection means such as GPS for detecting the position of the moving body on the map can be provided, and the display control means can switch and control the surrounding image of the moving body displayed on the display means and its position display. .

  As described above, according to the present invention, for example, an omnidirectional visual sensor is installed in the front and rear part of a vehicle (for example, on a front and rear bumper of a car or a corner part), and a portion that becomes a blind spot by a display means provided in a driver's seat is Since it can be easily confirmed, safety confirmation can be performed more reliably when checking surroundings at the time of departure, when turning left or right, when entering or leaving a parking lot or a garage, or when checking left and right rear.

  Even if the driver does not switch a plurality of cameras that output video displayed on the display means or switches the direction of the camera as in the conventional vehicle monitoring device, the display control means can display any display image or display direction. Since the image size can be switched and displayed, it is possible to easily check the safety by switching the display at the time of retreating to prevent a contact accident or the like.

  In addition, a bird's-eye-view bird's-eye view image is obtained near the center of the field of view when entering and leaving a parking lot or garage, and when looking at the width. Since such an image is obtained, a driver of a moving body such as a vehicle can easily make a safety check and prevent a contact accident or the like with an image with less discomfort.

Hereinafter, a case where the first and second embodiments of the moving body equipped with the moving body surrounding monitoring apparatus of the present invention are applied to an automobile will be described with reference to the drawings.
(Embodiment 1)
FIG. 1 (a) is a top view of a first embodiment of an automobile equipped with the moving body surroundings monitoring device of the present invention, and FIG. 1 (b) is a side view thereof.

  As shown in FIG. 1, an automobile (vehicle) 1 is provided with an omnidirectional visual sensor 11 at the upper center of a front front bumper 2 and a rear rear bumper 3. An image processing means 12 to be described later is provided in the front engine compartment or the rear cargo compartment of the vehicle 1. In the driver's seat of the vehicle 1, for example, a display unit 13 and a display control unit 14 are provided in the front panel or on the side of the panel.

  Each of the omnidirectional visual sensors 11 can obtain an image in a visual field region of 360 degrees around the center. For example, the omnidirectional visual sensor 11 installed on the front bumper 2 has a substantially horizontal direction of 180 degrees in the rear visual field region. The visual field area becomes a blind spot by the image of the host vehicle, and the effective visual field area is 180 degrees forward. Further, in the omnidirectional visual sensor 11 installed on the rear bumper 3, a visual field area of approximately 180 degrees in the horizontal direction in the front visual field area becomes a blind spot by an image of the own vehicle, and an effective visual field area is 180 degrees in the rear.

  FIG. 2 is a block diagram showing a schematic configuration of Embodiment 1 of the moving object surroundings monitoring apparatus of the present invention.

  As shown in FIG. 2, the moving object surroundings monitoring apparatus 10 includes an omnidirectional visual sensor 11 including an optical system 11a capable of central projective conversion and a video imaging unit 11b, an image converting unit 12a, and an output buffer memory 12b. Image processing means 12 for converting the first image data obtained by the imaging means 12b into a panoramic image or a fluoroscopic image, and the second image data (a surrounding image of the moving body) 13a output from the image conversion means 12a. In addition, the display means 13 that can switch and display the image data (position image of the moving body) 13b output from the vehicle position detection means 9 and the display image such as the display direction and the image size of the surrounding image of the automobile are controlled. In addition, the display control means 14 for switching and controlling the surrounding image of the vehicle and its position image by the control signal 14a, and the image processing means 12 And a warning generating means 15 for generating an alarm in a case where there is an obstacle around in response to the output of al. In addition, although the vehicle position detection means 9 is provided here, it does not need to be provided.

  Below, each part of this moving body surroundings monitoring apparatus 10 is demonstrated in detail.

  First, the omnidirectional visual sensor 11 including the optical system 11a and the imaging unit 11b will be described.

  FIG. 3 is a perspective view showing a configuration example of a main part of the optical system 11a of the moving object surrounding monitoring apparatus 10 of FIG. 2, and an image of a 360 ° field of view area is obtained, and central projection conversion is performed on the image. An example of a possible optical system 11a is shown.

  As shown in FIG. 3, the optical system 11a uses a hyperboloidal mirror 111a, and the first principal point of the imaging lens of the video imaging means 11b is arranged at the other focal position (focal point 2) of the hyperboloidal mirror 111a. By doing so, central projective transformation is possible. Since details of the optical system 11a are described in Japanese Patent Laid-Open No. 6-295333, only feature points will be described here.

In FIG. 3, the hyperboloid mirror 111a has a mirror surface on the convex surface of one of the curved surfaces (two-leaf hyperboloid) obtained by rotating the hyperbola around the axis J1 (Z axis) (Z> 0 region). Is formed. This two-leaf hyperboloid is
(X ² + Y ² ) / a ² −Z ² / b ² = −1
c ² = a ² + b ² (1)
Represented by In the above formula (1), a and b are constants that define the shape of the hyperboloid, and c is a constant that defines the position of the focal point. Hereinafter, these constants a, b, and c are referred to as mirror constants of a hyperboloid mirror.

  This hyperboloidal mirror 111a has two focal points (focal point 1 and focal point 2), and the light directed from the outside toward one focal point 1 is reflected by the hyperboloidal mirror 111a, and all is directed to the other focal point 2. . Therefore, the rotation axis J1 of the hyperboloid mirror 21 and the optical axis J2 of the video imaging means 11b are made to coincide with each other, and the lens principal point (first principal point) of the video imaging means 11b is located at the other focal point (focal point 2 in FIG. 3). ), The image of the surrounding area is converted into image data with one focal point (focal point 1) as the center of the viewpoint, without changing the viewpoint position depending on the viewing direction. It has a feature that it is possible (central projective transformation is possible).

  The video imaging means 11b is, for example, a video camera, and an optical image obtained through the hyperboloid mirror 111a is converted into image data (first image data) using a solid-state imaging device such as a CCD or a CMOS. . The converted image data is sent to the image processing means 12.

  Next, the image processing means 12 will be specifically described.

  FIG. 4 is a block diagram showing a configuration example of a main part of the image processing means 12 in FIG.

  As shown in FIG. 4, the image processing means 12 includes an A / D converter 121a, an input buffer memory 122a, a CPU (control unit; central processing unit) 123a, a look-up table (LUT) 124a, and image conversion logic 125a. The image conversion unit 12a having the output buffer memory 12b is connected to the bus line 12c.

  In the case of an analog signal, the image captured by the video imaging unit 11b (first image data) is converted into a digital signal by the A / D converter 121a and then input to the input buffer memory 122a. If the first image data is a digital signal, it is directly input to the input buffer memory 122a.

  Thereafter, the output from the input buffer memory 122a is converted into second image data such as panoramic image data or fluoroscopic image data by the image conversion logic 125a, or other image processing is performed and input to the output buffer memory 12b. The In the image conversion logic 125a, data in the LUT 124a is referred to as necessary during image conversion and image processing. In this image conversion means 12a, each part can be controlled by a so-called RISC CPU 123a.

  The image data conversion process by the image conversion logic 125a of FIG. 4 will be specifically described below.

  First, conversion from the circular input image 111 obtained by imaging by the imaging unit 11b to the 360 ° panoramic image 113 will be described with reference to FIG.

  As shown in FIG. 5, the donut-shaped image 112 is an image in the middle of cutting out and cutting out the circular input image 111 into a donut shape, and the image after the donut-shaped image 112 is stretched and converted into orthogonal coordinates is a panorama. This is an image 113.

When the circular input image 111 is represented by polar coordinates with the center as the origin, the coordinates of a certain point (pixel) P in the circular input image 111 are represented by (r, θ). A coordinate conversion formula for cutting this circular input image 111 into a donut shape, opening and expanding it with reference to PO (ro, θo), and converting it into a rectangular rectangular panoramic image 113 is the coordinates of a point on the panoramic image 113. If P (x, y),
x = θ−θo
y = r-ro
It is represented by If the coordinates of a point on the circular input image 111 are P (X, Y) and the coordinates of the center are O (Xo, Yo),
X = r × cos θ + Xo
Y = r × sin θ + Yo
So
X = (y + ro) cos (x + θo) + Xo
Y = (y + ro) sin (x + θo) + Yo
It is represented by

  Next, the coordinate position relationship of perspective transformation in the perspective transformation image data will be described with reference to FIG.

  For coordinate transformation by perspective transformation, it calculates which position on the input image plane the point in space corresponds to, and assigns the image information at that position to the corresponding coordinate position on the surrounding video after perspective transformation Like that.

As shown in FIG. 6, the coordinate of a certain point in the space is P (tx, ty, tz), and the coordinate of the corresponding point on the circular input image 111 formed on the light receiving portion of the video imaging means 11b is R ( r, θ), the focal length of the lens of the video imaging means 11b is F, and the hyperboloid mirror constants are a, b and c.
r = F × tan ((π / 2) −β) (1)
However,
β = arctan (((b ² + c ² ) × sin α−2bc) / (b ² −c ² ) × cos α)
α = arctan (tz / sqrt (tx ² + ty ² ))
θ = arctan (ty / tx) (2)
It becomes. If the above formula (2) is rearranged,
r = F × (((b ² −c ² ) × sqrt (tx ² + ty ² ))
/ ((B ² + c ² ) × tz−2 × b × c
× sqrt (tx ² + ty ² + tz ² ))) (2 ')
It becomes. Furthermore, when the coordinates of the points on the image are converted into orthogonal coordinates and expressed as R (X, Y),
X = r cos θ (3)
Y = rsinθ (4)
Than,
X = F × (((b ² −c ² ) × tx / ((b ² + c ² ) × tz−2 × b × c
× sqrt (tx ² + ty ² + tz ² ))) (5)
Y = F × (((b ² −c ² ) × ty / ((b ² + c ² ) × tz−2 × b × c
× sqrt (tx ² + ty ² + tz ² ))) (6)
It becomes.

  Note that the panoramic image 113 and the perspective image can be horizontally rotated (panned) and vertically rotated (tilted), and the perspective image is zoomed in and zoomed out in the same manner as in JP-A-2002-218451. Is possible.

  In the image conversion means 12a, the vicinity of the center of the visual field is converted into image data of a bird's eye view that looks down on the arrangement surface of the moving body (road surface in the case of an automobile). For example, when the video image pickup unit 11b is attached to the vehicle 1 so that the optical axis J2 of the video image pickup unit 11b is vertically upward with respect to the ground, the viewing direction of the fluoroscopic image is set to α = −90 degrees by vertical rotation movement. When is selected, the fluoroscopic image becomes an image (bird's eye view) in which the periphery of the vehicle 1 is viewed from directly above.

  Further, in the image conversion means 12a, the visual field portion other than the vicinity of the visual field center that is a fixed distance from the visual field center passes through the projective conversion center and is in a plane parallel to the arrangement surface of the moving body (road surface in the case of an automobile). A semi-cylindrical surface having an axis at a certain distance from the projective transformation center (hyperboloid mirror focal point 1) and a radius of the distance between the projective transformation center and the road surface, or a quarter cylindrical surface obtained by dividing the hemispherical surface into two. Alternatively, the image data is converted into image data projected onto a 1/4 spherical surface (a semi-cylindrical surface portion or a semi-spherical surface portion; a partial cylindrical surface portion or a partial spherical surface). Alternatively, the axes orthogonal to each other at a certain distance from the projective transformation center (hyperboloid mirror focal point 1) in a plane parallel to the arrangement plane of the moving body (road surface in the case of an automobile) through the projective transformation center. It has the same radius as the radius of the part of the cylindrical surface, centered on the intersection of the axes of the two part cylindrical surfaces or the two part cylindrical surfaces whose radius is the distance between the projective transformation center and the road surface. It is converted into image data projected onto a partial spherical surface connecting two partial cylindrical surfaces.

  First, the case of converting to image data projected onto the partial cylindrical surface will be described with reference to FIG. Here, it is assumed that the optical axis J1 of the optical system 11a (the rotation axis of the hyperboloidal mirror 111a) is vertically upward (Z-axis direction) with respect to the road surface (placement surface of the moving body), but is not necessarily vertically upward. There is no need.

Here, as an example, an axis K is provided in a plane parallel to the road surface through the projection transformation center (focal point 1 of the hyperboloid mirror 111a), and the axis K is l ₀ from the projection transformation center (focal point 1). apart, considered from the center of the axis K part cylindrical surface 114 of radius r ₀ (the semi-cylindrical surface portion 114) as the projection plane. Distance R from the projective transformation center (focal point of the hyperboloidal mirror 111a 1), the depression angle alpha, rotation angle about an axis parallel L to the optical axis (Z-axis) at a distance l ₀ from the projective transformation centered perpendicular to the cylindrical axis K The coordinate P (tx, ty, tz) of the point at the position of θ and the rotation angle φ around the cylindrical axis K is
tx = l ₀ + r ₀ cos φ (7)
ty = r ₀ tan θ (8)
tz = r ₀ sinφ (9)
Represented by Therefore, by substituting the above equations (7) to (9) into the above equations (5) and (6), the coordinates X and Y of the points on the input image plane are
X = F × (b ² −c ² ) × cos θ × (l ₀ + r ₀ cos φ)
/ (R ₀ × (b ² + c ² ) × cos θ × sin φ
−2 × b × c × (cos ² θ × (l ₀ ² + 2l ₀ r ₀ cos φ) + r ₀ ² ) ^1/2
... (10)
Y = F × (b ² −c ² ) × r ₀ sin θ
/ (R ₀ × (b ² + c ² ) × cos θ × sin φ
−2 × b × c × (cos ² θ (l ₀ ² + 2l ₀ r ₀ cos φ) + r ₀ ² ) ^1/2
(11)
Represented by

  Next, the case of converting to image data projected on the two partial cylindrical surfaces and the partial spherical surface connecting them will be described with reference to FIG. Here, it is assumed that the optical axis J1 of the optical system 11a (the rotation axis of the hyperboloidal mirror 111a) is vertically upward (Z-axis direction) on the road surface (placement surface of the moving body). There is no.

Here, as an example, it has an axis 50a parallel to the Y axis and an axis 50b parallel to the X axis in a plane parallel to the road surface through the projective transformation center (focal point 1 of the hyperboloid mirror 111a), Partially cylindrical surfaces 115a and 115b (half-cylindrical surface portions) having a radius r ₀ from the center (intersection N) whose axes 50a and 50b are separated from the projective transformation center (focal point 1) by a constant value l ₀ and a constant value m ₀ respectively. 115a, 115b) is a certain quadrant part of their cylindrical surface 115a, consider a projection surface which is coupled with a part spherical surface 116 having the same radius _{r 0} and 115b. Since the relationship between the points on the partial cylindrical surfaces 115a and 115b and the points on the image plane is the same as the expressions (10) and (11) described with reference to FIG. A relationship between a point on the (hemispherical portion 116) and a point on the image plane will be described.

The coordinates of the intersection N of the axes K1 and K2 of the partial cylindrical surfaces 115a and 115b are (l ₀ , m ₀ , 0), and the axis L parallel to the optical axis (Z axis) passing through the intersection N with the intersection N as a fulcrum. Assuming that the rotation angle around is θ and the rotation angle from the plane extending from the axes K1 and K2 is φ, the coordinates P (tx, ty, tz) of a point on the partial spherical surface 116 are:
tx = l ₀ + r ₀ cos φ cos θ (12)
ty = m ₀ + r ₀ cos φ sin θ (13)
tz = r ₀ sinφ (14)
Represented by Therefore, by substituting the above equations (12) to (14) into the above equations (5) and (6), the coordinates X and Y of the points on the input image plane are
X = F × (b ² −c ² ) × (l ₀ + r ₀ cos φcos θ)
/ ((B ² + c ² ) × r ₀ sinφ
−2 × b × c × (l ₀ ² + m ₀ ² + r ₀ ²
+ 2l ₀ r ₀ cos φ cos θ + 2 m ₀ r ₀ cos φ sin θ) ^1/2 )
... (15)
Y = F × (b ² −c ² ) × (m ₀ + r ₀ cos φsin θ)
/ ((B ² + c ² ) × r ₀ sinφ
−2 × b × c × (l ₀ ² + m ₀ ² + r ₀ ²
_{+ 2l 0 r 0 cosφcosθ + 2m} 0 r 0 cosφssinθ) 1/2)
... (16)
Represented by

  In each of the above cases, the projection surface above the plane parallel to the road surface through the projection center is assumed to be a plane extending from a part of the cylindrical surface, and a point on this surface and a point on the image surface Is the same as in the past, and the description thereof is omitted here.

  In the first embodiment, the omnidirectional visual sensor 11 is described as being installed perpendicular to the bottom surface of the installation space (for example, the road surface when installed in a vehicle), but is not necessarily installed perpendicular to the bottom surface. Even when the projector is installed tilted in one direction, by converting it into an equation that takes into account the tilt of the optical axis of the omnidirectional visual sensor 11, a distortion close to a horizontal visual field is obtained although it is an overhead image. It is possible to easily obtain a surrounding image with little.

  Next, as a bird's-eye view, as in the case of the prior art, a case where the viewing direction of the perspective image screen is simply −90 ° (vertically downward) and a position away from the central viewing direction as in the first embodiment are predetermined. FIG. 9A and FIG. 9B show the difference in display state between the pseudo bird's-eye view projected onto a partial cylindrical surface or a partial spherical surface.

  When the viewing direction of the fluoroscopic image screen is simply −90 ° (vertically below), as shown in FIG. 9A, the object 20 away from the viewpoint greatly extends in the viewing direction. Is displayed. For this reason, the driver (observer of the screen) has a very uncomfortable image.

  On the other hand, in the pseudo bird's-eye view of the first embodiment, as shown in FIG. 9B, the object 20 located away from the viewpoint is displayed in a form close to a horizontal visual field. For this reason, an image with less distortion and less discomfort can be obtained.

  Next, the display means 13 and the display control means 14 will be described in detail.

  The display means 13 is a monitor such as a cathode ray tube, LED, or EL, for example, and receives the second image data 13a output from the output buffer memory 12b of the image processing means 12 to display an image, or a vehicle position such as GPS. The position of the own vehicle detected by the detection means 9 is displayed on the map screen. At this time, an image displayed on the display unit 13 by the display control unit 14 including a microcomputer (a panoramic image or a perspective transformation image obtained from the image processing unit 12, a pseudo bird's-eye view, the vehicle position detection unit 9). (Position display, etc.) can be switched and displayed, and the display direction, image size, and the like can be controlled.

  FIG. 10 is a diagram showing an example of a display screen displayed by the display unit 13 of FIG. Here, it is assumed that the omnidirectional visual sensor 11 is installed at the front or rear bumper or bonnet center position of the vehicle 1. Here, the pseudo bird's-eye view display is set so that a portion of the vehicle at a certain distance from the left and right is closer to the horizontal field of view.

  As shown in FIG. 10, on the display screen 131 of the display means 13, an explanation display unit 131A for explaining the perspective image display unit 131a directly below, a perspective image display unit 131a for displaying a perspective image in front of the vehicle by default, An explanation display part 133A for explaining the pseudo bird's eye view display part 133a directly below, a pseudo bird's eye view display part 133a for displaying the front of the vehicle by default, an up / down / left / right direction key 135, an image enlargement key 136 and an image reduction key 137 are displayed. Yes.

  The explanation display sections 131A and 133A are active switches of the image display sections 131a and 133a immediately below the explanation display sections 131A and 133A, respectively, and when the user operates the explanation display sections 131A and 133A, the image display sections 131a and 133a immediately below the display sections 131A and 133A. The display colors of the explanation display portions 131A and 133A change to indicate that the image display portions 131a and 133a are in the active state. By operating the direction key 135, the enlargement key 136, and the reduction key 137 when the image display units 131a and 133a are in the active state, the image can be moved up and down, left and right, enlarged or reduced, and the like.

  For example, when the user operates the explanation display unit 131A, the signal is sent to the display control unit 14, and the display control unit 14 changes the display color of the explanation display unit 131A to a color representing an active state or the explanation. The display unit 131A blinks and the fluoroscopic image display unit 131a is activated. When the user operates the direction key 135, the enlargement key 136, and the reduction key 137, a signal is sent from the display control means 14 to the image conversion means 12a of the image processing means 12 according to the signal, and corresponding to each key operation. The converted image data is sent to the display means 13 and displayed on the fluoroscopic image display unit 131a.

  As described with reference to FIGS. 7 to 9, the pseudo bird's-eye view display unit 133 a displays a bird's-eye view that overlooks the arrangement surface of the moving body (road surface in the case of an automobile) around the center of the field of view. For a part away from the center by a certain distance, it is located at a position away from the projective transformation center (mirror focus 1) within a plane that passes through the projective transformation center and is parallel to the plane of placement of the moving body (road surface in the case of an automobile). An image projected on the partial cylindrical surfaces 115a, 115b or the partial spherical surface 116 having an axis and having a radius between the projection transformation center (focal point 1) and the road surface is displayed. As a result, an object located away from the viewpoint is displayed in a form close to a horizontal field of view, and an image with less distortion and less discomfort can be obtained.

  FIG. 11 is a diagram showing another example of the display screen displayed by the display means 13 of FIG. Here, it is assumed that the omnidirectional visual sensor 11 is installed at a bumper or bonnet corner position in front of or behind the vehicle 1. Here, the pseudo bird's-eye view display is set so that a portion (a left or right side of the vehicle, the side where the sensor is attached, and the front or rear) of the omnidirectional visual sensor 11 is close to a horizontal visual field. Has been.

  As shown in FIG. 11, on the display screen 132, explanation display units 132 </ b> A to 132 </ b> C for explaining the directly below fluoroscopic image display units 132 a to 132 c, a fluoroscopic image display unit 132 b that displays a fluoroscopic image on the left side of the vehicle by default, A fluoroscopic image display unit 132a that displays a fluoroscopic image of the front of the vehicle by default, a fluoroscopic image display unit 132c that displays a fluoroscopic image of the right side of the vehicle by default, an explanation display unit 134A that describes a pseudo bird's-eye view display unit 134 directly below, and a vehicle by default A pseudo bird's eye view display unit 134a for displaying a front perspective image, up / down / left / right direction keys 135, an image enlargement key 136, and an image reduction key 137 are displayed.

  Similarly to the display screen 131 shown in FIG. 10, the explanation display sections 132A to 132C and 134A shown in FIG. 11 are active switches of the image display sections 132a to 132c and 134a directly below, respectively. By operating the units 132A to 132C and 134A, the image display units 132a to 132c and 134a immediately below the respective units can be moved up and down, left and right, and enlarged and reduced, and the display colors of the explanation display units 132A to 132C and 134A Changes to indicate that the image display units 132a to 132c and 134a are in the active state. By operating the direction key 135, the enlargement key 136, and the reduction key 137 when the image display units 132a to 132c and 134a are in the active state, the image can be moved up and down, left and right, enlarged and reduced, and the like. .

  In the first embodiment, an example in which one hyperboloidal mirror 111a is used as an optical system capable of central projective transformation as shown in FIG. 3 is shown, but in addition to this, as shown in FIG. Center projective transformation is also possible for an optical system using two hyperboloid mirrors 111a and 111b.

  As shown in FIG. 12, the omnidirectional vision sensor 11A includes one focal position (focal point 1) of a hyperboloid mirror (main hyperboloid mirror) 111a that hits the main mirror and a hyperboloid mirror (sub hyperboloid mirror) 111b that hits the sub mirror. Is coincident with the other focal position (focal point 1). Further, the hyperbolic rotation axis (optical axis of the hyperboloid mirrors 111a and 111b) J1 and the optical axis J2 of the imaging means 11b coincide with each other, and the other focal position (focal point 1) of the hyperboloid mirror 11b and the imaging means 11b. The lens principal points are arranged so as to coincide with each other. The tip of the convex portion of the hyperboloidal mirror 111a is not provided with a mirror surface so that light can be transmitted therethrough.

In this omnidirectional visual sensor 11, light directed to the focal point 1 of the hyperboloidal mirror 111a is reflected by the hyperboloidal mirror 111a and directed to one focal position (focal point 2) of the hyperboloidal mirror 111b. The light traveling toward the focal point 2 is reflected by the hyperboloidal mirror 111b, travels toward the other focal position (focal point 1) of the hyperboloidal mirror 111b, and is imaged by the imaging unit 11b. Therefore, the image captured by the imaging unit 11b does not change the viewpoint position depending on the viewing direction, and is an image with the focal point (focal point 1) of the hyperboloid mirror 111a as the viewpoint.
(Embodiment 2)
FIG. 13 (a) is a top view in Embodiment 2 of the automobile of the present invention, and FIG. 13 (b) is a side view thereof.

  As shown in FIGS. 13 (a) and 13 (b), an automobile (vehicle) 1 includes a rear bumper on a corner upper portion of the front bumper 2 (left corner portion in the second embodiment) and a diagonal line of the installation position thereof. The omnidirectional vision sensors 11 are respectively installed at the upper corners of the three corners (the right corner portion in the second embodiment). An image processing means 12 is provided in the front engine compartment or the rear cargo compartment of the vehicle 1. In the driver's seat of the vehicle 1, for example, a display unit 13 and a display control unit 14 are provided in the front panel or on the side of the panel.

  Each omnidirectional visual sensor 11 can obtain an image in a 360-degree viewing field centered on itself. For example, the omnidirectional visual sensor 11 installed on the front bumper 2 has an oblique rear viewing field (right side). The visual field area of 90 degrees in the substantially horizontal direction becomes a blind spot by the image of the host vehicle, and the effective visual field areas are 270 degrees on the front and left sides. Further, the omnidirectional visual sensor 11 installed on the rear bumper 3 has a visual field area of 90 degrees in the horizontal direction in the oblique front visual field area (left side) as a blind spot by the image of the own vehicle, and the effective visual field areas are the rear and right side. 270 degrees. Therefore, the field of view of both omnidirectional vision sensors 11 can be combined, and a field of view of approximately 360 degrees can be obtained in a region immediately adjacent to the vehicle that is likely to be a blind spot for the driver.

  FIG. 14 is a diagram illustrating an example of a display screen displayed by the display unit 13 according to the second embodiment.

  As shown in FIG. 14, on the display screen 138, an explanation display unit 138A for explaining the pseudo bird's-eye view display unit 138a directly below, and a pseudo bird's-eye view display unit 138a for displaying an image of a field of view around 360 degrees by combining the pseudo bird's-eye view. An up / down / left / right direction key 135, an image enlargement key 136, and an image reduction key 137 are displayed.

  In the pseudo bird's-eye view display unit 138a, among the areas 1 to 4 divided by the thick dotted line, the area 1 is displayed using the image data obtained by the omnidirectional visual sensor 11 installed on the front bumper 2, In the area 4, display is performed using image data obtained by the omnidirectional visual sensor 11 installed on the rear bumper 3. In areas 2 and 3, image data obtained by the omnidirectional visual sensor 11 installed on the front bumper 2, image data obtained by the omnidirectional visual sensor 11 installed on the rear bumper 3, and Any one of these is selectively used for display.

  Also in this pseudo bird's-eye view display unit 138a, as described with reference to FIGS. 7 to 9, a bird's-eye view overlooking the road surface is displayed around the center of the field of view, A cylindrical surface having an axis at a certain distance from the projective transformation center (mirror focus) in a plane parallel to the road surface through the projective transformation center and having a radius between the projective transformation center and the road surface, or An image projected on a partially spherical surface is displayed. As a result, an object located away from the viewpoint is displayed in a form close to a horizontal field of view, and an image with less distortion and less discomfort can be obtained.

  As described above, according to the first and second embodiments, an image in the 360 ° field of view is obtained, and an optical image obtained through the optical system 11a capable of central projective transformation on the image is obtained by the imaging unit 11b. Convert to image data. The image data is converted into panoramic image data or fluoroscopic image data by the image processing means 12, and an image corresponding to the second image data is displayed by the display means 13. The image near the center of the field of view is converted into image data that looks down on the road surface, and the part away from the center of the field of view through the projective transformation center has an axis at a position away from the center of projection transformation within a plane parallel to the road surface. And converting into image data projected onto a partial cylindrical surface or a partial spherical surface having a radius of the distance between the projection conversion center and the road surface. As a result, an image with a little uncomfortable feeling for the driver is obtained, and a moving body surroundings monitoring device that can easily check the surroundings of the moving body to reduce the burden on the driver and improve safety can be obtained. be able to.

  In the first and second embodiments, the description has been given by taking a passenger car (automobile) as an example of the moving body of the present invention, but in addition to this, a large vehicle such as a bus, a freight vehicle (truck), etc. The present invention can also be applied to other automobiles. In particular, in a freight vehicle, the driver's rear view is often obstructed by the cargo compartment, so that it is very useful to mount the moving object surroundings monitoring device of the present invention. In addition, the present invention is effective not only for automobiles but also for vehicles such as trains that are long before and after. Furthermore, the present invention can be effectively applied to all mobile objects, whether manned or unmanned, such as aircraft and ships, not limited to vehicles.

  Further, as described above, the present invention has been exemplified by using the above-described preferred embodiments 1 and 2 of the present invention. However, the present invention should not be construed as being limited to the first and second embodiments. It is understood that the scope of the present invention should be construed only by the claims. It is understood that those skilled in the art can implement an equivalent range based on the description of the present invention and the common general technical knowledge from the description of the specific preferred embodiments 1 and 2 of the present invention. Patents, patent applications, and documents cited herein should be incorporated by reference in their entirety, as if the contents themselves were specifically described herein. Understood.

  According to the present invention, it is possible to easily confirm a blind spot from the driver's seat of the moving body with an image that is less uncomfortable for the driver, so when checking the surroundings at the time of departure, when turning left or right, when parking, Safety confirmation can be more reliably performed at the time of entering / exiting from the garage, at the time of confirming left and right rear. Further, since it is not necessary for the driver to switch between a plurality of cameras displayed on the display means or to switch the direction of the cameras, safe driving can be reliably performed. INDUSTRIAL APPLICABILITY The present invention is very effective for ensuring the safety of all moving objects, whether manned or unmanned, such as vehicles such as automobiles, buses, and trains, airplanes, and ships.

(A) is the top view of Embodiment 1 of the motor vehicle carrying the mobile body surroundings monitoring apparatus of this invention, (b) is the side view. It is a block diagram which shows schematic structure of Embodiment 1 of the mobile body surroundings monitoring apparatus of this invention. FIG. 3 is a perspective view illustrating a configuration example of a main part of the optical system in FIG. 2. FIG. 3 is a block diagram illustrating a configuration example of a main part of the image processing unit in FIG. 2. FIG. 4 is a plan view for explaining conversion from an input image to a panoramic image in the moving object surrounding monitoring apparatus of FIG. 2. FIG. 4 is a perspective view for explaining conversion from an input image to a perspective projection image in the moving object surroundings monitoring apparatus of FIG. 2. (A) is a perspective view for demonstrating the conversion from the input image to the image projected on the partial cylindrical surface in the moving body surroundings monitoring apparatus of FIG. 2, (b) is the top view. (A) is the perspective view for demonstrating conversion to the image projected on the partial cylindrical surface and the partial spherical surface from the input image in the mobile body surrounding monitoring apparatus of FIG. 2, (b) is the upper surface. FIG. (A) And (b) is a schematic diagram for demonstrating the difference in the image displayed on a display means in the moving body surroundings monitoring apparatus of Embodiment 1 of this invention and this invention. FIG. 3 is a schematic diagram for explaining an example of a display form of a display unit in the moving object surrounding monitoring apparatus of FIG. 2. FIG. 10 is a schematic diagram for explaining another example of the display form of the display means in the mobile object surrounding monitoring apparatus of FIG. 2. It is a perspective view which shows the other structural example of the optical system in the moving body surroundings monitoring apparatus of FIG. (A) is the top view of Embodiment 2 of the motor vehicle carrying the moving body surroundings monitoring apparatus of this invention, (b) is the side view. FIG. 14 is a schematic diagram for explaining an example of a display form of a display unit in the moving object surrounding monitoring apparatus of FIG. 13.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Vehicle 2 Front bumper 3 Rear bumper 9 Vehicle position detection means 10 Moving body surrounding monitoring device 11 Omnidirectional visual sensor 11a Optical system 111a, 111b Hyperboloid mirror 11b Imaging means 111 Circular input image 112 On the way of cutting out and cutting out a circular input image into a donut shape State 113 of panorama image after enlargement 114, 115a, 115b Partial cylindrical surface (half-cylindrical surface part)
116 Partial spherical surface (hemispherical surface)
12 image processing means 12a image converting means 12b output buffer memory (image comparison distance detecting means)
12c Bus line 121a A / D converter 122a Input buffer memory 123a CPU
124a LUT
125a Image conversion logic 13 Display means 13a Output from image processing means 13b Output from vehicle position detection means 131, 132 Display screen 131A, 132A, 133A, 134A Explanation display part 131a, 132a-132c Perspective image display part 133a, 134a Bird's eye view display unit 135 Direction key 136 Expand key 137 Reduce key 14 Display control means 14a Control signal 15 Alarm generating means 20 Object located far from viewpoint J1, J Rotation axis of hyperboloid mirror K Partial axis 115a, 115b Partial cylindrical surface L Axis parallel to Z axis N Intersection of partial cylindrical surface axis

Claims (14)

An omnidirectional display including an optical system capable of obtaining an image of a 360-degree field of view and capable of central projective conversion with respect to the image, and imaging means for converting an optical image obtained through the optical system into first image data A visual sensor;
Image processing means for converting the first image data into at least panoramic image data or perspective image data;
Display means for displaying an image corresponding to the second image data obtained by the image processing means,
The image processing means converts the arrangement surface including at least a part of the moving body into an image data overlooking the vicinity of the visual field center, and the moving body in a visual field portion other than the vicinity of the visual field center at a certain distance from the visual field center. A moving body surroundings monitoring device having image conversion means for converting image data when viewed from above the arrangement surface in a lateral direction from the moving body side.   The image conversion means includes a cylindrical axis and a spherical center at a position apart from the projective transformation center in a plane parallel to the arrangement plane of the moving body including the projective transformation center in a visual field portion other than the vicinity of the visual field center. 2. The image data having at least one of the points and converted to image data projected onto at least one of a partial cylindrical surface and a partial spherical surface having a radius of a distance between the projective transformation center and a moving object arrangement surface. The moving body surrounding monitoring apparatus described.   The image conversion means, in a field portion other than the vicinity of the field center, in a plane that includes the projection conversion center and is parallel to the arrangement surface of the moving body, at each position spaced apart from the projection conversion center by a certain distance. At least one of each cylindrical surface and each spherical surface that has at least one of a cylindrical axis and a spherical center point, and whose radius is the distance between the projective transformation center and the moving object placement surface The moving body surroundings monitoring apparatus according to claim 1, wherein the moving object surrounding monitoring apparatus converts each of the projected image data.   The projection plane has two partial cylindrical planes each having a cylindrical axis perpendicular to each other at a position spaced apart from the projection transformation center within a plane that includes the projection transformation center and is parallel to the arrangement plane of the moving object. The partial spherical surface is a spherical surface having the same center radius as the intersection of the cylindrical axes of the two partial cylindrical surfaces and connecting the two partial cylindrical surfaces. The moving body surroundings monitoring device according to claim 2 or 3.   In the omnidirectional visual sensor, the optical system includes a hyperboloid mirror in which a convex surface of one curved surface of a two-leaf hyperboloid obtained by rotating a hyperbola around an axis is formed, and the imaging unit The optical axis thereof coincides with the rotation axis of the hyperbola, and the principal point of the lens is disposed at the other focal position of the two focal points of the hyperboloid mirror. Mobile object surrounding monitoring device.   In the omnidirectional vision sensor, the optical system is a mirror surface except for a convex portion that can transmit light on a convex surface of one curved surface of a two-leaf hyperboloid obtained by rotating a hyperbola around an axis. And a sub-hyperbolic mirror in which the convex surface of the other curved surface forms a mirror surface, and the imaging means has an optical axis that coincides with the rotation axis of the hyperbola, The moving body surroundings monitoring apparatus according to any one of claims 1 to 4, wherein the principal point of the lens is disposed at the other focal position of the two focal points of the hyperboloidal mirror. The display means also serves as a position display means for displaying a position image of the moving body on a map screen,
Moving body position detecting means for detecting the position of the moving body; and the display means for switching between the position image of the moving body and the surrounding image of the moving body based on position information detected by the moving body position detecting means. The moving body surroundings monitoring device according to claim 1, further comprising display control means that can be controlled.   A moving body on which the moving body surroundings monitoring device according to claim 1 is mounted.   The mobile body according to claim 8, wherein the mobile body surrounding monitoring device is installed on at least one of the front and rear bumpers.   The moving body surroundings monitoring device is installed at one of the positions on the right end and the left end of the front bumper, and is also installed on a rear bumper on a diagonal line with respect to the installation position. Item 10. The moving object according to Item 9.   In the vicinity of the center of the field of view, the first image data obtained by capturing an optical image obtained through an optical system capable of central projective transformation with respect to the image of the 360-degree field of view around the image is captured by the imaging means as the second image data. The image data including the at least part of the moving object is converted into image data for bird's-eye view, and the visual field part other than the vicinity of the visual field center that is a fixed distance away from the visual field center is above the moving object from above the moving object side. An image conversion method for converting image data when viewed in the horizontal direction.   When converting the first image data into the second image data, in a field portion other than the vicinity of the field center, within the plane including the projection conversion center and parallel to the arrangement surface of the moving object, the projection conversion center At least one of a cylindrical axis and a spherical center point at a certain distance away from each other, and at least one of a partial cylindrical surface and a partial spherical surface having a radius between the projective transformation center and the moving object placement surface The image conversion method according to claim 11, wherein the image is converted into the projected second image data.   When converting the first image data into the second image data, in a field portion other than the vicinity of the field center, within the plane including the projection conversion center and parallel to the arrangement surface of the moving object, the projection conversion center Each cylindrical surface having at least one of a cylindrical axis and a spherical center point at each position separated by a certain distance in the opposite direction, each having a radius between the projective transformation center and the moving surface The image conversion method according to claim 11, wherein each image is converted into the second image data projected onto at least one of the partial spherical surfaces.   The projection plane includes two partial cylindrical planes having cylindrical axes orthogonal to each other at positions spaced apart from the projection transformation center by a predetermined distance in a plane that includes the projection transformation center and is parallel to the arrangement plane of the moving object. The partial spherical surface is a spherical surface having the same radius as the radius of the partial cylindrical surface and connecting the two partial cylindrical surfaces with the center of the intersection of the cylindrical axes of the two partial cylindrical surfaces. The image conversion method according to claim 12 or 13.

Images