JP2005234224A - All azimuth imaging system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an all azimuth imaging system capable of attaining the correspondence of objects between stereoscopic images with simple processing. <P>SOLUTION: The system is provided with an all azimuth camera, a gyro-sensor, a portable PC (personal computer) and a video server. The all azimuth camera is provided with an all azimuth mirror 32 capable of reflecting light incident from all the directions of 360° and a camera. Regarding the all azimuth mirror 32, six small semi-spherical mirrors 32b are integrally formed on a semi-spherical mirror 32a. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to an omnidirectional imaging system, and more particularly to an omnidirectional imaging system capable of simultaneously acquiring images and distance information at low cost.

Conventionally, research and development of portable surveillance cameras and fixed point surveillance cameras have been promoted in order to monitor the surrounding environment of people and prevent crimes.
The role of the surveillance camera for monitoring the surrounding environment of such a person is to find and record the object that appears in the field of view that the person does not notice. A person is aware of the front vision by his vision, but unconsciously has not acquired information on the side and rear vision. For this reason, a malicious criminal who suddenly appears from outside the human eye, or a perpetrator who collides inadvertently even if it is not malicious, can detect an accident in advance and avoid or deal with it. The incident can be prevented in advance.

An omnidirectional camera has been developed to acquire such wide-angle field of view information including the side and the back, and studies on reflective optical systems that are conscious of the side field of view are being actively studied in the field of robotics. A feature of the omnidirectional camera is that a side image of 360 degrees around can be acquired by one shooting.
In this way, taking advantage of omnidirectional observation, the omnidirectional camera can be used for remote video conference (for example, see Non-Patent Document 1), the omnidirectional camera can be used as a fixed point monitoring camera, There are attempts to use it for tracking a moving object or extracting an event (for example, see Non-Patent Document 2).

In addition, as one of various technologies proposed as an application of robotics, there is a method of observing omnidirectional depth information using an omnidirectional camera. In this method, two mirrors having curved surfaces are superimposed on the optical axis of the camera. Thereby, the depth distance of all directions can be measured in real time from the input image obtained with one camera (for example, refer nonpatent literature 3). The depth distance of the object is obtained from the correspondence of the stereo method, but there is a problem that it is difficult to obtain the correspondence such as false correspondence of the object. In order to simplify such association, there is a technique called voting. Furthermore, there is a technique in which a plurality of images captured by a moving camera are arranged in time and space, and the trajectory is cut along a plane called EPI (Epipolar Plane Image) to obtain distance information from the trajectory.
Nishihara, T., Yabe, H., and Oka, R., "Indexing of Human Motion at Meeting Room by Analyzing Time-varying Images of Omni-directional Camera," Proc. 4th ACCV, vol. I, pp. 1- 4, 2000. Onoe, Y., Yokoya, Y., Yamazawa, K., and Takemura, H., "Visual Surveillance and Monitoring System Using an Omnidirectional Video Camera," Proc. 14th ICPR, vol. I, pp. 588-592, 1998 . Southwell, D., Reyda, J., Fiale, M., and Basu, A., "Panoramic Stereo," Proc. ICPR, 1996.

As described above, an omnidirectional camera is effective for observing the entire periphery. However, in order to obtain images and distance information (depth information) simultaneously, a plurality of omnidirectional cameras are required, which is problematic in terms of cost.
In addition, in order to obtain distance information, it is necessary to associate objects between stereo images obtained from a plurality of omnidirectional images, but there is a problem that processing is complicated.

  Furthermore, the video recording the situation is evidence of the situation of the accident or incident, but there is a problem in terms of cost to always record the video of the entire surroundings. For this reason, there is a request for detecting an object approaching the camera and cutting out and recording only an image in which the object is captured.

  In addition, in the application of the conventional omnidirectional camera to remote video conference and fixed point monitoring, it is necessary for a remote user to interactively specify a location where the user wants to know the state of the conference and a location to be monitored. Thereafter, a perspective projection image (image viewed with a normal camera) of the designated place is presented to the user. However, when considering application to monitoring of a person's surrounding environment, it is not known from which direction the object appears, and the cutout location cannot be specified interactively. For this reason, a perspective projection image cannot be generated in real time.

  Moreover, there exists an omnidirectional camera in which two mirrors having curved surfaces are superimposed on the optical axis of the camera as in Non-Patent Document 3. However, there is a known problem that a false correspondence occurs when stereo viewing is performed using the triangulation principle from images observed from two viewpoints (for example, M. Yachida, Y. Kitamura, M Kimachi, "Trinocular vision: New approach for correspondence problem," Proc. 8th Int'l Conf. Pattern Recognition, pp.1041-1044, 1986).

The present invention has been made to solve the above-described problems, and an object thereof is to provide an omnidirectional imaging system capable of acquiring an image and distance information at a low cost.
It is another object of the present invention to provide an omnidirectional imaging system capable of associating objects between stereo images with simple processing.
It is another object of the present invention to provide an omnidirectional imaging system that can record an image of an object at low cost.

It is another object of the present invention to provide an omnidirectional imaging system capable of generating a reverse perspective projection image in real time.
It is another object of the present invention to provide an omnidirectional imaging system capable of specifying an accurate three-dimensional position of an object without causing false correspondence of the object.

In order to achieve the above object, an omnidirectional imaging system according to the present invention includes three or more mirrors each having a shape capable of reflecting 360 degrees of surrounding light, and light reflected by the three or more mirrors. And a single camera for imaging.
A single camera captures light reflected by three or more mirrors. Distance information can be obtained by associating images reflected by three or more mirrors. For this reason, the omnidirectional imaging system which can acquire an image and distance information at low cost can be provided.

  Preferably, the omnidirectional imaging system further includes a mirror combination extraction unit that extracts a combination of two mirrors from the three or more mirrors, and each of the mirror combinations extracted by the mirror combination extraction unit. A distance calculating means for calculating a distance to the object based on the reflected images of the two mirrors included in the combination obtained from the image captured by the camera and the baseline length of the two mirrors; Prepare.

The distance to the object is calculated based on images obtained from three or more mirrors. Therefore, it is possible to provide an omnidirectional imaging system that can specify an accurate three-dimensional position of an object without causing false correspondence of the object.
More preferably, the above-described omnidirectional imaging system further determines whether or not the object has entered a virtual sphere having a predetermined radius centered on the camera, based on the distance to the object. And a warning generation means for issuing a warning when the object has entered the phantom spherical surface.

  According to this configuration, it is possible to issue a warning and call attention to an object that has entered the virtual spherical surface. For this reason, accidents and incidents can be prevented in advance.

  The omnidirectional imaging system further includes a line of sight for obtaining a line-of-sight direction from the camera to an object based on each of the reflected images of the three or more mirrors included in an image captured by the camera. Vote for direction acquisition means, virtual sphere generation means for generating a virtual sphere centered on the camera and divided by voxels of a predetermined size, and voxels intersecting the line of sight obtained by the line-of-sight direction acquisition means A voting unit that performs the voting, and a three-dimensional position specifying unit that specifies a three-dimensional position of the object based on a voxel that has been voted above a predetermined threshold based on a voting result. .

  The position of a voxel subject to voting is limited to a virtual sphere, and a voxel that intersects the line of sight from the camera is voted as the position where the object exists. For this reason, the omnidirectional imaging system which can perform the matching of the target object between stereo images by simple processing can be provided.

  In general, when voting, a plurality of viewpoints are provided around an object, and voting is performed from the viewpoint in the line-of-sight direction with respect to the object. This is because if the camera is installed in the center and the vote is made in the direction of the entire circumference, the voting target position will spread infinitely. Therefore, the space for voting must be narrowed by some method.

  In this configuration, the target space for voting is limited to a virtual sphere. For this reason, voting with the camera at the center is possible.

  More preferably, the above-mentioned omnidirectional imaging system further determines the line-of-sight direction from the camera to the object based on each of the reflected images from the three or more mirrors included in the image captured by the camera. Intersects with the gaze direction obtained by the gaze direction obtaining means, virtual sphere surface generation means for producing a virtual sphere surface divided by cells of a predetermined size around the camera, and the gaze obtained by the gaze direction obtaining means Voting means for voting on a cell, and three-dimensional position specifying means for specifying a three-dimensional position of an object based on a cell on which voting of a predetermined threshold value or more is performed based on a voting result May be.

  The voting position is limited to a virtual sphere surface cell. For this reason, there are few voting locations compared with the case of voting on a voxel. For this reason, the omnidirectional imaging system which can perform the matching of the target object between stereo images by a further simple process can be provided.

  More preferably, the above-mentioned omnidirectional imaging system further determines whether or not the object has entered a virtual sphere having a predetermined radius centered on the camera based on the three-dimensional position of the object. A determination unit for determining, and when the object has entered the virtual spherical surface, based on an image captured by the camera, the object is perspective-projected and converted to an image viewed from a normal camera; Perspective projection conversion means for presenting the converted image to the user.

  The three-dimensional position of the object is obtained by a simple method called voting. For this reason, the line-of-sight direction from the camera to the object can also be obtained immediately. Therefore, it is possible to provide an omnidirectional imaging system capable of generating a reverse perspective projection video (video viewed from a normal camera) in real time.

  More preferably, the above-described omnidirectional imaging system further includes recording means for recording the image after the perspective projection conversion.

  Provided is an omnidirectional imaging system capable of recording an image of an object at a lower cost than when recording a raw image captured by a camera because only a reverse perspective projection image can be recorded. it can.

  More preferably, the above-mentioned omnidirectional imaging system further includes a stereoscopic image generating unit that generates a stereoscopic image by arranging the reflection images of the three or more mirrors included in the image captured by the camera in a line, and Curve extraction means for extracting a curve from the stereoscopic image, and determination means for determining whether or not an object exists within a predetermined distance from the camera based on the extraction result of the curve extraction means. May be. Preferably, the curve extracting means includes a removing unit that removes a straight line from the stereoscopic image, and an extracting unit that extracts a curve from the stereoscopic image from which the straight line has been removed.

  If parallax occurs between images, that is, if the object is in the vicinity of the camera, whether or not the object exists in the vicinity using the property that the object is extracted as a curve in the stereoscopic image Judgment. For this reason, the distance to a target object can be measured by a simple method.

  More preferably, the omnidirectional imaging system further includes an inclination detection unit that detects the inclination of the camera, and an inclination correction unit that corrects an inclination of an image captured by the camera based on the inclination of the camera. Is provided.

  By correcting the tilt of the image captured by the camera based on the tilt of the camera detected by the tilt detection means such as a gyro sensor, even if the camera is subjected to vibration or the like, An exact distance or an exact 3D position of the object can be determined.

  By using the omnidirectional imaging system as described above, the following effects can also be obtained. In other words, depending on crime prevention for fixed-point surveillance cameras in the city, if there are many obstacles that block the camera's line of sight, blind spots are likely to occur even if the number of installed cameras is increased. For this reason, it is difficult to determine the arrangement position of the camera. Also, fixed-point surveillance cameras have a privacy infringement problem. Therefore, even in crime prevention, it is possible for an individual to carry a small camera, so that it is possible to constantly monitor the proximity area of the individual user, and the effect on crime prevention without excessively infringing on the privacy of others. Can be realized.

As described above, according to the present invention, it is possible to provide an omnidirectional imaging system capable of acquiring an image and distance information at a low cost.
It is also possible to provide an omnidirectional imaging system that can associate objects between stereo images with simple processing.

  Furthermore, it is possible to provide an omnidirectional imaging system that can record an image of an object at low cost.

Furthermore, it is possible to provide an omnidirectional imaging system capable of generating a reverse perspective projection image in real time.
Furthermore, it is possible to provide an omnidirectional imaging system capable of specifying an accurate three-dimensional position of an object without causing false correspondence of the object.

Hereinafter, an omnidirectional imaging system according to an embodiment of the present invention will be described with reference to the drawings.
FIG. 1 is a block diagram showing a hardware configuration of an omnidirectional imaging system according to an embodiment of the present invention. The omnidirectional imaging system 20 includes an omnidirectional camera 22, a gyro sensor 24, a portable PC (Personal Computer) 26, and a video server 28.

  The omnidirectional camera 22 is a camera that can capture an image of 360 degrees around. Details of the omnidirectional camera 22 will be described later. The gyro sensor 24 is a sensor for detecting the tilt of the omnidirectional camera 22. Based on the tilt of the omnidirectional camera 22 detected by the gyro sensor 24, the portable PC 26 corrects the tilt of the video imaged by the omnidirectional camera 22 and cuts out the video. The portable PC 26 is connected to the video server 28 via a wireless network or the like, and transmits the cut image to the video server 28. Further, an alarm is output based on the monitoring judgment result obtained from the video server 28. Furthermore, an image obtained from the video server 28 is displayed.

  The video server 28 is connected to the portable PC 26 via a wireless network or the like, analyzes the video received from the video server 28, determines whether or not an object has entered the periphery of the omnidirectional camera 22, and Or generate an image. The video server 28 transmits the determination result and the image of the object to the portable PC 26.

  FIG. 2 is an external view showing the configuration of the omnidirectional camera 22. The omnidirectional camera 22 includes an omnidirectional mirror 32 and a camera 34. The omnidirectional mirror 32 is a mirror that reflects light incident from the direction of 360 ° around. Details of the omnidirectional mirror 32 will be described later. The camera 34 captures an image reflected by the omnidirectional mirror 32.

  FIG. 3 is a diagram illustrating an example of the installation positions of the omnidirectional camera 22 and the gyro sensor 24. As shown in FIG. 3, an omnidirectional camera 22 and a gyro sensor 24 may be installed on the ceiling of an automobile to monitor approaching vehicles and approaching persons from the surroundings.

  FIG. 4 is a diagram illustrating another example of the installation positions of the omnidirectional camera 22 and the gyro sensor 24. As shown in FIG. 4, an omnidirectional camera 22 and a gyro sensor 24 may be installed at the shoulder position of the shoulder bag to monitor an approaching person from the surroundings.

  FIG. 5 is a diagram showing still another example of the installation positions of the omnidirectional camera 22 and the gyro sensor 24. As shown in FIG. 5, an omnidirectional camera 22 and a gyro sensor 24 may be installed in the school bag to monitor an approaching person from the surroundings.

  FIG. 6 is a diagram showing still another example of the installation positions of the omnidirectional camera 22 and the gyro sensor 24. As shown in FIG. 6, an omnidirectional camera 22 and a gyro sensor 24 may be installed on the cap to monitor an approaching person from the surroundings.

  FIG. 7 is a diagram illustrating yet another example of the installation positions of the omnidirectional camera 22 and the gyro sensor 24. As shown in FIG. 7, an omnidirectional camera 22 and a gyro sensor 24 may be installed on a motorcycle to monitor an approaching object from the surroundings. By placing the portable PC 26 at a position where the driver can visually recognize the driver, the driver can monitor the side and rear approaching persons.

8A and 8B are external views showing the configuration of the omnidirectional mirror 32, where FIG. 8A is a side view and FIG. 8B is a bottom view. In the omnidirectional mirror 32, six small semispherical mirrors 32b are integrally formed on a semicircular mirror 32a.
In this way, the user is informed in advance of the presence of an unconfirmed object to be approached from the acquired information by carrying the omnidirectional camera 22 and the gyro sensor 24 that are miniaturized. The portable PC 26 also has a display system that calls attention. The video acquired by the omnidirectional camera spreads in all directions with the user as the center, but the resolution decreases as the distance from the user increases, making discrimination difficult. However, in the vicinity of the user, a high-resolution and recognizable video can be obtained.

  Further, as described above, the omnidirectional mirror 32 is formed using a plurality of semicircular spherical mirrors 32b. For this reason, the distance of a target object can be detected similarly to the stereoscopic vision using a some camera. As shown in FIG. 8, the distance connecting the center of the curved surface of each mirror 32b is the base line length, and parallax occurs between the mirrors 32b.

  FIG. 9 is a diagram for explaining a method for obtaining a depth distance from an image obtained from two mirrors 32b. Here, an image obtained from the two mirrors 32b is an image 42a and an image 42b. Assuming that the base line length is d, the relationship between the point 44 in the space and the video 42a and the video 42b can be expressed by the following equations (1) and (2). The depth from each mirror 32b can be calculated from this equation. Although the distance that can be determined from two curved surfaces is shown in FIG. 9, the same object is projected onto a plurality of curved surfaces, so that the distances from all combinations can be calculated.

カメラから複数の全方位を投影する曲面に対する光軸は、それぞれのミラー３２ｂに対して異なった角度から投影される。そのため、図１０（ａ）に示すように無限遠方の直線は同じ方位となるが、図１０（ｂ）に示すように距離が近くなるに従い、方位にずれを生じる。それに加え、光軸の傾きにより、同じ高さの物体に対しても光軸中心から放射方向までの距離が異なる。ただし、ミラーは設計段階で基線長が決められ、カメラに設置した後、事前にキャリブレーションが可能である。このため、光軸の傾きを仮想的に光軸が平行であるように予め変換することが可能である。

The optical axis for the curved surface that projects a plurality of omnidirectional directions from the camera is projected from different angles to the respective mirrors 32b. Therefore, the straight line at infinity has the same azimuth as shown in FIG. 10A, but the azimuth shifts as the distance becomes shorter as shown in FIG. 10B. In addition, due to the inclination of the optical axis, the distance from the center of the optical axis to the radiation direction differs even for an object of the same height. However, the base length of the mirror is determined at the design stage, and the mirror can be calibrated in advance after being installed in the camera. For this reason, the inclination of the optical axis can be converted in advance so that the optical axis is virtually parallel.

Next, two methods will be described as methods for actually obtaining the distance to the object.
The first method is three-dimensional detection by voting. In binocular stereoscopic vision, finding the corresponding points between images requires complicated processing, but in multi-view vision, the space is divided into voxels of a predetermined size, and the line of sight obtained from the images is converted into voxels. When voting, a true three-dimensional point results in multiple votes. With one conventional omnidirectional camera, the line of sight is radiating from the center, so it is impossible to vote for a plurality of lines of sight, but voting is possible by using a plurality of mirrors 32b. This is shown in FIG. A virtual sphere having a detection limit radius in the vicinity of the camera is generated. The inside of the sphere is divided into voxels 52 as shown in FIG. 12, and for each of the images obtained from the plurality of mirrors 32b, a vote is given to the voxel through which the line of sight of the object passes. When an object is in a sphere, votes from more than one line of sight are always in the corresponding voxel. Therefore, a voxel having a plurality of votes (a vote exceeding a predetermined threshold) is set as a three-dimensional position of the object. However, since voting is limited to nearby spheres, the processing becomes simple.

  As shown in FIG. 13, the surface of a virtual sphere having a radius of detection limit may be divided into a plurality of cells 54, and voting may be performed on the cells 54. In this way, voting can be performed at the boundary where an object enters the virtual sphere, and the voting is further limited to the surface of the sphere, thereby further simplifying the processing.

  Next, in the second method, a simple method for detecting the presence or absence of parallax in an image obtained from a plurality of mirrors 32b will be described instead of the distance. As shown in FIG. 14, the omnidirectional images acquired from the plurality of mirrors 32b are arranged in a line to generate a stereoscopic image (three-dimensional image) 62. An object far away from the camera 34 has the same orientation as described above. For this reason, since no parallax appears between the images, a straight line 64 is drawn in the stereoscopic image 62. However, since an object existing in the vicinity of the camera 34 generates parallax, the parallax appears depending on the baseline length. For this reason, a curve 66 is drawn in the stereoscopic image 62. Therefore, in the stereoscopic image 64, by using the straight line removal filter and removing the straight line 64, it is possible to easily extract only a region with parallax and determine a nearby object.

  The distance accuracy of the depth distance decreases as the distance from the user increases. On the other hand, the distance of the object near the user can be obtained with high accuracy. Therefore, as shown in FIG. 15, according to the omnidirectional imaging system 20, a barrier 72 is formed in the vicinity of the user, the distance to an unconfirmed object that has entered the interior is measured, and the object is displayed to the user as an image. You can be informed.

Note that the image captured by the omnidirectional camera 22 is different from the real world for the user, and the surrounding omnidirectional image is displayed on a rectangular imaging surface. Therefore, the user can acquire information in all directions without being aware of the direction of the camera. However, since the omnidirectional image is a refracted image, it is difficult for the user to recognize the object even when viewing the image.
For this reason, in order to convey a video without a sense of incongruity to the user, it is necessary to convert the video into a perspective projection image similar to an image taken with a normal camera. In order to acquire the line of sight of the user when converting to a perspective projection image, the object approaching the user is automatically detected as described above, and the direction to be viewed by the user (the line of sight of the user) is set.

When the distance from the user to the object is closer than a certain distance and enters the inside of the virtual barrier 72, the user is warned by an alarm or the like as an approach of the unconfirmed object, and the direction is presented. At the same time, based on the direction, the image of the omnidirectional camera is cut out to be the same as the image of the normal camera, converted into a perspective projection image, and displayed.
The omnidirectional camera 22 carried by the user captures the surroundings of the user, but is set so that there is no inclination of the omnidirectional camera 22 when carrying it. Further, since the omnidirectional camera 22 moves with the user's movement, the user carries the gyro sensor 24 together with the omnidirectional camera 22 in order to correct the inclination due to the movement. Based on the output of the gyro sensor 24, video segmentation with the tilt corrected is performed. These processes are performed by the portable PC 26 owned by the user. On the other hand, in order to store the video, the video obtained from the omnidirectional camera 22 and the output of the gyro sensor 24 are transferred to the video server 28, and the user's action history is always stored in the video server 28. The data is transferred to the portable PC 26.

  As described above, according to the present embodiment, the light reflected by the six mirrors is captured by one camera. Therefore, distance information can be obtained by associating the images reflected by the mirror. For this reason, the omnidirectional imaging system which can acquire an image and distance information at low cost can be provided.

  Further, the distance to the object is calculated based on images obtained from the six mirrors. Therefore, unlike the case where the objects are associated based on the images obtained from the two mirrors, it is possible to specify an accurate three-dimensional position of the object without causing false correspondence of the object. An imaging system can be provided.

  Further, the distance to the object can be obtained by a technique of voting or extracting a curve from a stereoscopic image. Therefore, the omnidirectional distance can be measured at high speed, and a highly accurate measurement result can be obtained near the user. For this reason, a warning can be issued to an object approaching within a certain distance, and the video can be displayed.

  Moreover, the arrival time to the user can be predicted and detected in advance based on the distance information. Therefore, this system can be used for vehicles including bicycles and motorcycles, and can be used for ensuring safety from nearby traffic during driving. Especially on highways, etc., if you change the lane without noticing the vehicle running on the adjacent lane, it will lead to a major accident, but by using this invention that monitors the rear side, the driver is alerted in advance, The danger can be confirmed. Moreover, since this video is a running record, it can be used as a proof of the situation when an accident occurs.

The omnidirectional imaging system according to the embodiment of the present invention has been described above, but the present invention is not limited to this embodiment.
For example, although a hemispherical curved mirror is used as the mirror 32b, any shape may be used as long as it is a mirror capable of reflecting 360-degree light such as a hyperboloidal mirror and a parabolic mirror.
Further, the number of mirrors 32b is not limited to six, and may be more or less.

  The present invention can be applied to an omnidirectional imaging system, and in particular, can be applied to monitoring a surrounding environment of a vehicle including a user, a bicycle and a motorcycle.

It is a block diagram which shows the hardware constitutions of the omnidirectional imaging system which concerns on embodiment of this invention. It is an external view which shows the structure of an omnidirectional camera. It is a figure which shows an example of the installation position of an omnidirectional camera and a gyro sensor. It is a figure which shows another example of the installation position of an omnidirectional camera and a gyro sensor. It is a figure which shows another example of the installation position of an omnidirectional camera and a gyro sensor. It is a figure which shows another example of the installation position of an omnidirectional camera and a gyro sensor. It is a figure which shows another example of the installation position of an omnidirectional camera and a gyro sensor. It is an external view which shows the structure of the omnidirectional mirror 32, (a) is a side view, (b) is a bottom view. It is a figure for demonstrating the method of calculating | requiring the depth distance from the image | video obtained from a certain two mirrors. It is a figure for demonstrating the relationship between the distance to an object, and an azimuth | direction. It is a figure which shows the mode of voting. It is a figure which shows a mode that the inside of the bulb | ball was divided | segmented into the voxel. It is a figure which shows a mode that the surface of the bulb | ball was divided | segmented into the cell. It is a figure for demonstrating the method to obtain the distance to a countermeasure. It is a figure for demonstrating the virtual barrier provided in the vicinity of a user.

Explanation of symbols

20 Omnidirectional Imaging System 22 Omnidirectional Camera 24 Gyro Sensor 26 Portable PC
28 Video Server 32 Omnidirectional Mirror 34 Camera 32a, 32b Mirror 52 Voxel 54 Cell 62 Stereo Image 72 Barrier

Claims (11)

Three or more mirrors each having a shape capable of reflecting 360 degrees of ambient light;
An omnidirectional imaging system comprising: one camera that images light reflected by the three or more mirrors. Furthermore, mirror combination extraction means for extracting a combination of two mirrors from the three or more mirrors;
For each of the mirror combinations extracted by the mirror combination extraction means, based on the reflected video from the two mirrors included in the combination obtained from the image captured by the camera and the baseline length of the two mirrors The omnidirectional imaging system according to claim 1, further comprising a distance calculating unit that calculates a distance to the object. A determination unit configured to determine whether or not the object has entered a virtual spherical surface having a predetermined radius centered on the camera based on a distance to the object;
The omnidirectional imaging system according to claim 2, further comprising warning generation means for issuing a warning when the object enters the virtual spherical surface. Furthermore, based on each of the reflected images of the three or more mirrors included in the image captured by the camera, a line-of-sight direction obtaining unit that obtains a line-of-sight direction from the camera to the object;
Virtual sphere generating means for generating a virtual sphere centered on the camera and separated by voxels of a predetermined size;
Voting means for voting on voxels that intersect the line of sight obtained by the line-of-sight direction obtaining means;
The three-dimensional position specifying means for specifying a three-dimensional position of an object based on a voxel for which voting equal to or greater than a predetermined threshold is performed based on a voting result. Omnidirectional imaging system. Furthermore, based on each of the reflected images of the three or more mirrors included in the image captured by the camera, a line-of-sight direction obtaining unit that obtains a line-of-sight direction from the camera to the object;
Virtual sphere surface generation means for generating a virtual sphere surface divided by cells of a predetermined size around the camera;
Voting means for voting on a cell that intersects the line of sight obtained by the line-of-sight direction obtaining means;
The three-dimensional position specifying means for specifying a three-dimensional position of an object based on a cell in which voting equal to or greater than a predetermined threshold is performed based on a voting result. Omnidirectional imaging system. A determination unit configured to determine whether or not the object has entered a virtual spherical surface having a predetermined radius centered on the camera based on a three-dimensional position of the object;
When the target object enters the virtual spherical surface, based on the image captured by the camera, the target object is perspective-projected into an image viewed from a normal camera, and the image after the perspective projection conversion is converted to a user. The omnidirectional imaging system according to claim 4, further comprising: a perspective projection conversion unit presented in The omnidirectional imaging system according to claim 6, further comprising recording means for recording the image after the perspective projection conversion. Furthermore, a stereoscopic image generation means for generating a stereoscopic image by arranging the reflection images of the three or more mirrors included in the image captured by the camera in a line;
Curve extracting means for extracting a curve from the stereoscopic image;
2. The omnidirectional imaging according to claim 1, further comprising: a determination unit configured to determine whether or not an object is present within a predetermined distance from the camera based on an extraction result of the curve extraction unit. system. The curve extracting means includes
A removal unit for removing a straight line from the stereoscopic image;
The omnidirectional imaging system according to claim 8, further comprising: an extraction unit that extracts a curve from the stereoscopic image from which a straight line is removed. 10. The apparatus according to claim 8, further comprising warning generation means for issuing a warning when the object enters within the predetermined distance from the camera based on a determination result of the determination means. Omnidirectional imaging system. Furthermore, an inclination detecting means for detecting the inclination of the camera;
The omnidirectional imaging system according to any one of claims 1 to 10, further comprising an inclination correction unit that corrects an inclination of an image captured by the camera based on an inclination of the camera.

Images