WO2004084542A1

WO2004084542A1 - Panoramic picture creating method and device, and monitor system using the method and device

Info

Publication number: WO2004084542A1
Application number: PCT/JP2003/003403
Authority: WO
Inventors: Seijiro Tomita
Original assignee: Seijiro Tomita
Priority date: 2003-03-20
Filing date: 2003-03-20
Publication date: 2004-09-30
Also published as: AU2003236050A1

Abstract

A panoramic picture creating method and device. A tilt-rotatable and/or pan-rotatable mirror (2) is inclinedly provided in a position opposed to the imaging face of a single-chip CCD camera (1). Division images (1-16) created every rotational angle of the mirror (2) are stored. A panoramic picture is created by connecting the division images (1-16) on one screen of an image display device (27). Thus a wide range of monitoring using a camera (1) is achieved. The created panoramic picture is as sharp as that created by a high-precision wide camera. The panoramic picture creating device can be inexpensive and small and can, if the scene changes, automatically indicate the change to the monitoring person. A monitor system using the method and device is also disclosed.

Description

Specification

Panoramic image generation method and device, and monitoring system using these methods and devices

Technical field

The present invention relates to, for example, a panoramic image generating method and a panoramic image generating method capable of capturing a monitoring range of 360 degrees around the entire circumference with a single camera or the like and connecting and displaying the panoramic images. .

Technology background

At present, most conventional monitoring devices use a video camera to photograph the periphery of a monitoring target and monitor or record images.

In such a conventional monitoring device, when monitoring a wide area, the video camera must be slowly rotated in the tilt direction (up and down direction) and the pan direction (horizontal direction) for monitoring. Since the rotary drive device becomes large and the power supply cable and the image transmission cable must be wired, the cable is often worn or disconnected, and is not durable. However, since the video camera rotates slowly, there is a problem that blind spots are easily generated.

For this reason, for example, when monitoring the surroundings of nuclear power facilities or military facilities, to eliminate blind spots, multiple cameras equipped with wide viewing angle lenses are arranged around the facility at required intervals, and At present, the images are projected on multiple image display devices placed in the monitoring room and monitored.

However, it is not possible to monitor a plurality of image display devices while maintaining tension for a long time, physically or mentally, and there is a risk that a serious accident may be overlooked.

As means for solving such a conventional problem, there is, for example, an all-around panoramic image forming method and apparatus as disclosed in Japanese Patent Application Laid-Open No. 11-43373.

In this conventional method and apparatus, a mirror is inclined and installed at a position facing an imaging surface of an imaging device, and the mirror is rotated 360 degrees around the optical axis of the imaging device as a rotation axis. The state and the final state rotated 360 degrees from the initial state are appropriately divided according to the rotation angle of the mirror. Each of the image information stores the image information on the right side, performs a rotation conversion on the stored image information about the center of the image corresponding to each rotation angle, and inverts the image information after the rotation conversion to a mirror image. This configuration is such that parallel movement and pattern matching are performed on the inverted image information, and the image information is connected to obtain a panoramic image all around. However, in such a conventional technique for obtaining an entire panoramic image, since each image to be stored is distorted in a fixed pattern at each rotation angle, even if a pattern matching process is performed, In actuality, the panorama is displayed without correcting the distortion, so when compared to a single screen shot with a high-precision wide camera, an image that is different from the actual scene is displayed, making it impractical. Had the problem that

In addition, the above-described conventional technique for obtaining an all-around panoramic image has a structure that does not involve matching processing between the displayed image corresponding to each rotation angle and the updated image captured at the same rotation angle. Even if there is a different scene between the previous image and the image to be updated, the observer has no means to confirm this, so it is easy for the observer to overlook the above change, and to prevent an accident from occurring. There was also a problem that it was not possible. The present invention was conceived in view of such circumstances, and its purpose is to enable a single camera to monitor a wide range, and to capture the obtained image with a high-precision wide camera. It is possible to obtain a clear image that is comparable to the original image, and if the scenery changes, it is possible to automatically inform the observer of the change. An object of the present invention is to provide a generation method, a device therefor, and a monitoring system using the method and device.

Disclosure of the invention

In order to achieve the above object, the invention described in claim 1 tilts a mirror configured to be tiltable and / or pannable at a position facing an imaging surface of a single camera. The divided images obtained according to the rotation angle of the mirror are accumulated, and the divided images are joined on one screen of an image display device to generate a panoramic image. The image is characterized in that the distortion is corrected by the correction coefficient for the imaging position corresponding to the rotation angle of the mirror and displayed. As a result, according to the present invention, it is possible to obtain a clear panoramic image having no blind spots, which is comparable to an image captured by a high-precision wide camera. In addition, when an image is obtained by rotating the mirror in the tilt direction and the pan direction, it is necessary to correct distortion in both directions and the distortion correction circuit becomes complicated. When the camera is also rotated in the pan direction in synchronization with the rotation in the pan direction, it is only necessary to correct the distortion in the tilt direction, so that the distortion correction circuit can be simplified and the processing can be performed at high speed. Can be In this case, the image captured during the period from the stationary state due to the synchronous rotation of the mirror and the camera in the pan direction to the next stationary state has a large play distortion due to the movement. It is desirable to perform the erasing process because the storage capacity can be reduced and the image processing can be simplified.

In the invention according to claim 2, after the divided images are connected and displayed on one screen, the mirror is rotated in the same procedure by tilting rotation and panning or panning to divide the same position. It is characterized in that the image is configured to be sequentially updated. With such a configuration, a nearly real-time panoramic image can be obtained.

Furthermore, the invention described in claim 3 is based on the panoramic image generation method described in claim 2, compares the displayed divided image with the updated divided image, and finds different parts. In some cases, the different portions are displayed in different colors, or a warning sound is automatically generated.

With this configuration, the observer can automatically know the change in the landscape without having to look at the image display screen for a long time, so there is no need to worry about overlooking the change leading to the accident. It can be reliably prevented.

According to the present invention, when there is a change in the updated image, as described in Claim 4, only the updated divided image is enlarged on one screen, and a clear image is processed and displayed. It can also be configured so that the cause of change can be analyzed quickly.

Further, according to the present invention, as described in claim 5, when an updated divided image having the different portion is displayed, a changed portion in the updated divided image and Alternatively, an image adjacent to the updated divided image is automatically tracked and displayed.

This allows for faster analysis without losing the cause of the change.

Further, according to the present invention, as described in claim 6, only the updated divided image having the different portion or only the panoramic image including the updated divided image is recorded and stored. Is what you do.

As a result, the memory capacity of the image can be reduced, and long-term recording and storage can be performed. In this case, in order to identify the time at which the change occurred later, the invention records and saves the time simultaneously on the screen in addition to the image and save as described in Claim 7. It is desirable to do.

Further, according to the present invention, as described in claim 8, the image matching processing is performed by overlapping edges of the divided images adjacent to the respective divided plane images. is there.

With this configuration, it is possible to easily obtain a panoramic image in which the seams are not visible, such as a panorama image captured by a high-precision wide camera.

Furthermore, in the present invention, as described in claim 9, the rotation angle of the mirror is linked to GPS, and map information is displayed on the obtained divided image or / and panoramic image so as to overlap. It is characterized by doing.

Thereby, the location of the image can be specified more accurately.

Further, according to the present invention, as described in Claim 10, the overlapped portion of each of the divided images and the adjacent divided image can be enlarged and displayed.

With this function, without this function, in general, even if the joint part of the split screen is displayed with one of the joint parts missing, both split parts will look at the center of each split screen. The overlap of the images can be seen enlarged.

And, according to the present invention, as described in claim 11, the image picked up by the camera is configured to be shake-reduced by a camera shake preventing means. . As a result, a clearer image can be obtained without being affected by weather or surrounding vibration.

In the panoramic image generating method configured as described above, as described in claim 12, the parallax between two continuous screens of the divided image or the panoramic image is used to provide a depth and a sense of distance. It is also possible to obtain a certain stereoscopic image.

As a method for obtaining a stereoscopic image using the present invention, in addition to the panorama image generation method according to claim 13, as described in claim 12, May be arranged at a fixed distance from each other, and a stereoscopic image can be obtained by using the parallax of an image obtained by synchronously operating each of the mirrors and the camera.

According to another aspect of the present invention, there is provided a panoramic image generation apparatus for achieving the above object, wherein one camera and one panoramic image generation apparatus are tilted to a position facing an imaging surface of the camera. A mirror arranged in a tilted manner, a driving unit for controlling the rotation of the mirror in the tilt direction and the Z or pan direction, and a unit for storing a divided image corresponding to the tilt angle set by the driving unit; Means for sequentially displaying the divided images at predetermined positions on one screen of the image display device; and control means for correcting distortion of each of the divided images with a correction coefficient for an imaging position corresponding to each rotation of a mirror. It is characterized by having been configured.

By configuring the panorama image generating apparatus in this way, a single inexpensive camera can obtain a high-precision panoramic image comparable to a high-precision wide-camera image. In the panoramic image generation device according to the present invention, when the mirror is rotated in the tilt direction and the pan direction to obtain an image, distortion correction in both directions is required, and the distortion correction circuit is complicated. Therefore, if the camera is also rotated in the pan direction in synchronization with the rotation of the mirror in one direction, preferably in the pan direction, only distortion in the tilt direction need be corrected. Therefore, the distortion correction circuit can be simplified, and the processing can be speeded up. In this case, the mirror and camera are synchronized in the pan direction. The images captured during the period from the stationary state due to the rotation to the next stationary state have a large amount of distortion due to the movement. As a result, the storage capacity can be reduced, and the image processing is simplified, so that the device can be downsized and provided at a lower cost. In addition, as described in claim 15, the one camera can easily and quickly digitally process an image by being configured with an inexpensive digital camera.

Further, as described in claim 16, a clearer image can be obtained from the image captured by the digital camera by correcting the image disturbance by the camera shake preventing means.

Further, in the panoramic image generation device according to the present invention, as described in claim 17, an ultrasonic motor is used as a motor for driving the mirror in the tilt direction and the Z direction or the burn direction. It is characterized by the following.

When a stepping motor pulse motor is used, it is difficult to stop the mirror precisely at a predetermined position.On the other hand, an ultrasonic motor can perform high-precision positioning control, so that clear images without press Can be easily obtained.

Further, in the present invention, as described in claim 18, the control means converts the image data of the tilted captured image into image data of an erect image based on the rotation angle information of the mirror. Since an image processing unit for correction processing is provided, even an image captured by an inexpensive camera can be displayed as an accurate image without distortion.

Further, according to the present invention, as described in claim 19, a function of linking the rotation angle information of the mirror and the information obtained from GPS is added to the control means, and the obtained division is provided. It is characterized in that map information based on GPS information is overlapped and displayed on an image or / and a panoramic image.

As a result, since the real image and the map can be accurately overlapped, the position of the real image can be accurately specified.

Further, in the present invention, as described in claim 20, the control means is configured to be able to enlarge and display an overlap portion of each divided image and an adjacent divided image. It is characterized by the following.

Therefore, if this function is not provided, generally, even if one of the joint portions is a joint portion of the split surface displayed with a chipped portion, both the joint portions are displayed as if looking at the center of each split screen. High-precision panorama that allows you to see the enlarged wrap of the divided image Similar to a single panoramic image captured by a camera, necessary parts can be enlarged and displayed without worrying about joints.

Further, in the panoramic image generation device according to the present invention, as described in claim 21, the control unit stores the same number of image data as the number of captured divided images, Since the apparatus is provided with the storage unit for updating and storing the image data every time the image is rotated forward, the image can be recorded and stored. As a method of storing and storing, all of the captured divided image or panoramic image may be stored and stored. However, as described in claim 22, only the divided image of the different part is It is desirable to configure so that only a panoramic image including a divided image of a different part is stored and stored, so that one memory can be kept small and long-term storage can be performed. In this case, it is desirable to record the update date and time together with the image, as described in the range of the request 23, so that it is easy to check later.

Still further, in the panoramic image generation device according to the present invention, as described in Claim 24, the control unit compares the stored divided image with the updated divided image, If there are different parts, these different parts are displayed in different colors, or a means is provided to alert the user with a warning sound, so that the same image is monitored for a long time. There is no need to change the scenery easily.

In this case, when the divided image to be updated is different from the divided image at the same shooting position where the updated divided image is stored, the control unit enlarges and displays the updated divided image as described in claim 25. This makes it easy to analyze the cause of the change, which is useful as a means to prevent accidents.

Further, in the present invention, as described in claim 26, when an updated divided image having a different portion is displayed, the control means may change the updated divided image and / or Means for automatically tracking and displaying an image adjacent to the updated divided image.

This makes it possible to track automatically without losing the cause of the change, unlike a watchman who tracks with experience and feeling. Further, in the panoramic image generation device according to the present invention, as described in claim 27, the control means causes the edge of each divided image to overlap with each divided image and overlaps each other. Since the means for performing the matching processing is provided and configured, a seamless high-precision panorama image similar to a panorama image captured by a high-precision wide camera can be easily obtained.

In the panoramic image generation device according to the present invention, as described in claim 28, the control unit uses the parallax of two continuous screens of the divided image or the panoramic image. A function of generating a stereoscopic image can be added. In this case, a stereoscopic image having a depth and a sense of distance can be obtained, and there is no much difference from seeing an actual scenery, so that it is hard to be tired. Monitoring can be made easier. Further, in the panoramic image generation device according to the present invention, as described in claim 29, the mirror and the camera are arranged in a pair at a predetermined distance from each other, and these mirrors and the camera are arranged in pairs. A stereoscopic image can also be obtained using the parallax of the image obtained by performing the synchronous operation.

The panoramic image generation method and the apparatus configured as described above are used in a system for monitoring a peripheral portion of a monitoring target location as described in claim 30, so that a plurality of cameras and monitors can be connected. Compared to a conventional monitoring system that must be connected and arranged with many wires, it can be provided as a very simple and inexpensive system, and its accuracy is the same as that of the conventional monitoring system described above. In comparison, higher-precision security can be obtained.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 is an explanatory diagram illustrating a schematic configuration of a panoramic image generation device according to an embodiment of the present invention.

FIG. 2 is an explanatory diagram of a state in which a divided image captured by the panoramic image generation device is converted into an erect image without distortion.

FIG. 3 is a block diagram showing the configuration of the panoramic image generation device.

FIGS. 4A and 4B are explanatory diagrams showing other display modes of the panoramic image.

BEST MODE FOR CARRYING OUT THE INVENTION

Hereinafter, based on one embodiment of the monitoring device according to the present invention shown in the accompanying drawings, Details will be described.

A digital camera (CCD camera) 1 is installed in the monitoring equipment (not shown) to capture the surrounding scenery of the target nuclear facilities and military facilities, etc., and the following equipment is attached as shown in Fig. I. Is provided.

That is, the mirror 2 is provided adjacent to the digital camera 1 in an inclined state. The mirror 2 is fixed to a motor shaft 3 ′ of a mirror elevation motor 3, and the elevation angle (tilt angle) of the mirror 2 is adjusted by the operation of the motor 3.

Further, a motor shaft 4 ′ of a mirror rotation motor 4 is fixed to the mirror 2, and the mirror 2 is configured to rotate in the pan direction (horizontal direction) by the operation of the motor 4.

In the illustrated embodiment, the optical axis of the digital force camera 1 and the motor axis 4 ′ of the mirror rotation motor 4, which is the rotation axis of the mirror 2, are located on the same axis. Both the mirror elevation angle adjustment motor 3 and the mirror rotation motor 4 are connected to the motor control circuit 5 by conducting wires. Note that an ultrasonic motor is used for the mirror elevation angle adjustment motor 3 and the mirror rotation motor 4 used in this embodiment. Since the configuration of this ultrasonic motor is publicly known, a detailed description thereof will be omitted here. However, this ultrasonic motor is capable of accurately stopping the mirror 2 at a predetermined angle as compared with a publicly known stepping motor or pulse motor. It is desirable to use an ultrasonic motor in the present invention because of its excellent function of causing the vibration.

Further, a conducting wire is connected to input the mirror rotation angle signal 6 from the motor control circuit 5 to the signal processing and control circuit 7.

A lead wire is connected to the digital camera 1 so that the digital camera control signal 8 is input from the signal processing and control circuit 7, and the digital camera control signal 8 adjusts the focus, aperture, and shutter speed of the digital camera 1, and adjusts the zooming. In addition, image distortion correction adjustment and the like by a camera shake prevention circuit are performed.

On the other hand, an image signal 9 (in this embodiment, 16 divided image signals) from the digital camera 1 is input to the signal processing and control circuit 7 via a conductor. Based on this, the signal processing and control circuit 7 performs each signal processing of the 16 divided images.

The processed 16 image signals 10 from the image signal processing and control circuit 7 are sequentially displayed at respective corresponding locations of the image display device 27. FIG. 2 shows the image signal processing in the signal processing and control circuit 7. The upper part of FIG. 2 shows an image signal 9 from the digital camera 1 at every rotation angle 22.5 degrees of the mirror 1. In this embodiment, the number of divided images is set to 16. However, the present invention is not limited to this, and the number of divisions is arbitrary. Set according to the number of divisions.

By the way, the image signal 9 from the digital camera 1 at every rotation angle 22.5 degrees of the mirror 2 can be obtained by the digital camera 1 rotating continuously and taking an image under the operation of the high-speed shutter, or It can also be sent by stopping the digital camera rotation momentarily every 22.5 degrees of mirror rotation angle and capturing an image.

The digital force image is tilted or distorted with respect to the erect image according to the rotation angle of the mirror 2. Accordingly, in this case, the divided images projected on the screen of the image display device 27 are displayed at different angles, and each of the actually captured divided images is distorted by the rotation angle of the mirror 2. Displaying the image as it is on the screen 27 as it was captured would not be able to generate a uniform panoramic image, which would be inconvenient for visual monitoring. For this reason, the rotation circuit 12 of the image processing unit in the signal processing and control circuit 7 corrects distortion, angle, and the like based on a correction coefficient that is calculated and stored in advance in accordance with the rotation angle of the mirror. It is configured so that the erect image is corrected as shown in the lower part of FIG. 2 so that the erect image can always be observed on the subsequent VTR and the screen of the image display device 27. . In addition, the mouth-station circuit 12 performs matching processing of the image by overlapping the edge of each divided image with the edge of the adjacent divided image, and forms a joint similar to a panoramic image captured by a high-precision wide camera. It is configured to display no high-precision panoramic image on screen 27.

The same number of image data as the number of divided images captured in this way are all stored and stored in the signal processing and control circuit 7, and the signal processing and control circuit 7 stores the image data at every forward rotation of the mirror, that is, 1 After capturing the six divided images, a storage unit that captures the scene of the same part as the re-divided image 1 and thereafter sequentially captures the divided images 2 to 16 again, and updates and stores each divided image data is provided. ing. At this time, the update date and time are recorded together with each divided image. Further, the signal processing and control circuit 7 compares the stored divided image with the updated divided image, and if there is a different portion, displays the different portion in a different color, for example, red. It is designed to alert the observer by alerting them to the fact that they are different, or by giving a buzzer or other warning sound. This eliminates the need to monitor the same image for a long time, making it easy to recognize changes in the scenery.

Further, when the divided image to be updated is different from the divided image at the same shooting position where the updated divided image is stored, the signal processing and control circuit 7 enlarges and displays the updated divided image on the screen 27 at an arbitrary magnification. It is controllable to be able to. As a result, the cause of the change can be easily analyzed, so that an accident can be prevented before it occurs.

As described above, the image stored in the storage and storage circuit may store and store all the divided images and panoramic images. However, it is preferable that the divided image in which the change occurs or the divided image in which the change occurs It is preferable to store only panoramic images that contain a character string, so that the memory capacity can be reduced and, as a result, recording can be performed for a long time.

When a change appears on the screen, the control circuit 7 controls the operation of the mirror 1 and the camera 1 so as to automatically track the target on the screen where the change has occurred. Thus, the target can be automatically tracked without having to follow the rules of thumb or feelings, so that the analysis can be easily performed. As a screen display mode in the case of the automatic tracking, a display mode shown in FIG. 4B is preferable. Since this automatic tracking means is known in, for example, Japanese Patent Application Laid-Open No. 2000-224744, Japanese Patent Application Laid-Open No. 2000-290962, the detailed description thereof will be omitted. The description is omitted here.

In this embodiment, although not shown, the position of the image obtained as described above is linked to the rotation angle information of the mirror 2 and the GPS information, so that the map information is displayed on the screen. Overlapping display can be performed, and the degree of latitude of the target object can be easily specified. Of course, an overlap image with this map may be stored and stored.

In the present embodiment, the signal processing and control circuit 7 is configured to be able to enlarge the overlapping portions of the split image adjacent to each divided image _c Therefore, if this function is not provided, in general, even if the joint part of the split screen is displayed with one of the gaps missing, both parts will be seen as if they were at the center of each split screen. The overlap of the divided images can be magnified and viewed. Just like a single panorama image captured by a high-precision panorama camera, the necessary parts can be magnified and displayed without worrying about joints.

Next, the operation of the signal processing and control circuit 7 configured as described above will be described.

The digital image of the scene reflected by the continuously rotating mirror 2 is guided to the rotation circuit 12, and the digital camera image at each predetermined rotation of the mirror is rotated and processed by the image processing unit of the signal processing and control circuit 7. The erect image is obtained by inverting the image and correcting the image distortion and distortion due to camera shake (Fig. 2).

At a rotation angle of 0 ° of the digital camera 1, an image obtained by inverting the digital camera image of the target scene is shown, and the image is inverted in the image processing unit to correct distortion and erect. The image "1" is obtained. When the rotation angle of the mirror 2 is 22.5 °, the image is inverted and tilted to the right by 22.5 degrees. The image processing unit performs the inversion processing and corrects the distortion to correct the erect image. "2" is obtained.

At a rotation angle of 45 ° of the mirror 2, the image is inverted and further tilted to the right by 45 degrees, and the upright image “3” is obtained by the above procedure of the image processing unit.

Thereafter, the images 4 to 16 obtained at the phases of the respective rotation angles of the mirror 2 are sequentially processed by the image processing unit in accordance with the above procedure, thereby sequentially erecting the images “4” to Γ ΐ 6 ” Is obtained.

That is, during the continuous rotation or intermittent rotation of the mirror 2, the image field of view at every rotation angle of 22.5 degrees is monitored in real time on the screen 27, or after a certain period of time, reproduced by the VTR and monitored at any time. Can be

FIG. 3 shows a configuration of a monitoring device according to the present embodiment. The monitoring device is configured by a digital camera unit Α and a VTR process unit B.

The digital camera unit A includes a digital camera 1, a mirror 2, a mirror elevation angle adjustment motor 3, a mirror rotation motor 4, and a motor control circuit 5. On the other hand, in the VTR process unit B, the digital camera image signal 9 is corrected in image orientation by the mouthpiece circuit 12, and then analog-converted by the digital / analog converter 25 and sent to the VTR 26. , And the digital camera image signal, the image attitude is corrected by the rotation circuit 12, stored in the frame memory 13 A to 13 P, and then the digital to analog converter 14 A to 14 P It is configured to be sent to each of the designated positions on screen 26 after being converted.

That is, the digital camera image signal 9 output from the digital camera unit A is converted into a rotation circuit (including a camera shake prevention circuit) of the image signal processing unit in the image signal processing and control circuit 7 in the VTR process unit B 12. Thus, the image disturbance due to the posture of the plane image and the vibration of the camera-mirror is corrected, and is updated and stored in the frame memory indicated by the reference numerals 13 A to 13 P for each rotation of the mirror 2.

After that, the digital-to-analog converters 14-A to 14-P convert them into analog signals, output the VTR process unit B as outputs 1 to 16 and output the positions (signs) corresponding to the mirror rotation angles on screen 27. 1 to 16). The VTR output is input directly to the VTR 26 via the digital-analog converter 25 after the attitude of the image signal 9 of the digital camera is corrected in the signal processing and control circuit 7.

In the illustrated example, the case where the present panoramic image generation apparatus is applied to the monitoring apparatus shown in FIG. 3 is shown. However, the present invention is not limited to this, and it is not limited to this. It is also effective as a monitoring device for places where many people enter and leave. For example, in places where many people come and go, each image data captured by this panoramic image generation device is recorded and saved on the spot, and each of the recorded and saved divided image data is transmitted to a management center via a communication line. However, the management center may compare the transmitted divided image data with, for example, the face data of a criminal to use it so as to quickly determine the safety of places where many persons enter and leave. It is possible.

Further, in the above embodiment, the case where the mirror is rotated in the tilt direction and the pan direction to obtain an image has been described as an example. However, the present invention is not limited to this, and one of the mirrors may be used. Direction, desirably, the camera is also rotated in the pan direction in synchronization with the rotation of the mirror in the pan direction. It is desirable that the image correction processing is performed only by the distortion correction of the above, the distortion correction circuit is simplified, and the processing is speeded up. In this case, the images captured during the period from the stationary state due to the synchronous rotation of the mirror and the camera in the pan direction to the next stationary state have much blur distortion due to movement, so the images during this period are deleted. By performing the processing, the storage capacity can be reduced, and the image processing can be simplified, so that the device can be reduced in size and provided at a lower cost.

Further, according to the present invention, a stereoscopic image can be obtained by using a parallax amount of a continuous divided image or a panoramic image. Further, it is possible to obtain a stereoscopic image based on the parallax amount of an image obtained by installing a pair of mirrors and a pair of cameras at separate places and operating these mirrors and the camera in synchronization. The method of generating a stereoscopic image using the amount of parallax is the same as, for example, PCT / JP02 / 124244 proposed by the present applicant and the inventor. Omitted.

Industrial applicability

As described above, according to the panorama image generation method and the apparatus thereof according to the present invention, a single digital camera can monitor and image a field of view of 360 degrees around the entire circumference at a high speed. It can be observed on an image in real time, or recorded and stored on a VTR. In this case, since the digital camera does not rotate but the mirror rotates, the cables and cords connected to the digital camera do not move, so that there is no trouble such as disconnection of the cables.

In addition, when an image is obtained by rotating the mirror in the tilt direction and the pan direction, distortion correction in both directions is necessary, and the distortion correction circuit becomes complicated. Therefore, either one direction, preferably, the mirror pan direction is used. When the camera is also rotated in the pan direction in synchronization with the rotation of the camera, it is only necessary to correct the tilt direction distortion, so the distortion correction circuit can be simplified and the processing can be performed at high speed. Can be In this case, since the image captured during the period from the stationary state due to the synchronous rotation of the mirror and the camera in the pan direction to the next stationary state has a large play distortion due to the movement, the image during this period is deleted. As a result, the storage capacity can be reduced, and the image processing can be simplified, so that the device can be reduced in size and provided at a lower cost. In addition, an image processing unit is provided that corrects the image data of the tilted captured image into image data of an erect image based on the rotation angle information of the mirror. Since the image blur is eliminated by the prevention means, an accurate and clear image can be easily obtained without distortion even if the image is taken by an inexpensive digital camera.

Furthermore, by linking the mirror rotation angle with the GPS, the map information can be overlapped with the actual image, and, for example, the latitude of the portion where the change has occurred can be accurately grasped.

In addition, according to the present invention, the control means includes a storage unit that stores the same number of image data as the number of captured divided images and that updates and stores the image data each time the mirror is rotated forward. The image can be recorded and saved together with the update date and time, and the stored divided image is compared with the updated divided image. If there is a different portion, the different portion is displayed in a different color. Can alert you with alert sounds, so there is no need to monitor the same image for a long time, and you can easily recognize changes in the scenery, and such changes have occurred In this case, the cause of the change can be easily analyzed by enlarging and displaying the updated divided image. In addition, when saving images, saving only the changed divided images or panorama images containing the changed divided images can reduce the memory capacity and enable long-term storage and storage. Become.

Furthermore, by automatically tracking the part that has changed, tracking can be performed without losing sight of the target object, without relying on experience, so analysis becomes easier.

Further, in the present invention, since the image matching process is performed by overlapping the edges of each divided image and the adjacent divided image, the same seamless height as a panoramic image captured by a high-precision wide camera is used. Accurate panoramic images can be easily obtained.

Further, according to the present invention, since a stereoscopic image can be generated by using the parallax of a continuous divided image or a panoramic image, a sense of distance can be easily grasped, and a sense of seeing an actual scenery can be obtained. Since monitoring can be performed on a computer, it is easy to perform monitoring work without fatigue. The panoramic image generation method and the apparatus configured as described above are used in a system for monitoring the periphery of a monitoring target location, so that a plurality of cameras and monitors must be connected and arranged with many wires. Compared with the conventional monitoring system, the system can be made very simple and inexpensive, and the accuracy can be higher than that of the conventional monitoring system. . Of course, the monitoring device of the present invention is suitable not only for crime prevention but also for monitoring any moving object such as an animal.

Claims

The scope of the claims

1. At a position facing the imaging surface of a single camera, a mirror configured to be tiltable and / or pannable is arranged at an angle, and is obtained according to the rotation angle of the mirror. In a method of generating a panoramic image by storing divided images and joining each divided image on one screen of an image display device,

A panoramic image generating method, wherein each of the divided images is displayed with distortion corrected by a correction coefficient for an imaging position corresponding to a rotation angle of a mirror.

2. After the divided images are displayed on one screen, they are sequentially updated by rotating the mirror and tilting or panning in the same procedure. 2. The panoramic image generation method according to claim 1, wherein

3. In the panoramic image generation method according to claim 2, the displayed split image is compared with the updated split plane image, and if there is a different portion, the different portion is displayed in a different color. Or a method of generating a panoramic image, characterized by emitting a warning sound. .

4. The method according to claim 3, wherein when there is a different portion between the displayed divided image and the updated divided image, the different updated divided image is enlarged and displayed on one screen of an image display device. Panorama image generation method.

5. When an updated divided image having the different portion is displayed, a changed portion in the updated divided image and / or a surface image adjacent to the updated divided image is automatically tracked and displayed. The panorama image generation method according to claim 3 or claim 4, wherein

6. The panoramic image according to any one of claims 2 to 5, wherein only the updated divided image having the different portion or only the panorama image including the updated divided image is recorded and stored. Generation method.

7. The panoramic image generation method according to claim 6, wherein in the recording and saving, the time is simultaneously recorded and saved on the screen.

8. The panoramic image generation method according to any one of claims 1 to 7, wherein an image matching process is performed by overlapping an edge of each of the divided images and an adjacent divided image. .

9. The rotation angle of the mirror is linked to GPS, and the obtained divided image or panorama image is displayed with map information overlapping. The panoramic image generation method according to any one of the above.

10. The panorama image generating method according to any one of claims 1 to 9, wherein an overlap portion of each of the divided images and an adjacent divided image can be enlarged and displayed.

11. The panoramic image generation method according to any one of claims 1 to 10, wherein the camera corrects an image disturbance by a camera shake preventing unit.

12. A panoramic image according to any one of claims 1 to 11, wherein a stereoscopic image is obtained by using a parallax between two continuous screens of the divided image or the panoramic image. Image generation method.

13. A pair of mirrors and a camera are arranged at a fixed distance from each other, and a stereoscopic image is obtained by using parallax of an image obtained by synchronously operating each mirror and the camera. A method for generating a panoramic image according to any one of claims 1 to 12.

14. A camera, a mirror arranged at an angle to a position facing the imaging surface of the camera, drive means for controlling the rotation of the mirror in the tilt direction and / or the pan direction, and the drive means Means for storing a divided image corresponding to the set tilt angle; means for sequentially displaying the divided images at a predetermined position on one screen of the image display device; and distorting each of the divided images according to each rotation of a mirror. And a control means for correcting the image pickup position with a correction coefficient for the image pickup position.

15. The panoramic image generation device according to claim 14, wherein the one camera is a digital camera.

16. The panoramic image generation method according to claim 15, wherein the image captured by the digital camera corrects image distortion caused by camera shake prevention means.

17. The panoramic image generation device according to claim 14, wherein the motor that drives the mirror in the tilt direction and / or the burn direction is an ultrasonic motor. apparatus.

18. The control means includes an image processing unit configured to correct image data of the tilted captured image into image data of an erect image based on the rotation angle information of the mirror. The panoramic image generation device according to any one of claims 14 to 17, wherein:

19. The control means links the rotation angle information of the mirror and information obtained from GPS, and displays map information based on GPS information on the obtained divided image or / and panoramic image in an overlapping manner. The panoramic image generation device according to any one of claims 14 to 18, wherein:

20. The panoramic image generation device according to any one of claims 14 to 19, wherein the control unit enlarges and displays an overlap portion of each divided image and an adjacent divided image. .

21. The control means is provided with a storage unit for storing the same number of image data as the number of captured divided images or the constructed panoramic image information, and updating and storing the image data each time the mirror is rotated forward. The panoramic image generation device according to any one of claims 14 to 20, characterized in that:

22. The panoramic image generation method according to claim 21, wherein the control means stores and saves only the divided image of the different portion or only a panoramic image including the divided image of the different portion. apparatus.

23. The panoramic image generation device according to claim 21, wherein the stored image is stored together with time information.

24. The control means compares the stored divided image with the updated divided image. If there is a different portion, the different portion is displayed in a different color, or a warning sound is output. The panoramic image generation device according to any one of claims 14 to 23, further comprising means for calling attention.

25. The control device according to claim 14, wherein the control means enlarges and displays the updated divided image when the updated divided image is different from the stored divided image at the same shooting position. The panorama image generation device according to any one of

26. When an updated divided image having a different portion is displayed, the control means automatically tracks and displays a changed portion in the updated divided image and / or an image adjacent to the updated divided image. The panoramic image generation device according to any one of claims 14 and 25, characterized by comprising means.

27. The method according to claim 1, wherein the control means includes means for performing image matching processing by overlapping a part of each divided image and an adjacent divided image. 28. The panoramic image generation device _c 28 according to claim 26, wherein the control unit generates a stereoscopic image using parallax between two continuous screens of the divided image or the panoramic image. The panoramic image generation device according to any one of claims 14 to 27, characterized in that:

29. The mirror and the camera are arranged in a pair at a fixed distance from each other, and the control means generates a stereoscopic image by using the parallax of an image obtained by synchronously operating each of the mirror and the camera. The panoramic image generation device according to any one of claims 14 to 27, wherein

30. A monitoring system characterized by monitoring a peripheral part of a monitoring target place using the panoramic image generation method or the panoramic image generation device according to any one of claims 1 to 29.