JP2011070123A - Fish eye camera, system, method, and image processing program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a device, method and the like having a zoom function for expanding an optional range of a fish eye image. <P>SOLUTION: A fish eye camera body S including a fish-eye lens group 1 includes a zoom lens group 2 for expanding a photographed image. The zoom lens group 2 is arranged at a rear stage of the fish-eye lens group 1. A rotation mirror 3 for specifying a zoom range which can shift the positional relationship between the optical axis A of the fish-eye lens group 1 at the front stage and the optical axis D of the zoom lens group 2 at the rear stage is provided. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a zoom function of a fish-eye camera, and more particularly to a technical field such as a fish-eye camera, an image processing system, an image processing method, and an image processing program having a zoom function capable of enlarging an arbitrary range of a fish-eye image.

  A camera (for example, a fisheye camera) to which a wide-angle lens (for example, a fisheye lens) is applied can accommodate a wide-angle range with a viewing angle of more than 180 degrees as an image without a mechanical operation unit.

  With a normal camera to which a wide-angle lens is not applied, the viewing angle that can be captured is extremely limited (the effective viewing angle is narrow), so that the same wide-angle range as the above wide-angle lens can be obtained as one image using such a normal camera. In order to capture an image, for example, the effective viewing angle can be expanded by increasing the number of cameras installed, or the camera can be panned (swinged and rotated) by a mechanical mechanism for each effective viewing angle to capture a wide-angle range. I was trying to do it.

  In order to reduce the total number of cameras installed and to realize maintenance-free by using the fisheye lens, such a wide-angle lens is used for a surveillance camera or the like.

  However, the fish-eye camera has a problem that, as the viewing angle becomes wider, far-field objects appear smaller and the resolution decreases. Since the resolution of the original image of the fisheye camera is insufficient, even if digital zoom (digital enlargement) is performed, the image becomes rougher (resolution does not improve), and the object indicated in the image Cannot be recognized and has no effect. Therefore, it is necessary to zoom the fish-eye image with a lens (optical zoom).

  Japanese Patent Application Laid-Open No. 2004-228561 describes a fisheye camera that includes a zoom lens at the rear stage of a fisheye lens group. FIG. 7 is a diagram illustrating a structure of a fisheye camera provided with a zoom lens in the latter stage of the fisheye lens group disclosed in Patent Document 1.

JP 2009-103790 A

  However, in the fish-eye camera provided with the zoom lens in the latter stage of the fish-eye lens group disclosed in Patent Document 1 and FIG. 7, the fish-eye lens group and the zoom lens group are fixed so that the optical axes coincide with each other, and the optical axis is centered. You can only enlarge the image.

  Further, if an image is to be magnified at a portion other than the optical axis, the camera body must be magnified by a zoom lens group in an arbitrary direction. In other words, the direction of the camera body must be moved. The fish-eye camera originally has an advantage that it can acquire a wide range of images at a time and can be fixedly installed, so that no mechanical operation is required. If the direction of the fisheye camera body must be moved, a mechanical mechanism is required, and the original meaning and advantages of the fisheye camera are lost.

  Therefore, the present invention has been made in view of the above problems, and an example of the object relates to a zoom function of a fish-eye camera. An object is to provide an apparatus and method having a zoom function capable of expanding the range.

  In order to solve the above-described problem, the invention according to claim 1 is a fish-eye camera including a wide viewing angle lens, and is provided with a zoom mechanism for enlarging a photographed image, and the zoom mechanism is arranged at the rear stage of the fish-eye lens group. In addition, by providing a shift mechanism that can shift the positional relationship between the optical axis of the front-stage fisheye lens group and the optical axis of the rear-stage zoom lens group, an arbitrary range of the fisheye image formed by the front-stage fisheye lens group can be zoomed It is characterized by doing.

  According to the present invention, a zoom mechanism is provided for zooming (enlarging) a captured image, and the zoom mechanism disposes a zoom lens group at the rear stage of the fisheye lens group, and the zoom lens group is a front fisheye lens group. The positional relationship between this optical axis and the optical axis of the subsequent zoom lens group can be shifted.

  Therefore, the arbitrary range of the fish-eye image formed by the front-stage fish-eye lens group can be zoomed.

  The invention according to claim 2 is the fisheye camera according to claim 1, wherein the shift mechanism is provided with a mirror between the front-stage fisheye lens group and the rear-stage zoom lens group.

  A third aspect of the present invention is the fisheye camera according to the first aspect, wherein the shift mechanism shifts the rear-stage zoom lens group by a driving mechanism.

  According to a fourth aspect of the present invention, the fisheye camera according to any one of the first to third aspects and a photographed image captured by the fisheye camera are used as input data, and output data is obtained based on the input data. An image processing system including an image processing device that generates and displays the output data as a display image on a display unit, wherein the image processing device includes a detection unit that detects a specific part of the display image, and the detection Zooming means for performing the zoom on the specified specific part.

  According to this invention, the image processing apparatus includes a detection unit that detects a specific part of the display image, and a zoom unit that performs the zoom on the detected specific part.

  Therefore, since the specific part of the display image is automatically detected and the zoom is performed on the detected specific part, it is possible to reliably grasp the change in the environment of the imaged location.

  The invention according to claim 5 is an image processing method in the image processing apparatus according to claim 4, wherein a detection step of detecting a specific part in the display image and the detected specific part are detected. And a zoom step for performing the zoom.

  According to a sixth aspect of the present invention, there is provided a computer included in the image processing apparatus according to the fourth aspect, wherein a detection unit that detects a specific part in the display image, the zoom with respect to the detected specific part. It functions as zoom means for applying

  As described above, according to the present invention, the fish-eye camera is provided with a zoom mechanism for enlarging a captured image, and the zoom mechanism arranges the zoom lens group after the fish-eye lens group. Since the positional relationship between the optical axis of the fisheye lens group and the optical axis of the zoom lens group at the subsequent stage can be shifted, the direction of the fisheye camera body is fixed, and an arbitrary range of the fisheye image can be zoomed in particular. it can.

It is a conceptual diagram which shows the expansion result in the case of enlarging the conventional fisheye image. It is a conceptual diagram which shows the expansion result of the fisheye image at the time of applying this application. It is a block diagram which shows the structure of the image processing system concerning this embodiment. It is a flowchart which shows operation | movement of the control part 9 concerning this embodiment. It is a figure which shows the example of a display screen concerning this embodiment. It is a block diagram which shows the structure of the image system of other embodiment concerning this embodiment. It is a figure which shows the structure of the fisheye camera provided with the zoom lens in the back | latter stage of the fisheye lens group currently disclosed by patent document 1. FIG.

[1. Principle of the application]
First, the principle of the present application will be described with reference to FIGS. FIG. 1 is a conceptual diagram showing an enlargement result in the case of enlarging a conventional fisheye image (a photographed image taken by applying a fisheye lens).

  As described above, in the prior art, when enlarging a fish-eye image, only the portion of the image centered on the optical axis could be enlarged. FIG. 1A is a view showing the fisheye image a and the optical axis a1 before the enlargement, and FIG. 1B is a view showing the fisheye image a and the optical axis a1 after the enlargement. .

  As shown in FIG. 1B, the fish-eye image a can only enlarge (zoom) a partial image centered on the optical axis a1.

  Therefore, the inventors have invented a fisheye camera or the like having a zoom function that can enlarge an arbitrary range of a fisheye image. Specifically, the inventors have invented an apparatus that takes an arbitrary point of a fish-eye image as an enlargement center and captures an enlarged image based on the enlargement center.

  FIG. 2 is a conceptual diagram showing a fisheye image enlargement result when the present application is applied.

  As shown in FIG. 2A, the fisheye camera of the present application can set an arbitrary point of the fisheye image b as the expansion center a1. FIG. 2 (A) shows a fish-eye image when the enlargement center is set to b1.

  FIG. 2B shows a fisheye image enlargement result (fisheye image b1) when the enlargement center is b1.

  In the following, a system and apparatus for realizing the principle of the present application will be described.

[2. Best Embodiment of the Present Application]
The best embodiment of the present application will be described with reference to FIG. The embodiment described below is an embodiment when the present application is applied to an image processing system.

  First, the configuration and functional overview of the image processing system according to the present embodiment will be described with reference to FIG.

  FIG. 3 is a block diagram showing the configuration of the image processing system according to the present embodiment.

  As shown in FIG. 3, the image processing system T includes a fisheye camera V, a control unit 9 (an example of the detection unit and zoom unit of the present application), a display unit 10 that displays an image as output data, and the like.

  The fisheye camera V includes an optical lens group S, a camera body unit 7 and the like.

  The optical lens group S is a group of lenses arranged to form light taken from outside as an optical image, and includes a fisheye lens group 1, a zoom lens group 2, and a zoom mirror for designating a zoom range 3 (the shift of the present application). An example of a mechanism and a mirror).

  The optical lens group S includes an array of a fish-eye lens group 1, a zoom range designating rotating mirror 3, and a zoom lens group 2 in order from the object to be imaged to the image side. That is, the zoom lens group 2 is arranged at the rear stage of the fisheye lens group 1.

  For convenience of explanation, the optical axis of the fisheye lens group 1 is the optical axis A, the optical axis of the zoom lens group 2 is the optical axis C, the direction parallel to the optical axis A is the X axis, the X axis is vertical, and the zoom lens group 2 side. A coordinate system having the Y axis as the Y axis, the X axis and the Y axis, and the Z axis as the vertical axis is defined as a coordinate system B.

  The fish-eye lens group 1 is a kind of optical lens and has an angle of view close to 180 degrees. The fisheye lens group 1 has four types of methods: orthographic projection, equidistant projection, stereoscopic projection, and equisolid angle projection. In the present embodiment, such a method can be arbitrarily selected and is not limited to this.

  The zoom lens group 2 includes a zoom mechanism that is arranged to zoom (enlarge) the image taken by the fisheye lens group 1. Since the zoom mechanism is a well-known technique, a detailed description thereof will be omitted. However, the zoom mechanism is photographed by the fisheye lens group 1 by moving the zoom lens group 2 in the Y-axis direction (front and rear) and changing the focal length. Magnify the image.

  In other words, the zoom lens group 2 can enlarge an image photographed by the fisheye lens group 1 around the optical axis C of the zoom lens group 2.

  The zoom range designating rotary mirror 3 is a mirror provided to allow the zoom lens group 2 to capture an image photographed by the fisheye lens group 1. An image photographed by the fisheye lens group 1 is reflected in the Y-axis direction by the zoom range designating rotating mirror 3 and taken into the zoom lens group 2.

  In addition, the optical lens group S is configured to variably set the positional relationship between the optical axis A of the fisheye lens group 1 and the optical axis D of the zoom lens group 2 (described later between the fisheye lens group 1 and the zoom lens group 2). In order to rotatably arrange the zoom mirror for designating the zoom range 3), the optical axis A of the fisheye lens group 1 and the optical axis C of the zoom lens group 2 are arranged to be orthogonal to each other. The zoom mirror 3 for specifying a zoom range according to the present embodiment is rotatably provided between the fisheye lens group 1 and the zoom lens group 2. Therefore, zooming the arbitrary range of the fisheye image photographed (formed) by the fisheye lens group 1 by controlling the direction of the zoom mirror 3 for specifying the zoom range and using the zoom mechanism of the zoom lens group 2. Become.

  In other words, the zoom mirror for designating the zoom range 3 moves the optical axis C to an arbitrary position of the image photographed by the fisheye lens group 1 in order to variably set the optical axis C of the zoom lens group 2, and the image Can be enlarged.

  Specifically, the zoom range designating rotating mirror 3 is fixed so as to be rotatable in the Z-axis direction and the axis D direction provided on a virtual plane passing through the X-axis and the Y-axis. Thus, by being fixed so that rotation is possible, the optical axis D of the zoom lens group 2 can move the arbitrary point of a fish-eye image. Then, by using the zoom mechanism of the zoom lens group 2, it is possible to take an enlarged image with an arbitrary point of the fisheye image as the enlargement center and the enlargement center as a reference.

  The means for fixing the zoom range designating rotary mirror 3 to be rotatable can be arbitrarily selected. For example, the zoom range designating rotary mirror 3 is provided with a rotary shaft, and a power source (for example, A motor or the like).

  Further, in FIG. 3, the fisheye image formed by the front-stage fisheye lens group 1 is viewed by looking at the fisheye image formed by passing through the zoom lens group 1, and the position to be zoomed is designated using the cursor 4. (Zoom range 5 in FIG. 3) shows a state (magnified image 6 in FIG. 3) in which the direction of the zoom mirror 3 for specifying the zoom range is controlled and enlarged using the zoom mechanism of the zoom lens group 2.

  In the camera body 7, the light taken in from the optical lens group S is adjusted by a diaphragm mechanism (not shown) and formed on the image sensor 11 as an optical image. Such an optical image (including a still image and a moving image) is converted into an electrical signal by the image sensor 11 and output to the interface unit 8.

  The image sensor 11 converts the imaged optical image into an electric signal proportional to the brightness of the optical image by a photoelectric exchange function using, for example, photodiodes arranged in a matrix and outputs the electric signal to the interface unit 8. For example, a CCD (Charge Couple Semiconductor) image sensor, a CMOS (Complementary Metal-Oxide Semiconductor) image sensor, or the like is applied.

  The camera body 7 further includes an optical low-pass filter (not shown) for limiting the spatial frequency input to the interface unit 8 and an infrared (not shown) for cutting long wavelength components other than the visible light region input to the interface unit 8. A cut filter or the like is provided.

  The camera body 7 also includes a return mirror 12 for switching and transmitting the light taken from the optical lens group S to the image sensor 11 and the finder optical system. When observing the finder subject image, the optical image formed on the focusing plate 13 via the return mirror 12 is converted into an erect image by the prism 14 and then magnified and observed by the finder 15.

  When an optical image is formed on the image sensor 11, the return mirror 12 rotates in a direction approaching the focus plate 13. In addition, a sub mirror 16 and a focus detection device 17 are provided.

  The interface unit 8 is an interface for outputting a captured image to the control unit 2.

  Specifically, the interface unit 8 outputs the captured image to the control unit 9 as input data composed of pixel data indicating pixels constituting the image and color data corresponding to the pixels. To do.

  The control unit 9 is a processing circuit for performing image processing based on the input data and generating output data indicating a two-dimensionally visible planar image and a fisheye image as a moving image. Not including a CPU, a RAM, and a ROM.

[3. Operation of the control unit 9 according to the present embodiment]
Next, the operation of the control unit 9 according to the present embodiment will be described with reference to FIGS.

  FIG. 4 is a flowchart showing the operation of the control unit 9 according to the present embodiment.

  First, the control unit 9 outputs a fisheye image as output data to be displayed on the display unit 10 based on the fisheye image (captured image) captured by the fisheye camera body V (step S1).

  FIG. 5 is a diagram showing a display screen example according to the present embodiment. In FIG. 5A, the fisheye image is displayed on the display unit 10.

  Next, the control unit 9 detects the presence or absence of a moving object (moving object, an example of a specific part of the present application) among the objects displayed in the fisheye image (step S2).

  For the detection of the presence or absence of the moving object, a detailed description is omitted because a known image recognition technique is applied. For example, the control unit 9 recognizes the subject of the first photographed image and then captures the image after that. The moving amount of the subject is calculated by comparing with the subject of the image. If the amount of movement is equal to or greater than a certain value, the control unit 9 detects the subject as a moving object.

  In the above image recognition technology, the process of recognizing the subject of the first photographed image and comparing it with the subject of the image photographed thereafter is generally called tracking, and such tracking is performed by the Lucas-Kanade method. This is realized by calculating the optical flow using.

  In FIG. 5A, the person 22 is moving in the direction of the fisheye camera main body V. Therefore, the control unit 9 detects the person 22 as a moving object.

  Next, the control unit 9 enlarges and displays the subject recognized as a moving object by zooming (step S3).

  Specifically, the control unit 9 designates a position to be zoomed (zoom range 5 in FIG. 3), controls the zoom mirror for designating the zoom range 3, and zooms in using the zoom mechanism of the zoom lens group 2. Give instructions to.

  FIG. 5B shows an image 23 in which the control unit 9 moves the optical axis D of the zoom lens group 2 to the position of the person 22 and displays the enlarged image of the image.

  When the person 22 (moving object) goes out of the frame (when the person 22 exceeds the effective display range of the enlarged image 23) (step S4: YES), the enlarged display is terminated, and the fish An eye image is displayed (step S1).

  On the other hand, when the person 22 does not go out of the frame (step S4: NO), the image 23 in which the above is enlarged and displayed is continuously displayed (step S3).

  For example, when an instruction to end the image processing system is input to the control unit 9, the control unit 9 ends the system.

  As described above, in the present embodiment, the fisheye camera main body S including the fisheye lens group 1 is provided with the zoom lens group 2 for enlarging the photographed image, and the zoom lens group 2 is located at the rear stage of the fisheye lens group 1. By providing a zoom mirror for designating a zoom range 3 that is arranged and can shift the positional relationship between the optical axis A of the front-stage fisheye lens group 1 and the optical axis C of the rear-stage zoom lens group 2, the front-stage fisheye lens group 1 An arbitrary range of the formed fisheye image can be zoomed.

  Therefore, the arbitrary range of the fisheye image formed by the front fisheye lens group 1 can be zoomed.

[4. Other Embodiments]
Next, another embodiment of the present application will be described with reference to FIG.

  FIG. 6 is a block diagram illustrating a configuration of a fish-eye camera according to another embodiment of the present embodiment.

  For convenience of explanation, the optical axis of the fisheye lens group 1 is the optical axis A, the optical axis of the zoom lens group 2 is the optical axis E, the direction parallel to the optical axis A is the X axis, the X axis is vertical, and the zoom lens group 2 A coordinate system in which the vertical axis is the Y axis, the X axis and the Y axis, and the vertical axis is the Z axis is a coordinate system F.

  A fisheye camera Q according to another embodiment includes an optical lens group R, a camera body 7, an interface unit 8, and the like according to another embodiment. In addition, about the thing which shows the same code | symbol as FIG. 3, the description is abbreviate | omitted as having the same function.

  In another embodiment, the zoom lens group 2 includes the entire zoom lens group 2 in the Y-axis direction and the Z-axis in order to variably set the positional relationship between the optical axis A of the fisheye lens group 1 and the optical axis E of the zoom lens group 2. In the direction, a drive mechanism is provided and is movably installed.

  Thus, the zoom lens group 2 is movably installed, so that the optical axis E of the zoom lens group 2 can move at any point of the fisheye image. Then, by using the zoom mechanism of the zoom lens group 2, it is possible to take an enlarged image with an arbitrary point of the fisheye image as the enlargement center and the enlargement center as a reference.

  The drive mechanism (means for movably installing the zoom lens group 2) can be arbitrarily selected. For example, the Y-axis direction and the Z-axis are provided on the housing (outside the housing part) of the zoom lens group 2. A rack portion movable in the direction may be provided, and a power source (for example, a motor or the like) provided with a pinion may be provided.

  As described above, in the present embodiment, since the zoom lens group 2 is shifted by the drive mechanism, an arbitrary range of the fisheye image formed by the front fisheye lens group 1 can be zoomed.

  In the above-described embodiment, an example in which the present application is applied to an image processing system has been shown. However, for example, for a digital video camera, a digital camera, a personal computer, a household electronic device, or the like. Is applicable.

DESCRIPTION OF SYMBOLS 1 Fisheye lens group 2 Zoom lens group 3 Rotating mirror for zoom range specification 4 Cursor 5 Zoom range 6 Enlarged image 7 Camera body 8 Interface unit 9 Control unit 10 Display unit 11 Image sensor 12 Return mirror 13 Focus plate 14 Prism 15 Viewfinder 16 Sub mirror 17 Focus detection device Q Fisheye camera R according to another embodiment Optical lens group S according to another embodiment Optical lens group V Fisheye camera U Optical lens group T Image processing system

Claims (6)

  In a fish-eye camera including a wide viewing angle lens, a zoom mechanism is provided for enlarging a captured image, and the zoom mechanism is arranged at the rear stage of the fish-eye lens group, and the optical axis of the front-stage fish-eye lens group and the rear-stage zoom lens group A fish-eye camera having a zoom function characterized in that an arbitrary range of a fish-eye image formed by the front-stage fish-eye lens group is zoomed by providing a shift mechanism capable of shifting the positional relationship with the optical axis. The fisheye camera according to claim 1,
The fish-eye camera, wherein the shift mechanism includes a mirror between the front-stage fisheye lens group and the rear-stage zoom lens group. The fisheye camera according to claim 1,
The fish-eye camera, wherein the shift mechanism shifts the rear zoom lens group by a drive mechanism. The fisheye camera according to any one of claims 1 to 3 and a captured image captured by the fisheye camera are used as input data, output data is generated based on the input data, and the output data is displayed as a display image. As an image processing system comprising an image processing device for displaying on a display unit as
The image processing apparatus includes:
Detecting means for detecting a specific part of the display image;
Zoom means for applying the zoom to the detected specific part;
An image processing system comprising: An image processing method in the image processing apparatus according to claim 4,
A detection step of detecting a specific part of the display image;
A zooming step for performing the zoom on the detected specific part;
An image processing method comprising: A computer included in the image processing apparatus according to claim 4,
Detection means for detecting a specific part of the display image,
Zoom means for applying the zoom to the detected specific part;
An image processing program that functions as an image processing program.

Images